If you open a basement window, the cold air will quit coming in around the fireplace. Before you do a Vega-matic on the fireplace, fix the house. You obviously have a negative pressure problem or else the outdoor air would not be trying to get inside. Seal upper level penetrations first. Provide makeup air for all powered exhaust fans. Seal and balance ductwork. Ensure full coverage with insulation.
As for the fireplace, you can get some cold air in even if the vent termination is properly sealed to the outside wall through the seams in the DVP pipe. If this is an older unit, there would need to be a fiberglass rope gasket around the vent pipe siliconed to the rear outlet of the fireplace. If the wall over the mantel is insulated but the outer perimeter is not, I recommend eventually weatherizing these walls as JTP discussed. This entails foaming and caulking as many seams and gaps as possible. Then, fiberglass batts without Kraft paper in the stud bays. On this sheath with Thermoply with seams sealed with UL 181-B/ FX foil tape.
Around here, this work can cost 10X the price JTP quoted. Understand, he works for a major mfr.s installing distribution channel where pricing is geared towards builders and not retro. While his Division sells retail the company's pricing is low margin for the most part. To do this job properly, I would pull the fireplace from the inside, weatherize, re-install with RTV silicone on the outer pipe joints and firestops then seal it up. Note: you cannot pack insulation in the space btw the top of the Fp and the header, which is why it is useless to insulate over the mantel---it leaves a huge uninsulated hole in the thermal envelope of the home.
back from EXPO