I never really took the gun outside. My outside temps came from a separate sensor that's pretty accurate. So I shouldn't have had too much of an issue with that.
Your thoughts about air leaking in from the upstairs & bathroom fans could definitely apply. The house is far from airtight (it's not drafty, but not airtight either). I'm sure there are plenty of places for air to leak in to the house. Hopefully most of that will get addressed when I rip out the upstairs ceiling and redo the insulation/drywall.