If there any ductwork on this unit? Sounds like it's a package unit, and either heats hot water with a desuper heater or it's a combination water/air unit that alternates heating water or air.
I agree, I wouldn't want a compessor anywhere but in a basement.... not in my home, it's too quiet. I can clearly hear the water circulating on my recirculation loop on my water heater it's so quiet inside. A compressor... no way.