Page 1 of 2 12 LastLast
Results 1 to 13 of 15
  1. #1
    Join Date
    Oct 2005
    Location
    Buffalo
    Posts
    195

    Looking for some comments on the Consumer Reports ranking of forced air equipment. The footnote basically explains there was no significant difference between the different manufactured units (with the exception of Goodman). I always trusted the opinion of CR but after this study I must reconsider. How can Heil and Tempstar be different?!?!

    Anyone have experience with private labled brands? (Xenon, Everest, etc.). We have been approached about selling these brands but I am more than hesitant to take them on. Their profit model looks attractive...but what about delivering quality to the consumer?!?
    Eager to learn, willing to teach, reluctant to admit defeat.

  2. #2
    Join Date
    Oct 2005
    Location
    Buffalo
    Posts
    195


    ....and I forgot to ask...anyone have input on Maytag furnaces?
    Eager to learn, willing to teach, reluctant to admit defeat.

  3. #3
    Join Date
    Apr 2002
    Posts
    1,606
    My father lives and breaths by what Consumer Reports says.

    Not so much me. I'm just one of the many in the real world of svc & repair of those highly rated engineered, lab tested equipment.

    Would be interesting to go back and see how CR rated some of those early model prizes before they got some real field
    exposure. You know the early Pulses or the great Rheem Drum type models.
    "The Bitterness of poor quality remains long after the sweetness of low price is forgotten"

  4. #4
    Join Date
    Mar 2004
    Posts
    1,042
    When you run statistics on a data set like that, there is always some noise. It's just the same as when they do opinion polls on who people will elect to be the next President. Joe Blow will get 48% of the vote, with a margin of error of plus or minus 3%. The statistics will actually tell you your margin of error for a given data set.

    CR does good statistics on their survey data. So while they list the actual numbers they will also tell you the margin of error. Let me make up some numbers for illustration- say that 7% of respondents said they had a problem with their Heil so-and-so in the first year, but 9% of those with Tempstar had a problem. If the margin of error is big enough, you can't say with statistical confidence that there is a difference in the real world. Their data set is the survey responses that come from their readers, which do not perfectly represent the whole world.

    For example, if they measure 7% for Heil and have a margin of error of 3%, what they have really determined the actual number (real world) is almost certainly between 4% and 10%. If Tempstar measures 9% (meaning a range of 6% to 12%), the two ranges overlap. It could be that the actual failure rate is 6% for both. The only way to measure the 6% directly, though, is to poll every single person who had a Heil and every single person who had a Tempstar, and somehow get every single one of them to respond to your survey (and not make any mistakes). Thus, given my imaginary numbers here, it could be that the 7% is Heil and the 9% is Tempstar, but the reality is that both are actually 6%. Or 7, 8, 9, or 10%. Knowing the margin of error makes the all the difference in knowing what you can conclude from the data. That's why when CR publishes a table like the one they have on furnaces, there will be a footnote that says "Differences of less than X points aren't meaningful." That's their way of helping people who don't know statistics to understand the limitations of the data. If you don't read the footnote, though, you can easily go too far in drawing conclusions from the data.

    If they could get more responses to their survey, the margin of error would be smaller. As it is, they already send a survey to every magazine subscruber every year, so they're doing the best they can with the data that they can get. This kind of research is actually pretty challenging. They try to present the results clearly and fairly, without getting into the statistics too much ('cause they just make people's brains hurt). Sometimes it's hard to be a good consumer of that kind of data without having a lot of statistical training, though.

    In the case of their furnace reliability ratings, the numbers were clustered pretty closely together for all of the brands. The only brand that was not in the zone where the margins of error were overlapping was Goodman. So for all the excitement that table brought to this board, the only conclusion you can safely draw from it is that all brands are about the same, except that Goodman is a couple percent less reliable (I'm quoting that from memory, so no promises on that number being exactly right).

    They have the same issues with their car reliability ratings. You can find differences in their ratings of the Ford Taurus and Mercury Sable, for example. Of course they are identical cars, but that's how the numbers fall out. That's as tightly as they can predict the actual number (the "if you asked the entire world and the whole world responded" number) given the number of responses they have in their data set.

    [Edited by wyounger on 10-04-2005 at 04:21 PM]

  5. #5
    Join Date
    Nov 2000
    Location
    Eastern PA
    Posts
    68,933
    The CR ratings of furnaces is the same as their ratings of cooling equipment; useless survey.

    Why do I state "uselesss"? As has been mentioned already, CRs own footnotes shows that the margin of error is greater then the degree of difference in the brands.

    Also as has been noted, the same exact product with different names is rated with different results. It is a survey. There was no equipment tested. No one ever even looked at a piece of equipment.

    A survey is an opinion based on how other opinions are put into a data sheet. As the saying goes; "opinions are like buttholes, everybody has one."

    A survey is also in the catagory of statistics. Again, as the saying goes; "there are liars, there are damned good liars, and then there are statisticians."
    Government is a disease...
    ...masquerading as its own cure…
    Ecclesiastes 10:2 NIV


  6. #6
    Join Date
    Mar 2004
    Posts
    1,042
    Actually, the difference between the unqualified who do surveys and statisticians doing surveys is that the unqualified just assume that the results are accurate. The statistician can calculate how accurate the results are, and will tell you so. Unfortunately the general public generally isn't prepared to understand the results of the statistical analysis without quite a bit of coaching. That's why I say that the best statisticians spend most of their time telling people the details about what they don't know.

    But Robo, I have to disagree that there was no equipment tested. There are tens of thousands of furnaces running in CR readers homes. That's a great natural experiment, and it makes sense to see what you can do with the data. CR would have been best off to not post their usual bar graph reliability ratings, though, since the margin of error was wide enough as to make the numbers basically useless for finding differences. The value of the numbers, with the right analysis of their meaning, is that the data indicates what we always say here- that there isn't really a difference from one brand to the next. It's the quality of the installation and maintenance that matters. They did actually say that in the body of their article, by the way ;-)

  7. #7
    Join Date
    Mar 2003
    Location
    PA/DE area
    Posts
    1,535
    Consumer Reports also picked THE FLAMING FILTER Aprilaire 5000 and Pulse furnace,Glowcore and Hydrotherm boilers,so you tell me!!!
    It's NOT the BRAND,it's the company that installs it!!!!!

  8. #8
    Join Date
    May 2005
    Posts
    996
    i heard about the flaming filter! i also had a call where the ho tried to change the 5000 filter and he goes, i tried to change this thing 10 times this morning and every time i touched it, i got shocked. its called turn off the power and wait 30 seconds for discharge!

  9. #9
    Join Date
    Nov 2000
    Location
    Eastern PA
    Posts
    68,933
    CR used to actually test products themselves. Here lately they have been quite slack in their standards. I also am not buying the "no influence" agenda any more after seeing some of the obviously paid for picks on a number of products.

    It used to be that I would not buy a product without consulting CR. I used to have a catagorized collection of their products testing. I think it was about 15 years ago that I read a completely bogus report on air cleaners that got me realizing that they have really changed for the worse.

    Let's face it, CR is not in business as a consumer advocate, CR is in business to make money.
    Government is a disease...
    ...masquerading as its own cure…
    Ecclesiastes 10:2 NIV


  10. #10
    Join Date
    Feb 2005
    Location
    Toms River, NJ
    Posts
    425
    They were right about one brand...need I say more???

  11. #11
    Join Date
    Aug 2004
    Posts
    52

    Goodman Last! That's no a brainer






    [Edited by rosc on 10-11-2005 at 02:05 PM]

  12. #12
    Can anyone list these in order, according to CR?

    I have a Thermo-pride NG forced air unit and am wondering how it stands. I have heard it is a good unit.

  13. #13
    Join Date
    Aug 2002
    Location
    Office and warehouse in both Crystal River & New Port Richey ,FL
    Posts
    18,836
    Originally posted by wyounger
    Actually, the difference between the unqualified who do surveys and statisticians doing surveys is that the unqualified just assume that the results are accurate. The statistician can calculate how accurate the results are, and will tell you so. Unfortunately the general public generally isn't prepared to understand the results of the statistical analysis without quite a bit of coaching. That's why I say that the best statisticians spend most of their time telling people the details about what they don't know.

    But Robo, I have to disagree that there was no equipment tested. There are tens of thousands of furnaces running in CR readers homes. That's a great natural experiment, and it makes sense to see what you can do with the data. CR would have been best off to not post their usual bar graph reliability ratings, though, since the margin of error was wide enough as to make the numbers basically useless for finding differences. The value of the numbers, with the right analysis of their meaning, is that the data indicates what we always say here- that there isn't really a difference from one brand to the next. It's the quality of the installation and maintenance that matters. They did actually say that in the body of their article, by the way ;-)


    Asking readers their experience with a brand ,doesn't factor in the design,install and competence of the installers and service techs.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Comfortech Show Promo Image

Related Forums

Plumbing Talks | Contractor Magazine
Forums | Electrical Construction & Maintenance (EC&M) Magazine
Comfortech365 Virtual Event