Are the folks at Consumer Reports a bunch of idiots?

In this thread:

There is talk of how Consumer Reports says electric burners heat up faster than gas ones. Obviously that’s just plain stupid (though since there is no direct cite, I can’t say if it’s poor memory or CR being dipshits), because when I turn on the gas, I get heat NOW, whereas if I turn on an electric burner I get heat in a very short time.

I also have experience buying a “recommended” dishwasher from their list. One that supposedly does a good job of cleaning and is efficient, but wasn’t 800 - 1000 bucks. Yeah. I have to run the dishwasher with a shitload of extra detergent and on the “heavy soil” setting (where it runs for about 2 hours or more) if I want clean dishes. There goes my efficiency. Oh, and my old dishwasher didn’t require as much time or as much detergent as my new one.

In a lot of ways, reading articles, they seem to come off more as snobs than of knowledgeable people.

So, does anyone have anything GOOD to say about Consumer reports?

My experience has been hit-and-miss. We bought a car seat based on their recommendation, and it was the hardest tangle of straps I’ve ever had to deal with. Yeah, I want my kid to be safe, but if I can’t figure out the system correctly, then nothing has been accomplished.

They also miss the point, it seems to me, on cars, often. They always focus on the profit that a dealer is getting or manufacturer. I get that, within a brand and model, that makes sense, but they even do it across models. At that point, who cares about profit? If you rate two cars the same, and can get X at a lower price than Y, what’s the problem if X gives the dealer a higher profit margin?

They love the protein dishwasher detergents, but I can’t tell the difference.

All in all, I’d say you have to really consider what they’re looking for in a good widget (which they are usually pretty good about explaining), and then decide if that lines up with what you’re looking for in a good widget.

That’s not poor memory, I had the article in front of me - May 2006, WRT the gas stove thing.

I do hope they’re not entirely full of shit, as I’m buying a… gas stove they suggested. They do, however, like Electrasol for dishwasher detergent, and I just had a repairman come over and tell me that’s why my dishwasher leaks - Electrasol can cause a suds lock and the thing can’t drain. He had me run a cycle with white vinegar and then start using Cascade, and indeed the leak is no more.

I believe that the actual claim is that electric will heat up a pot of water faster than gas. Their test for heating speed is to time a 6L pot of water to a boil. Therefore, when they say electric is “faster” they’re referencing that test. I can absolutely believe that electric can do this faster than gas. Direct contact with the element vs. an inch gap with a flame under it.

You definitely have to take their ratings with a grain of salt. They liked the blender that I threw away because it sucked so much. It’s a data point at least, and a decent place to start when thinking about a purchase.

I use it before a big purchase but only in conjunction with other review sites, like amazon or epinions. I usually narrow it down a little with Consumer’s to get the real lemons out and then read the ‘real life’ reviews on other sites. Usually I get more helpful info from the real people reviews since you can look for trends in the reviews and they tend to give more practical advice.

It is helpful for things like safety ratings and other tests that people can’t rate on their own.

I subscribe to this rag, and like others, I think it’s best to use as a starting point. When I first started reading it, I thought it was great. So I bought one of their recommended best buy vac. cleaners. Now, this VC sucks like nothing you’ve ever seen, but

  1. It’s VERY heavy, and difficult to impossible to use on stairs.
  2. It sucks so hard, it actually took carpeting fuzz/fibers/whatever out of the not-so-new carpeting I used it on. I mean, I want it to clean, but leave the carpet!

Ahem. Now then. Thank goodness I didn’t throw away our old canister vac. that still works just fine. Sort of. The weird thing is, this new sucking monster doesn’t seem to work too well on hardwood, tile, whatever. But it should, as it sits low enough to work. I haven’t figured this out yet. (CR said it was good for this use)

But there is one thing I’ve always wondered about C.R. They NEVER seem to rate expensive products/appliances well. I’m talking about house appliances, like Viking and SubZero, and vacs like Kirby. These things are always rated low, and they are so over the top expensive, that I wonder how they stay in business. That makes me believe that C.R. does target a certain audience, older folk of average means. If Viking products suck as bad as CR says (expensive, repair prone, etc, etc), the company would shut their doors.

Someone is buying them. I can’t believe rich people are all stupid when it comes to this stuff.

I have considered Consumer Reports unreliable (at best) since reading their comparison of two VCRs some years ago. They raved about how wonderful one of the machines was, while utterly panning the other one.

Well, I was an electronics tech at the time; I spent a substantial amount of time repairing VCRs, and knew the VCR industry very well. As a result, I knew that, although the two models in question had different brand names and model numbers, they were in fact identical. They were made by the same company, on the same assembly line, deviating only for the brand-labeled cosmetic components and model number stickers. There was no difference in quality, functionality, or price between the two machines.

Even if they got a lemon of a machine for testing, that article indicated sloppy research at best, and bias at worst. I’ve ignored CR ever since.

Their ranking formula seems to be highly price sensitive, and they generally seem to ignore long-term life of an item.

As for Kirby vacs, I’ve got a 22-year old Kirby that looks and works like new. The local Kirby dealer said so last year, when I brought it in for a brush roller. CR will look at a vacuum for a month or so in testing, and ignore that a $1500 Kirby will outlast, outclass and outperform the $89 “Best Buy” flimsy plastic vac that will last a year if you’re gentle with it. If you can front the $1500, the Kirby will be a far better purchase than $100 per year on disposable vacs.

Of course, my new Dyson leaves the Kirby in the dust. :smiley:

I have subscribed for the last several years, and find them often wanting.

We always used them back in the 1970s with nary a complaint. We felt they routinely led us to quality products. Not so much these days. It seems their criteria have changed.

Most annoying however, is that the models they test are almost always gone by publication time.

I just got my CR within the last couple of days, but I can almost guarantee that at least 70% of any electronic equipment models they rated in that issue will not be on the shelves of any store I care to consider buying them at. Their ratings in these categories are often less than worthless.

Considering the title of this thread and some of the responses, I think this is more of a collective bitch session than an actual poll.
Moving thread from IMHO to The BBQ Pit.

OK, just to demonstrate what I was talking about, instead of doing my job I compared CR’s list of Digital Cameras (one of the categories where they claim to keep things up to date in their online version, regardless of when they publish their reports).

Here are their top five picks for compact (as opposed to sub-compact) digital camera as of right now (it’s in the current issue), and what I found on various potential resellers’ sites regarding availability and price (my own experience is that these sites have a fairly accurate correspondence with what’s on the shelves at my local store).

1 Sony CyberShot DSC-W5 (A CR Best Buy) $220

Sony: site lists refurbished models in limited quantities only
Best Buy: Not listed
Circuit City: Not listed
Fry’s Outpost: Not listed
Ritz Camera: Not listed
Samy’s Camera: Not listed
Sears: Not listed
Target: Not listed
Wal-Mart: Not listed
Amazon: unavailable directly, no price
2 Canon PowerShot A620 $325

Canon: has page for product, no indication of availability
Best Buy: Not listed
Circuit City: $249
Fry’s Outpost: $249
Ritz Camera: Not listed
Samy’s Camera: $249 -rebate
Sears: Not listed
Target: $299
Wal-Mart: Not listed
Amazon: $Click-and see!

  1. FujiFilm FinePix F10 $315

Fujifilm: has page for product, no indicaton of availability
Best Buy: Not listed
Circuit City: Not listed
Fry’s Outpost: Not listed
Ritz Camera: Not listed
Samy’s Camera: Not listed
Sears: Not listed
Target: Not listed
Wal-Mart: Not listed
Amazon: $331, marked down from $399

  1. Kodak EasyShare Z700 $245

Kodak: $199
Best Buy: Not listed
Circuit City: Not listed
Fry’s Outpost: Not listed
Ritz Camera: $299 for camera AND printer dock
Samy’s Camera: Not listed
Sears: Not listed
Target: Not listed
Wal-Mart: Not listed
Amazon: $149
5. Olympus C-5500 Sports Zoom $280

Olympus: lists camera in “archived products”
Best Buy: Not listed
Circuit City: Not listed
Fry’s Outpost: Not listed
Ritz Camera: Not listed
Samy’s Camera: Not listed
Sears: Not listed
Target: Not listed
Wal-Mart: Not listed
Amazon: unavailable directly, no price

Not only are their top five picks unavailable, but their prices are way off. I might dismiss researching the Canon A620 due to the price, not knowing it’s actually $75 less at most of the places that even sell it.

I realize that CR buys models direclty from the shelves, and that there is testing and publishing delay involved, but these results are so pathetic they might as well not bother.

One thing they do that makes the whole subscription price worthwhile is that annual poll of cars and repairs.

Of course, nothing matters more to me in car ratings than reliability.

Years ago, back when the Canon AE-1 was a current camera, I picked up their 35mm SLR camera special issue. As an avid photog for many years by that time, and working at a photographic equipment retail store at the time, I thought I would find it good fun reading.

However, apparantly the people they had write that issue were not photographers, not having any familiarity with the processes involved. Also, it was painfully evident that they hadn’t even read the instruction manuals of the cameras they “tested,” as they got virtually every feature wrong on some of the most popular 35mm SLRs ever. (OM-2, FE, ME-Super, XD-11, R-3, A-1, FS-1, RTS, etc…)

I took it back to the store and all of us had some great laughs reading their reports.

I’ve never taken that magazine seriously since.

Perhaps Consumer Reports uses the manufacturer’s recommended retail price for cameras rather than the “street price” that cameras actually sell for in stores. I noticed this phenomenon in Popular Photography–a camera price mentioned in the articles (such as in a review) was always higher than what you’d see for that same camera in a store or even in the ads in the back of that very magazine!

My grandmother swore by Kirby, and that woman was a clean freak. She abused those machines and they never let her down. So, I’ll take that as two satisfied customers. But you are right. Dropping $1500 on a vac. is a tough thing to do. Come to think of it, I’m wondering where old gramma got the cash for that?

Now, tell me about your Dyson. CR rates them average at best, but I want to know more. My carpet needs a break!

I don’t think they’re “a bunch of idiots.” Their approach is to attempt to objectively compare products. In order to do that, they need to establish standard criteria, such as the amount of time needed to bring six liters of water to a boil. it’s not a perfect approach, but what is the alternative? (Someone suggested sites like Amazon or Epinions, but my impression is that the reviews on those sites aren’t from people who have tried all the competing products.)

I know a big part of their reputation is made testing cars. I’m not saying it was a great car or anything, but they claimed the Dodge Omni was a dangerous car. Why, you may ask? Because if you were in a turn with the steering turned all the way to one side, and you suddenly turned the steering all the way to the other lock as fast as you could, then the car would lose control. They offered no reason why you would pull a stunt like this, and they didn’t perform this test on any other car in that issue. They really don’t know very much about cars.

Except that some of their “reviews” are actually estimated reliability ratings based on previous model years, which by itself isn’t bad, but when you do dumb-ass stuff like rate a vehicle highest based on data from only one previous year(Honda Ridgeline) when there are others with many years of data (Ford Ranger, Dodge Dakota, Toyota Tundra), it makes me wonder how much of their “reviews” and stuff is just guesstimation to make people comfortable, and not really to show anything interesting.

There is, I suspect, another problem which is what I call the “Choice” effect (that being the name of the Australian magazine of the same type).

When they do a test and find a particular widget that is cheap and very good, it sells out very quickly and I suspect that the manufacturer makes sure they discontinue it for reasons that are counter intuitive but not so much if you understand how the widget market works.

The sale price of widgets is set by marketing people according to what the market will bear, and how the most money can be extracted from the most people, not by price of manufacture (which merely sets a floor). Manufacturers sell (as a typical example) an entry price widget, a mid price widget and an upper price widget. The disparity in price in no way reflects the disparity in manufacturing cost.

The entry price widget typically lacks a few features but in no way does that lack justify the price drop: the features are missing just so that it justifies the purchase of the model above by those that can or will. At the same time, the manufacturer can still make some money out of those with with little money to spend, which it wouldn’t if it priced all its widgets high.

The upper price widget similarly has a few additional features, but again in no way does the cost of those features necessitate the price hike. One of the big features that the upper price widget often has is prestige and reputation. This is achieved by selling upper price widgets under a different brandname upon which the manufacturer has lavished marketing expenditure in order to convince people that the brand is higher quality, more prestigious etc. even though the upper price widgets may well differ only in terms of a few stickers and mouldings from the lower price widgets, and come from the same factory. This allows the manufacturer to extract more money from the rich while selling them nothing much more than they sold the poor for half the price.

If Choice/CR does a report that shows that [low price point] Type A Brand X widgets are cheaper, better, more reliable and easier to use than [high price point] Type B Brand Y widgets, then that will kill the sales of the latter. And the latter is where the the real profit is. So if one manufacturer owns Brands X & Y, they certainly don’t want Type A Brand X widgets on the market.

What marketing people know, but most consumers don’t like to admit to themselves as much as they should, is that in the absence of objective information people assume expensive things must be better.

I have heard (but admittedly never actually from first hand sources) that an experiment was done in which catfood sales were proved to increase by increasing the price while controlling all other variables.

How much objective information does the average consumer have in order to determine that a particular product is actually better quality?

Many rich people got where they are by being savvy about appearances. They care that they are seen to have expensive widgets, not whether they actually work better.

The thing about Viking and Sub-Zero and Wolf though, is that, while they are readily available to the general public, most people don’t have the need for a professional grade 46" range. Viking and Wolf make most of their money with contracts for restaurants or on TV shows that you see on food network (pay attention to the credits at the end of Good Eats or whenever Emeril is sucking off his Viking cooktop). What CR does is they take this information into account; since most people won’t EVER need a commercial grade range, they’re not going to rate it as high because the damn thing is so expensive and you could get a quality Kitchenaid range that LOOKS commercial grade for half (or sometimes even a quarter) the price.

As for electric heating up water faster than gas, yes it’s true on some makes of radiant heating cooktops, and most certainly true on the newer induction ones (see my comments in the linked thread).

FWIW, the Bosch dishwashers that came out on top in CR for 2006 are definitely worth every penny (though they don’t have built-in hard food disposals). Remarkably quiet, and unbelievably efficient, if I had the money I’d definitely spring for one.