How useful are they? are they worth paying for? How much does it weight do you put on it for a car buying decision? Is there another guide that you like better?
Here’s a fairly critical discussion of Consumer Reports.
The opinions online are that CR is the end-all be-all if they like <MY CAR>, and if they don’t then they are obviously in the pocket of <MY CAR’S COMPETITORS>. I remember last year (during the unintended acceleration thing) when CR posted a “do not buy” on the Lexus GS truck because they didn’t like the way the stability control worked, and more than a few people on the Toyota boards were unironically convinced that Consumer Reports was being paid off by GM/Ford/etc to smear Toyota. :rolleyes:
Overall, I think CR’s survey data is as useful as anyone else’s survey data, and generally do reflect reality, but you really have to take their “recommendations” with a very critical eye. I’ve listened to interviews with various people who work there, and the rationale behind their recommendations are extremely dubious. A prominent example from memory - they “recommend” the Honda Ridgeline, despite its fairly modest towing and hauling capabilities, over other pickup trucks, because it was their opinion that most pickup truck buyers did not need the capabilities of a full size pickup truck anyway. That was straight from an interview with one of their editors, and it stuck in my mind as being particularly odd. I wonder if their “recommended” minivan would be a Geo Metro, since in their opinion no one really needs an actual minivan either? They also seem to be rather cavalier in their treatment of year by year model changes. A particular model might have one bad model year, and subsequently be improved, but it is not clear that CR takes this into account although they claim to.
I would say use their survey data as objective raw data and ignore any of their recommendations that are anywhere remotely subjective.
I agree with their logic.
I also agree with that logic, and purchasers who really need the towing capacity know that they do and buy accordingly.
Also, that critical discussion starts of flawed. I’ve done surveys before (hated them) and a 6% response to an unsolicited survey is about right. The bigger question is if his ‘suggestions’ would give more than a 50% increase in responses (up to 9%). Personally, I don’t think so. The rest of his discussion is similarly flawed. Sounds like someone who’s purchase was panned by CR, and is hiding behind a PhD while discussing a topic he’s unfamiliar with.
I saw a lot of the same Toyota griping on Edmunds, where Toyota owners* were screaming conspiracy. All that does is forward my theory that the dumber one is, the louder one is (including ‘loud’ in the online sense). I was happy to see Toyota mostly vindicated, but of course that didn’t get the air time the alleged problems did.
*I am a Toyota SUV owner, but I know the only conspiracy was among ambulance chasers (with apologies to BrianGlutton 
) looking for a big class action payday.
Something else to understand with any survey system is that first they generate a score for each vehicle, and then they generate a ranking by sorting th evehicles into order by score.
If the scores range from 1 to 100 and the top 5 vehicles in the class all score between 71 and 73, guess what: Statistically speaking that’s a 5-way tie for first place. There is ZERO useful information that #1 is actually any better than #5.
But that’s sure not the way they spin the results, nor is it the way naive readers interpret them.
And also understand that their score depends very much on how they weight the factors. If they care about handling 4.3 times more than they care about air conditioning airflow volume, and you personally care 3.8 times more for handling than airflow, well then *their *scores won’t reflect *your *preferences. And their ranking really won’t reflect your preferences.
Bottom line: pay most attention to the individual items you care about, just a little attention to the total score of each vehicle, and zero attention to the ranking of vehicles.
Yep, that’s basically it. CR’s data may have a grain of truth to it, but they insist on presenting everything in the format of a cracked.com top 10 list when reality is a bit more nuanced.
The counter point it that they are no worse, I suppose, than every other car magazine on the market. Car & Driver has a similar way of always finding a reason to put a Honda or BMW at the top of every ranking list.
What did those people say when Toyota came up with a software fix, and then CR tested the truck with the fix, and it passed? Then CR took off the “do not buy” warning on the vehicle as long as it had the updated software.
You can probably get a copy at the library to see if you like the information.
Thank you all.
I agree with this.
I agree with your bottom-line assessment, but I don’t think it’s quite fair to say that" that’s sure not the way they spin the results", regarding the ranking.
Sure, they list the cars in the order of their scores but they don’t make a big deal out of it. They don’t say you should buy the car ranked number one. The main thing they emphasize is the checkmark indicating that a vehicle is recommended. If it’s recommended then the ranking is not important.