We got a new customer satisfaction reporting system at work.
I was reading on of the reports today and I saw that we had fixed all but two cars correctly on the first visit.
Looking a bit further the report broke down where the issues were. Here is the breakdown:
Exterior 50%
Steering 50%
Electrical 50%
So I wrote them the following:
Um guys that is 150%. You can’t have 150%. You can have three problems each at 33.3% but everything has to total 100%
They wrote back:
The reason that section does not add up to 100% is because the question is multiple answer question so the customer can select several options. Therefore that section will probably never equal 100%.
That answer sounds perfectly cromulent to me. Maybe it’s not the specification you would like. Was this some kind of a custom-designed software system, or was it some kind of off-the-shelf software that you bought? If custom, you could ask them to do things differently to your liking.
What that report tells you is that 50% of (just those two?) unhappy customer chose “Exterior”, 50% chose “Steering”, and 50% chose “Electrical”. Since a customer can choose more than one problem in the same report, it follows that a single report could contribute to more than one of those statistics at the same time. Thus, it’s certainly possible for them to total up to any absurdly-large-looking number. Properly understood, that makes sense.
The original report probably has a lot more options than just electrical, steering, and exterior. There might well be a dozen or many more choices. Offering a list like this of every possible combination would be entirely impractical.
However, they COULD have offered the user a multiple-choice menu, like they do, but then analyzed it to give a breakdown of every combination of choices that any user actually selected. That could result in a report like obfusciatrist suggests here.
Yeah, but that would be only a little better for Rick to read than for the customer to fill out. With ten choices there’s over 1000 possibilities. Who wants to read a report wwith fifteen pages of tiny percentages and try to come to some useful conclusion?
Really, what they provided is what Rick wants, even if he doesn’t know it yet.
Let me toss in a bit more info.
The report does not tell me directly that I fixed all but two cars.
The report says there were 14 reviews and in another place it says I fixed 85.7% of the cars right the first time.
So I to do some math to come up with how many weren’t fixed right the first time.
So going in it is not the numbers aren’t clear.
For example looking at a different report I have 438 reviews 94.5% fix right first time and the following issues:
Engine 12.5%
Trans 16.7%
Tire 4.2%
Audio 8.3%
Exterior 12.5%
Steering 8.3%
Brakes 8.3%
Seats 0.0%
Heat AC 4.2%
Interior8.3%
Electrical 37.5%
For a total of 120.8%
Without doing some math these numbers aren’t user friendly to me.
Conceivably I could fix all but one car and if that guy ticked off every category I would have 1100% problems.
Sure, the total of all the various percents isn’t a useful number; but no one using these reports should really care about that number either.
If I was trying to use the report I’d worry about the ‘fixed right the first time’ percent, and I’d be watching for any unusually high (for the industry) problem areas in the breakdown. Or I’d be watching for changes in the numbers and then identifying the reason for the change so I could either fix the problem (numbers are trending up) or share the solution with others in the company (numbers are going down).
The total of all the percents doesn’t give you any useful information so I’d say ignore it entirely.
Why sum the percents? What are you expecting that to tell you?
From the data you provided, I can tell that 24 cars were not fixed correctly the first time (5.5% of 438), and nine of those cars had electrical problems (37.5% of 24).
If they were all 100%, then that would mean all 24 had electrical problem, all 24 had seat problems, etc.
ETA: More to the point, perhaps, what do you want the report to tell you?
Yeah I can do the math, but why doesn’t the report do it for me?
What I would like is either
What % of the cars not fixed correctly had A problem in X category. Or alternatively how many customers (number) had a problem with X?
Do you have any chance of getting the report customized to include the actual numbers like that?
Also: Is it clear that these reports only include cars that were not fixed right the first time? Or does it also include data for cars (like what their problems were), even for cars that were fixed right the first time?
Have you seen the actual on-line (I assume) survey that your customer are filling out (even a blank unfilled-out form)? Is it clear to you that the problem breakdown only includes cars that weren’t fixed right the first time?
And is it clear to your customer who are filling out this form that only cars not-fixed the first time should be included?
Well the way I am looking at it is there were 29 issues reported. 29 is 100% in my mind, so 9 of them were electrical. That is not 37.5% it is 31%
Yes I have seen it. It is part of the manufacturer’s customer satisfaction report that is 31 questions long and has yes/no answers, 1-10 answers and multiple choice answers. I get scored and rated (and paid) based on these results. I have a very vested interest in these numbers.
They just shifted over to this brand new program, and I have previously caught them fucking up some math, and frankly I think they are making it up pretty much as they go along.
Going back to the original example, to my way of thinking, we had 2 cars were that were not fixed properly. Three areas were identified as not fixed. All of something is 100%, therefore in this case 3 = 100%, so no one problem can be 50% of a three item set.
I am also somewhat concerned that someday I will get a real doorknob from the factory that does not understand the numbers and will rake me over the coals for having 37.5% of the cars I work on have unfixed electrical problems (no nimrod, I worked on 438 cars and 9 had issues, that is about 2%)
I think you need to be clear in your own head where the numbers are coming from and what they mean. When you say
and also
those are two different things. 9 is 37.5% of 24 cars and 9 is also 31% of 29 issues. At the same time.
This is your real issue. Math isn’t going to solve that one, but understanding the math might help you when the doorknobs come calling. (Or not, since they may have no interest in understanding the math when it saves you money, but costs them something.)
I think you’re thinking of the wrong denominator for these problems. There are three relevant numbers here that can be used as the denominator for the percents - the number of cars, cars with problems, and the number of problems the cars with problems had.
You have reviews on 438 cars. 94.5% were fixed correctly on the first try. Using 438 as the denominator, we can calculate that 414 were fixed correctly. (414/438 = .945)
Subtracting 414 from 438 gives us 24, which is the number of cars with problems - the second denominator of interest. ZenBeam has already done the calculations using this denominator to figure out how many cars with problems had a particular type of problem. For example, 12.5% of cars with problems had problems with the engines. 24 problem cars * .125 engine problems = 3 cars with engine problems.
We can repeat this calculation for each category (but ZenBeam already did) and then total up the number of problems. There are a total of 29 problems. If you want to know what percent of your problems are engine problems, you need to do an additional calculation using this new denominator. 3/29 = 10.3.
Here’s all of this in a little table that I sure wish I could format better.
Problem Area…% of Cars…# of Cars With Problem…% of Problems
Engine…12.5%…3…10.3%
Trans…16.7%…4 …13.8%
Tire…4.2%…1…3.4%
Audio…8.3%…2 …6.9%
Exterior…12.5%…3 …10.3%
Steering …8.3%…2 …6.9%
Brakes…8.3%…2 …6.9%
Seats…0.0%…0 …0.0%
Heat AC…4.2%…1 …3.4%
Interior…8.3%…2 …6.9%
Electrical…37.5%…9…31.0% Total …X…29 …100.0%
If I’m understanding everything correctly, the only percentages I would expect to see are 0, 50, and 100 for each category if you have two not properly fixed cars. The percentages, to me, say of the 2 that were not fixed, what was/were the problem(s). I don’t see where you are getting 3 from, nor why they should add up to 100%. Each category is independent. Either 0/1/2 of the 2 non-fixed cars had a problem in that category, so only 0%, 50%, or 100% are the possible values. If I saw 33%, I would be confused as to how that’s possible with only two customers.
