Discussion thread for the "Polls only" thread (Part 2)

And I don’t generally go around recommending household fuel oil providers, anyway. I might, I suppose, if the subject comes up, but it doesn’t usually. There’s only about three options around here and anybody who uses fuel oil has generally either settled on one or calls around and buys from whoever’s cheapest at the moment.

Every time I used the IT help desk at work, I got a survey asking me how likely I was to recommend the IT help desk. We had no options. I wasn’t ever going to recommend it to anyone because everyone already knew about it. I always said I was unlikely, and then had to fill in the text to say why.

Why?

Maybe somebody loves the place and doesn’t want to tell anybody about it because they don’t love crowds. Maybe somebody loves the place but doesn’t have any friends; or it’s a hamburger place and most of their friends are vegans; or it’s a car sales place and most of their friends don’t have cars; or they’re only in town twice a year and their friends are never there at all. Maybe somebody thinks the place is meh, or even hates it, but it’s the only place for thirty miles where you can hang out with a friend at two in the morning drinking coffee, even if it’s terrible coffee. Maybe they think the place reeks of bad perfumes and they’re never going back, but they know a person who likes that kind of thing so they’re going to recommend it to them.

Wouldn’t they find out how their customers see their business a whole lot more accurately if they asked them that, instead of asking them something else entirely?

But they don’t actually care if you like it. They care if you are likely to return, and they care if you will recommend it to others.

  1. Since only large companies are using this, they aren’t interested in individuals. They want to know the general trend.
  2. If it’s for a retail store, that store is the only place that will be affected by the ratings
    3a. Upper management is actually horrible at statistics
    3b. Stores are rated only by the number of 5 star ratings they get.

So, if you’re going to fill in a rating, give them 5 stars or don’t rate them. Or 1 star if there’s something dreadfully wrong with that store.

Related, a few years ago, Jack in the Box changed their mayo packets from Hellman’s to their own brand, which has a noticeable mustard taste.
I went to their website to complain and there was absolutely no way to complain without choosing which location I bought from. I’m sure that the only people who got my complain were the district and store managers

I enjoy rating things/places I truly enjoyed and liked. I’ll do a full review with pictures, etc. Google Maps tells me I’m in the top 5% of restaurant reviewers!

It seems possible that they were using a third-party support service, and rather than looking for a literal answer they were looking for some sort of understanding of whether it was working for the company’s users or not.

Just saw a television series which included a death doula as one of the secondary characters.

Regarding the surveys, If it’s Microsoft, asking me yet again if I have time to answer just two questions, I will normally close the window. I have no choice about using their products. And if I do answer it will be extremely negative.

Same with HoxHunt. Do they really expect that anyone would recomend HoxHunt to their friends?

This, I suspect.

If that’s the case, then they are asking the wrong question, as @thorny_locust suggests.

I mean, the counterarguments are, 1) the user understands what’s implied by the question even if they dislike the semantics, and 2) everyone using exactly the same NPS question allows for some calibration across surveys. From my perspective, if the vendor did a good job I want them to have a good score, if they didn’t, I don’t.

Making a separate note because it’s a separate topic, but I get these requests from Amazon deliveries if they did a good job. Usually they do and I give them a good rating, but. sometimes they leave it in a weird location…but I’m not gonna downrate someone busting their ass up and down the streets just because they stuck in front of my garage instead of what the instructions said.

If I answer at all, I’m going to answer the question as asked, not the question that I might guess they mean to ask instead. Or if possible I’m going to skip that one altogether.

I don’t think it’s a bright idea to assume that all or even most of the people taking a survey are going to guess that the survey means the same thing by asking a different question entirely.

For (nearly) every issue there’s an answer that is simple and wrong –

If you get easy calibration of a meaningless question, or of one with seriously confusing meaning, your results will be meaningless.

(I also always want to say to those people publicizing results that say ‘X percent of people say the country is headed in the wrong direction!’ “Did you ask them which wrong direction they think the country’s headed in?”)

I am. I’ve had a couple of packages damaged by rain. And i can’t get into the garage if they leave a package in the way.

If the garage door is open and they tuck it in there instead of climbing up my front steps, fine. But if they leave it in the middle of the driveway, yah, I’m going to complain.

I’m with @thorny_locust on this. For a burger place, they didn’t actually care whether you are there because you like the burgers or whether it’s a convenient place to hang out with your friends. The two things they want to know are
Will you return?
Will you encourage others to go there?

So that standard question is exactly the right question to hit the second of those. But for your Microsoft operating system or your employer’s help desk, it really isn’t. And they ought to ask the question they want the answer to.

Unless they’re interested in comparing across a wide range of similar vendors who ask the exact same question across many companies. I’ve seen a lot of the backend results from NPS surveys, there are plenty of people who will interpret the question implicitly and answer it that way.

I used to answer surveys from businesses if (a) there was something about the transaction that was particularly good or bad, (b) I was in the mood for it, (c) I felt I had time for it, and/or (d) there was some other factor that was favorable, such as sunspot activity, etc.

That all came to a screeching halt across the board when my bank asked me if I would recommend the teller I had dealt with on a recent visit to deposit a check. WTF? Who on God’s green earth recommends a bank teller? And why? And how would the friend follow up on that recommendation if they wanted to? All I ask is that they process the transaction in a reasonably competent and efficient manner so I can be on my way, which they always do, and not be overtly rude, which they never are. And I’d rather they not be proactively friendly either, because I don’t want to spend energy processing how to deal with it. Just be professional. Get me in and out so I can go to Burger King down the street and get a Whopper for lunch, where I hope they’ll treat me the same way.

In other words, my optimum hope for a bank teller transaction is that it not be memorable in any way. If I were a branch manager, my metrics for great job performance for tellers, ideally, would be that I could somehow see that they’re consistently providing forgettable service over an extended period of time, if the corporate office would let me do it that way. (Maybe that’s why I’m not a bank manager.)

And also, what’s my recommendee supposed to do?

Teller: I’ll take who’s next.
Customer: Are you Bertha?
Teller: No, I’m Jane.
Customer: Well, my friend dirtball recommended Bertha. I’ll wait until she’s available.
Teller: This is her day off.
Customer: All right. I’ll come back tomorrow.

So, stupid question. Stupid, stupid question. It can’t possibly provide any meaningful information because recommending a bank teller is something that’s simply not under consideration regardless of the service, which I’m pretty sure is what they really want to know about.

Yes, you might say it bothers me more than it should, but that’s an easy thing to do these days when I get so many of these survey requests. Staples: “You purchased a mouse pad at our xxx store. How was your experience?” Well, I found one on the shelf, took it to the cash register, and the guy rang it up correctly. What’s there to say about a guy ringing up a mouse pad purchase? He didn’t spit in my face, so there’s that.

Just stop it, people. Just stop it. Rant over.

Based on my own experiences, and lots of knowledge about how companies work:

  • Wrord-of-mouth and personal recommendations are widely understood to be more powerful than paid marketing programs
  • NPS is a straightforward question (in their minds, at least), and an easy-to-understand metric; they think that most respondents will give a straightforward answer to the question
  • They don’t really care about corner-case reasons why particular consumer go to (or don’t go to) particular locations, or won’t recommend something
  • Because NPS is used so widely, and because consultants publish NPS “benchmarks” for various industries, it gives them the warm-and-fuzzy feeling that they have an objective measure of how their customers feel about them

Also:

This. They want and need someone to tell them, “look at this, and you will have understanding.” Even if that’s not actually true.

They may have the feeling. They don’t have the actuality.

And thinking that you know what your customers think about you when you don’t actually know any such thing is worse than knowing that you don’t know what your customers think about you.

For the record, I completely agree with you.