One thing has perplexed me and I can’t find an answer on TSD board anywhere. TV stations and networks are always claiming ‘2 million people watched program X’ and ‘8 million people watched program Y’. How the hell do they know?
The ways I thought they might know:
1: Maybe TV signal deteriorates or changes the more people receive it. So they can measure this and work out the viewing patterns. Umm…seems unfeasible from a scientific/physics point of view.
2: Your TV acknowleges its reception to a base station somewhere. I can’t see how this is possible, but scary from a privacy point of view
3: They monitor something like electricity grid usage and can see when people turn their TVs on. Hmm…not accurate enough as some other appliances might confuse the issue and how do they distinguish between channels then?
4: They have random phone samplers who call people up and ask what they watched, take an empirical sample and extrapolate it along some curve of some kind. I dont know anyone who has ever received one of these calls. Do you?
5: They make it all up.
In Australia (and I guess in most places) they have the “TV ratings”. This is by the same type of companies that run political polls. Traditionally, it was done by giving a few thousand people a log book into which they’s record their television viewing habits over the course of a weeks or so. In recent years, it has been done using a “black box” connected to the TV, which automatically records the information.
A problem with the old book system was that people weren’t entirely honest. When I was selected in 1990 or so, I claimed that I’d watched favourite shows which I’d actually missed, or claimed I didn’t watch stuff which I’d seen and didn’t like, in an effort to keep the things I enjoy on the air.
Ever hear of the Neilson Ratings? Neilson puts black boxes on a sample of TVs and extrapolates from there. It’s not magic; it’s statistics – experience has shown that a certain size sample chosen correctly give an accurate picture of the entire viewing audience.
There are also the Arbitron ratings, which are random phone calls and can give a number overnight.
I once took part in the Neilson ratings for radio – had to fill out a diary about what I was listening to. The diaries were the older method; they switched to the boxes because people didn’t always record things accurately. However, there are less than 2000 Neilson families in the U.S. (and they’re not supposed to talk about it), so it’s rare that you know anyone doing it.
I never understood Neilson. How can 2000 families give any sort of indication about what people in the country watch? How can they extrapolate from that 2000 to say that 5 million people tuned in to a certain show on Sunday night?? That seems like a bit of a stretch.
The regular, day-to-day ratings and the overnights are calculated with the Nielsen set-top boxes called “People Meters.” The boxes send the information to the Nielsen computers every night at something like 4:00 a.m., and the overnights are calculated and available by 9:00 a.m. The People Meters have become increasingly sophisticated in what they can measure: they can determine whether you are watching a broadcast station or a cable station, using your VCR, or playing a video game. Each family member has an assigned button they are supposed to press whenever they are watching, so that they can match demographic information to the ratings numbers.
There are currently around 5,000 People Meters in use, out of a total of 98 million television households (TVHH) in the U.S. Statistically speaking, the sample size is probably large enough to yield an accurate figure, but critics argue that the demographics of who gets the boxes are not representative of the population. There are also, in 49 local markets, simpler boxes which measure only tuning information but not demographic information. These boxes are in around 20,000 homes.
Nielsen diaries are still used, during the four sweeps periods (February, May, July, November). They are provided to around 1 million viewers in all markets, and each family member is asked to log when they watched, for how long, what channels, etc.
Nielsen is also currently working with Arbitron on a Portable People Meter, a pager-type device that can detect encoded information in a broadcast signal and thus measure ratings information even when the viewer is not at home.
If you take a course in statistics you soon find out that the accuracy of your data (assuming a good random sampling) is strictly dependant on the size of the sample, not the percentage of the population sampled. Therefore a poll of 2000 people can give you a high confidence level, whether there is 200,000 or 200,000,000 in the population.
sample size: 270,000,000
margin of error: 1%
confidence level: 99%
…and was told an accurate sample would be 16640 people.
Now my numbers are flawed, as 270,000,000 may not be the population of the United States, and it’s certainly not the population of TV viewers in the USA. I don’t know if Nielson measures households or heads.
Another problem with Nielsen ratings are the people who refuse to participate.
Like us.
My parents & I were asked, & we turned them down.
How does this effect the statistics, I wonder?
We turned them down because–[list=a]
[li]we saw it as an invasion of our privacy[/li][li]We had no benefit from it, [/li][li]It was inconvienent[/li][li]It doesn’t report, or even take into account, the people who aren’t watching anything–if all the shows on stink, & your TV is off, this isn’t reflected well in the statistics.[/li][/list=a]
So, people who are smart enough to say “no” are eliminated from the sample.
In the UK, and so presumably also in the US, the participants are chosen to be representative according to the latest Census data, broken down by Age, Gender, Socio-Economic status and Region. And probably a few other things too (size of household and stuff…I can’t remember now).
So, Bosda, they didn’t just randomly pick someone to replace your family…it would be people similar to you (even if you wouldn’t consider them quite as intelligent )
99% confidence level is to high. 95% is a pretty standard confidence level for surveys. Margin or error at 1% is also much higher than most of these sorts of polls 3% or 4% is around where you see most polls. 95% and 4% margin of error with 270,000,000 people gives a sample of 600.
The ratings are a farce. You just can’t take a sampling like that and extrapolate it to assume the viewing habits and tendencies of an entire nation. It’s just not possible. We may like similar things, but there are zillions of similar things on television.
One problem is that it’s very subjective. People may fill stuff in their diary that they’re not watching, or they may forget to fill in stuff at other times. And even worse, maybe they generally don’t watch a particular program, but they’re in a funky veg out kind of mood, so they watch it. And record it in their diary. Poof! Ratings go up.
Six hundred people (as gazpacho said) might be enough if the survey is doing a yes/no question. But with literally hundreds of channels available (and that’s just on cable, not broadcast) you could easily have a comparatively popular show that none of the surveyed people are watching! How do they rate those kind of shows?
Question about how cable tv works: If a lot of people in my town are watching a certain channel, does the cable company notice any kind of degradation of the signal? If so, wouldn’t monitoring that sort of thing be wonderful for Neilsen? A whole bunch of cable broadcasters could send minute-by-minute estimates of how many people were watching a given channel, and then just add them all up! Is this feasible?
This is not true. Ratings are expressed as a percentage of the total audience population, which includes people who aren’t watching anything.
Share, another figure frequently used in reporting television statistics, is expressed as a percentage of the audience viewing television at that particular time.
Most reported ratings are shown in a rating/share format, making clear the difference between those who are watching and those who aren’t. For instance this chart shows that out of people who had their T.V.'s on that night, 17% were watching the All-Star Game, but only 9.5% of the total available audience watched the game.
Television producers are acutely aware of those people who have their sets turned off. Schedules take into account which nights of the week more or fewer people are likely to be watching anything at all. For instance, the Oscars were moved from Monday night to Sunday night some years ago because Sunday has the largest number of people watching overall.
Choosing not to participate in the ratings may skew the ratings, but non-viewers are definitely counted.
This is a good point. I will have to review how the sample size formulas work. If your survey shows that 3% of people watched the show and the margin of error is 4% you are basically saying that with 95% confidence we say that between -1% and 7% of people are watching this show. So using margin of error at 4% may be to high.
In a well designed TV tuner there should be almost no degradation of the signal based on what channel the TV is tuned to. So this is not feasible. I don’t know how the new digital cable services work but I would not be surprised if the tuner boxes that you need with digital cable sent back information on what you were watching. But I don’t know if they do, just speculation on what is feasible so don’t say you heard that the cable company was spying on you through the digital boxes.
The ratings are not just based on 600 people. They are based on thousands, although there are several ways of gathering the data. (I believe there are also some attempts now at gathering data from group viewing locations, like bars and dorms.)
The ratings do indeed work well on an overall basis. Nobody doubts than ER and Survivor are watched by far more people than Baby Bob, or that UPN and WB and Pax get far lower numbers than NBC. The difference between being show number 45 and 46 is negligible, so fine tuning of that magnitude just isn’t needed.
But decisions on retaining or cancelling shows are rarely made on such broad numbers. The reason that far more than 600 people are surveyed is to allow breakdowns into demographic groupings. Since I am no longer a member of the desirable 18-49 audience I can rail at this idiocy, but nobody really doubts that it is accurate either.
The networks have been at this job for 75 years. The system is surely inaccurate and inadequate in places, but it corresponds well to every other gross indicator of popularity – and from one rating service to another – and nobody knows a better one anyhow.
Dishonesty or forgetfulness in recording is a problem (less so, with the electronic monitoring boxes), but what makes you think that “a funky veg out kind of mood” makes the statistics a farce? If you’re watching a show, you’re watching a show. Sure, maybe you don’t usually watch that particular show, but some other day, there’s likely to be some other schmo who doesn’t usually watch that show, who’s watching it. Despite the common perception, statistics is actually an exact science, and it does work.
When they called to ask, I told them that we almost never watch any TV any more (OK, the kids watch Saturday morning cartoons until we get up, but I didn’t tell her that.) I haven’t turned on the tube in months except once to watch a video that a friend loaned me. Didn’t matter to the lady who called, she said that they want replies from people like us too. So I guess my vote that TV isn’t worth my time will be counted, in a way. As it turns out, we’ll be traveling on vacation for most of that period, so we won’t even record the Saturday morning usage.
They paid me five bucks for this. It was a surprise. She didn’t mention this in the phone call, the five bucks cash just fell out of the envelope, and none of the info in the package referred to the cash at all. Strange.
We were a Neilson family for one session, and it seemed endless.
Little log books to fill in, and no real way to show when you channel surfed or when people came and left the room.
After a few days we gave up and just filled in the book from vague memory the next day, and finally decided to “vote” for shows we liked but had not watched and “skip over” shows we saw but thought were lame.
)