Good going! I’m imagining the world – which is to say, science – is already now and certainly in the future will be learning much about the virus from the Herculean efforts being undertaken there.
One thing I’ve been wondering about – and I wrote one of the papers there but got no reply – is how many false positives they are receiving from all those tests, and how they are handling them. This seems like such an open question all around the world, with many jurisdictions perhaps not being in a place to even try to figure it out, so you’d think this would be another valuable data point to come from New Zealand. Are you hearing anything on this from following the news there? I imagine they are very carefully double-checking everything before they fully confirm a case? Would love to learn about their methodology. I recognize it might not be applicable to every other situation, but would still be interesting to learn about.
While false positives do occur, false negatives are much more prevalent for all three types of testing methods. So if the numbers get skewed, they’re probably too low. This is can be a problem because it gives people a false sense of security. They may go out and infect someone thinking that they are not contagious.
Interesting. Thanks for the link. I can’t say that this part inspires confidence!
What about accuracy? False negatives — that is, a test that says you don’t have the virus when you actually do have the virus — may occur. The reported rate of false negatives is as low as 2% and as high as 37%. The reported rate of false positives — that is, a test that says you have the virus when you actually do not — is 5% or lower.
Yikes. I’ve come across a number of other sources, though, that guess on rates like 99.5% for sensitivity and 98+% for specificity. But of course, and getting perhaps to what you said about false negatives, those accuracy rates are for conditions where the only source of error is the test itself, I’d imagine? If you get a bad swab, I guess that throws it all out the window.
I just wonder how well these issues are being addressed (if they do matter). Given how many people must be working, worldwide, on the hundreds and hundreds of thousands of tests each and every day (millions? tens of millions?), you’d have to think there is a pretty good sized margin of error.
If you’re lucky enough to live in a place like NZ, this isn’t an issue because they are keeping such close tabs on the virus.
In places that are not controlling or keeping tabs on the virus, like the US, the test positivity rate is what alerts authorities to a large amount of undetected community spread. Eventually, people start dying which, if not properly being tested for the virus, translates to excess deaths and discrepancies as discussed in another thread.
When San Antonio cases started surging in June, the test positivity rate went from 4% to eventually 24%. That means there was covid all over the place. Who knows what our real numbers were? All we know is that our hospitals were filling up. We probably have people who died of “pneumonia” who really died of covid.
But, really, false negatives is a much, much smaller problem than insufficient testing. That’s what the US has.
Do you think there is any chance the prevalence was well below 24%? How much chance do you think there is that it was significantly above 24%? Any chance it could have been double that number? (If so, would present testing capabilities really much matter?)
RE: False results -
Everybody coming into the country is sent to a quarantine facility for 14 days.
They are tested for the virus on day 3 and day 12
I hven’t been monitoring closely, but haven’t heard anything about false results.
Recalling highschool biology, and combining it with my rudimentary math skills, I’m not worried about false positives - we have the contact tracing capacity and quarantine setup to manage this.
It would be false negatives that are the concern -
while: chance of being infected x chance of false negative x chance of false negative may be low,
this may be what eventually causes another full scale community outbreak here.
And I would also need to add - it absolutely galls me everytime I hear about our “church cluster”.
I’m very much biased against any sort of organised religion and when hearing about the spread of a disease from a Church service it just fucks me off.
And this may be more suited to the politics thread - but my greatest hope is that Biden whacks Trump hard on stats from the virus - citing countries like NZ an South Korea where our test to positive ratio is so low, deaths are low etc etc and just absolutely crush him with simple stats.
I might be misunderstanding your question but test positivity means, in this case, that 24% of the tests came back positive. That means that only people who clearly have symptoms were getting tested. This is particularly true since many people may wait until their viral load drops to undetectable levels but they still be contagious and definitely had covid.
Think about it. Covid has a range of symptoms which could point to anything. Yet 24% of people were testing positive. Remember, there will be more false negatives than positives. And probably only a fraction of people were going in to get tested. So the real number is much larger. In certain regions of the US, where there is little control of the virus and testing is minimal, an outbreak probably has much more cases circulating than what is reported. I’ve seen estimates anywhere from 5-11X more when numbers overload testing and contact tracing.
Right, that’s what I’m getting at. If the positivity rate was that high, what do you think that says about the prevalence of the virus in the community at large. I do not know enough about the situation on the ground in San Antonio to guess how many people were getting tested, how likely those people were to be experiencing symptoms, and so on. I just know that 24% seems really high!
In New Zealand we have done 880,000 tests* and have 1,809 cases
So about 1 in 500 positive?
Again - it flabbergasts me when you can get a positivity rate above 10%.
Here - anybody that tests positive ALL of their close contacts are also tested. That alone would be more than 10 people (even if none of them are infected)
How can anybody possibly be doing any sort of contact tracing with those sorts of positivity rates?
Seems to me - we may need to change the slogan for US from “The Land of the Free” to “The Land of the Ostrich”
I’m not sure if “tests” means people or actual tests
…there were less than a handful of false positives from the latest outbreak from what I could gather. For example, from here:
So the procedure seems to be:
Person gets a positive test: its treated as a positive and the person goes into self-isolation.
Contact tracing starts: close contacts are asked to self-isolate and tested, casual contacts are notified and if symptomatic asked to come in for a test
The initial person is retested: if positive again then they are moved into managed isolation (or appropriate arrangements made.) If negative then they are retested again to make sure and if it comes back negative again the case is closed.
The Director General of Health Dr Bloomfield said that the Americold outbreak followed a very predictable pattern: and I suspect that any positive tests outside of that pattern (like the Wellington or Christchurch false positives) would get more scrutiny than those within.
Since I’m back in the thread I thought it would be interesting to look back over the last month just to give an overview of the outbreak and the NZ Government’s response.
Before the outbreak New Zealand had gone 100 days Covid-free: and we had gotten a bit complacent. Testing was at an all-time low since the start of the pandemic: outside of managed isolation my understanding was that the day before the index case there were less than 20 tests done on the day.
So it was very fortunate (and we are forever thankful) that the index case decided to get a test on the 10th of August. This was the start of the Americold cluster. Americold had a cool store in Mt Wellington where the index case worked. As of today how that person got infected is still a mystery. There is no genomic or epidemiological link to any past cases or any cases on the border. They are 99% sure it didn’t come in through the cool store. They checked the ports, they checked the airline crews. Nada.
Once the outbreak got detected NZ went back to their alert levels. Auckland went to Alert Level 3 and for the first time the police put up checkpoints to stop people getting into and out of the city. The rest of the country went to Alert Level 2. We learnt from Victoria’s mistake: we went for a regional lockdown instead of trying to lockdown streets and buildings. And we didn’t wait to introduce those measures, we pretty much introduced them over the course of 48 hours.
The system showed that it was capable of being ramped up substantially, in a way that would have been impossible back in March. The contact tracing teams were organised by the Regional Health Boards and included both Maori and Pacifica representation which was crucial as the initial outbreak was in South Auckland, an area dominated by these two communities. The labs worked overnight to get results back in 48 hours (any positive cases found in the lab were “flagged” and fast-tracked). If people returned a positive test both them and the people in their immediate bubble were moved to managed isolation: this reduced the risk of other people getting infected, but also reduced the burden of those at risk having to look after a person with Covid-19.
The first big wrinkle happened when a case was found to have been infected while travelling on a bus. This lead to a compulsory mask mandate on public transport nationwide: a mandate that is likely to continue even when the country drops back to Level 1.
The second big wrinkle was a subcluster, linked back to the Mount Roskill Evangelical Church. Members of the church allegedly held meetings over lockdown, then allegedly weren’t entirely co-operative with health officials with contact tracing, which lead to a subcluster of new infections. This subcluster lead to a “long tail” of infections that extended the Alert Levels (Auckland is now at Level 2.5, which allows travel but still limits group size, the rest of the country is at Level 2) that hopefully we will have finally stamped out. Its been 3 days with no new community cases. Hopefully today will be day 4.
But the outbreak has shaken (me at least) out of my complacency. I didn’t use the app before the new outbreak. But I now use it everywhere I go. I didn’t own a mask. We now have a set of disposable masks ready to go and I have my own custom crimson mask that I wear places to “look cool.” When we drop back to Level 1 next week nobody wants to go back to Level 2 or 3 again. So I’ll play my part.
I guess I’m not explaining myself well. The positivity rate is used to indicate the prevalence of the virus in the community at large relative to actual cases. When you have a high positivity rate, you have a lot of undetected covid in the community.
So I started figuring that for every new case that day, there may be 5-10 more out there. Some of those may already be quarantining themselves or they’re in the hospital. But maybe half of those are roaming around and possibly infecting people.
Twenty four % is not at all high compared to some places in the US at certain times. Some have reached over 80% positivity. My in-laws live in Oklahoma and I remember looking about at their rates. They had an 80% positivity rate in March-April. That means there were tons more cases. Even now, they’re reporting a lot of pneumonia deaths. My husbands, aunt had a friend who just died of pneumonia. I wouldn’t be surprised if it was really covid.
We’re barely doing any contact tracing, especially when we get a spike. There’s just not enough resources I moved to SA from the Miami area. One of my friends was moonlighting as a contact tracer in Broward Co. They put her up in a hotel to contact trace for about a month. While cases were still high but starting to decrease, they said “thanks” and let her go. I think Florida just gave up. Officially, 1 in 16 people have been infected in the Miami-Dade area. I bet at least 30% has been infected in some areas of the county. I know a lot people who have been infected. I know people who’s relatives have been hospitalized. I know of one death. Most of these happened in the less deadly ‘second wave’.
The controversial CDC recommendation last month that people without symptoms didn’t need to be tested wasn’t written by CDC scientist, but by HHS officials.
A heavily criticized recommendation from the Centers for Disease Control and Prevention last month about who should be tested for the coronavirus was not written by C.D.C. scientists and was posted to the agency’s website despite their serious objections, according to several people familiar with the matter as well as internal documents obtained by The New York Times.
The guidance said it was not necessary to test people without symptoms of Covid-19 even if they had been exposed to the virus. It came at a time when public health experts were pushing for more testing rather than less, and administration officials told The Times that the document was a C.D.C. product and had been revised with input from the agency’s director, Dr. Robert Redfield.
But officials told The Times this week that the health department did the rewriting itself and then “dropped” it into the C.D.C.’s public website, flouting the agency’s strict scientific review process.
“That was a doc that came from the top down, from the H.H.S. and the task force,” said a federal official with knowledge of the matter, referring to the White House task force on the coronavirus. “That policy does not reflect what many people at the C.D.C. feel should be the policy.”
A new version of the testing guidance, expected to be posted Friday, has also not been cleared by the C.D.C.’s usual internal review for scientific documents and is being revised by officials at Health and Human Services, according to a federal official who was not authorized to speak to reporters about the matter.
Interesting, my mother died of dementia (in a nursing home) a little over a month ago. It was long expected, though.
I’m pretty sure she didn’t have COVID, they tested regularly and were very transparent with the results - I got a weekly robocall with the tallies. They never had a patient test positive (while mom was alive, I don’t know about after) but they had staff test positive.
The lack of contact was tough on the patients and I wonder if that’s a factor. Thanks for posting the link.
No, I think I’m following you fine. I certainly get that when the positivity rate is higher, that is very, very compelling reason to believe that the prevalence within the community is higher. But what seems less clear to me is how, exactly, that positivity rate – particularly at any one snapshot in time, say a day or week or even month – helps to answer the (million-dollar, in my mind) question of how many people in the community have been infected.
Any attempt at answering that question based on a positivity rate would of course need to include several variables. There could be a nice rule of thumb, like the one you mentioned of assuming 5-10 more cases for every one you find, except that you of course couldn’t use that rule of thumb if you were testing 20% of the population every day and getting a high rate of positives. And then of course there is the factor of whether the people coming forward to be tested are more likely to be infected, as you also pointed out.
And then, of course, there is the question of how you extrapolate all this. If, for example, 20% of college kids test positive on the week they come back to school, what does that imply about the rate that have been infected at any time, since the beginning? It’s got to be well higher than 20%, doesn’t it?
I guess I don’t understand why there isn’t more random sampling of the population happening, if indeed the true population-wide prevalence at any snapshot in time is a critical factor for driving policy. We seem to be relying on measures that are noisy at best. Unless there is more random sampling going on than I am aware of.
At any rate, to get back to the San Antonio example, I’m assuming that if there was a significant raw number of tests performed in the week where you got 24%, then you’d think that the prevalence within the community was a considerable number itself, even if the sample being tested was nowhere near random. (Outbreaks themselves are surely not truly random, but they probably aren’t fully clustered either.) What I don’t know, or really have an even a clue about, is whether it would be reasonable to guess that a 24% positive test rate on a given week in San Antonio would imply that 10% of the population was infected during that week. If that’s a reasonable range, then you don’t have to stack many of those weeks on top of each other before you start to run out of new people to infect, which is what I meant when I asked why new testing would really even matter if the spread had already been so high.