In it, Ed talks about “the wisdom of the crowds”, how take a random collection of people and ask them a question and then average/compile all the random answers, and the result will be fairly close to the truth.
He brings up the book, The Wisdom of the Crowds, and lists three examples, one of which is the stock market response to the 1986 destruction of the Challenger Space Shuttle. The book describes how four parent companies responsible for shuttle development had their stocks take an immediate hit, but by the end of the day three were starting to recover, but the fourth - Morton Thiokol, the company that built the Solid Rocket Boosters - was still falling. As we all know, it was later determined that the cause of the explosion was the failure of the o-rings on the boosters.
Ed and the book both wonder how did they know. But that is silly. The stock buyers didn’t have to be scientists, they were not operating in a vacuum. Everyone watching the explosion repeatedly on TV that day was able to see the fire start at the SRB. It was relatively quickly understood where the failure began. It doesn’t take a genius to connect a fire from SRBs to the SRB manufacturer as the cause. I’m much more impressed by the precision of the cow example (1 lb).
You understand that the point of the column was to poke fun at instances of magical thinking on the part of Web 2.0 enthusiasts. I wasn’t attempting a close rebuttal of what is, after all, a six-year-old book; I was merely making light of the fact that Surowiecki takes an uncritical attitude toward his tales. There are several possible explanations for the Morton Thiokol price drop, not the least persuasive of which is that it was happenstance - we are, after all, talking about a single case. Another plausible explanation is that trading in Morton’s stock, unlike that of the other companies, was briefly halted; this undoubtedly spooked investors. Perhaps it was the fact that the fire started at the booster, as you say. Absent SOME explanation, though, what’s silly is to to suggest that the crowd just knew.
This was an interesting subplot in the book Shockwave Rider by John Brunner from 1975. In the future they have established a betting board on various propositions happening in the future.
So how come corporations are totally dysfunctional foundering jellyfish? It seems to me the more people you add to a project, the less gets accomplished.
It comes down to the divideability of the task. A thousand people can guess the weight of a cow, or write Wikipaedia, but cure cancer? I don’t see it happening…
Sure, a large crowd can cure cancer (well, if anyone can). One way they can do that is by setting up systems for recognizing who among them have developed the necessary expertise in the relevant fields to work knowledgeably on the problem of curing cancer, and granting the appropriate level of recognition to their research on the matter.
If humanity can cure cancer, well, hell, the human community makes for quite a large crowd.
As a further explication of why this particular event is not a definitive example of “mass wisdom,” one should look at the reasons for why investors would hedge their bets against stoppage of the Shuttle program. Of the four major contractors involved in manufacturing the components of the STS launch system (Rockwell International, The Boeing Company, Martin Marietta, and Morton Thiokol), only the last two had a substantial involvement in consumable or refurbished components; for Martin, the External Tank (ET), for Thiokol, the Solid Rocket Boosters (SRB). The other two contractors provided prime integration and major structural and propulsive components of the Orbiter Vehicle. Aside from the rebuilds of the Space Shuttle Main Engines (SSME) this didn’t involved a lot of ongoing work.
While Martin built new ETs for each mission, this was done in a government-owned Michoud Assembly Facility in New Orleans, so the capital investment in facilities and tooling by Martin was limited. By contrast, Thiokol had invested heavily in facilities and IR&D for SRB production, and with the termination of UGM-96A Trident C4 production, stalls on UGM-133A Trident II D5 and LGM-118A ‘Peacekeeper’, and development and funding problems on the MGM-134 ‘Midgetman’ mobile ‘Small ICBM’, the Shuttle SRBs were the major source of anticipated revenue for Morton Thiokol. When the launch failure of STS-51L occurred, Thiokol was in negotiation for an order of flight sets of SRBs to support more than sixty future missions, and indeed, it was the pressure of this negotiation (and the threat of a Congressional mandate for dual sourcing the SRB, despite the cost) that has been claimed by some as the leverage used by NASA to force Thiokol management to concur with launch approval despite concerns by the Thiolokol program manager and a senior engineer about the field joint o-ring seal at cold temperatures.
A program hold like that which followed the Challenger failure was very likely to slow or stop production of the SRBs regardless of what the culpable body or failed component was. Whereas the other contractors had diverse business interests (defense and commercial), of which the STS program was at most a couple percent of total revenue, for Morton Thiokol the Shuttle SRBs were a substantial fraction of its revenue at that time, and between this and a couple of fires in Peacekeeper and SRB production facilities, the company was hurting badly, a fact not lost to investors following the aerospace field. That Morton Thiokol managed to survive and thrive in its current incarnation, Alliant Techsystems or ATK, is a combination of business acumen (read: buying up all of their competitors) and performing an exemplary job of implementing and testing the modified joint on the Redesigned (later: Reusable) Solid Rock Motors (RSRM), which included a number of other enhancements that NASA was previously unwilling to fund. Thiokol, through various incarnations (including being shed off from Morton Chemical Company, purchased by ALCOA, and then by Honeywell), ended up buying its major competitor, Hercules Aerospace, and becoming the predominant manufacturer of large solid rocket motors in the United States in the post-Cold War environment, a development nobody would or could have predicted in 1986.
The market, of course, does not make moral or punitive judgements, but merely invests in companies that appear (for reasons real or imagined) to be likely to offer a profit in the near term. And even if it did, it would have made little sense to punish Thiokol, which had already campaigned for a redesigned SRB that would have implemented most of the changes that ended up in the RSRM (and were integrated into the fiber-wound composite ASRM that would have been used in the USAF Blue Shuttle program had it not been cancelled). Equal or more culpability was due to NASA management that pressed forward with positive “go for launch” judgements even after multiple contractors (including Rockwell) expressed concerns about excessive ice buildup on the pad that (it was feared) would result in ice debris damage to the thermal tiles and delicate reinforced carbon composite leading edges that did ultimately result int the destruction of Columbia seventeen years later. (In fact, the worst debris damage prior to Columbia occurred during the STS-26 “Return To Flight” mission, and resulted in a delay between the following STS-27 mission and STS-28 flight of nearly ten months to investigate the root cause and severity of insulation damage.) The buildup of ice was later determined to be a combination of issues including venting from the cryogenics O[sub]2[/sub] tanks and the design of a ramp that attached the ET to the OV that allowed ice to build up and come free during launch. The same venting issue is very likely what causes such excessively low temperatures in the SRB joint that failed on -51L.
One more side point of note is that despite layman statements to the contrary, Challenger did not explode or was destroyed as the result of an explosion. Indeed, the SRBs, freed of the ET, continued to fly for nearly fifty seconds (one pitched end for end without breaking in half, which was impressive) before being sent the destruct command by the Range Safety Officer. The External Tank was torn apart and mostly liquified propellants released into the atmosphere to expand and combust locally, but the breakup of the OV occurred due to aerodynamic forces due to its sudden change of orientation and speed through the atmosphere. No explosion occurred other than the destruct charges on the SRBs.
Surowiecki and the Web 2.0 folks may be making some uncritical determinations, but there is something interesting in “the wisdom of the crowds”. Sure, a crowd of completely uninformed folk are not likely to provide a very meaningful average answer; e.g. if uninformed people are likely to underestimate the weight of a dressed cow, for instance, then averaging their reported numbers would give a low value for the final number. It would be interesting to see what percentage of the crowd were butchers or farmers who had experience guessing how much a cow would convert to when dressed, and how far apart their numbers were from the final answer.* And I think it fairly reasonable that the more knowledge and experience the crowd has on the topic, the more accurate the final averaged answer. I am under the impression that Galton set out to prove the opposite point, that the crowd would do worse than the experts (according to NOVA ScienceNOW). Unfortunately, I can’t confirm that.
I don’t think Galton felt that any crowd of random dummies would do as well on any topic, but I concede that the Web 2.0 folks may be reading it that way.
Karen Lingel, divideability of the task is only one issue of relevance. Actually, Surowiecki lists 4 characteristics that make a “wise crowd”:
[ul][li]Diversity of opinion - Each person should have private information even if it’s just an eccentric interpretation of the known facts.[/li][li]independence - People’s opinions aren’t determined by the opinions of those around them.[/li][li]decentralization - People are able to specialize and draw on local knowledge.[/li][li]aggregation - Some mechanism exists for turning private judgments into a collective decision[/li][/ul]
I think with your corporations example, aggregation is one key breakdown. The communication chain to make the aggregation falls apart. Stranger on a Train, excellent points about the reality of the situation and the various other factors involved in the stock market decisions not considered by the economists’ analysis.
*Paper online here. http://galton.org/essays/1900-1911/galton-1907-vox-populi.pdf?page=7 Apparently a fair number of butchers and farmers, but not exclusively so. Note also that Galton used and advocated the median, and the result was off by 9 pounds. Subsequent math showed the mean was just 1 lb off. I also note the dataset isn’t included, only his presentation of analysis. Can’t assess the results of in individual guess.
They get a little overenthusiastic. A lot of these guys have a utopian streak - the Web will enable us to unlock our potential, loosen the grip of corporations, empower the masses, etc. The wisdom of crowds thing plays to that. Don’t get me wrong, I acknowledge the basic (and ancient) principle: two heads are better than one, many hands make light work, and so on. But the mysticism bugs me. Crowds don’t just miraculously know things. If you read Surowiecki closely you can see he doesn’t really think a random collection of dopes is inherently wise. There’s a subtle but critical interplay between the brilliant few and the grunts. Get that figured out and you’ve got something. But there’s a tendency to overestimate the extent to which we’ve got it figured out.
There is what is known as Bayesian inference; an informal principle that a crowd of relative “experts” will inform and influence the whole group in ways that are subtle but which improve the overall evaluation of probability of a collection of hypotheses (largely by de-emphasizing false positives and false negatives). This principle has been used primarily in search theory (and to a limited extent in financial market modeling) but it is very difficult to implement and assess error in a rigorous fashion for more general subjects.
Such a principle seems intuitively correct–if Alice, Bob, and Carl put their heads together they’ll debunk each others’ flaky ideas and reinforce the most probable answers–but if you end up with Larry, Moe, and Curly, you’re more likely to knock down a building than find a gold mine. The idea of “crowds as geniuses” only works if the crowd has a mean knowledge basis that approaches the level of individual experts.
Some years ago there was a chess match of “Kasparov vs. the World”. Kasparov played black, and anyone in the world with an Internet connection could vote on White’s move. But the world team had four up-and-coming young chess stars as official advisors, and in practice, the World always went along with the advice of at least one of them. Interestingly, once the game reached endgame (an area where none of the official advisors was all that strong, since most chess games are conceded long before that point), Kasparov ended up crushing the world, even though the world team was allowed to use computer assistance, and computers are great at endgames.
OK, so I have thought about it and if my corporation were to try to cure cancer, it would gather together about a dozen scientists, and 200 marketing people. They would all start working, and the marketing people would come up with a catch phrase and a PowerPoint template (6 months). Every 6 months there would be progress meetings, where the marketing people would argue about requirements and deliverables. After 2 years, there would be an organization shake-up, new people would be put in charge, and a new PowerPoint template would be developed (stripes are out! squares are in!). After the third year the project would be canceled due to market conditions or profitability concerns.
I’d go a little further than that. There’s an advantage to having a smattering of folks who aren’t necessarily all that smart but know things nobody else knows. But you still need some somebody bright enough to pull it all together, plus a critical mass of other people with plain old common sense.
The web does all that. However, it also allows the crazy CT kooks to find like-minded kooks, makes un-vetted information and ill-informed opinions widely accessible, and can cater to the lowest common denominator (e.g. aryan web-sites). It all depends on what side you emphasize.
