Do search engines gobbling up threads and posts here on the SDMB resolve the text that has been spoilered by the teeming millions? We have to click to make it visible. Is that trivial or necessary for the search engines?
How about the extra secret double spoilered entries?
Browsers don’t immediately display “spoilered” text because the agree not to. It has full access to all the text in the downloaded page, but displays spoilers “spoilered” because the makeup says to, the same way it displays italics or green font or an image. Spiders don’t care about markup, so they have access to all the text with no restrictions.
nm. what gnoitall said ^
To make it more explicit, the raw text under the hood is something like [spolerthis]this is the text to hide[\spoilerthis].
I don’t know what the actual keyword is but that is the idea. Then the browser sees this and says “ah, I’m supposed to make this hidden and provide a way for the reader to unhide it”.
The web crawlers do not bother with interpreting these instructions. They just grab everything.
Thanks, there goes my plan to anonymously fade back into the bushes à la Homer.
There are ways of fooling dumber bots, like using substitution font ciphers, or letter replacements.
For example, this text:
Ѕtrаıɡһt Ⅾοре Меѕѕаɡе Воаrⅾ
Looks somewhat like “Straight Dope Message Board” to a person. But if you try to Google that, you get irrelevant results:
That just because it swaps out a few characters that look similar to the less discerning human reader, but are actually different in computer code. It’ll fool simple bots, but not more clever ones.