# Why do some places on the internet replace apostrophes with odd characters?

**URL:** <https://boards.straightdope.com/t/why-do-some-places-on-the-internet-replace-apostrophes-with-odd-characters/720889>\
**Category:** Factual Questions\
**Created:** [May 26, 2015, 4:23pm UTC](https://boards.straightdope.com/t/why-do-some-places-on-the-internet-replace-apostrophes-with-odd-characters/720889 "2015-05-26T16:23:02Z")\
**Posts on this page:** 5\
**Page:** 3

<div class="post-metadata">

**Author:** ![Derleth](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Derleth](https://boards.straightdope.com/u/Derleth)\
**Post date:** [May 28, 2015, 12:51am UTC](https://boards.straightdope.com/t/why-do-some-places-on-the-internet-replace-apostrophes-with-odd-characters/720889/41 "2015-05-28T00:51:32Z")

</div>

> [@LSLGuy](#):
>
> **Derleth** : Now let’s discuss collations in MS SQL Server, Oracle, etc. On second thought, let’s not. 🙂

Collation is inherently more complicated in Unicode than it is in ASCII. Unicode is closer to how complex it is in reality, once you move beyond only caring about a subset of English-language text.

Another thing which becomes more complicated is letter case: For example, what is the upper-case version of ß? Ask any German speaker, and they’ll tell you it’s SS. Two letters. However, the letter pair ss _does_ occur in German text, so there’s no way to always know what the lower-case version of SS is. If you’re trying to do it with a simple algorithm, you lose. The real world doesn’t work like that.

(Interestingly, there’s a very rare character which is a single-letter capital ß: ẞ. It does occur in German text. It was not invented for Unicode. It is vanishingly rare, and essentially never occurs in modern German text.)

The upshot is, nontrivial text processing needs to be done using specialized libraries. This has, really, always been the case, it’s just that now, multilingual text is technically feasible, making it impossible to ignore some things the ASCII-only world let us gloss over.

---

<div class="post-metadata">

**Author:** ![friedo](https://avatars.discourse-cdn.com/v4/letter/f/8edcca/32.png) [@friedo](https://boards.straightdope.com/u/friedo)\
**Post date:** [May 28, 2015, 12:55am UTC](https://boards.straightdope.com/t/why-do-some-places-on-the-internet-replace-apostrophes-with-odd-characters/720889/42 "2015-05-28T00:55:37Z")

</div>

The old roundtrip casefolding myth. It has led to much heartbreak. And did you know Unicode also has titlecase?

Unicode is hard. Really hard. Yet people still think they can handle text with cheap hacks from the punchcard days.

---

<div class="post-metadata">

**Author:** ![Derleth](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Derleth](https://boards.straightdope.com/u/Derleth)\
**Post date:** [May 28, 2015, 1:25am UTC](https://boards.straightdope.com/t/why-do-some-places-on-the-internet-replace-apostrophes-with-odd-characters/720889/43 "2015-05-28T01:25:17Z")

</div>

> [@friedo](#):
>
> The old roundtrip casefolding myth. It has led to much heartbreak.

The only round-trip Unicode cares about is Unicode to other character encoding to Unicode: The text _must_ be the same out the back-end as it was on the front-end. This has lead to Unicode preserving oddities and arguable mistakes from decades ago.

> [@](#):
>
> And did you know Unicode also has titlecase?

Yep. And the only way to justify such nonsense is the idiotic fact _real languages_ have it.

> [@](#):
>
> Unicode is hard. Really hard. Yet people still think they can handle text with cheap hacks from the punchcard days.

In a way, I blame ASCII for having been designed so well: So many hacks _work_ in ASCII: Want to go from a numeral to a number? Subtract ‘0’. Want to sort alphabetically? Sort numerically by character code. Want to flip case? Flip a bit. Easy-peasy, and you get enough of English to convince people that characters not in ASCII are weird or special somehow.

Now that world is gone, and good riddance. The habits die hard, though.

---

<div class="post-metadata">

**Author:** ![Just\_Asking\_Questions](https://avatars.discourse-cdn.com/v4/letter/j/ba9def/32.png) [@Just\_Asking\_Questions](https://boards.straightdope.com/u/Just_Asking_Questions)\
**Post date:** [May 28, 2015, 10:42pm UTC](https://boards.straightdope.com/t/why-do-some-places-on-the-internet-replace-apostrophes-with-odd-characters/720889/44 "2015-05-28T22:42:24Z")

</div>

Found another one in the wild today.

From a Yahoo! news article:

> [@](#):
>
> ?Tonight our beautiful Dana chose to go with the Lord. We don?t yet understand his plan for her, but she will make a perfect angel,? they wrote.

The next sentence had properly displayed double quote marks. But the (apparently) single quotes and apostrophe failed to convert.

---

<div class="post-metadata">

**Author:** ![LSLGuy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lslguy/32/5813_2.png) [@LSLGuy](https://boards.straightdope.com/u/LSLGuy)\
**Post date:** [May 29, 2015, 1:31am UTC](https://boards.straightdope.com/t/why-do-some-places-on-the-internet-replace-apostrophes-with-odd-characters/720889/45 "2015-05-29T01:31:44Z")

</div>

I always enjoyed the Turkish (dotted vs. undotted) \* (upper vs. lowercase) I. Case folding is, as you’ve said, very far from trivial.

[Previous page](https://boards.straightdope.com/t/why-do-some-places-on-the-internet-replace-apostrophes-with-odd-characters/720889.md?page=2)
