Hacker News

points by dataflow 1 day ago | 0 comments

Why?

They are not part of any existing writing scheme. I think they could be a different standard like ":+1:" and ":chainsaw:" but it makes no sense for them to be built into text rendering.

[–] p-e-w 1 day ago

> They are not part of any existing writing scheme

They are today, and there is now orders of magnitude more writing that uses emoji than the entire corpus of many of the scripts covered by Unicode.

Language and writing have always been continuously evolving and that process didn’t magically stop when the Internet was invented. The first emoji preceded their inclusion in Unicode, humans used them in writing, therefore they belong in Unicode.

[–] orbital-decay 21 hours ago

>They are today

Correction: Unicode emojis are, because they got added to Unicode.

>Language and writing have always been continuously evolving and that process didn’t magically stop when the Internet was invented.

Okay but why should Unicode be a part of it, and a judge on what to include? That sounds like linguistic prescriptivism, while the original role of Unicode was descriptive and the evolution was happening outside of it. Emojis before Unicode were perfectly fine as composites that weren't mandated by committee.

[–] p-e-w 20 hours ago

> Okay but why should Unicode be a part of it, and a judge on what to include?

How could it be otherwise? These debates have been a part of Unicode since the beginning, such as which Chinese characters to unify, which ancient scripts to include, which attested glyphs get their own code points etc.

That’s simply what it means to be a standard: Decisions need to be made, and not everyone will agree with all of them.

[–] orbital-decay 19 hours ago

Ancient glyphs and Chinese characters existed before Unicode, the committee didn't conjure a new form of language out of thin air. The decisions with Chinese writing were necessary due to practicality (there are various composable encodings, including IDC in the Unicode itself, but they're not very practical for a general-purpose text processing encoding).

Unicode emojis did not exist before Unicode, the existing composite emojis/kaomojis/etc were already perfectly expressible in existing symbols, and were nothing like Unicode emojis. This was a huge distraction from their mission, and definitely not inevitable.

Today's emotes became customized, and it's impossible to express them in any encoding. They diverged too much from writing to be called writing, it's a volatile signaling system at best. You can literally make an AI generation or upload your picture, how do you even encode or standardize that? But because of their blunder Unicode got stuck with its role of a judge and censor of a new entirely synthetic language it created, which wasn't the original purpose of the consortium.

[–] thunderfork 7 hours ago

The original set of emoji were from JP mobile phones (pre-unicode-emoji) and should not be confused with kaomoji/emoticons

[–] saulpw 22 hours ago

Let me put it another way: Unicode is for letters, not for words.

[–] thunderfork 7 hours ago

Unicode is for text symbols, some of which represent one word (e.g. &, U+2116) and some of which represent multiple words (e.g. U+FDFD). Other non-letter characters include numbers, punctuation, and so on.

[–] p-e-w 20 hours ago

It’s for many, many things besides letters, and was designed to be so from the beginning. Pictograms of various types have always been in Unicode, and even in many predecessor codesets.

[–] Timwi 8 hours ago

Imnotsureyouunderstandwhattextwouldlooklikeifyoucouldonlyuseletters

[–] ninjin 1 day ago

Can not speak for the original poster, but I tend to agree in that it greatly limits the speed of creativity due to the integration process, creates a very subjective bureaucratic process for inclusion, and that frankly emoji are not a script (yes yes, you can make an argument that they are akin to ideograms, but this analogy breaks down rather quickly). Thus, I think a different technological solution could have been warranted. The story I keep hearing is that it was necessary to get the Japanese telephone companies on board with Unicode rather than their own emoji hacks combined with local Japanese encodings, which I can sympathise with as a necessary (but unfortunate) evil. A word of caution though that I find text with heavy use of emoji largely unreadable as my eyes fixates on the emoji rather than following the text; thus I am likely biased (but possibly still correct).

All things aside though, Han unification [1] remains the worst part of Unicode to me as it makes it impossible to know whether the glyph I am seeing is the one intended by whoever wrote the text (Chinese names in e-mails for example is an utter mess for someone like me that has to have their system set up to mostly handle English and Japanese). The whole Han unification mess more or less necessitates markup and sometimes even the insertions of raster image glyphs for historical and regional texts which is just bloody daft. Yes, Unicode language markers do exist and can resolve this in part. But, they are invisible to the end user, can disappear while copying and pasting, and you have to rely on the original author even putting them there in the first place! Whenever someone struggles to see the issue, I usually propose that we unify the common subsets of the Cyrillic, Greek, Latin, and Mathematical glyphs which usually finally drives home the point.

[1]: https://en.wikipedia.org/wiki/Han_unification

[–] eviks 1 day ago

> I usually propose that we unify the common subsets of the Cyrillic, Greek, Latin, and Mathematical glyphs which usually finally drives home the point.

That would be an awesome compact encoding scheme (and also help with the next 4million emojis)! And given the relatively higher importance of those CyrGrLat, it would force wider proper support that would resolve the issues you raised, which are all app-dependent

> But, they are invisible to the end user

Apps make them visible, some even do that for whitespace chars to highlight the difference

> can disappear while copying and pasting

broken clipboards can be fixed? They somehow manage to retain rich text formatting, surely they cna retain language markers?

> you have to rely on the original author even putting them there in the first place!

Nope, the author's input app can handle that just like the original author doesn't need invisible joiners to use composed emojis

[–] bigstrat2003 1 day ago

I think that Unicode should have been limited to only depicting existing writing systems. Anything else is imo out of scope for a text encoding format.

[–] hnlmorg 1 day ago

So what about ASCII control codes? They are non-printable characters that are part of Unicode, grandfathered in, yet aren’t part of any writing systems.

And home computers have had pictograms as part of their character set since the late 70s/early 80s. For example code page 437 (MS-DOS) had smilies, card suites, and other emoji. Likewise for the PETSCII (Commodore Pet) too.

And not just emoji’s, but full on drawing characters. In fact that was the only way to draw graphics on some early systems, such as Teletext, in the early 70s.

And then you have the question of what pictograms are part of writing systems. For example the media control symbols (play, pause, fast forward, etc). Or how about the fact that some emojis had been part of written text long before Unicode, in the form of “smilies”, eg :-)

I hope this illustrates that the problem was never an easy one to define boundaries for inclusion.