Hacker Newsnew | past | comments | ask | show | jobs | submit | prescod's commentslogin

The whole Unicode brouhaha was triggered by Paul's description of the issue. If he had said: "Arc doesn't support Unicode yet but I expect it would be easy and I welcome patches" then it would have not been an issue at all.

Instead, he said: "I realize that supporting only Ascii is uninternational to a point that's almost offensive, like calling Beijing Peking, or Roma Rome (hmm, wait a minute). But the kind of people who would be offended by that wouldn't like Arc anyway."

There was no reason to take a swipe at the "kind of people" who care about internationalization, which is to say:

* people who are not mono-lingual anglophones

* people who want to build real-world applications

Why shouldn't people in those categories be interested in Arc? Why should they be excluded? And why treat it as a matter of political correctness rather than just a technological decision?

Paul threw the first punch and the blogosphere punched back.

Furthermore, his general tendency to divide and conquer the programing world in that way is why there was already a huge pool of haters ready to pounce on him. Every essay of his implies that there are people who get it and people who don't and one can distinguish between them by seeing which people agree with him and which do not.

He only needed to say: "Arc does not yet support Unicode" and the whole thing would have been avoided.


With the understanding that arc was released as a tech demo of a personal project at this stage, I took what pg wrote to mean that he thought he had better things to do than make it do X (in this case, add and test international support). I think there were enough disclaimers in the announcement, web site, and posts to conclude this is all he meant. He simply released his project as it was.

Why should PG say "unicode support will be easy and that he will accept patches"? Wouldn't making promises be contrary to the entire disclaimer he wrote? (Have you read disclaimers before? Paul's addresses at least as many facets as those written by lawyers.)

If bloggers want to influence PG into feeling guilty in order to get him to spend more time on arc regardless of his disclaimer of "when it's done, it's done," they can do it by phrasing things like adults. Or, writing their own languages.

(Great post. Maybe PG wouldn't have realized this perspective had you not posted it for users to upvote. I know I didn't notice your perspective. On the other hand, I wouldn't change a thing. You're expecting PG to plan for all potential hurt feelings, be it Mac users, Linux users, corporations, people in remote locations without access to the Internet because this means they can't download arc, and more. This is impossible to do, and leads to nothing being released at all.)


I'm not expecting PG to plan for hurt feelings. I'm expecting him simply not to go out of his way to hurt feelings. For no particular reason he took a technical time-to-release issue and turned it into a political correctness issue (and therefore made the whole debate around it political).

Simply take this paragraph:

"Which is why, incidentally, Arc only supports Ascii. MzScheme, which the current version of Arc compiles to, has some more advanced plan for dealing with characters. But it would probably have taken me a couple days to figure out how to interact with it, and I don't want to spend even one day dealing with character sets. Character sets are a black hole. I realize that supporting only Ascii is uninternational to a point that's almost offensive, like calling Beijing Peking, or Roma Rome (hmm, wait a minute). But the kind of people who would be offended by that wouldn't like Arc anyway."

And change it to:

"Currently, Arc only supports Ascii. MzScheme, which the current version of Arc compiles to, has some more advanced plan for dealing with characters. At some point (I don't know when) I or someone else on the Arc team will probably figure out how to take advantage of it."

That's all. Say less. Stick to the technology. Avoid politics. Controversy avoided.


It should be noted that the released version of Arc actually did support Unicode. The only reason that it became a controversy was because PG said it was ASCII-only and implied it was a design decision. It is purely a communication issue.

(Btw. same thing with the controversy over presentational markup vs CSS. AFAICT the Arc framework don't do any layout at all out of the box, including any presentational markup.)

Now PG seem to believe that controversy is inevitable when creating a new language so you might as well throw a few random punches. However if you want to build a vibrant community around a language I think it is very important to choose you battles carefully and your swipes wisely.


Do I understand correctly that Arc strings are sequences of octets?

If so: I really don't want to be a negativity guy but it seems like every language that has made an 8-bit string the default string type has regretted it later because it is so painful to change it without breaking code. Okay, Paul says that he won't mind breaking code. Maybe he means it, but it doesn't make any sense to me to knowingly and consciously repeat a design mistake that dozens of other people have made and regretted.

It really just takes one day to get this right. You need to distinguish between the raw bytes read from a device and the true string type (which needs to be 21 bit or greater). You need a trivial converter from one to the other (which you can presumably steal from MZScheme) and back.

That's it. You get this right at the beginning and you never have to backtrack or break code.

My apologies in advance if this post is based on incorrect premises. I'm trying to help.


Arc snarfs the string implementation from MzScheme which support Unicode in The Right Way, as code points rather than octets.


So should I infer that the only reason UTF-8 is mentioned is that the reader APIs do not let you select the codec? Or is even that provided in which case it is accurate to say that Arc supports Unicode-in-general?


Arc uses MzSchemes reader (it modifies the readtable slightly to support []-syntax). AFAIK you cannot access the reader API from inside Arc. The reason Utf-8 is mentioned is that it is the default encoding when MzScheme reads or writes files or streams.

I don't think anyone at this point would claim that Arc supports unicode-in-general.


Could you offer a better solution? What would your solution offer that octets do not? Random character access? No, because not a single unicode encoding offers easy random character access (because they are made of possibly several codepoints, which, in some encodings, are made of more than one basic "chars"). Gylph, word and sentence segmentation? I guess not.


Abstraction. Having Unicode strings (i.e. strings that are a sequence of Unicode code points rather than octets) allows you to work with strings without worrying about encoding (except when doing IO, which is where encoding matters).

If you treat strings as octets OTOH even simple operations like concatenating two strings might lead to headache if the strings are in two different encodings. And how do you keep track of the encodings of individual strings? Madness lies down that road.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: