Hacker Newsnew | past | comments | ask | show | jobs | submit | tkf's commentslogin

Hi, the author here.

One of the hidden messages of the introduction is: watch Guy Steele's talks [1][2][3] if you are interested in data parallelism! These talks are not Julia-specific and the idea is applicable and very useful in any languages. My libraries are heavily inspired by these talks.

Of course, if you haven't used Julia yet, it'd be great if (data) parallelism gives you an excuse to try it out! It has a fantastic foundation for composable multi-threaded parallelism [4].

[1] How to Think about Parallel Programming: Not! https://www.infoq.com/presentations/Thinking-Parallel-Progra...

[2] Four Solutions to a Trivial Problem https://www.youtube.com/watch?v=ftcIcn8AmSY

[3] Organizing Functional Code for Parallel Execution; or, foldl and foldr Considered Slightly Harmful https://vimeo.com/6624203

[4] Announcing composable multi-threaded parallelism in Julia https://julialang.org/blog/2019/07/multithreading/


I love Julia, and in many areas both the language and standard libraries are quite polished.

However, simple parallelism based on threads (e.g. a parallel map) surprisingly requires a third party library. ThreadsX, the one you posted about, is also my favorite option.

Besides, the standard pipe operator (|>) doesn't support partial application. This has lead to several third-party reimplementations via macros, transducers, etc. Since this is a basic feature, fragmentation is a bit worrying.


Yeah, I get the point that it's nice to have basic things in the core language and stdlib. I'm rather on the "minimal language core" side in this discussion and I think it'd be better to not start adding more things in Julia Base and stdlib. At least not right now, to avoid accumulating "dead batteries." There are some non-trivial things to figure out for a composable data-parallel interface. For example, what the data collection interface should be to support folds over them? I'm using a minimalistic interface defined in SplittablesBase.jl for this but it's not wildly used outside my packages. So I'm not super confident that it's enough yet.

[1] https://github.com/JuliaFolds/SplittablesBase.jl


Just to echo Tkf here, I think it'd be great to have something like ThreadsX.map in base someday, but when the multi-threading features in Julia first came out, it was unclear what the best interface to provide would be.

Since Julia is committed to semantic versioning, this is a kinda scary prospect because it means that any high level, exported interface we decide on, we'll be stuck with until at least 2.0 and probably forever.

So with things like this, it's important to be conservative about the interfaces we expose and instead, we've have people explore different approaches out in the package ecosystem. One day, if someone can make a strong enough argument, I'm sure we'll see one of these package solutions end up in Base.


Yes, I agree getting design choices right is important in order not to end up with a lot of legacy code in future versions of Julia.

I think the pipe operator is probably the area that needs most urgent attention. Lots of libraries have their own slightly different macros for improved pipes with partial application, and that introduces a bit of fragmentation and hurts composability.

There's an open issue for this with some background: https://github.com/JuliaLang/julia/issues/5571


Given your experience, what primitives would you advocate for JS for data parallelism?


For composable nested parallelism, I think spawn and sync like Julia has would be great. If there is a variant of spawn that asserts there is no concurrency is required for the child task, I'd imagine the runtime can do more optimizations (Julia doesn't have it but there was an experimental PR [1]). But concurrency sounds like a very basic thing in JavaScript so I'm not sure if it fits well in the language. But I don't have experience in serious JavaScript programming so I don't know what fits best there. I'd imagine you can build some data parallel libraries already with web workers API.

[1] https://github.com/JuliaLang/julia/pull/31086


> The takeaway is that Elisp needs to learn to interface with other languages better.

This is exactly why @kiwanami developed EPC (a general RPC protocol/libraries for Emacs) and why I wrote Python binding: https://github.com/kiwanami/emacs-epc

He was also inspired by the article.


I think "maximum extensibility" is not the philosophy of Emacs. Remember how long it took to have FFI support in Emacs (it was discussed in the article). Also, even with FFI, it is not possible to load non-free libraries. I think this shows some discrepancy from the philosophy to achieve "maximum extensibility". I would say GPL does not allow "maximum extensibility".

So the next question is: does the philosophy the author speaks of matches with the one of GPL, or rather, the one FSF and Stallman have in mind? Actually, I think these philosophies are different. If I remember correctly, in Emacs mailing list sometime ago, Stallman rejected idea of adding function to gcc to dump parse tree (or something similar) to use gcc from other software (I can't find the link right now). This was because non-free software could use gcc via CLI and "steal" the data from free software. Now, although it is the other way around, Emacs can use non-free external program. It matches with the philosophy by Matsuyama but I doubt that Stallman says it is the philosophy of Emacs.

I would say this philosophy is for author of Emacs Lisp program (though this was not what Matsuyama was talking about). You have many options. You could choose "maximum extensibility" but actually it is not doable due to GPL. You can follow FSF's philosophy (I am not sure what is it, but I believe there is). Somewhere in between, there is philosophy proposed by Matsuyama and it reflects current situation of Emacs very well, and I like it.


Some awesome Emacs Lisp programs that relies on external programs:

mu4e: http://www.djcbsoftware.nl/code/mu/mu4e.html

not much: http://notmuchmail.org/

EDBI: https://github.com/kiwanami/emacs-edbi

ropemacs: http://rope.sourceforge.net/ropemacs.html

RSense: http://cx4a.org/software/rsense/index.html

GCCSense: http://cx4a.org/software/gccsense/index.html

emacs-ipython-notebook: http://tkf.github.io/emacs-ipython-notebook

emacs-jedi: http://tkf.github.io/emacs-jedi

request.el: http://tkf.github.io/emacs-request/

Well, the last three is my projects so you could exclude that :)

Also, we could add version conrolling interfaces such as magit and VC (relying on git/svn/hg/...) and advanced interpreter such as SLIME/nrepl.el/geiser.

Ah, and don't forget dired!


And then again, they are pain in the arse (if not impossible) to use on Windows. One of the benefits of ELisp extensions is that they are multiplatform.


I agree some of them are harder to setup in Windows. But it helps (or forces) developers to make these tools and languages cross-platform, which is good, I think. And I guess making them cross-platform is actually easier than implementing them purely in Emacs Lisp.


Well, I am Japanese and I am almost sure my translation is right. But I hope calling the s/he "Eccentric" is not very rude. I just thought it is better than "unusual"... or not?

Ah, I just notice "unique" fits the context very well. Perhaps I will change that later.


I personally think that "eccentric" is more polite than "unusual," which could be either positive or negative. It's also better than "unique"; I mean, everyone is unique... Given Steve Yegge's hilarious, insightful, and somewhat cheeky rants (after learning Java for a few months, "Kingdom of Nouns" [1] expressed my nascent thoughts exactly), I think he would qualify for "eccentric." I bet he'd even like the term :)

[1] http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom...


Thanks for the explanation. It is hard feel this kind of nuance as my English is not that good. It's also good to know that he'd like that :)


Richard Stallman is a true eccentric. Steve Yegge has called himself "The drunken blogger," which is unusual, but he seems pretty normal otherwise. Just outspoken. No, I don't think it is rude to call him unusual, it still feels like you were trying to say JavaScript in Emacs was unusual and not Steve Yegge, but I'll take your word for it.


I agree with you that the title is a bit misleading and trolling. And I actually name it like you do if I were him. I just translated it directly.


I think you misread the philosophy he talked about. He actually thinks that having the problems about threading, libraries and low APIs is a good thing, as it helps us maximizing social value of software we develop.


He didn't say these problems are a "good thing" at all. He said externals in Emacs can be used elsewhere easily than Eclipse plugins, and that's the "good thing" (being decoupled, modular). Sometimes these problems are just a motivation, not a need, for using externals, and that's why he criticizes js2-mode and Semantic (as they're not externals). He's indeed talking about the Unix philosophy.


Guile integration may matter, assuming it happens (not everybody seems happy: https://lists.gnu.org/archive/html/emacs-devel/2012-04/msg00...)

Python in Vim has slowly become the go-to choice to talk to an external program from Vim, because VimScript is simply that bad.

I know little, however, about how hard ELisp makes it to interface with an external program. The examples he gives (a JS interpreter in ELisp, and Semantic) lead me to believe that the authors wrote all that in ELisp because it was easier than to play with an external program. Am I wrong?

Will Guile ease the task of interfacing with an external program?


It's actually pretty easy to run external programs from elisp. stdin comes from a buffer, and stdout/stderr go to a buffer, and you can use all the usual emacs facilities on the result. See the various interactive modes (e.g., M-x run-python, M-x shell) for a demonstration.

I suspect people who write packages entirely in elisp simply like elisp. And why not? The language itself is mostly OK to use, even if it doesn't always conform to modern standards.


To be honest, ELisp is not bad. Not in the same range as VimScript. In fact, ELisp with this weird concept of I/O through buffers makes a lot of sense for a text editor and I find it quite an elegant concept (not always practical, though). And while ELisp is not really CL, the lacking parts are the interpreter and runtime themselves, not the language.

Writing elisp is not hard. What's hard is integrating all the modes, keymaps and quirky behavior of coupling all these smalls bits of code together. This is what makes emacs hard. Emacs, the "core editor" itself, is incredibly small. Way smaller than vim (build it by hand if you don't believe it). Emacs as the editor that you use every day is almost pure extensions.

There's a /whole lot/ of global state in an editor. This is not going to change with Python, Guile or any other language. Users that complain about ELisp, most of the time actually never tried to write ELisp at all (they just glue some elisp around in .emacsrc). That's fine, but they don't realize what's going on under the hood. The C/C++ mode in emacs is completely different to the "stupid" syntax highlighting in Vim or most other editors.

I normally recommend switching to another editor if after several years you still don't appreciate the difference. IMHO emacs, as it is, is a great editor. ELisp is part of the success (though guile would be a step forward).


Actually it's not a complete word. So, it's like "understan" or "comprehen". Probably because it's still beta.


Nope, it actually is a complete word: http://tangorin.com/general/%E3%82%8F%E3%81%8B%E3%82%8A

You probably came to that conclusion from the fact that the word 「分かります」 means "to understand". Actually, 「分かります」 is made up of two parts, 「分かり」 (the same word that started this whole discussion), the nominalized form of the verb 「分かる」 ("to understand"), and the polite suffix 「ます」. 「分かり」 (or 「わかり」, as it's often written when nominalized) can of course be used by itself as well.

> Probably because it's still beta.

From a software branding perspective, that would be a really stupid thing to do. It would severely damage your SEO, and you'd be stuck renaming all kinds of resources and files after you go out of beta.


Whoa there cowboy. I'm pretty sure that tkf (if he's the same one who's submitted pull requests to my projects) is from the land of the rising sun. And his second statement was a joke. And funny IMHO


> I'm pretty sure that tkf (if he's the same one who's submitted pull requests to my projects) is from the land of the rising sun.

Him being Japanese doesn't automatically make him an authority on Japanese grammar. The small Midwestern town I grew up in is chock full of white-as-bleach European Americans whose ancestors have exclusively spoken English for several generations. Most of them couldn't tell you the difference between an adjective and an adverb if their lives depended on it.

The same goes for many of the foreign (read: white) English "teachers" in Japan. Most of them barely graduated from college, couldn't find any other work, and so they decided to go to Japan to teach English. In actuality, they should be going back to America and spending a couple years in middle school remedial English classes.

In fact, just to confirm, I got out my copy of Koujien (an authoritative Japanese dictionary comparable to Merriam-Webster's Collegiate Dictionary or the Oxford English Dictionary) to look up 「わかり」. And sure enough, there it is:

> わかり【分り】

>  わかること。さとること。のみこみ。会得えとく。了解。

> And his second statement was a joke. And funny IMHO

Well, that went over my head. If so, then his first statement was probably meant as a joke as well.


You can connect to REPLs over ssh using `ipython console`. Even without EIN, you can use this command (plus several options for ssh connection) for python-mode's interactive shell. With EIN I think it's a little bit easier (if you are using python.el), as it helps you to setup these options.


I've never tried, but VIM already got nice looking IPython binding (https://github.com/ivanov/vim-ipython). I think it doesn't support notebook editing though.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: