> In other languages like C, Ruby or Python there is the expectation that whatever happens on line 1 will finish before the code on line 2 starts running and so on down the file. As you will learn, JavaScript is different.
A lot of beginner guides to various programming languages make this mistake of associating a certain property with a language as if it's inherent. In this case, async code - while sold as a main feature of the NodeJS platform - is in no way exclusive to, or even an inherent part of, Javascript/ES the language.
This may seem a nitpick, but I think it's an incredibly important distinction for beginners (or at least, I think it's incredibly important not to mislead beginners into believing in this limitation early on).
Another common example was, up until recently, that Javascript "wasn't powerful enough" to do filesystem access, hardware operations, etc. A simple side-effect of the environment the language was most commonly executing in (the browser) was turned into an inherent limitation of the language in order to "simplify things for beginners".
While bombarding beginners with a lot of info at the start is a bad idea, these kind of misconceptions can be very damaging. They leads to a very narrow idea of what's possible with (any) languages in general, and uninformed decisions on what to learn as a result.
> In other languages like C, Ruby or Python there is the expectation that whatever happens on line 1 will finish before the code on line 2 starts running
actually, if you are into asynchronous programming (I am), I would claim the opposite. When writing an async program in C# you need to be acutely aware that in-between any two sequentially executed lines, entire threads of execution could have been invoked and completed. If you don't design for that, you WILL have race conditions.
Moving to nodejs 2 years ago, and something that I am still mindful of, is that you will NEVER have race conditions, as the entire user code will execute in a single thread, with interrupts only occurring between callbacks.
I like it because it's very easy to construct high performance professional apps in NodeJs (most devs are not multithreading experts) but sometimes I miss the idea of leveraging tons of cores. But that's why we scale horizontally (more 1 or 2 proc servers, less 32proc servers).
This is correct. Race conditions are still the most common vulnerability in web applications.
Take a classic example: a one-time use coupon. Without some kind of locking, two simultaneous requests would be able to use the same coupon successfully. Roughly:
0.0s - [1] Client 1: Apply coupon
0.0s - [2] Client 2: Apply coupon
0.1s - [1] Server to database: Is coupon marked as used?
0.1s - [2] Server to database: Is coupon marked as used?
0.2s - [1] Database to server: Nope! All good.
0.2s - [2] Database to server: Nope! All good.*
* Since the question was asked for both requests at the same time, the answer was "Not used" in both cases.
0.3s - [1] Server to database: Update coupon as used.
0.3s - [2] Server to database: Update coupon as used.*
* Coupon has been used twice :(
Node's async model is not going to help you here. You will need something more, such as a mutex.
Nope, your ACID database will ensure that two writes in the same time for the same record will not happen. You don't even need to ask database if coupon is used. You just try to use coupon, if already used return message to user.
Node give you freedom from Thread Hell. You enjoy similar performance without locks, threads, synchronisation. Race conditions happen but mostly with cache invalidation where you can have limited consistency guarantees.
The problem isn't "two writes at the same time", it's the server thinking everything's OK for two separate users.
If can model your database differently for this rudimentary example, then good for you – you're thinking about race conditions. Many web programmers do not think about them at all, and race conditions are usually not trivial to fix.
agree. the coupon problem would be best fixed with atomic database transactions.
in fact, an entirely synchronous, 1-user-at-a-time server wouldn't fix this, if there were two or more servers. and honestly, who would design any system that only supports 1 concurrent user?
I agree. Asynchronous programming is orders of magnitude easier to grasp than concurrency. Especially in low lock scenarios. Interlocked.Increment(yourPost) :P
The claim about line 2 executing right after line 1 breaks for any program running in concurrent context. Which is probably every nontrivial program dealing with side effects ever written, does not really matter whether that is C#, C++, JS or bare metal assembly.
> something that I am still mindful of, is that you will NEVER have race conditions, as the entire user code will execute in a single thread
If your runtime (kernel, VM, etc - not "user code") has theoretical possibility to change user memory/variable (call user code) concurrently, there is theoretical possibility to have race conditions. Concurrent runtime must have certain entry points to user code and everything that can be theoretically touched from these entry points must be treated dirty, which can get pretty expansive with complex callbacks. The only way to NEVER* have race conditions is to use polling - explicitly mark memory dirty (semantically in code) between "synchronisation points" having only flags/counters (in my view they are the same) for communication between userland and runtime.
* If we allow for settling time and do not treat situations "read flag --> have flag value flipped --> execute based on old flag value" as race conditions.
While it's not "true multithreading", there are a lot of options for using multiple cores in JS now.
In the browser, there are web workers. When combined with the "transferrable" argument in the postMessage function [0] you can get zero-copy transfers (with the caveat that you can no longer access that variable from the process that sent it, so they are actually "transfers" not "copies" or "references"). They are not only here now, but they have pretty stellar browser support (back to IE10 and even safari 5.1).
Node had it's own thing for a while as well, and there are many libraries on top of it to make it more capable or close to the web worker spec if you want. Sadly none of them seem to fully implement the web worker spec with the transferable objects ability, but there's no reason why that can't happen.
Personally I prefer this method MUCH more than just giving you classic multithreading primitives and telling you to not shoot your eye out. Going along with what you said, it's significantly harder to get race conditions when you are either copying everything, or are actually transferring the data to someone else. It seems limiting at first, but I haven't hit anything personally that requires more than that to use 100% of the cores on a system. (the few times i've used it in production situations was setup to basically have the main thread split the data, then transfer them off to worker threads to be processed and either transferred back, or the result collected in the main thread when ready)
That being said, there is some work happening on "SharedArrayBuffers" and Atomics [1] which are the real deal "you'll shoot your eye out" shared memory. And I believe they are well into the standardization process. Personally I'm going to avoid them like the plague, but I know that there are use cases out there that can't be solved with "transferrable objects". Mozilla has an MDN page on SharedArrayBuffer at [2] which is pretty useful.
If I accept your claims that the distinction is important, and that misconceptions can be damaging, is the actual implication here doing any actual damage? Is what you're talking about relevant to this article or to JavaScript?
The fact of the matter is that JavaScript was born with first class functions and event handlers, from the very beginning. In the browser, the majority of projects of any substantial size involves using event handlers, AJAX requests and requestAnimationFrame/setTimeout, among other things.
Even on NodeJS, if you're serving something you're using async. If you're interacting with process events, you're using async. If you're reading files, you're using async. (even the sync functions for files wrap async ones) If you've downloaded anything from NPM, you're almost certainly using async. There is next to nothing that is pure synchronous.
Even if what you said is true, you are suggesting a misconception that could be damaging in the name of avoiding damaging misconceptions. Almost all real world usage of JavaScript uses async code & callbacks, and to split hairs over whether async is native to the language for the sake of beginners is to mislead them.
> A lot of beginner guides to various programming languages make this mistake of associating a certain property with a language as if it's inherent. In this case, async code - while sold as a main feature of the NodeJS platform - is in no way exclusive to, or even an inherent part of, Javascript/ES the language.
Exclusive no, but considering JS's original environment and constraints it is in fact pretty inherent. `setTimeout` is part of the core language and has been for a long time, a blocking `sleep` is not.
> This may seem a nitpick, but I think it's an incredibly important distinction for beginners (or at least, I think it's incredibly important not to mislead beginners into believing in this limitation early on).
No, I think it's very important, maybe even more important for some of us old dogs still marvelling at having virtual SMP in a desktop rig. While it might be easier to teach people new to programming the old model of a series of instructions that goes step by step, I feel it's also important to express that statements may get run out of order, even in languages not designed for it (optimizing compilers have been doing this forever).
> While bombarding beginners with a lot of info at the start is a bad idea, these kind of misconceptions can be very damaging. They leads to a very narrow idea of what's possible with (any) languages in general, and uninformed decisions on what to learn as a result.
It's also dangerous to always assume that statement A will execute before B just because that's how they appear in the code. Getting people acquainted with the tools necessary to properly do "threading" early on would be a step in the right direction.
Are there JS runtimes that are primarily synchronous? Certainly not any popular/widely used ones. Asynchrony is for all practical purposes a property of JS.
Maybe not, but it's irrelevant. The point I was making is that the async features are platform APIs, they have no inherent attachment to the language.
And I personally think not knowing the distinction between the language and APIs provided to it is extremely limiting and misleading for someone approaching learning programming (it was for me, I wish this had been clearer to me earlier).
I'm not sure it is. What's available, without custom work on your own, drives the ecosystem.
I can build an asynchronous application with PHP, but there's not much established practice, existing tools, shared learning and so forth. And, whatever I build that way is limited to an audience that will go out of their way to build my custom php interpreter.
You can run Javascript on the JVM with Rhino and Nashorn. There you have access to all the classical synchronous Java APIs - and you need to care about synchronization and real multithreading, which is not really javascripty.
I'm not sure if you mean implementation of JS or of something that supports executing JS. If you start with the bare interpreter and add say, global functions, through the interpreter's extension API, they CAN be synchronous. You COULD produce a JS environment this way and run scripts in it. Your extensions could include synchronous usage of sockets. Your programs in this environment would be synchronous. Writing a decent web server in that environment would be a challenge.
We're saying the same thing. The nature of the environment is slanted to async, thus, commonly available runtimes don't have much functionality that is synchronous.
You can find some of it, but it's the exception, not the rule.
Yes, you can build something specific, just as you can leverage async libraries in languages where it's not the common pattern.
As a curious and perhaps naive aside, I've been wondering why the programmer should need to care about asynchronous execution of code at all. Can't it all be abstracted under a procedural layer and let the OS worry about not blocking anything? The advent of promises, async.js, and other paradigms tell me that people still kind of want to write code that does one thing after another, then another, then another.
That's exactly what monadic code in Haskell does --- a Haskell `do` block sets up a callback chain, and then when the runtime evaluates the chain, all the entries in the chain happen in the right order. Dependencies between items in the chain and between different chains happen automagically.
Because all mutable state is encapsulated inside the monad and only actually takes effect when the state changes get applied to the outside world, it also allows really cool things like abandoning and retrying state changes if the state's not right. This allows really cool things like STM's 'atomically' operation. Behind the scenes it'll roll back and retry the operation whenever necessary --- but you never need to care: the effect of the operation gets applied to the outside world exactly once.
> Can't it all be abstracted under a procedural layer and let the OS worry about not blocking anything?
No. There's no way around understanding that some of your code will run now, and some will run later. It's imperative to understand this in places where you mix sync and async code. It's not possible to avoid mixing them, after all, the async functions have to be called by something.
You could hide all async in a procedural layer if you accepted the constraint that once an async call starts, none of your own code will run until it returns. It wouldn't block the browser or OS, but it would block you. That's the only way to abstract async away, but that's a constraint I think most people would not be willing to accept.
> ...people still kind of want to write code that does one thing after another
I have no choice in the matter. I can't make use of the result of a REST call until I actually have the result.
But, you're right at a fundamental level as well; people do want strict ordering and simple to understand execution with predictable results. The ideal is being able to read a piece of code and see & understand everything about it based on the function you're looking at, and not a bunch of context outside that function. It's the reason functional programming paradigms are favored by so many.
No. Making it look imperative and being abstracted away are two different things.
Co-routines are not imperative, and you cannot treat them as though they are, you still have to understand they're async to use them at all. That is not "abstracted away", that is syntactic sugar.
Furthermore, this syntactic sugar only works locally when examining your coroutines, or promises, or asyc/await code. You still have to trigger that async code from somewhere, and the place it's triggered is always mixing sync and async code, so you can't sugar coat all of it.
There is no getting around knowing and thinking about the async factors when writing async code. You cannot hide it with coroutines or anything else.
> You could hide all async in a procedural layer if you accepted the constraint that once an async call starts, none of your own code will run until it returns. It wouldn't block the browser or OS, but it would block you ... I have no choice in the matter. I can't make use of the result of a REST call until I actually have the result.
The core.async library for Clojure makes the code look imperative, but handles locking and selecting threads to run under-the-covers ... you may find it an interesting compromise
I think one of the reasons people don't appreciate the importance of explicit effect management is that any one unmanaged effect in your program is usually ok. It's the interaction of multiple unmanaged effects that causes problems.
From a mathematical perspective, synchronous code is strictly more powerful and say-what-i-mean. Asynchronous code is closer to how a (modern) computing system functions in practice.
And yes, a large percentage of our use-cases for callbacks tend to be of the "yield execution of a multi-step task" form. But it isn't just callbacks, it's anything of a "concurrent-and-branching" type flow, so game AIs, UI, etc. also run into this stuff a lot. Sometimes it can be dealt with informally, other times it needs more structure for engineering with confidence to be possible.
A formalized finite state machine is one solution to making sense of the problem in a more generalized way: instead of handing off execution flow based on a morass of polling and callbacks, there's a big switch statement to represent the FSM. The end of each branch of the FSM dictates what the next branch is by setting a branching variable. And every branch ultimately calls into the FSM again, either via a callback or an external loop. This style has the upside of an easier debug trace and the downside of another variable to track and potentially mishandle. (overall, a net gain as the FSM scales up)
A more composable form of FSM used in game AI is the "behavior tree", which encodes blocks of logic in each node of the tree, each node returning a status code: success, failure, in-progress, and optionally taking an action affecting external state like "play animation" or "fire weapon". The general progression is a walk from roots to leaves, with some nodes used to determine the exact sequencing(left to right, pick random, etc.) and other nodes used to tell the tree to yield execution or restart processing by sending back a status code. With behavior trees, there's a substantial win for reusability since the nodes cleanly delineate their needs for parameters, allocation, yielding, etc.
At the language level we're still mostly catching up on strategies that give the idioms comfortable syntax. Yielding iterators (such as those in Python) and promises can sometimes substitute for the simple case of calling a sequence. Full-blown continuations exist in several languages and are very powerful but also encode too much program scope to be used at scale or for long-running, persistent tasks.
>people still kind of want to write code that does one thing after another, then another, then another.
For the most part - yes. And not because they want it but because there is no other way. You usually need to do something before you do that next thing anyway.
The problem is that most languages are not powerful enough to easily express this. You basically need call/cc in some form or built-in coroutines. And a library/framework designed for this.
That's how it's done in Go. You write plain, "blocking" code, and if it blocks, the Go runtime will just schedule another goroutine (green thread). In Go there's no need for callbacks or littering your code with `await` keywords to write concurrent software.
I never checked that out but ; https://www.golang-book.com/books/intro/10 looks messy to me. Less messy than callbacks but more messy than async/await imho. Maybe there are nicer examples?
Also I agree with the coroutine LUA rationale you can read in the link in this same thread. It appears that with Go I still need to alter my own code to use libraries that are written to run async or am I wrong?
>Less messy than callbacks but more messy than async/await imho. Maybe there are nicer examples?
That book chapter doesn't really illustrate the utility of Go's concurrency very well, it just explains the basic components that make it work. Let's say you wanted to fetch three text documents via HTTP and print each of them. Here's the regular, blocking way to do that (error handling, package imports, etc. omitted for brevity):
For each URL in the documentURLs list, make a GET request to that URL, read the body of the HTTP response, and convert it to a string (from an array of bytes) and print it. First it'll fetch the first document and print, then the second, then the third. Of course, we'd prefer not to wait for the previous request to finish before we perform the next, so let's make it concurrent.
func main() {
documentURLs := []string{
"http://example.org/foo.txt",
"http://example.net/bar.txt",
"http://example.com/quux.txt",
}
// `documents` is a channel of values of type `string`.
// A channel is a safe FIFO queue.
documents := make(chan string)
for _, url := range documentURLs {
go func() {
response, err := http.Get(url)
body, err := ioutil.ReadAll(response.Body)
documents <- string(body)
}()
}
for doc := range documents {
fmt.Print(doc)
}
}
OK, let's see what's new here. First, we're making a "channel" which we will put our text documents into as we receive them. Second, the loop over documentURLs is a little different. We put the code into an anonymous function and run it with the `go` keyword. This starts the function in a "goroutine", which is like a light-weight thread. Because we run the function in a new goroutine, the loop does not wait for the anonymous function to complete and the loop continues immediately to the next URL, for which a function is again launched in a new goroutine, and so on for all the URLs. Anonymous functions are closures in Go, so we don't need to explicitly pass in the `documents` and `url` variables (actually this program is buggy, but that's a minor detail).
In the closure we make an HTTP request and read the response body, just like before, but instead of printing the result directly, we send it to the `documents` channel. Basically, we start jobs on three new threads and tell them to put the result of the work in a queue. When we have started the jobs with the first loop, we proceeed to the next loop where we read the strings being sent to the `documents` channel and print them as we receive them. Reading on a channel blocks until there is something to receive.
So I hope you'll agree that this is a pretty simple way to run things concurrently: just use multi-threading. Except we're not using OS threads, which are expensive, we are using goroutines—light-weight green threads managed by the Go runtime. You can have thousands or hundreds of thousands of goroutines running at once, OS threads don't scale that well. If you block a goroutine (e.g., when performing an HTTP request), the Go runtime will just schedule another goroutine, which is cheap. The Go runtime may use a single thread (in which case the program is concurrent but not parallel) or it may use multiple threads (in which the program is both concurrent and parallel).
If instead you were writing a network server with Go, here's how you'd do it (example from https://golang.org/pkg/net/):
ln, err := net.Listen("tcp", ":8080")
for {
conn, err := ln.Accept()
go handleConnection(conn)
}
You listen for new TCP connections, and when you get one, you hand over the connection to a function started in a new goroutine, and then you go back to listening for new connections again. It's the simple model of one thread per connection, except with goroutines it's actually scalable. You can block as much as you want in the handler goroutine and it won't block other goroutines. You can even start additional goroutines inside your handler goroutines. For example, if you wanted to fetch the text documents from the concurrent GET example and send them to your clients, you could adapt the `main` function from the concurrent GET example just a little bit and use that as your `handleConnection` function.
>It appears that with Go I still need to alter my own code to use libraries that are written to run async or am I wrong?
You can make asynchronous library APIs with goroutines and channels: when a function is called, start the work in a goroutine and return a channel. The goroutine sends the return value on the channel. It's kind of ugly and generally frowned upon; it's cumbersome for users who want to use it synchronously, and it's no better than doing it yourself if you do want to use it asynchronously. Instead it is preferred to expose synchronous APIs, which can then be made to run concurrently as desired.
For most people thinking in the fourth (time) dimension should come naturally. It becomes complicated when there are multiple time-lines (threads) though, but JavaScript only has one time-line. You can have multiple time-lines in JavaScript via child processes and web workers though.
I think this is a fantastic article in many ways. It clearly describes the issues of callback hell and gives simple examples on how to clean it up.
Modern JavaScript has gotten very complicated lately because it's so flexible. Many people are developing interesting frameworks to solve niche problems; however, it feels like many of these solutions are overly complicated outside the niche. Yet, developers are adopting these frameworks due to it being trendy instead of choosing the correct tool for the job.
Many times, the correct "tool" for the job can be a mixture of an effective standard library, small specific libraries, and good conventions like the author describes.
I've been a programmer for 18 years now, and I am convinced that simple proven solutions are the way to go.
Maybe it's because I'm a C programmer and not familiar with js but I felt this article did a very poor job of describing "callback hell." I still don't have a clear idea of what it is.
The author shows some code with lots of nested if/else clauses and claims this is bad because it blocks; fair enough. Then they explain what callbacks are and that some people have trouble understanding the asynchronous nature.
Then they immediately go into an explanation called 'How do I fix callback hell?' and meanwhile I'm scrolling up to see if I've missed something.
Perhaps people unfamiliar with this particular problem are not the target of this article, but that doesn't really make sense to me.
> See the pyramid shape and all the }) at the end? Eek! This is affectionately known as callback hell.
In other words, "callback hell" is simply the state of having too many nested inline callbacks. It makes the code uglier and for some harder to read. That's why pulling callbacks out into their own functions (i.e., making them no longer inline) fixes callback hell.
I believe the primary issues are a mistaken belief that "every line of code that gets executed should be read from top to bottom in the same order" (an issue with any code that isn't modular) and the debugging issue of using anonymous functions defined inline.
Readability of code is very important. Inline callback definitions hurt this.
Extracting inline callbacks leads to indirection, especially if you're only extracting to avoid indentation. It doesn't make the code simpler or easier to follow.
Promises and especially async/await let you read code top to bottom once more, with other benefits like monad chains that catch synchronous failure.
Any one section of code should have a "need to know" format. While you want to know "what's happening" in order of execution, having all the "how it's happening" code shouldn't be necessary.
For example, you want to know that the code is about to "getAccountData" but you shouldn't need to know how. You see that it is called, and you see that afterward, a callback called "loadAccountDataInReport" will be called. Why should you see all the guts of those two functions inline?
Now let's say you're in a route handler where the buck stops, and you need to compose any number of async calls, some branching on if-else logic and/or the results of other async calls.
This is where the "just wrap it" argument breaks down.
Though I think promises are only slightly better when you have if-else branches that add to the async chain. It's not til async/await where Javascript finally hits its stride.
Javascript uses "function references" in a similar fashion to C's "function pointers". This...
function fooAllTheBars() { /* ...quux... */ }
...is a function declaration statement which binds a callable reference to my function to the name `fooAllTheBars`. That said, we can express an "anonymous function declaration" as an rvalue, so we can also bind the function to a name ourselves:
This is, admittedly, a bit of a pathological example than the simple "nested callback problem", and the code makes even less sense because "bazzify" and "quuxify" don't actually mean anything, but I think it's a better illustration of the problem at hand.
Basically, if you get into callback hell, your control flow starts bouncing around like a pinball hitting bumpers, and figuring out why is nearly impossible (unless you want to dig through fifteen levels of anonymous functions). And the sad part is that in most cases, you can't avoid it altogether because there's something fundamentally asynchronous like a `setTimeout` or an `xmlHttpRequest` waiting at the end of the call stack.
"I've been a programmer for 18 years now, and I am convinced that simple proven solutions are the way to go."
I would rephrase this slightly to "simple proven solutions that solve the immediate problem at hand, and nothing else."
Bringing in lots of extraneous dependencies, or DSLs offering functionality way beyond whats actually needed for the immediate problem at hand, can quickly lead to code that is very hard to understand, maintain, and debug.
The 'asynchonous problem' in JS is twofold. It makes it more difficult to reason about the program, and is hell on readability, and readbility matters. Support, maintenance, etc. The lack of an ability to structure async ops in a simple way is the biggest drawback in the language.
Callbacks of course have the drawbacks mentioned. Sure, you can separate the functions, smaller functions and that too is preferred, but still you get code that can be difficult to reason about and read.
Promises, for all the hype, in my view clearly did not help matters much. Promise code can be as weird looking or worse than callback code. Promises did not solve the problem.
Async/await IS the solution. I've started converting callback and promise code to it. It's a great new enhancement.
Promises did solve the problem; async/await is just syntactic sugar for them. The important thing is being able to start a task and pass around its value on completion without worrying whether that completion has happened yet. If you didn’t see an advantage to promises, you might have just been using them like callbacks.
I used to think that promises were pretty bad, but then i started using async/await. Now I think of promises as async/await with a lot of extra cruft. I have a colleague who prefers promises because they force programmers to break logic into small functions.
I can't disagree with you more strongly. Async/await and promises both take control flow complexity, of which callback hell is a symptom, and sweep them under the rug.
Whereas before you had a clear (if convoluted and in need of refactoring) control flow, now with async/promises you shove it all into an invisible state machine, with transient data structures that are either hard to inspect (promises) or totally inscrutable (async/await).
No thanks. I like callbacks.
Here's my test: can you reason about what a function might do with control flow just by looking at its call site, not its source code? With callbacks you can. With async/await you can't.
Promises are just as bad as callbacks in most of the code organization issues and I agree with you that they don't actually solve "callback hell".
However, one thing that promises (and other library-based solutions) do better than hand-written callbacks is exception handling. By default, promises will capture and propagate and exceptions thrown inside them, which is something that is very easy to forget to do with hand-written callbacks.
These techniques seem to help code cleanliness in the small without attacking the root issue. Names and error handling help, but modularization requires more care than is demonstrated. The new module example separates the boilerplate, yes, but it also hides document selectors that assume page structure. So it's not actually modular. It's not reusable and it's brittle for the one page it was written for (if you change the markup and don't remember to change the module, the page breaks). It needs a way to pass in the elements queried in callbacks.
Also, generally good advice for everything and something to keep in mind when designing systems.
Excel has one of the worst cultures when it comes to naming things. Easy to blame the users, but tools end up being used the way their design encourages. Excel gives you meaningless names. So most things have meaningless names.
As much as I enjoy async/await, the filesize penalty once Babel-ified can bloat up your bundles; better stick to Promises-only for now, even if it's slightly more verbose.
However they can already be used in Node with the "--harmony-async-await" flag (and without flag in the upcoming Node 8).
Callback hell has confused me for a long time. I kept assuming there was a good reason people write code that way, and there was something wrong with naming my functions and passing them by name. The latter approach always made more sense to me - it's more readable, and more intuitive.
If there is no real benefit to the nested anonymous function style, why do people do it that way? It would never even have occurred to me to write code like that if I didn't see it in javascript all over the place.
I can think of one advantage - lexical closure: when your callback's body is defined inside the calling scope, you can use variables from that scope inside the callback; and the callback can assign to those variables & have the change persist in the caller (only really relevant when the callback's invoked synchronously, but can be very useful in those cases - eg, with Array.prototype.forEach).
It is possible to replicate these effects without closures, but it can get pretty messy: the former by prepending the variables to the callback's parameter list and .bind()ing them where you use it; the latter similarly, but you also have to introduce a layer of indirection via an object to get a "pass-by-reference"-like effect.
So sometimes it really is simpler to just use a function expression in-place.
A lot of one-time-use functions aren't necessarily more readable either. You can start to the lose the context of when, where, and why it's being called. Many small functions or nested anonymous functions are just two ways of dealing with a difficult situation.
It's interesting that JS is still trying to figure out ways to make callback-based asynchronicity less-painful while Go managed to skirt the issue altogether. Go is readable, asynchronous and parallelizable largely without callbacks of any kind. Of course Go has its own problems, but I think they're largely orthogonal to its async story (i.e., JS could adopt Go's async strategy without adopting Go's more controversial features).
JS became a language it was never meant to be not because of its merits, but because it is the language of the web.
Go is the language it was always meant to be, and a significant part of that was handling async.
Once async/await comes along, JS will have a really nice solution, and to be honest, I think it's pretty impressive that JS, with the history of its development, has been able to acquire new technologies of this nature.
Async/await will be an improvement to be sure, but I think it still leaves a lot to be desired. In particular, I'm not a fan of bolting (a)synchronicity onto the function definition; instead, it's nice to be able to run a function in either a sync or async context. But yes, props to JS for continuing to improve.
I agree with the structure the author is putting out there and have been using it myself.
It was honestly a mystery whey people would use this hard to read nested structure when you can write things in a more readable way.
The best theory I can come up with is that it is easier to write that way if, for instance, you are answering a question on StackOverflow, and these examples just find their way into production code via copy/paste without people understanding how they work.
This is where syntactic sugar can be helpful. CoffeeScript can make the example a lot more readable by removing the unnecessary braketing when the last argument of a function is a callback (common case in node.js). The only thing that's still ugly is the .bind() applied to a function, which somewhat messes with the style. Other than that, this makes callbacks flow a lot more naturally when reading source code.
(Of course, as it is for all stylistic considerations, it's a matter of taste, not absolutes.)
Please don't perpetuate the else silliness of the (intentionally bad) first example there. Stick a return in front of the log calls, lose the else, and unindent all following.
I absolutely disagree. Brackets are a big part of the reason the JS version is more readable than the CoffeeScript version, IMO. I don't understand why anyone would think readability would increase with syntactical ambiguity. This isn't even the traditional braces-vs-indentation argument either; I much prefer Python's syntax to CoffeeScript's, and I use both regularly.
Lot of blabbling for a simple concept: do not abuse nesting.
It's the case with control statements, it's still the case with functions, and even more with async functions.
Oh, and callbacks definitely exist in other languages, like C.
What's the deal with this trend of setting up a whole website for a (basic) blog post?
"Simple concept" is subjective. I'd bet that discussing "simple concepts" with an engineer or scientist would likely cause your mind to grasp and instinctively try to relate to your own memorization of facts. But that shouldn't preclude you from discussing what you do know, or for that matter, sharing it with the world.
(answering you, but also for otherz saying the same thing below)
Yeah you're probably right, I've been a bit pedantic.
I feel this article is unnecessary long for the subject, it would be a good length for something like "setting up a hadoop cluster", but having all sorts of articles about all subjects is a very good thing. My bad.
Callbacks exist in C. But inline functions (where a parameter to another function would be placed), don't. Actually, we don't even have closures in standard C (there is a non-standard extension provided by Apple that maybe you can use to recreate JavaScript's callback hell if you're so inclined...)
I would've loved an article this well written and explicit when I was first learning JS (or even any language, really). Really well done with good, practical examples.
> what is the threshold that I can assume that my code will run sync, and when it cant?
Generally you won't have a choice. If something in your function calls another async function, the function as a whole is forced to be async. If not, then it won't be.
It's possible to make a function fake-async even if it doesn't call any async functions, by just calling the callback at the end. But there's not much point -- and callers may make assumptions about the the callback being called in a separate iteration of the event loop, so doing that could cause subtle bugs. That latter problem is solvable by calling the callback in a setImmediate callback, but again, no point unless you have a good reason to.
(Exception: there are a few apis where you do have a choice, e.g node's randomBytes can be either sync or async. Generally they're things that might block, where the sync version returns an error if it can't satisfy immediately. But generally you don't.)
Anything that suspends processing and waits for the OS to provide some data (file, time, socket, or piping usually) should be behind a callback. Beyond that benefit is more limited (in a single threaded app like NodeJS), since you are no longer using async to do othe stuff while you wait for resources. However, if the computation locks up the thread for too long, it cannot service callbacks, and so timers might be delayed, heartbeat messages might be skipped leading to dropped connections, and other bad stuff. If that's the case you want to either split your large computation into many smaller ones (allowing other callbacks to fire in between) or offload the computation into another process or thread.
In other words, it's an engineering decision, based on the expected consequences of the overall design in the context of understanding how the event reactor works.
After working with React Native for a while now I really began to appreciate the usefulness of async/await. By expressing asynchronous operations in a more linear fashion it becomes much easier to reason about them.
Agreed. It tags the complexity with identifiers, which is of some use, but it doesn't change anything about the execution path which can still get wildly out of hand.
Probably better, but the callback hell stems from a broken up control flow; where we cannot read / understand what's going on with a linear (top to bottom) scan of the source code (even for small sections of code).
I think programming where we have state and things can happen at any time, where programs wait for events or something similar has proven to be difficult to write/understand/maintain. The 30+ years it has taken to write the (unfinished) Herd micro-kernel is a good example.
I don't consider myself a huge code style pusher but when I see two space indentation I have to really ask... seriously... why???!!!
Its funny to me because the languages where you get massive nesting you almost want greater indentation as it gets extremely confusing as to what scope you are in.
While I prefer either 4 spaces or tabs (obviously not mixed) I'm not completely revolted with 2 spaces in Algo languages like Java, C or Go (although I suppose Go you have no choice) because you just don't have much nesting.
But Scala and Javascript.... wholly crap is there a lot of nesting. Please for god sakes if you absolutely need that much indenting either refactor or give into the glorious evil that is TAB.
In all serious there is an accessibility thing with 2 spaces. You are hindering people with bad eye sight. I just don't have the visual acuity anymore to discern two spaces easily and I imagine there are many more. It is sort of like building a building and making the stairs extremely steep because you don't like taking lots of steps with your long legs.
If your lines are too long with 4 spaces, then you either have a tiny screen, or aren't line-breaking when you should be. Or you're in callback hell...
Let me guess... It is hard for you to read long lines or your monitor isn't big enough? You might call that an accessibility thing right? You get what I'm getting at. You traded one accessibility that is fairly easy to fix (at worse case you have line wrapping) for one that is almost impossible to fix (zooming and large font don't really help).
To me some parts of FE development kind of require deep nesting, e.g. css, but in javascript if you are 4/5 levels deep in the nest (with the "long lines symptom"), then you should do as proposed in the article: split into modules and functions.
And sometimes (often?) long lines comes from the bad habit to do 3/4 things in one statement. To me good code does exactly one thing per statement, and except for config or urls, is should not be long.
Yes but like I said before long lines is an extremely easy thing to fix in most PLs (ignoring config of course). You can refactor code so that the line is not as long but you can't refactor indention spacing.
I don't think 4 spaces makes it easier to know in what scope I'm in when there are a lot of scopes. The amount of white space is a bit overwhelming to me.
But you know, it's no big deal, just a matter of taste. Maybe 3 spaces is a better answer, which I guess we'll never know...
How long of a line are we talking here? Most coding standards are 80-120 with maybe some at 140.
If you are say 4 levels deep the additional characters used over 2 spaces is 8 characters. You got whopping 8 additional characters to make an even longer line of code (code to be interpreted or read) and screwed over the people who need the indenting.
If long lines are a problem than just refactor. I agree long lines are hard to read but there is a simple fix. Either refactor or just put in a line feed (or however the PL deal with long lines aka \ for bash or python).
I used to feel the same way. Any confusion goes away after you get used to it. Used 4 spaces for ~14 years, used 2 spaces now for 3 years. Now feel the same way about 4 as I once felt about 2 spaces.
Your try catch is useless here, AFAIK JSON.stringify doesn't throw. and you can't catch someCustomHttpGet() "exceptions" if the latter is supposed to be asynchronous.
Yeah, I messed it up when quickly typing the code without thinking, it should've been JSON.parse, I fixed it now.
Also regarding that article: you can call async functions from non-async functions - you will simply get the Promise object instead of the data.
If you're using TypeScript, your editor will immediately highlight the error if you try to use that Promise object as though it were the data, so zero chance of getting it wrong. Try it yourself - https://goo.gl/vDk3Gz - hover over the `n` inside the `formatName` call.
> Also regarding that article: you can call async functions from non-async functions - you will simply get the Promise object instead of the data.
async function() getZ(){ return promise }
function() getX(){
return x = y * getZ()
}
// sync call globally.
let x = getX(); // NaN
async/await makes things more readable, it doesn't change the issue of leaky abstraction and you need to know whether the call is blocking or non blocking when calling getZ and using a function that calls getZ. What should you do? assume everything is asynchronous by default?
Imagine you have a validation lib, you don't need it to be asynchronous to validate a string of length > 10, but you need to rewrite your whole API if you need to validate the fact a unique index has to be validated from a database.
That's kind of exactly what the second half of my comment was about - if you have static typing, then this problem just doesn't exist.
If not, then the problem is really no different from not knowing what your functions are returning (are they returing a Promise object? are they returning the data itself?). Not knowing the return type of the functions you use is hardly something new. You would have the exact same issue with functions that take callbacks.
See the link I sent there https://goo.gl/vDk3Gz - you can hover over all the variables and functions (once the editor fully loads), try typing "n.", hover over the highlighted errors, etc. The tooling takes care of everything.
"The purpose of async/await functions are to simplify the behavior of using promises synchronously and to perform some behavior on a group of Promises. Just like Promises are similar to structured callbacks, async/await is similar to combining generators and promises."
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
He addressed more modern solutions at the bottom.
I think, he is trying to evangelize "you should write readable code, no matter of the limitations of the language."
As a C programmer working with a large codebase, I have come to HATE callbacks.
Seriously, the worst feeling ever is tracing through a huge function tree, only to run into function pointer dereference. Then you have to go on a wild goose chase to find out when, where and what it will be assigned to.
C/C++ callbacks are fairly different in practice than what the article is talking about. You might like callbacks in JavaScript. You might really like promises -- no wild goose chase to track down the chain, because promises chain things explicitly and make async code look more synchronous.
What's most interesting about your point is that what the article suggests -- naming and de-nesting callbacks -- is in a way what you're warning against. By naming it and physically moving it, execution that was contained within a single block of code is now jumping around, and can move to other places where you have to go looking.
Even though it's considered an anti-pattern for good reasons, there are some advantages to nested callbacks.
I feel this async computational burden has been passed on to the developers without any abstraction, making the life of the foundational platform developers easier.
IMO, platform IO libraries should support async behavior natively. Joe Armstrong [0] explains this beautifully.
> I don't know how you would write this in Javascript. I've written quite a lot of jQuery and know how to setup and remove callbacks. But what happens if an event is triggered in the time interval between removing an event handler and adding a new one. I have no idea, and life is too short to find out.
If you are manipulating inside a single function call it is atomic, JS is single threaded and doesn't interrupt functions.
A lot of beginner guides to various programming languages make this mistake of associating a certain property with a language as if it's inherent. In this case, async code - while sold as a main feature of the NodeJS platform - is in no way exclusive to, or even an inherent part of, Javascript/ES the language.
This may seem a nitpick, but I think it's an incredibly important distinction for beginners (or at least, I think it's incredibly important not to mislead beginners into believing in this limitation early on).
Another common example was, up until recently, that Javascript "wasn't powerful enough" to do filesystem access, hardware operations, etc. A simple side-effect of the environment the language was most commonly executing in (the browser) was turned into an inherent limitation of the language in order to "simplify things for beginners".
While bombarding beginners with a lot of info at the start is a bad idea, these kind of misconceptions can be very damaging. They leads to a very narrow idea of what's possible with (any) languages in general, and uninformed decisions on what to learn as a result.