I'm the architect for Fuzed. I thought I'd explain how powerset uses it production for the main site.
Fuzed sits in front of processes that run Ruby, and are bound to PARC's parsing and grammar analysis libraries. These self-identify to the system and each report their own unique API (the services are heterogenous in both function and version).
Every time you fire a query at powerset, it is sent via JSON-RPC to the frontend of our internal fuzed instance, which then does query analysis before forwarding it to the part of the system that actually does search retrieval. For every one query fired at the powerset frontend, our fuzed cluster takes 2-3 hits, so it has to scale pretty well. So far it's performed admirably even under load. The Erlang runtime is proving to be extremely stable.
Hey. I started playing with Fuzed after the Railsconf presentation and I have to say that I'm impressed. I got a 10 node instance running in no time and it was nicely balanced. I'll be submitting some patches in the future before I fork it on Github. Major kudos to you guys for releasing it! Now with Fuzed and PoolPartyRb, there's no more excuses for not being able to scale easily.
PS: The headline of this post is a bit misleading since Fuzed is tied to Rails as well (Chassis binding)... not just Erlang.
Well, it is and it isn't. Powerset doesn't use the rails functionality at all right now.
P.S., we want more language and framework bindings. I have about half a mzscheme erlang binding done (Erlenmeyer on my github) because I'd love to see arc bindings.
What are the advantages of fuzed over other load balancing options already available? That you can have more complicated logic in the frontend to backend mapping?
This is one. Unlike a lot of existing systems that are "implicit" clusters (e.g., nginx+mongrels, HW load balancer over nginxs over mongrels, etc in that vein), Fuzed is an "explicit" cluster.
So it does things that normal routing doesn't do very well:
1. Fuzed restarts rack processes that become unresponsive or die suddenly as part of its normal operation.
2. Fuzed's dispatcher actually knows when your rack instances are busy and when they aren't. It won't route requests to a busy instance. This ends up being a very good scheduling algorithm for things like rails because of the variable return times.
3. Fuzed clusters reqire almost 0 configuration, they need to be able to do a DNS lookup to a master node, and that's pretty much it.
4. Fuzed can simultaneously host multiple versions of a site and monitor all of them individually OR in groups. Powerset uses the api-mode to host multiple versions of software as we roll out or do performance testing on new hardware or software.
5. Fuzed natively supports multiple frontends that serve out over the same hardware. Usually traditional approaches don't scale horizontally over this dimension without cloning configurations.
I could name some other features that normal clustering approaches need a lot of secondary software to achieve, but I've got a blog post coming up soon that should go over it in more detail.
Well, Powerset's current frontend manages over 2^9's cores, which is even more processes to monitor and route traffic to. Fuzed manages about a 3rd of these and ensures that they are up and traffic is rerouted in the event that the C code that constitutes our NL analysis crashes or consumes too much memory (it can), or a machine drops, or a nic crashes.
Fuzed is cool for this, and a big win for Powerset, because you can scale as you go, you can very nearly just drop more hardware in with practically no global configuration. Search Engines are a special case of websites because they have to scale very broad, very early.
But Fuzed is useful and usable from one box to hundreds, and it's idea for dynamic environments like EC2 where you need to stay very light on your feet to minimize cost. That's originally why I wrote it.
Fuzed sits in front of processes that run Ruby, and are bound to PARC's parsing and grammar analysis libraries. These self-identify to the system and each report their own unique API (the services are heterogenous in both function and version).
Every time you fire a query at powerset, it is sent via JSON-RPC to the frontend of our internal fuzed instance, which then does query analysis before forwarding it to the part of the system that actually does search retrieval. For every one query fired at the powerset frontend, our fuzed cluster takes 2-3 hits, so it has to scale pretty well. So far it's performed admirably even under load. The Erlang runtime is proving to be extremely stable.