potential_victim(Minions) ->
fun (Pid) ->
not(pman_process:is_system_process(Pid))
and not lists:member(Pid, Minions)
end.
death_from_above(Minions) ->
Pids = lists:filter(potential_victim(Minions), erlang:processes()),
case Pids of
[] -> none;
_ ->
Victim = lists:nth(random:uniform(length(Pids)), Pids),
Name = pman_process:pinfo(Pid, registered_name),
exit(Victim, kill),
{ok, Victim, Name}
end.
The idea is to run it during load tests. Afterwards run your normal unit tests to check that nothing got permanently broken. It's good for finding broken supervisor trees.
Yeah, I've been playing around with the idea of Chaos Monkey at the code level, rather than at the systems level, but you can only truly do it with independent actors. I'm hoping to have something to show soon, probably on Akka/Scala.
There is a similarity with mutation testing, but mutation testing is trying to throw things too far up the chain; it wants your program to crash and die so the test fails. Really, we want it the other way: proof that the test would have failed, but the program is still running effectively.
I've worked with runtime repair in the past, which is also sort of similar, but, IMHO, less effective than Erlang-style Let It Crash. [1]
