I don't think you need active learning to get results like this, just decent statistical analysis. There are parallels here with distributed query planning.