You can set up Workload Management[1] to restrict the amount of compute / query_slots each query/user can use. It splits the memory/compute into slices, and queries can use multiple slices, so you can get some fine-grained control, but it takes a bunch of work.

[1] http://docs.aws.amazon.com/redshift/latest/dg/cm-c-modifying...