We update the table schema as we run into new fields, up to a limit. We also store the unstructured part of the data in a column that can be queried via json_extract_path_text.

Amplitude here - Most of our dashboards are powered separately from Redshift. We offer Redshift access as a way for our customers to answer more complex questions not offered by the dashboards.

Why not power your dashboards with it? What do you use?

I am considering using a columnar data store (maybe redshift) with a BI tool like bimeanalytics.com specifically to do dashboards.

My guess is latency - using Redshift for short lived, small queries might not be the best.