Grafana/Prom/Loki is an awesome stack - overall I'd say that we try to correlate more signals in one place (your logs <> traces <> session replay), and we also take an approach to go more dev-friendly to query instead of going the PromQL/LogQL route.
It's a stack I really wanted to love myself as well but I've personally ran into a few issues when using it:
Loki is a handful to get right, you have to think about your labels, they can't be high-cardinality (ex. IDs), the search is really slow if it's not a label, and the syntax is complex because it's derived from PromQL which I don't think is a good fit for logs. This means an engineer on your team can't just jump in and start typing keywords to match on, nor can they just log out logs and know they can quickly find it again in prod. Engineers need to filter logs by a label first and then wait for a regex to run if they want to do full-text search.
Prometheus is pretty good, my only qualm is again the approachability of PromQL - it's rare to see an engineer that isn't fluent with time-series/metric systems to be able to pick up all the concepts very quickly. This means that metrics access is largely limited to premade dashboards or a certain set of engineers that know the Prometheus setup really well.
Grafana has definitely set the standard for OSS metrics, but I personally haven't had a lot of success using their tools outside of metrics, though ymmv and it's all about the tradeoffs you're looking for in an observability tool.
Grafana/Prom/Loki is an awesome stack - overall I'd say that we try to correlate more signals in one place (your logs <> traces <> session replay), and we also take an approach to go more dev-friendly to query instead of going the PromQL/LogQL route.
It's a stack I really wanted to love myself as well but I've personally ran into a few issues when using it:
Loki is a handful to get right, you have to think about your labels, they can't be high-cardinality (ex. IDs), the search is really slow if it's not a label, and the syntax is complex because it's derived from PromQL which I don't think is a good fit for logs. This means an engineer on your team can't just jump in and start typing keywords to match on, nor can they just log out logs and know they can quickly find it again in prod. Engineers need to filter logs by a label first and then wait for a regex to run if they want to do full-text search.
Prometheus is pretty good, my only qualm is again the approachability of PromQL - it's rare to see an engineer that isn't fluent with time-series/metric systems to be able to pick up all the concepts very quickly. This means that metrics access is largely limited to premade dashboards or a certain set of engineers that know the Prometheus setup really well.
Grafana has definitely set the standard for OSS metrics, but I personally haven't had a lot of success using their tools outside of metrics, though ymmv and it's all about the tradeoffs you're looking for in an observability tool.