Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Not sure why there are all the down votes (if OP is asking why they don't support parquet as an input format). If Google support avro, why not parquet? JSON isn't exactly efficient if it's about efficiency.

We dump the output of spark jobs into BQ for exploration and having to produce JSON in addition to parquet is an irritating (and expensive) overhead.



The post is about the [adaptive] format BQ uses internally for fast queries. Your problem is having BQ ingesting data in Parquet format. While BQ should support Parquet ingestion, it should also rewrite it on-the-fly into whatever format brings faster query performance.


I agree.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: