tl;dr – iceberg is pervasive, the real fight is for the catalog, concurrent transactional writes are a bitch, append-only tables still rule, and trino is widely adopted
Tag Archives: iceberg
hive to iceberg migration tool (rev1)
they had a need for an iceberg migration tool, I wrote an iceberg migration tool — i committed it as a github project, then i promoted a github project (i’ve got macklemore’s thrift shop in my head as i write this excerpt)
data universe 2024 workshops (feedback appreciated)
feel free to come and test drive my four trino/starburst workshops i will be delivering at data universe 2024
apache iceberg table maintenance (is_current_ancestor part deux)
as a follow-on to my earlier post about iceberg versioning (and the is_current_ancestor flag), i thought it would be useful to show working examples of the maintenance activities that are needed to manage the sprawl of data lake files that come with more and more versions
iceberg snapshot is_current_ancestor flag (what does it tell us)
i’ve noticed the is_current_ancestor column of the apache iceberg $history metadata table for a while now – it wasn’t until I got a direct question about it that i realized it was time to find out for sure
dbt cloud & starburst galaxy workshop (beta testers welcome)
interested in building a data pipeline with dbt cloud and starburst galaxy? if so, then this post presents recorded videos of 7 lab exercises plus the lab guide itself so you work through them on your own & at your pace
z-order (visualized)
when asked to compare sort-by with z-order for data lake tables i realized i finally needed to have a better understanding of what z-order is all about and my goal with this blog post is to present a simplified visualization of what’s going on and how it can help
hive acid transactions work on trino (can even update a partitioned column)
it seems that folks who haven’t used hive in production are always quick to say that hive doesn’t have classic crud operations, much less the merge statement, and that simply isn’t true – this post shows you that you can create a hive acid table and mutate its contents with trino
building a sql-based data pipeline with trino & starburst (5 slick videos)
a collection of videos presented as an overview of how you could build a sql-based data transformation pipeline utilizing trino/starburst and automating it with dbt
better iceberg materialized views in galaxy (no staleness check)
i’m happy to report that some code changes were made since my last post on materialized views in starburst galaxy and the (mostly useless) “staleness check” is not being executed any more