understanding iceberg deletion vectors (and enjoying some humble pie)
for or a given iceberg snapshot, there can be 0 or 1 deletion vector per data file & a deletion vector cannot span more than one data file
my freewrite alternative (under $200)
a smart typewriter, freewrite, is made of a simple keyboard, a tiny e-ink display, a low capacity flash card, and a low-bandwith wifi card – my first car didn’t cost $750
building trino data pipelines (with sql or python)
trino is well-known as a fast query engine, but it is also a robust transformation processing engine that allows data engineers to developer in sql and/or python
yarp: yet another rag post (this time using sql)
you don’t have to know python or bother your data scientists to start exploring genai concepts like rag; you just need a tool that offers these features in a familiar sql interface
trino query plan analysis (video series)
query plan analysis is critical for getting every single ounce of performance & scalability out of your trino cluster; my 3-part video series will get you started with the basics
logo to company match game (data engineering open-source projects)
can you match the open-source data engineering project logos with the company names who are most affiliated with each?
delta lake time-travel (just reference the version)
trino’s delta lake connector offers features around versioning to include comparing versions and time-travel querying
unstructured docs in ai (the wild west)
rag ai apps can only be as good as the parsed and chunked data that fuels them – testing, testing, and more testing the outputs of all the various available libraries with the front-end apps is critical
success (ability, fortitude & luck)
success calls for ability and fortitude, but if you take a deep look, you will see that luck plays a part in it, too
optionality and common sense (why i returned to starburst)
i’m so excited to have returned to starburst and be focused on rebooting the devrel function, not to mention staying active in the trino and iceberg communities — long live the icehouse
Follow My Blog
Get new content delivered directly to your inbox.