batch as a “special case” of flink streaming (yes, now we’re mv’ing streaming back to batch)

the third part of a loosely coupled trilogy on flink batch and streaming that take us full-circle with the collapse of the DataSet API into the DataStream API — i’m not sure Run-D.M.C. could make this less tricky

big data api’s look a lot alike (code comparison with flink, kafka, spark, trident and pig)

exploring the similarity of the APIs from flink, kafka streams, spark (RDDs & DFs), storm’s trident and yes, even good old pig by implementing the canonical word count solution with each framework

joining spark dataframes with identical column names (not just in the join condition)

a quick walkthru of spark sql dataframe code showing joining scenarios when both tables have columns with the same name; this includes when they are used in the join condition as well as when they are not

how do i load a fixed-width formatted file into hive? (with a little help from pig)

presents a couple of options for converting a fixed-width formatted file a a delimited one to prepare it to be exposed as a hive table

generalizing specialists (or is it specializing generalists?)

There are a ton of different beliefs of what it takes to be successful in the software development profession.  Some folks out there would tell you to become the very best “Xyz” developer you can possibly be.  While I’m not against be very competent at any particular skill, I’d sure advocate being very good atContinue reading “generalizing specialists (or is it specializing generalists?)”