We love to share our work and interesting technology. Plus, there are just too many things to remember so we like to write them down!
Building DataLakes can become complicated when dealing with lots of data. Even more so, when working with heavily skewed data. In this post, we'll be looking at how we can improve your long running Spark jobs caused by skewed data.
From X-raying bones to debugging code, this radiographer is ready to take on a new kind of exposure in the world of IT!
Using AWS SSO (IAM Identity Center) to set up CLI and Athena JDBC access for the day
A brief article covering a method to dynamically calculate the ideal number of Spark partitions at runtime
A journey into SharePoint Online and event driven file uploads.