This tutorial will give examples that you can use to transform your data using Scala and Spark. The focus of this tutorial is how to use Spark Datasets after reading in your data, and before writing it out… the Transform in Extract, Transform, Load (ETL).

One of the benefits of writing code with Scala on Spark is that Scala allows you to write in an object-oriented programming (OOP) or a functional programming (FP) style. This is useful when you have Java developers who only know how to write code in an OOP style. However, Spark is a distributed processing engine…

Eric Tome

Senior Data Engineer working with Scala, Spark, Docker, and Kubernetes at IBM. Currently getting a Math degree at the University of Texas at Austin.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store