Using Apache Spark for big data processing offers also a possibility to use Scala. Despite Python being more popular than Scala, Scala is still THE language in Apache Spark world. It is time to start writing code in it.
The interactive tool sbt helps you build Scala and Java projects. It is similar to Java’s Maven or Ant. It offers native support for compiling Scala and, among other things, offers support for mixed Scala/Java projects.
Run the following to install sbt.
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 sudo apt-get update -y sudo apt-get install sbt -y
Test sbt by running
Should return something like this
[info] Set current project to ubuntu (in build file:/home/ubuntu/) [info] 0.1-SNAPSHOT
The tool is now installed and ready to use.
You can run sbt by simply typing
Creating an example Scala project that works with Apache Spark is described in this post.