Compiling Scala code is a pain. For testing and even deployment, there is a much faster method to deploy Scala code. Consider a recent project I am working on. The addition of three large fat jars and OpenCV caused a compile time of roughly ten minutes. Obviously, that is not production quality. As much as it is fun to check how the president is causing stocks to roller coaster when he speaks or yell explitives around co-workers, it is probably not desirable.
- A Quick Note on SBT Packaging: Packaging Library Dependencies, Skipping Tests, and Publishing Fat Jars
- Avoiding Duplication Issues in SBT
Repeated Compiling and Packaging
Sbt offers an extremely easy way to continuously package sources. Use it.
sbt ~ package
The tilde works for any command and forces SBT to wait for a source change. For instance, ~ compile has the same effect as ~ package but runs the compile command instead of package.
This spins up a check for source file changes and packages sources whenever changes are discovered. Pressing enter will exit the process.
Running the Packaged Sources
We were using sbt assembly for standalone jars. In test, this is fast becoming a bad idea. Instead, place a large dependency jar or multiple smaller dependency jars in a folder and add them to the class path. The following code took compilation from 10 minutes to under one. Packaging takes mere seconds but a full update will take longer.
$ sbt clean $ sbt package $ java -cp ".:/path/to/my/jars/*" package.path.to.main.Main [arguments]
This set of commands cleans, packages, and then runs the Main class with your arguments.
This works miracles. The classpath is system dependent, a colon separates entries in Linux and a semi-colon separates entries in Windows.
It is possible to avoid building fat jars using this method. This greatly reduces packaging times which can grow to minutes. In combination with a continuous build process, it takes a fraction of the time to run Scala code as when using the assembly command.
Make Sure You Use the Right Jars
When moving to running with the classpath command and continual packaging in Scala, any merge strategy will disappear. To avoid this, try creating an all in one jar and only placing this in your classpath alongside any jar containing your main class.
This article reviewed a way to more efficiently package and run scala jars without waiting for assembly. The method here works well in test or in a well defined environment.