Reduce number of fat jars built

[patchwork01]

Background

When building Sleeper, Maven spends a significant amount of time creating fat jars. We currently have 28 invocations of maven-shade-plugin to create fat jars.

Description

We'd like to reduce the number of fat jars we build, to aim to reduce the time spent building.

This should save us a significant amount of time when deploying the system for tests.

Analysis

There are some modules that look like they're building a fat jar unnecessarily:

• statestore-committer-core
• system-test-data-generation and system-test-drivers are both built

The main reason we build jars separately is to reduce the number of dependencies included, keep the size of the jar low, and hopefully reduce the startup time of the JVM. In practice, most jars include Hadoop, which is much larger than any of the other dependencies.

13 of the fat jars are built to deploy lambdas. We could try building one jar that does include Hadoop for lambdas, and one that does not.

[m09526]

Our last attempt at reducing un-needed shading is here: #1245