Our microservices are implemented as Dropwizard applications. The configuration and dependency injection comes from a YAML file that is read on startup. Our projects typically contain 3 slightly different copies of the file: one for local development, one for component testing, and one (templated) for deployment.
Did you notice a smell there? It’s not DRY (I like the antonym WET – “We Enjoy Typing”, or “Write Everything Twice”), and from time to time – even with peer review – we’ve all made the mistake of adding new keys to the two “fixed” copies of the file and forgetting to make the corresponding changes in the template file. Guess what? The templated version is never exercised by the developer or by the build process, so we don’t spot errors until deployment time: it’s hardly fail-fast.
Recently I’ve been involved in implementing a new microservice for our publish notifications feed, and I’ve taken the opportunity to DRY this out and ensure the template file is tested. The solution is to treat the configuration file in just the same way as an executable artefact: use an appropriate test harness to validate it during the build process. In the actual deployment process, the template is processed by a Puppet module, with values coming from a Hiera database. It’s always a worthy ambition to make lower environments as production-like as possible, so why not generate the YAML file by applying local or test case parameters to the template? This fixes the problem of maintaining independent copies of the file, ensures the template itself passes through testing as a natural part of the build process, and makes the environment-specific differences more explicit and easier to validate and visually scan.
Mocking up Puppet and Hiera sounds like a big deal, but in fact almost everything we need to process an ERB template in the Java world comes bundled in JRuby. Our use of the templating is quite simple – there are a couple of variations but essentially it’s things like:
All we need do is inject a fake object called scope, that can respond appropriately to function_hiera calls and backed by an in-memory map of key-value pairs, into the templating engine, which is then called with a minimal Ruby script to render the YAML.
We don’t have a test case solely to check the template, but as our component tests make use of Dropwizard’s support for JUnit rules to start the application with test config, it’s easy to use the template to generate the config for that test case as part of the process.
public static final DropwizardAppRule<NotificationStoreConfiguration> appRule = new DropwizardAppRule<>(NotificationRWApplication.class, CONFIG_FILE);
As long as the file is generated before this line is executed, we’re fine – a class initializer block solves this quite nicely. Another bonus is that it becomes easy to generate multiple different configurations if needed for different scenarios. Now, the component tests won’t pass unless the combination of template and test data produce a valid configuration file.
As an aside, we’re always experimenting and so some of our services have Mustache templates for the YAML file instead. No problem: rinse and repeat with JMustache.