Introducing Sparkler - A REST API microservice reference implementation in Java
I recently started a project in which I need to build several RESTful API’s and microservices in Java. When I started scoping out the architecture, I thought about the fact that the last Java REST API I’d built from scratch was in 2012. I’d developed a few in Rails and Sinatra-based API’s in the meantime, though, and I wanted a similarly happy and productive developer experience in Java.
Solving for developer happiness
Now happiness is subjective, especially among developers. To me, being able to quickly iterate with immediate feedback is key. Everything else follows from that principle:
- An architecture that takes a software tools approach, particularly the Unix Philosophy of simplicity, parsimony, and modularity.
- Easy to get started with a working application, going from zero to deployed in less than ten minutes.
- Ideally, live-reloading of code in development against a running server. Barring that, server build and startup loops that can be measured in seconds. I don’t want the overhead of having to compile, deploy a WAR, and wait for an application server to launch an app before I see results. If I redeploy the application 20 times a day, waiting a minute or two for each iteration means I could lose almost an hour. This rules out approaches that involve application servers like Tomcat in favor of an application with an embedded Jetty or Netty.
- Support for a REPL for small, fast experimentation. Dynamic languages like Ruby, Python, and Clojure have set the standard, but there are ways to get a REPL in Java without waiting for Java 9.
- Support for continuous integration and continuous delivery with end-to-end, fast functional tests and an automated deployment.
- If a relational database is needed, support for migrations and easy object-relational mapping without massive cognitive overhead.
- A workflow that is independent of any IDE. If the work can’t be automated on a command line, you’ve locked yourself and your team into an exercise of point and click in Eclipse or IntelliJ. It’s doesn’t scale, and point and click workflows that rely on memorization and mouse dexterity aren’t fun to me.
Point number 1 bears special attention. I want to be able to combine small tools that each do one thing well into a solution that meets my needs for the situation. In contrast, consider something like Spring MVC. What started off with good intentions has become a grand complication. The Spring Framework Reference Documentation is 626 printed pages at the time of this writing, April 2016. The opening line of the overview states, “The Spring Framework is a lightweight solution and a potential one-stop-shop for building your enterprise-ready applications.” Sorry, but “lightweight” and “one-stop-shop” don’t sound like compatible goals to me. Again, I’m looking for a solution that follows the Unix Philosophy. I don’t know about you, but simplicity and parsimony are words I don’t often hear from myself or others when speaking of Spring.
While not as bloated as Spring, I would put other “full stack web application frameworks” like Ninja and Play in the list of things I don’t want as the basis of a microservice. There’s simply too much there to consider. Small things can be understood, mastered, and combined in many different ways.
Given the very specific vision of what I was looking for in an application architecture, I realized the simplest way forward was to build a proof-of-concept to show my colleagues and the world to get feedback. If someone tells me, “Hey, didn’t you know about X? It does everything you just did!”, I’ll be very interested in taking a look. For now, I wanted an API with a dead simple domain just to show what it would take to build something deployable in a real-world way.
Allow me to introduce you to Sparkler! Sparkler was developed by two software engineers, Sarah Aslanifar (that’s me) and Bobby Norton. Rather than starting with a framework, we started with a stub of a domain that would illustrate the other “plumbing” we’d need for our RESTful microservices. You can check out the README for details about how to get started, but this introduction is more about some of the more interesting implementation choices.
Building the Server and REST API with the Spark Web Framework
Not to be confused with the distributed computing framework Apache Spark, “Spark Web” is a relatively new Java Web Application “microframework”. Based on the design principles of Sinatra and leveraging the functional programming potential of Java 8, Spark proved to be a fantastic foundation for the stack I wanted to build. From Spark’s introduction,
“Microservices work best with micro frameworks, and Spark has your REST API ready to serve JSON in less than ten lines of code. Spark is mainly used for creating REST API’s, but it also supports a multitude of template engines. Why not create one Spark application for your backend and one for your frontend?”
This is exactly the kind of modular thinking we wanted. Don’t need a front-end? Great…it isn’t there. Don’t need a database? Don’t think about it…it isn’t there. After spending a couple of hours experimenting with one of Frederico Tomassetti’s well-written tutorial, I was ready to extend Spark with my own ideas.
Spark includes an embedded Jetty server, which is perfect for my needs. From the Jetty documentation:
Jetty has a slogan, “Don’t deploy your application in Jetty, deploy Jetty in your application!” What this means is that as an alternative to bundling your application as a standard WAR to be deployed in Jetty, Jetty is designed to be a software component that can be instantiated and used in a Java program just like any POJO. Put another way, running Jetty in embedded mode means putting an HTTP module into your application, rather than putting your application into an HTTP server.
Functional Testing…with JUnit! Semantic Versioning
For those that have found functional testing to have become “Cucumbersome”, seeing an approach for end-to-end testing using nothing more than JUnit may be interesting.
Test-driving the service’s public RESTful API allowed us to carefully define our API’s contract. Defining the public API this way is essential for Semantic Versioning: If the public API changes, i.e. the semantics of the functional test in this case, that’s a major revision. If there’s a bug fix or some refactoring anywhere in the application, that’s a patch. Just about anything else is a minor version.
Ready for Continuous Delivery
Sparkler is a 12 Factor App. As a way to put that to the test, Sparkler includes a fully automated deployment to Heroku with a single
make heroku-deploy command. The artifact deployed to Heroku is the same as an alternative tarball-based deployment artifact that you can deploy just about anywhere that runs Java 8. How we did that could be an article of its own, but have a look at the makefile and Procfile for now if you’re curious about how we did it.
Immediate feedback with the REPL
Unit tests can be a great way to get fast feedback, but REPL-driven development also has a place for even faster line-by-line experimentation. There isn’t a built-in REPL in Java, though one is coming in Java 9. In the meantime, we can leverage Groovy to provide a REPL through Maven using
maven groovy:shell. There’s a shortcut for this in the Sparkler
True REPL-driven development like you would see in, for example, a Clojure workflow, involves functions being composed piece by piece sent from a text buffer for evaluation. The Maven Groovy shell doesn’t support integration with an IDE like IntelliJ, but does offer the benefit of immediate evaluation. For Java developers that are completely new to dynamic languages, a Groovy shell can be a gentle introduction. Again, REPL-driven Java is a big enough topic to deserve its own article, but this provides a glimpse.
So will this be a framework eventually?
Maybe some day. A framework implies a level of extensibility through the open-closed principle that is currently missing in Sparkler. This application isn’t closed at all, in fact. It’s completely open for you to inspect, criticize, and most importantly, understand. I think of Sparkler as a cookbook for me and other developers with like-minded design principles to easily build Java-based REST API’s from a worked example.
After we’ve deployed several production applications based on Sparkler, it might be time to think about introducing a framework. For now, I think the value is in seeing how Java is a viable alternative for iterative, adaptive REST API development. I think there are some interesting design alternatives in Sparkler for the SparkJava community to consider. On one hand, there’s nothing new here. On the other, we couldn’t find any other project where all of these ideas had been combined.
Maybe you don’t agree with some of the choices we made. Maybe you think configuration files should be used in favor of environment variables, or perhaps you like Gradle instead of Maven. That’s the beauty of open source: You’re welcome to fork Sparkler and modify it in any way you wish. Building your own template is a great way to get to know the libraries you need to use. Rather than taking an off-the-shelf package of someone else’s choices, build your own!
In addition to a write-up for how to build a more complicated domain in Sparkler, some follow-up articles might be in order for the functional testing approach, support for different databases in dev and prod-like environments, or continuous delivery. In the meantime, take a look at getting started with Sparkler and tell me what you think! I’d love to hear from you please contact me by email, firstname.lastname@example.org.