Faster gradle builds when working with docker compose

In this post, I will outline a technique to make local docker compose builds faster by pre-fetching dependencies.

For Spinnaker, we provide a docker compose configuration.

Recently, I wanted to add the ability to build and run images locally by providing a developer configuration. This would allow someone to test a change that spans multiple services by baking an image locally.

One of the issues I ran into was that when the Dockerfile is ran, it downloaded all of my Spring Boot dependencies along with the gradle wrapper, adding significant time to my builds.

If you are unfamiliar with docker-compose, the build command basically takes the dockerfile defined for your project and creates an image from it. There isn’t really a straightforward way to mount a volume at build time for me to point to an existing .gradle directory.

To rectify this, I changed the line I used to build in my Dockerfile file from

RUN ./gradlew buildDeb -x test

to

RUN GRADLE_USER_HOME=cache ./gradlew buildDeb -x test

This simply tells gradle to download all my dependencies to the cache directory in my source code.

Since my Docker build copies the contents of the build directory into the image first, I can then cache all my dependencies in the folder ( outside of docker compose ):

GRADLE_USER_HOME=cache ./gradlew compileGroovy clean -x test --refresh-dependencies

Now, all the libraries and wrapper files my build needs is in the cache directory. The next time I try to build my image with docker compose, it will find that all the jar files and gradle wrapper exists and will not download them.

In the CI system, since the cache directory is not checked in, it will download it fresh, which is the behaviour we want.

Collection of Links for Greach 2015 from Twitter

Spock Workshop – Fernando Redondo
Slides – http://www.slideshare.net/fredondo/greach-2015-spock-workshop

Grooscript books demo
Source Code -https://github.com/chiquitinxx/books-demo

Groovy and Grails in a Spring Boot App – Fátima Casaú
Source Code – https://github.com/fatimacasau/spring-boot-talk
Slides – http://www.slideshare.net/fatimacasau/use-groovy-grails-in-your-spring-boot-projects

GSPs in Spring Boot – Fátima Casaú
Source Code – https://github.com/grails/grails-boot/tree/master/sample-apps/gsp/gsp-example

Idiomatic Gradle Plugin Writing – Schalk Cronjé 
Slides – http://www.slideshare.net/ysb33r/idiomatic-gradle-plugin-writing

GPars Data Workflow – Russel Winder
Slides – http://www.russel.org.uk/Workshops/Dataflow/Notes/

Reactive Options for Groovy – Steve Pember
Slides – http://www.slideshare.net/StevePember/groovy-options-for-reactive-applications-greach-2015
Source Code – https://github.com/spember/greach2015-reactive-demos

Gradle Puppet Workshop – Andrey Adamovich
Slides – http://www.slideshare.net/aestasit/infrastructure-automation-with-gradle-and-puppet-at-greach-2015

DSL’ing Your Groovy – Alonso Torres
Slides – http://www.slideshare.net/alotor/greach-2015-dsling-your-groovy
Source Code – https://github.com/Alotor/2015-greach-groovy-dsls

Securing Ratpack – Jeff Beck
Slides – http://beckje01.com/talks/greach-2015-sec-ratpack.html#/

No-nosense NoSQL – Jenn Strater
Slides – https://github.com/jlstrater/No-Nonsense-NoSQL

Groovy or Scala: Friends or Foes – Marco Vermeulen
Slides – http://marcovermeulen.github.io/groovy-and-scala-talk/#/

Groovy Environment Manager ( GVM ) – Marco Vermeulen
Slides – http://marcovermeulen.github.io/gvm-talk/#/

Gr8CRM – Göran Ehrsson
Slides – http://gr8crm.github.io

Cassandra and Grails – Jeff Beck
Slides – http://beckje01.com/talks/greach-2015-cassandra-grails.html#/

Grails Goodness – Mr Haki
Slides & Code – https://github.com/mrhaki/greach2015-grails-goodness

Advanced Microservice Concerns – Steve Pember
Slides – http://www.slideshare.net/StevePember/advanced-microservices-greach-2015

Hacking the Spring Security 2 Plugin – Burt Beckwith
Slides – http://www.slideshare.net/burtbeckwith/hacking-the-grails-spring-security-20-plugin
Code – https://github.com/burtbeckwith/hacking_madrid

Groovy AST Transformations: More that meets the eye – Iván López
Slides – http://www.slideshare.net/ilopmar/greach-2015-ast-groovy-transformers-more-than-meets-the-eye
Code – https://github.com/lmivan/greach2015

Building Spring Boot cli apps with Docker

The Spring Boot CLI is quite useful when building small groovy scripts.

I recently created a small dockerfile based on Java 8 and ubuntu that makes it easy to use the Spring Boot CLI in Docker.

This is available via the docker hub.

To start, simply add

FROM tomaslin/spring-cli

to your docker file and it will provide you with a docker image with Spring boot CLI, Java 8 and ubuntu to work on your applications.

This post contains a full example on how to use this.

Spring Boot Shannanigan: Overriding properties via the command line when using Gradle

If you use Spring Boot and Gradle, you might sometimes find yourself in a homicidal rage when looking stuff up in the Spring Boot Reference Manual and encounter lines like ‘or specify on the command line using the switch –spring.profiles.active=dev,hsqldb.‘ because command line switches don’t bloody work in Gradle.

You can overcome this problem by adding the parameters to you application.properties file and reverting it before commit. It is a terrible solution that should be tied up and shot with arrows.

Better yet, you can use this dirty little trick to provide your own properties and it works with the Gradle command line.

Let’s say you want to provide your own value to spring.profiles.active, all you need to do is capitalize the letters and change the dots to underscores and Spring Boot will pick up the change.

./gradlew bootRun --spring.profiles.active=dev,hsqldb 

does nothing, but

SPRING_PROFILES_ACTIVE=dev,hsqldb ./gradlew bootRun

works like a charm.

The reason this works is that while Gradle won’t pass along system properties to Spring Boot, it does pass along environment variables. Spring Boot is able to pick up the environment variable when it resolves the property and the last ten months of changing property files by hand makes me look like a total and absolute moron.

You can use this mechanism to set server ports,

 SERVER_PORT=9000 ./gradlew bootRun 

Set your own properties

 MY_SERVER_URL=http://www.springtoot.com/ ./gradlew bootRun

And specify the name of the config you want to use:

 SPRING_CONFIG_NAME=killmenow ./gradlew bootRun

Yipikaye!

Spring Boot Recipe: Embedding local instances of datastores

A technique we found very useful in our Spring Boot development process is to embed a local version of Redis, Cassandra or Elastic Search that starts when we call bootRun.

Embedding these datastores also simplifies the setup needed for Continous Delivery and testing, since all the bits needed to run all our tests is available within our source code.

In this post, I will give an example configuration of how we launch a local elastic search instance and how this can then be used.

Spring Boot Recipe: Turn beans on and off by setting a property

In Spring Boot, you can use the @ConditionalOnProperty annotation to enable or disable a particular bean based on the presence of a property. This is very useful if you want to provide optional features to your microservice.

To do so, add the annotation either at the class or bean level, as follows

@ConditionalOnProperty(value='mybean.enabled')
@Bean 
MyOptionalClass optionalBean(){
}

Any place where you want this bean used, you should specify that is it optionally required:

@Autowired(required=false)
MyOptionalClass optionalClass

And that’s it. Your optionalClass bean should resolve to null when you specify mybean.enabled=false in your application.properties or system property file, or if it does not exist.

This mechanism is used extensively in Spring Boot itself, for example to turn on autoconfiguration in Spring social.

Warning, this blog post applies to Spring Boot 1.1.x and below, the annotation is slightly different in Spring Boot 1.2.

Spring Boot and Webjars – overwriting configuration during runtime

In a previous post, I highlighted how we use Gradle to create content jars that can be used in your Spring Boot application.

In this post, I will show you how to overwrite individual files within your content jar to let your server provide a custom configuration.

Spring Boot Recipe: Reading and Validating Lists of Configuration Properties

In one of my Spring Boot microservices, I want to keep track of a list of servers that can be externally configurable.

I can have zero to infinity servers, so I want my application to allow me to easily define this.

I also want to be able to check that the configuration is correct, so I don’t accidentally forget to enter a url or a name for my server.

In this post, I will show you how you can use the Spring ConfigurationProperties to organize and validate a map of configuration values.

Adding an optional internal artifactory repository to Gradle to speed up builds

For Asgard 2, we’re developing heavily with Spring Boot, Groovy and Gradle.

For our jenkins jobs, we found that our internal artifactory repository is much faster than Bintray’s jcenter. Due to the large number of dependencies in the project, a project that takes 30 minutes downloading and building with jcenter only takes 3 minutes using an internal artifactory.

Since our internal repository is only available to us within our network, we don’t really want to expose those credentials in the project we eventually want to open source.

In this post, I will show you how we add an optional internal artifactory repository to our open source builds.

Setting a Tomcat version with Spring Boot and Gradle

Spring Boot allows you to easily specify a tomcat version when using Maven (Docs). But there doesn’t seem to be an equivalent mechanism for Gradle.

To lock down a specific tomcat version for war deployment, I found that I had to exclude the starter-tomcat module and add the dependencies specified in the tomcat starter POM in my Spring Boot project’s build.gradle file.

compile('org.springframework.boot:spring-boot-starter-web:1.0.0.RC4'){
   exclude module: spring-boot-starter-tomcat
}
providedRuntime 'org.apache.tomcat.embed:tomcat-embed-core:7.0.52'
providedRuntime 'org.apache.tomcat.embed:tomcat-embed-el:7.0.52'
providedRuntime 'org.apache.tomcat.embed:tomcat-embed-logging-juli:7.0.52'
providedRuntime 'org.apache.tomcat:tomcat7-websocket:7.0.52'

Note: embed-el doesn’t seem to be included in versions below 7.0.52.

I imagine a similar approach would be needed to use Jetty 9.