[Howto] Setting up a Continous Integration Server for Grails with Hudson on VMWare

At work, we’ve used the Hudson continous integration engine to manage all our Maven project builds. I had to set up our Grails deployments so they rely on Hudson recently, and spent a few days playing with VMWare, Grails and Ubuntu. This guide will walk through all the steps needed to get a complete build server running Apache, Grails, Tomcat, Hudson on the VMWare server.

If you already have hudson deployed and grails running, you should be able to skip to step 7. When I first started this process, I was an absolute newbie to Ubuntu, so there might be more elegant Linuxy ways of doing stuff. Any advice or recommendations will be greatly appreciated

I used a VM image because much of our software was dependent on a Windows environment, but we needed a way to quickly deploy into a linux machine. It also allows us to stop and start quick deployment environments in a portable way without having to start and stop for each project.

1. INSTALLING UBUNTU ON VMWARE

If you have Ubuntu installed, skip this section. Just note that we’re going to use the login tomcat55 and password tomcat55 in the rest of the example

• NOTE: there is a neat tool called VMWare converter that will convert a virtual image to a real boy if you want to muck around with Ubuntu and then putting it on a real machine
• NOTE: If you have a windows machine, you can use the Wibu installer ( which is bundled in the Install CD ) so that your Ubuntu distribution lives on your machine and you can dual boot into it without affecting the file system

1. Download and install the free VMWare player ( or the free VMWare Server, VMWare Workstation or ESX Server ). For this guide, we assume VMWare player.
2. Create a disk. For VMWare player, you can download disk images from this website . You need both a vmx and a vmk file.To get you started faster, the headers of your vmk file should look like this

   #!/usr/bin/vmware
   displayName = "Ubuntu"
   guestOS = "ubuntu"
   
   memsize = "512"
   scsi0:0.fileName = "20G.vmdk"
   ide1:0.fileName = "cdrom.iso"
   
   # DEFAULT SETTINGS UNDER THIS LINE
   
   

3. Download Ubuntu server 8.04 from here, rename the file cdrom.iso and put it in the same directory as the disk images.Your directory should look like this.

   |-20G.vmdk
   
   |-cdrom.iso
   
   |-ubuntu.vmk

   
   

4. Double click on the vmx file, this should open in VMWare player.
5. You should see the Ubuntu install screen. Run through the installer
   • select your installation language
   • choose your language – I use English
   • choose your keyboard layout and country screens – I skip the detection and select America for the country screens
   • enter a hostname – in my case, I stick with the default ubuntu
   • select Time Zone – in my case, Eastern
   • select Guided, entire disk
   • Choose to write to disk, select Yes
   • Ubuntu will go away and think for a while. It is installing the base system. It’s working. It’s done!. Then it will ask you for an username.[ NOTE:I type in tomcat55 and enter tomcat55 as my password. The reason I do this is so that I don’t have to then modify Tomcat settings later on to change the login user. ( The apt package for tomcat logs in as tomcat55 ) ]
   • In the next screen, skip all the choices ( we will install the packages we need later for the sake of the good people who skipped this section, despite all the blood, sweat and tears I poured into it)

Ubuntu works its mojo, and you have a fresh Ubuntu install. Click Continue and watch your beautiful VM Machine restart

2. INSTALL ALL NECESSARY DEBIAN PACKAGES
To get Hudson running and testing, we need the following debian packages

• curl
• ssh
• Java 6 SDK and JDK ( Java 5 craps out Hudson )
• Apache Tomcat 5.5
• MySQL Server 5
• Subversion or CVS – or some wild and awesome other source management system

To install, follow these directions exactly.

1. Login to your virtual machine ( tomcat55 / tomcat55 )
2. Type in

    sudo -i 

3. Enter your password
4. Install curl and SSH
   • type

      apt-get install curl ssh 

     enter Y

   • This will allow you to enter your machine with the SSH shell ( you can find the IP by typing

      ifconfig 

     , it is the first IP address. Remember to type sudo -i after you login )

5. Install JAVA
   • type
     

      apt-get install sun-java6-jdk

   • Enter Y
   • Accept the DLJ Licence terms by clicking enter, selecting the yes button with arrow clicks
   • type

     nano /etc/environment

   • enter

     export JAVA_HOME="/usr/lib/jvm/java-6-sun-1.6.0.06"

   • add

     PATH = JAVA_HOME/bin:$PATH

   • Exit the file via ctrl-X, save by typing S
   • Exit the shell by typing

     exit

     twice. Re-login with tomcat55/tomcat55 and type

     sudo -i

   • Verify java installed correctly by typing

     $JAVA_HOME/bin/java -version 

     . This should give you the java version

6. Install MySQL
   • type

     apt-get install mysql-server-5.0 mysql-client-5.0

     select Y

   • Enter password admin for the root user
   • See that it is installed correctly by calling

     mysql -uroot -p 

     enter

     admin 

     . You should be able to login to mysql. Type

      exit 

     to exit mysql

7. Install Tomcat
   • type

     apt-get install tomcat5.5 tomcat5.5-admin tomcat5.5-webapps

     enter Y

   • Type ifconfig, you should see an IP address in the eth0 controller like this

      inet addr:192.168.235.128 

   • In a web browser outside of the VM Machine, type in the address:8180, i.e.

     192.168.235.128:8180

     You should see a tomcat welcome splash page

8. Install Subversion (SVN)
   • type

     apt-get install subversion

     enter Y

9. Install CVS
   • type

     apt-get install cvs 

     enter Y

3. SETUP GRAILS
You have the choice of installing grails from SVN or from the tar releases. In this guide, we will install grails from the tar releases, but you can find the SVN directions here

1. If you are not already logged in, login to your virtual machine ( tomcat55 / tomcat55 ), type

   sudo -i

   , enter your password ( tomcat55 ) and click enter

2. type in the following ( each new line is a new command ) :

   cd /usr/share
   curl http://dist.codehaus.org/grails/grails-bin-1.0.2.tar.gz > grails.tar.gz
   tar -xvf grails.tar.gz
   rm grails.tar.gz
   mv grails* grails
   

3. Note: the address http://dist.codehaus.org/grails/grails-bin-1.0.2.tar.gz is the latest release of grails at this writing. You should check the grails website and click on the

   Download tar/gz: Binary 

   for the latest copy.
   
   Note 2: there is probably a more elegant way of untaring this stuff. Leave me alone! I’m tired, this guide is getting long. Get off my back!

4. Now, we will add the GRAILS_HOME variable and the PATH
   type

   sudo nano /etc/environment

5. Add these lines

   export GRAILS_HOME="/usr/share/grails"
   PATH=GRAILS_HOME\bin:$PATH 

6. Your final

   /usr/environment 

   file should look like this

   PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
   export JAVA_HOME="/usr/lib/jvm/java-6-sun-1.6.0.06"
   PATH=JAVA_HOME/bin:$PATH
   export GRAILS_HOME="/usr/share/grails"
   PATH=GRAILS_HOME/bin:$PATH
   

7. Save that bad boy ( ctrl-X, S )
8. Test that you haven’t broken anything by typing

    exit 

   twice, login again ( tomcat55/ tomcat55 ). Type

   grails

   If the universe is fair, you should see:

   
   Welcome to Grails 1.0.2 - http://grails.org/
   Licensed under Apache Standard License 2.0
   Grails home is set to: /usr/share/grails
   

   
   No script name specified. Use 'grails help' for more info 

We’re MF’ing done installing grails!

4. MODIFY SUDOERS FILE
This step is here to bypass the prompt that happens when you issue the sudo command for the password. It is a matter of personal preference. Personally, I like the idea of sudo, but it gets in the way of later executing scripts from the command line. Let’s roll:

1. type

    export EDITOR=nano ; visudo -f /etc/sudoers

   this should bring up a nano window with the sudoers file in it.

2. In the file, change

   
   
   root    ALL=(ALL) ALL

   to

   root    ALL=(ALL) NOPASSWD:ALL

   and

   %admin ALL=(ALL) ALL 

   to

   %admin ALL=(ALL) NOPASSWD:ALL

   So your last few lines should read:

   
    # User privilege specification
   root    ALL=(ALL) NOPASSWD:ALL
    

   
   # Members of the admin group may gain root privileges
   %admin ALL=(ALL) NOPASSWD:ALL
   
   

3. save and exit ( ctrl-X, S )
4. type

   exit

   you should be at your tomcat user, if you type

   sudo -i

   , it should not ask you for a password.

If you are terrified about permissions, you can assign NOPASSWD to individual commands used in your scripts. I assign this to all commands because I am lazy and unsafe, and assume that my build server is well protected behind gigantic firewalls that shoot lasers ( or given that it is a VM machine, no contact with the outside world ).

5. CONFIGURAMACATE TOMCAT
We need to change the security setting ( on by default ) to off to make . See the tips section for additional Tomcat config information.

1. you type :

    sudo nano /etc/init.d/tomcat5.5 

2. you search line :

   # Use the Java security manager? (yes/no)
   TOMCAT5_SECURITY=yes

3. you change line :

    TOMCAT5_SECURITY= no 

4. you save file
5. you call

   sudo /etc/init.d/tomcat5.5 restart

6. you done

VERY VERY IMPORTANT: Tomcat config — for hudson, use Java 6 and turn security off. Or life will be filled with tragic and malevolent exceptions.

6. DEPLOY HUDSON

1. login to your vm machine ( tomcat55/tomcat55 ). Type

    sudo -i 

2. type

    cd /usr/share/tomcat5.5/webapps 

3. and then you call this and enjoy :

    curl -k https://hudson.dev.java.net/files/documents/2402/94697/hudson.war > hudson.war 

   the -k param tells it : I know you want https, but I just want the file, so ignore the certificate, mmmkay

4. get your ip address ( first one from ifconfig ) and open hudson in a browser by navigating to http://address:8180/hudson ( i.e. http://192.168.235.128:8180/hudson/ )

Note: that long ass url in the curl is the latest version of hudson. You should go to this page and get the link to the latest stable release of Hudson instead of trusting some random dude on the Internet.

7. CREATE A HUDSON JOB

Ok, step 7, you have made it so far. It’s actually a little anti-climatic.

Because you probably don’t want to build my company’s app, I will use the grails pet store in this example

Make a new job

1. Open your favorite web browser and navigate to your hudson build — again, for those who weren’t paying attention, get your ip address ( first one from ifconfig ) and open hudson in a browser by navigating to http://address:8180/hudson ( i.e. http://192.168.235.128:8180/hudson/ )
2. Click on the New Job link on your left hand side
3. In the next screen, define a new project called Grails Pet Store of type Freestyle project
4. Click OK

Set up SVN ( or CVS )

1. In the next screen, provide a description for your pet store and select SVN under the Source Code Management section. Enter your URL, for the petstore, we will use http://grails-petstore.googlecode.com/svn/trunk/
2. Click Save
3. Click on Build Now on your left hand menu to get SVN to fetch your files. You should see a build in process link on the left hand side. You can click on the console output link to see the progress of your build, but this should retrieve the code from CVS

Do initial setup of Grails app

1. Open a terminal window and SSH to your machine. Login with your username and password ( tomcat55 / tomcat55 ). Before we go any further with grails, we need to call a

   grails upgrade

   on the directory ( see this file for details ).

2. type

    cd .hudson/jobs/Pet\ store/workspace/trunk

   hudson creates a workspace that is stored by default in your .hudson directory. You can change this by setting a variable called HUDSON_HOME in your /etc/environment file

3. type

    grails upgrade 

   Answer yes to any prompts

4. at this point, you would run all the initial project steps needed to make sure you project builds correctly. For the petstore, follow this wiki page ( short version is to call : grails install-plugin searchable;grails install-plugin feeds; grails get-dependencies ) .
5. Call grails prod war
6. Call grails test-app
7. Call sudo cp *war /usr/share/tomcat5.5/webapps/petstore.war

Set up build script

1. return to your browser window, navigate to hudson and click on the “Pet Store” job
2. click “Configure”
3. Click build -> Execute Shell
4. Enter the following in the dialog box:

   sudo bash
   . /etc/environment
   cd trunk
   grails prod war
   grails test-app
   
   sudo cp *war /usr/share/tomcat5.5/webapps/petstore.war
   sudo /etc/init.d/tomcat5.5 restart
   

5. Click Save.

At this point, I should probably explain some of the steps here. So here is the annotated version of this script

# calls the bash shell as the superuser, which prevents many of the issues. In step 4 : Change Sudoers file, we got rid of the password checking prompt

sudo bash

# there wasn’t a clean way in Ubuntu to ensure the environment variables were set in a shell, so I call this file to set Grails_home
. /etc/environment

# make sure we are in the right directory
cd trunk

# generate war file
grails prod war

# the reason we call test-app after prod war is because the war step deletes all the test results, which sucks. This way, we can still keep all the test results — this step took me about a day to figure out, and would have probably been prevented if a GANT script was used instead
grails test-app

# replace the tomcat petstore war and restart – if you are really hardcore, you might want to incorporate the Cargo ant packages here.
sudo cp *war /usr/share/tomcat5.5/webapps/petstore.war
sudo /etc/init.d/tomcat5.5 restart

Store artifacts

One of the nice things about Hudson ( and every other continous build environments, I suppose ) is the ability to store the built war file.

To do so, go to your post build actions – select the archive the artifacts checkbox and enter

trunk/**/*.war

Also check publish JUnit test result report and enter

trunk/**/test/reports/TEST*.xml

Hudson is smart enough to validate your artifact links. So play with that if you are building into a different SVN or CVS directory.

There are other Hudson features, such as the ability to poll the source management system and automatically starting builds, find out more at the Hudson website

And we’re done! And here you have an explicitively and ridiculously detailed guide to getting Grails set up on Hudson.

You should be able to hit your live war file in your http://192.168.235.128:8180/petstore ( albeit I couldn’t verify this since one of the tests fails )

TIPS AND BLATANTLY OMITTED STEPS

• This stuff is a lot easier on the Ubuntu desktop because of the synaptic package manager, the intent of this post is to have a complete step by step so that all the sarcastic comments can be removed and it can be handed off to someone else with little ubuntu, vmware or hudson knowledge to deploy.
• Hudson has a beautiful Gant plugin that I was too lazy / ignorant to integrate
• I cheat because I create a tomcat55 user. If you need to change the user Tomcat logs in with, do a

  sudo nano /etc/init.d/tomcat5.5

  and change the value of the string

  TOMCAT_USER=Tomcat55

  . You need to add this user to the admin group.

• You probably want to create roles into Tomcat’s tomcat_users.xml file, this is located in /usr/share/tomcat5.5/conf
• A nicer way to integrate Grails projects into Hudson to generate a POM file with the Grails Maven Plugin, and then do an actual hot deploy into Tomcat with a tool like Cargo
• Grails 1.1 promises continous integration. I promise to be drunk this Friday
• Tomcat craps out a lot during the war redeploy. I found that wrapping the war file copy between a sudo /etc/init.d/tomcat5.5 stop and a sudo /etc/init.d/tomcat5.5 start will help this
• You need to change the way VMWare player maps ports if you want the app to be visible to anything outside of your host machine. Look at this pretty advanced NAT Port Forwarding tutorial for more details ( I map ports 22 and 8180 to outside ports 722 and 780 )
• Restart tomcat : sudo /etc/init.d/tomcat5.5 restart
• Restart mysql : sudo /etc/init.d/mysql restart