[Howto] Setting up a Continous Integration Server for Grails with Hudson on VMWare

At work, we’ve used the Hudson continous integration engine to manage all our Maven project builds. I had to set up our Grails deployments so they rely on Hudson recently, and spent a few days playing with VMWare, Grails and Ubuntu. This guide will walk through all the steps needed to get a complete build server running Apache, Grails, Tomcat, Hudson on the VMWare server.

If you already have hudson deployed and grails running, you should be able to skip to step 7. When I first started this process, I was an absolute newbie to Ubuntu, so there might be more elegant Linuxy ways of doing stuff. Any advice or recommendations will be greatly appreciated

I used a VM image because much of our software was dependent on a Windows environment, but we needed a way to quickly deploy into a linux machine. It also allows us to stop and start quick deployment environments in a portable way without having to start and stop for each project.


1. INSTALLING UBUNTU ON VMWARE

If you have Ubuntu installed, skip this section. Just note that we’re going to use the login tomcat55 and password tomcat55 in the rest of the example

  • NOTE: there is a neat tool called VMWare converter that will convert a virtual image to a real boy if you want to muck around with Ubuntu and then putting it on a real machine
  • NOTE: If you have a windows machine, you can use the Wibu installer ( which is bundled in the Install CD ) so that your Ubuntu distribution lives on your machine and you can dual boot into it without affecting the file system
  1. Download and install the free VMWare player ( or the free VMWare Server, VMWare Workstation or ESX Server ). For this guide, we assume VMWare player.
  2. Create a disk. For VMWare player, you can download disk images from this website . You need both a vmx and a vmk file.To get you started faster, the headers of your vmk file should look like this
    #!/usr/bin/vmware
    displayName = "Ubuntu"
    guestOS = "ubuntu"
    
    memsize = "512"
    scsi0:0.fileName = "20G.vmdk"
    ide1:0.fileName = "cdrom.iso"
    
    # DEFAULT SETTINGS UNDER THIS LINE
    
    
  3. Download Ubuntu server 8.04 from here, rename the file cdrom.iso and put it in the same directory as the disk images.Your directory should look like this.
    |-20G.vmdk
    
    |-cdrom.iso
    
    |-ubuntu.vmk


  4. Double click on the vmx file, this should open in VMWare player.
  5. You should see the Ubuntu install screen. Run through the installer
    • select your installation language
    • choose your language – I use English
    • choose your keyboard layout and country screens – I skip the detection and select America for the country screens
    • enter a hostname – in my case, I stick with the default ubuntu
    • select Time Zone – in my case, Eastern
    • select Guided, entire disk
    • Choose to write to disk, select Yes
    • Ubuntu will go away and think for a while. It is installing the base system. It’s working. It’s done!. Then it will ask you for an username.[ NOTE:I type in tomcat55 and enter tomcat55 as my password. The reason I do this is so that I don’t have to then modify Tomcat settings later on to change the login user. ( The apt package for tomcat logs in as tomcat55 ) ]
    • In the next screen, skip all the choices ( we will install the packages we need later for the sake of the good people who skipped this section, despite all the blood, sweat and tears I poured into it)

Ubuntu works its mojo, and you have a fresh Ubuntu install. Click Continue and watch your beautiful VM Machine restart

2. INSTALL ALL NECESSARY DEBIAN PACKAGES
To get Hudson running and testing, we need the following debian packages

  • curl
  • ssh
  • Java 6 SDK and JDK ( Java 5 craps out Hudson )
  • Apache Tomcat 5.5
  • MySQL Server 5
  • Subversion or CVS – or some wild and awesome other source management system

To install, follow these directions exactly.

  1. Login to your virtual machine ( tomcat55 / tomcat55 )
  2. Type in
     sudo -i 
  3. Enter your password
  4. Install curl and SSH
    • type
       apt-get install curl ssh 

      enter Y

    • This will allow you to enter your machine with the SSH shell ( you can find the IP by typing
       ifconfig 

      , it is the first IP address. Remember to type sudo -i after you login )

  5. Install JAVA
    • type

       apt-get install sun-java6-jdk
    • Enter Y
    • Accept the DLJ Licence terms by clicking enter, selecting the yes button with arrow clicks
    • type
      nano /etc/environment
    • enter
      export JAVA_HOME="/usr/lib/jvm/java-6-sun-1.6.0.06"
    • add
      PATH = JAVA_HOME/bin:$PATH
    • Exit the file via ctrl-X, save by typing S
    • Exit the shell by typing
      exit

      twice. Re-login with tomcat55/tomcat55 and type

      sudo -i
    • Verify java installed correctly by typing
      $JAVA_HOME/bin/java -version 

      . This should give you the java version

  6. Install MySQL
    • type
      apt-get install mysql-server-5.0 mysql-client-5.0

      select Y

    • Enter password admin for the root user
    • See that it is installed correctly by calling
      mysql -uroot -p 

      enter

      admin 

      . You should be able to login to mysql. Type

       exit 

      to exit mysql

  7. Install Tomcat
    • type
      apt-get install tomcat5.5 tomcat5.5-admin tomcat5.5-webapps

      enter Y

    • Type ifconfig, you should see an IP address in the eth0 controller like this
       inet addr:192.168.235.128 
    • In a web browser outside of the VM Machine, type in the address:8180, i.e.
      192.168.235.128:8180

      You should see a tomcat welcome splash page

  8. Install Subversion (SVN)
    • type
      apt-get install subversion

      enter Y

  9. Install CVS
    • type
      apt-get install cvs 

      enter Y

3. SETUP GRAILS
You have the choice of installing grails from SVN or from the tar releases. In this guide, we will install grails from the tar releases, but you can find the SVN directions here

  1. If you are not already logged in, login to your virtual machine ( tomcat55 / tomcat55 ), type
    sudo -i

    , enter your password ( tomcat55 ) and click enter

  2. type in the following ( each new line is a new command ) :
    cd /usr/share
    curl http://dist.codehaus.org/grails/grails-bin-1.0.2.tar.gz > grails.tar.gz
    tar -xvf grails.tar.gz
    rm grails.tar.gz
    mv grails* grails
    
  3. Note: the address http://dist.codehaus.org/grails/grails-bin-1.0.2.tar.gz is the latest release of grails at this writing. You should check the grails website and click on the
    Download tar/gz: Binary 

    for the latest copy.

    Note 2:
    there is probably a more elegant way of untaring this stuff. Leave me alone! I’m tired, this guide is getting long. Get off my back!

  4. Now, we will add the GRAILS_HOME variable and the PATH
    type

    sudo nano /etc/environment
  5. Add these lines
    export GRAILS_HOME="/usr/share/grails"
    PATH=GRAILS_HOME\bin:$PATH 
  6. Your final
    /usr/environment 

    file should look like this

    PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
    export JAVA_HOME="/usr/lib/jvm/java-6-sun-1.6.0.06"
    PATH=JAVA_HOME/bin:$PATH
    export GRAILS_HOME="/usr/share/grails"
    PATH=GRAILS_HOME/bin:$PATH
    
  7. Save that bad boy ( ctrl-X, S )
  8. Test that you haven’t broken anything by typing
     exit 

    twice, login again ( tomcat55/ tomcat55 ). Type

    grails

    If the universe is fair, you should see:

    
    Welcome to Grails 1.0.2 - http://grails.org/
    Licensed under Apache Standard License 2.0
    Grails home is set to: /usr/share/grails
    
    
    No script name specified. Use 'grails help' for more info 

We’re MF’ing done installing grails!

4. MODIFY SUDOERS FILE
This step is here to bypass the prompt that happens when you issue the sudo command for the password. It is a matter of personal preference. Personally, I like the idea of sudo, but it gets in the way of later executing scripts from the command line. Let’s roll:

  1. type
     export EDITOR=nano ; visudo -f /etc/sudoers

    this should bring up a nano window with the sudoers file in it.

  2. In the file, change
    
    
    root    ALL=(ALL) ALL

    to

    root    ALL=(ALL) NOPASSWD:ALL

    and

    %admin ALL=(ALL) ALL 

    to

    %admin ALL=(ALL) NOPASSWD:ALL

    So your last few lines should read:

    
     # User privilege specification
    root    ALL=(ALL) NOPASSWD:ALL
     
    
    # Members of the admin group may gain root privileges
    %admin ALL=(ALL) NOPASSWD:ALL
    
    
  3. save and exit ( ctrl-X, S )
  4. type
    exit

    you should be at your tomcat user, if you type

    sudo -i

    , it should not ask you for a password.

If you are terrified about permissions, you can assign NOPASSWD to individual commands used in your scripts. I assign this to all commands because I am lazy and unsafe, and assume that my build server is well protected behind gigantic firewalls that shoot lasers ( or given that it is a VM machine, no contact with the outside world ).

5. CONFIGURAMACATE TOMCAT
We need to change the security setting ( on by default ) to off to make . See the tips section for additional Tomcat config information.

  1. you type :
     sudo nano /etc/init.d/tomcat5.5 
  2. you search line :
    # Use the Java security manager? (yes/no)
    TOMCAT5_SECURITY=yes
  3. you change line :
     TOMCAT5_SECURITY= no 
  4. you save file
  5. you call
    sudo /etc/init.d/tomcat5.5 restart
  6. you done

VERY VERY IMPORTANT: Tomcat config — for hudson, use Java 6 and turn security off. Or life will be filled with tragic and malevolent exceptions.

6. DEPLOY HUDSON

  1. login to your vm machine ( tomcat55/tomcat55 ). Type
     sudo -i 
  2. type
     cd /usr/share/tomcat5.5/webapps 
  3. and then you call this and enjoy :
     curl -k https://hudson.dev.java.net/files/documents/2402/94697/hudson.war > hudson.war 

    the -k param tells it : I know you want https, but I just want the file, so ignore the certificate, mmmkay

  4. get your ip address ( first one from ifconfig ) and open hudson in a browser by navigating to http://address:8180/hudson ( i.e. http://192.168.235.128:8180/hudson/ )

Note: that long ass url in the curl is the latest version of hudson. You should go to this page and get the link to the latest stable release of Hudson instead of trusting some random dude on the Internet.

7. CREATE A HUDSON JOB

Ok, step 7, you have made it so far. It’s actually a little anti-climatic.

Because you probably don’t want to build my company’s app, I will use the grails pet store in this example

Make a new job

  1. Open your favorite web browser and navigate to your hudson build — again, for those who weren’t paying attention, get your ip address ( first one from ifconfig ) and open hudson in a browser by navigating to http://address:8180/hudson ( i.e. http://192.168.235.128:8180/hudson/ )
  2. Click on the New Job link on your left hand side
  3. In the next screen, define a new project called Grails Pet Store of type Freestyle project
  4. Click OK

Set up SVN ( or CVS )

  1. In the next screen, provide a description for your pet store and select SVN under the Source Code Management section. Enter your URL, for the petstore, we will use http://grails-petstore.googlecode.com/svn/trunk/
  2. Click Save
  3. Click on Build Now on your left hand menu to get SVN to fetch your files. You should see a build in process link on the left hand side. You can click on the console output link to see the progress of your build, but this should retrieve the code from CVS

Do initial setup of Grails app

  1. Open a terminal window and SSH to your machine. Login with your username and password ( tomcat55 / tomcat55 ). Before we go any further with grails, we need to call a
    grails upgrade

    on the directory ( see this file for details ).

  2. type
     cd .hudson/jobs/Pet\ store/workspace/trunk

    hudson creates a workspace that is stored by default in your .hudson directory. You can change this by setting a variable called HUDSON_HOME in your /etc/environment file

  3. type
     grails upgrade 

    Answer yes to any prompts

  4. at this point, you would run all the initial project steps needed to make sure you project builds correctly. For the petstore, follow this wiki page ( short version is to call : grails install-plugin searchable;grails install-plugin feeds; grails get-dependencies ) .
  5. Call grails prod war
  6. Call grails test-app
  7. Call sudo cp *war /usr/share/tomcat5.5/webapps/petstore.war

Set up build script

  1. return to your browser window, navigate to hudson and click on the “Pet Store” job
  2. click “Configure”
  3. Click build -> Execute Shell
  4. Enter the following in the dialog box:

    sudo bash
    . /etc/environment
    cd trunk
    grails prod war
    grails test-app


    sudo cp *war /usr/share/tomcat5.5/webapps/petstore.war
    sudo /etc/init.d/tomcat5.5 restart

  5. Click Save.

At this point, I should probably explain some of the steps here. So here is the annotated version of this script

# calls the bash shell as the superuser, which prevents many of the issues. In step 4 : Change Sudoers file, we got rid of the password checking prompt

sudo bash

# there wasn’t a clean way in Ubuntu to ensure the environment variables were set in a shell, so I call this file to set Grails_home
. /etc/environment

# make sure we are in the right directory
cd trunk

# generate war file
grails prod war

# the reason we call test-app after prod war is because the war step deletes all the test results, which sucks. This way, we can still keep all the test results — this step took me about a day to figure out, and would have probably been prevented if a GANT script was used instead
grails test-app

# replace the tomcat petstore war and restart – if you are really hardcore, you might want to incorporate the Cargo ant packages here.
sudo cp *war /usr/share/tomcat5.5/webapps/petstore.war
sudo /etc/init.d/tomcat5.5 restart

Store artifacts

One of the nice things about Hudson ( and every other continous build environments, I suppose ) is the ability to store the built war file.

To do so, go to your post build actions – select the archive the artifacts checkbox and enter

trunk/**/*.war

Also check publish JUnit test result report and enter

trunk/**/test/reports/TEST*.xml

Hudson is smart enough to validate your artifact links. So play with that if you are building into a different SVN or CVS directory.

There are other Hudson features, such as the ability to poll the source management system and automatically starting builds, find out more at the Hudson website

And we’re done! And here you have an explicitively and ridiculously detailed guide to getting Grails set up on Hudson.

You should be able to hit your live war file in your http://192.168.235.128:8180/petstore ( albeit I couldn’t verify this since one of the tests fails )


TIPS AND BLATANTLY OMITTED STEPS

  • This stuff is a lot easier on the Ubuntu desktop because of the synaptic package manager, the intent of this post is to have a complete step by step so that all the sarcastic comments can be removed and it can be handed off to someone else with little ubuntu, vmware or hudson knowledge to deploy.
  • Hudson has a beautiful Gant plugin that I was too lazy / ignorant to integrate
  • I cheat because I create a tomcat55 user. If you need to change the user Tomcat logs in with, do a
    sudo nano /etc/init.d/tomcat5.5

    and change the value of the string

    TOMCAT_USER=Tomcat55

    . You need to add this user to the admin group.

  • You probably want to create roles into Tomcat’s tomcat_users.xml file, this is located in /usr/share/tomcat5.5/conf
  • A nicer way to integrate Grails projects into Hudson to generate a POM file with the Grails Maven Plugin, and then do an actual hot deploy into Tomcat with a tool like Cargo
  • Grails 1.1 promises continous integration. I promise to be drunk this Friday
  • Tomcat craps out a lot during the war redeploy. I found that wrapping the war file copy between a sudo /etc/init.d/tomcat5.5 stop and a sudo /etc/init.d/tomcat5.5 start will help this
  • You need to change the way VMWare player maps ports if you want the app to be visible to anything outside of your host machine. Look at this pretty advanced NAT Port Forwarding tutorial for more details ( I map ports 22 and 8180 to outside ports 722 and 780 )
  • Restart tomcat : sudo /etc/init.d/tomcat5.5 restart
  • Restart mysql : sudo /etc/init.d/mysql restart

5 thoughts on “[Howto] Setting up a Continous Integration Server for Grails with Hudson on VMWare

  1. Flüge Bangkok

    Thank u for taking the time to write this article.
    I have already UBuntu on my computer, so I have followed ur instructions from step 2 and suceeded to
    to get a complete build server running Apache, Grails, Tomcat, Hudson on the VMWare server.I have also set up Subversion but I have some problems by using Subversion to exchange files with others.
    I would be very thankful, when you cover it in your next post.

    Reply
  2. Guillaume

    Hello thank u for the tutorial.
    I have a little question.
    Is it possible to update the war without reboot tomcat?

    Reply
    1. Tomas Lin Post author

      Yes. You can hot deploy with Cargo. Look into their documentation. However, I would recommend that you restart tomcat on each deployment, as there are some memory issues when changing war files that result in a PermGenSpace Out of Memory exception after a few deployments. One solution is to use the JRockit JDK, which has better memory management.

      Reply
  3. Pingback: Hudson setup to execute scripts with root privileges (safely) « Never Say Never

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s