In this post, I want to share my experience building a content management solution for a Grails application using Adobe AIR and XML bundles. This solution allows semi-static content to be kept externally. Main benefits are ease of deployment and desktop integration.

The Problem with Semi-static Content in Grails domain objects

Back when I was working for Mindblossom, I explored using Grails domain objects to build content management views for content-intensive websites. This approach worked fine as long as the content we were mapping were fairly deep and consistent. A list of wines, travel destinations or babecues usually had a set of attributes that were well known and defined prior to building the application.

There were two types of data, however, that constantly gave us problems.

1. Products with arbitrary attributes: Many of these features were optional and not well-mapped. For a large automotive manufacturer, for example, these attributes would change from year to year and model to model. Adjusting to them would either mean building a robust dynamic attribute management system into the domain model ( complex ), or extending the domain model each time a new product arrived.I remember long nights at work when a client would send us car information at the last minute, and hearing cursing in German from across the desk from the poor guy that was constantly changing our domain models to match the new information.
2. Information was that used to support user-interface elements. Page information needed for SEO, structured information needed for just a handful of pages, etc. This information was too big to keep in configuration files and didn’t warrant building a specific Grails domain object for them.

Two questions always came up when we were discussing pure Grails domain object-based CMS solutions. Our slick talking sales and product people will always talk their way around them with clients, but they always left a nagging question in my head as a technologist.

1. Deployment. How are we going to ensure that we’re testing content with at least a similar set of values? While this didn’t matter much for product information, it was important for UI content that was constantly updated and changed. Can we make changes without re-deployment?
2. Flexibility. The second one was flexibility. Despite all the advances in Grails, GORM and its hibernate based infrastructure still statically maps domain objects to databases. This infrastructure is heavily biased towards initial development. If I was building my own application from scratch, adding or removing attributes simply means changing my domain object and redeploying. To maintain this form of development, however, would mean that we would have to use a tool like autobase to manage database changes.

A Sketch of our current Editorial Content solution

When I first arrived at Pixsta and started working on the Empora fashion search website, we already had an editorial solution in place based on XML bundles and Plain Old Java Objects. Any content that was not classified as product information was kept in XML files that were pulled in by the Grails application.

At first, this system seemed a little clunky. As a geek, it was very hard to resist solving the problem by throwing more technology at it. We could solve the XML bundle problem by using a schema-less NoSQL  solution like CouchDB. Then we can store all this stuff in a database!

As I got to work with this setup for the next few months, I began to appreciate the simplicity of this design. Having XML bundles stripped a database dependency for content that was constantly changing. We could deploy new versions of this content by simply copying this information over. If you have ever worked with external images in Grails applications, this was the same mechanism, except that instead of just jpg and png files, you were moving structured data that contained information about views and products.

I have nothing good to say about the POJO part of the implementation. It worked, but adding a new attribute would involve modifying both the POJO and the XML serializer it used to convert the XML into something usable by Groovy. Over the next few months, we quickly migrated the POJO solution to a simple Groovy XML slurper solution, where the data structure would be loosely defined as an XML document that was passed around.

At the end, we ended up with a system for editorial content that looked like this:

We still kept our important product information in a database and performed our proprietary image search operations on them, but this setup allowed us to add additional editorial content to results based on the request URL. Our Grails controllers simply aggregated the information that best fit in a database with the information from the XML files.

Building a CMS

Our editorial content solution worked for the most part. However, because the editorial information was kept in XML files, it meant that our content team was constantly editing XML files by hand. This was not good. We saw encoding errors, malformed XML and problems that arrived from copy and pasting from Word files.

Our next task was to build a Content Management solution to handle this editorial content. Most of our content already lived in the form of XML files and local files. We wanted a platform that could easily be updated, allow us to use existing skills in-house and be easy to extend. We chose Adobe Air.

Benefits of Adobe AIR

After considering a web-based Grails approach, Griffon/JNLP and Adobe AIR, we settled for Adobe AIR for the following reasons.

• Direct access to the file system.
• Already know AIR / Flex programming.
• Easy to update via the automatic update mechanism.
• E4X support for XML editing.
• No need convert XML to domain objects.
• Good library of direct UI Manipulation Libraries.

A few screenshots of the CMS:

After about two weeks of work ( sandwiched between other priorities ), I had written an initial version of our editorial content CMS. It had a few interesting features:

• Direct image map manipulation using the ObjectHandles library – I wrote a simple imagemap renderer that allowed our content team to define and edit existing imagemaps and generate the necessary HTML.

• Automatic updates – Adobe AIR provides an automatic update mechanism, it enables changes to made quickly and pushed out to clients ( think JNLP ). This proved very useful in the initial use of the content tool, as it allowed us to make changes to the CMS independent of other product tools. It also enables us to extend the CMS quickly whenever new features are added to the web application.
  
• Read and write from XML - instead of converting objects into ActionScript objects, I simply used e4x. This allowed me to quickly map XML fragments to ActionScript components and edit files directly.

• Links to image and content check directories. One of the benefits of a custom CMS is to be able to create links to a data test bed. The CMS had direct links to a staging server that pointed to the data being edited directly. This allows our content team to see their data exactly as it would appear live. This functionality also means less changes to the web application, since there was not a messy cycle needed to support preview and direct GSP manipulation.

• File Locking - added a simple file lock wrapper around each file being edited. Since our content team is small, the chances of the same people editing the same file was very slim. A file-based lock would simply notify them that the file was being edited by someone else.

Lessons Learned In This Approach

• Sometimes, the database is just not the right solution.
• Deployment as static files is much easier than from a database.
• Configuration files can be made much easier by externalising into files as well.
• Web services get more complicated, but not by much.
• It doesn’t take that much time. With a pure server-based approach, you have to constantly worry about migrating data, deploying and setting up the right infrastructure. Sometimes simple is better.
• Auto-update is a joy.

Possible Helpful Grails Plugins if you want to do it yourself

In the course of investigating this solution, I came across a few plugins that could potentially make this process easier. None of these plugins were used in this solution because our infrastructure already supported most of this functionality. But I would definitively consider these when starting a new project where I would be creating a new set of dynamic attributes from scratch.

• Static Resources Plugin – Runs a web service that deploys your static content as an XML file. Combined with the new XML form creation features of Flash Builder 4, this might allows us to use client-side scaffolding of data.
• XSL Transformation Tag – Allows you to apply a XSLT to a XML fragment and render into a GSP view.
• Export Plugin – Enables the export of grails data into other formats like Excel.
• Weceem – A CMS that provides a easy way to manage changing content. If Weceem had attribute management, most of these problems would go away.
• Gorm-couchdb – Another approach would be for to use the AIR app to connect to couchDB and have our grails app retrieve the data it needs via CouchDB. However, this still has the drawback of statically mapping our domain objects to a database definition.

Static Content Wish List

So what would make managing this semi-static content easier? I can see a couple of plugins that might make all the work and effort done here obsolete:

1. A Data export / import plugin. A simple plugin that would enable backing up and exporting a grails project done at the UI level.
2. A Dynamic Attribute Management Plugin. A plugin that will allow for the creation and management of domain object attributes at runtime.

Conclusions

I like this approach much better than a pure Grails approach. Content management, to me, fits better within a client. Using Flex and Air means that I would be able to re-use many of the display components. It enables separating the portions of the web application used for display and the tools needed to edit these.

There are many usability benefits available to a desktop interface: drag and drop of files into the editor, ability to save drafts and revisions via the built-in SQLLite database and offline/online content synchronisation.

Moreover, compared to a CMS built purely in Grails GSPs, a system like this can be built to edit GSPs and domain objects on the fly without worrying about application re-loading and making dramatic changes to Grails core.

I still would like to see a robust attribute management mechanism similar to the one available for Elastic Path.

It would also be interesting to start experimenting with either a Griffon or AIR-based desktop application for managing Grails content for the Weceem CMS ( I can dream, can’t I? ).

Maybe someday I’ll find time ( or a sponsor ) to write one.

How do you get rid of the product backgrounds?

This was a problem that has plagued me and the other user interface developers at Pixsta ever since I started working here. Our efforts to build nice homepages and cool Facebook / Twitter apps seems almost hampered every try by the limitation that our product images always had a white background.

I decided to spend a little time investigating how we could remove backgrounds on the client-side, and immediately thought of Pixel Bender. In this post, I share the results of my experimentation and show that Pixel Bender is a viable alternative to server-side image manipulation and calculation. It really highlights the power of Flash and Flex over other client technologies like JavaScript.

Pixel Bender is a technology by Adobe that enables image processing at the pixel level introduced with Flash 10. It is fast and uses the native graphic capabilities of the machine. It is much faster and optimized than plain ActionScript.

I started with a naive approach, copying a colour-based pixel removal algorithm from an old stackoverflow thread. This algorithm simply replaces the white pixels of the image within a certain threshold with a transparent pixel. The kernel code is as follows:

input image4 src;
output float4 dst;

// How close of a match you want
parameter float threshold
<
  minValue:     0.0;
  maxValue:     1.0;
  defaultValue: 0.4;
>;

// Color you are matching against.
parameter float3 color
<
  defaultValue: float3(1.0, 1.0, 1.0);
>;

void evaluatePixel()
{
  float4 current = sampleNearest(src, outCoord());
  if((distance(current.rgb, color) < threshold) ){
      dst = float4( 0.0 );
  } else {
      dst = current;
  }
}

This algorithm works well for a big chunk of the time and uses Pixel Bender’s capabilities fully. However, it fails when the item itself has a lot of white and doesn’t do a very good job of removing the edges. The other problem here is that a threshold that might work for some images might look horrible on other images.

After finally getting the hang of Pixel Bender, I walked over to my well respected collegue Tuncer, who is a mad image processing PhD genius and asked him if he could recommend something we could use for  client side background removal. He sent me a version of the background removal algorithm we currently use in the Empora site, and I went ahead trying to see if it could work on Pixel Bender.

After tweaking it for a day and converting it from Matlab and Java into ActionScript, I arrived at a nice solution that worked with Pixel Bender, Flex and Flash.

Unfortunately, I cannot post the code for this as it involves some algorithms that a) I don’t really understand, b) are the brainchild of somebody else and c) are probably proprietary for the company. However, I can share some results.

The following rows show the original image, the white background removed image with an arbitrary threshold, and the automatically calculated image.

As you can see, this works really well and shows the possibility of the Adobe Pixel Bender technology in processing images on the client side.

There were a few limitations and challenges that I ran into implementing these algorithms in Pixel Bender, which were the following:

• Pixel Bender works at a per pixel level. Some of the pre-computations in these algorithms worked with the entire image. This means that a pure Pixel Bender implementation would be almost impossible to do. However, once these automatic thresholds were calculated, the rendering speed of Pixel Bender was simply amazing.
• There are a few kinks in the Flash player implementation of Pixel Bender. Most annoyingly, it doesn’t allow For loops. I had to write a simple groovy script to unravel my scripts. Hopefully this technology matures more and the language becomes a little more than a crippled toy language.
• The path to integrating Pixel Bender Shaders into Flex were not very clear. Neither the Pixel Bender page nor the official Adobe documentation goes into these technologies in depth. Fortunately, Mike Chambers has blogged about integrating Pixel Bender into Flex, and I would be completely lost without his blog posts. [ 1 ] [ 2 ] [ 3 ]
• To make it better, I would probably add another shader that will blend the edges into the image more. But this would be easy with Pixel Bender. The background in the example is very dark, we would display our images in a white or semi-white background, reducing the contrast of the edges.
• The algorithm is a little aggressive sometimes, removing more than it needs. But the Flex UI would allow for either a slider to be incorporated to manage individual thresholds or enable a more dynamic background removal approach such as the one presented here.

It was really neat to be able to explore these kind of image manipulations and see the limits of this technology. Makes me really appreciate working with people that actually know how to use Mathematics for good. It’s really exciting to see Pixel Bender and the possibility it brings to image processing.

If you are interested in reading more about the image processing work done at Pixsta, visit the homepage of our research team, Dr. Tuncer Aysal and Dr. Daniel Heesch. It contains all their publications to date.

To see some of this stuff at work in a real live product, visit Empora.com.

Please note: I cannot and will not share the code to the Pixel Bender Kernel for background removal.

It’s a little overdue, but I am excited to announce that I’ll be speaking at the Groovy / Grails Exchange in London on Dec 9 and 10.

The topic of my talk will be ‘Building Killer RIAs with Flex and Grails’, and you can read all about it on the Conference website. Building Killer RIAs with Flex and Grails.

It’s my first conference presentation, so please be gentle…

See you all there!

In this post, I share my experiences building a simple Content Host with Grails and the Google App Engine.

We’ve been pumping up the volume at Pixsta promoting Empora – our visual search engine for fashion items written in Grails.

One of the recent tasks I’ve worked on was to build a series of banner ads in Flash to showcase the colour search feature. Within the site, you can specify a colour or category, and it would let you browse similar items via our unique PhD-powered visual search algorithms.

It was very interesting that nobody in our teams had worked with ads or ad networks before. So we had to quickly learn all about this business very quickly. Fortunately, it also meant that we could make some mistakes – which fortunately there were few – and learn from them.

Building the Ads

The final ad that Mike, our graphic designer, and our product team came up with looks like this. ( Warning – flash file, original size is 300 x 250pxs ). You can see one in action at my lovely big feet.

The ads themselves were not difficult and took less than a couple of days to do in Flash. I used the standard Flash components for the display objects and the Tweener library to animate the transitions.

What was challenging was really meeting the different and undocumented requirements from different ad networks about Flash ad formats. It was hard not to lose your temper after getting two or three conflicting versions from the same company, or getting documentation sent to you by an ad network a couple of days after you have finished building your flash files. I definitively learned a lot more about clickTags, flash versions and the world of ad networks than I ever really wanted to know.

I had built an initial prototype in Flex, but quickly rewrote it in Flash as we couldn’t get file sizes small enough even after using the RSL mechanism. Flex still has a lot of work to do to make Swf files smaller.

Hosting the Ads.

Most of the bigger ad networks had their own tracking mechanisms and would host our ads on their networks as long as they met certain size restrictions ( 40k and 100k ). Some of our ads needed to be retooled to fit within certain guidelines, and I had to create several versions of each ad in different sizes, colour combinations and click out requirements.

For smaller blogs, they requested that we host the swf and backup image files ourselves. We had to come up with a solution that was quick, efficient and powerful enough to handle large amounts of traffic.

I had come across an article about using the google app engine as a content delivery network about a few months ago and have been wanting to try out this approach on Flex / Flash files. I’ve been playing with Grails and the App Engine a lot on my spare time, and suggested this to the team.

Google App Engine – Advantages

There were several advantages I saw to using the App Engine that made it perfect for our simple ad server:

• Almost unlimited bandwidth on demand: We could always buy more, and with an initial allotment of 4 gigabytes free a day (paid quota ) and $.12 per gigabyte transfer over that, it seemed like a perfect solution for us. We weren’t sure how much traffic we were gong to use, so the app engine provided a no-commitment answer for our ad hosting.

  Compared to cloud solutions like Amazon ( $70 per month ) or  content hosting like Akami ( contracts / service terms required ), the app engine made sense as an initial testing bed for our ad hosting.

• No Server Setup Required: With a traditional hosting solution, we would have had to set up a FTP account or an Apache server to deliver this content. If we wanted any sort of server side scripting, this would have had to been set up and would have taken time away from our system administrator.

• Grails integration: We have a lot of in-house expertise in Grails – and it was nice that the AppEngine plugin allowed us to use this expertise, just in case we needed to add something more robust. With this project, we were uncertain about some of the  tracking requirements, so it was nice to have the option of having GSPs, templates and all the other Grails goodies just in case we needed to expand the functionality of the ad host. It’s never a bad idea to have an enterprise level solution in the back pocket, just in case.

• Quick deployment: The app engine plugin allowed us to quickly add our content to be hosted, package it into a war file and have it running on the server within less than 20 minutes of creating the initial application.
  
• Ease of Administration: Compared to many build-your-own solutions, it was very easy to add additional developers into the application.

Embedding Flash Files

In the code we provided our ad hosts, we simply used SWFObject to include our SWF file. Using SWFObject allowed our ads to fallback into gif files in the case where Flash or JavaScript was disabled. We wrote a little bit of additional javascript so that we could differentiate which client and campaign were being tracked.

Ad Tracking

I wrote a simple grails page that would invoke google analytics with the correct tracking details. We were already using Google Analytics to track clicks, and it was very easy to just include a piece of javascript that would track all our ad impressions. [ code here ]

Did it Work?

What started as a simple experiment in hosting a few flash files that had to go up quickly rapidly became a test of how the Google App Engine would perform as a RIA hosting server.

Our marketing team got our banners on one of the newspapers websites, and we went from serving about one thousand ads a day to five digits ad impressions in one day.

The App Engine performed admirably. The only hiccup we experienced was setting the initial quota too low, and we quickly ran out of bandwidth. After a simple billing change on the server that took effect in 15 minutes, the ad serving continued.

On some days, the outbound traffic would hit more than fifteen gigabytes, but we had not had to restart our Grails server at all since the initial deployment and while we see spikes in memory and CPU use, there seems to be very few issues with the app engine as a content delivery server. Also, it is reassuring to see that fifteen gigabytes cost less than $2 in the billing reports.

It was definitively a good choice to do this in Grails. We had vague tracking requirements at the beginning of the project, and were able to implement a fairly robust if simple tracking system in less than a day.

Going Forward

This project shows that the Google App Engine is a solid solution for hosting dynamic, high bandwidth content such as Flash movies. Given the ease by which we can develop Grails applications, it would be interesting to build out a nice simple GUI for managing and uploading ads into the Engine without requiring a server restart.

One of my main frustrations working with Rich Media ads is that the specs given by current ad hosts make them nothing more than glorified animated gifs. It would be interesting to explore the building of more robust tracking and clickTag mechanisms for RIAs and widgets. It would also be interesting to explore alternative ways of displaying ads such as via HTML5’s canvas and video tags and see how they compare to Flash.

The prices of the ad engine definitively make it a very good candidate for rich, dynamic content. We see that Grails is able to scale up to many clients for simple GSPs that spit out data. However, it is still unclear as to how versatile dynamic flex applications with a complex domain model would perform on a platform such as the App Engine. Given that the app engine would time out a request if it takes too long to complete, it would be interesting to see how data-heavy RIAs would perform in this environment.

I’ve been thinking a bit about how to use the Google App Engine in a Grails deployment infrastructure. In the following post, I want to sketch out how to use the Google App Engine as an aggressive cache mechanism for Grails applications to reduce the cost of Grails experiments.

Let’s look at some of the pros and cos of using the Google App Engine for Grails Application Development.

Good: Lots of memory, memcache, bandwidth and disk space.
Bad: No threads, no hibernate, no full JRE, limited GORM, can’t write to file. Many plugins don’t work.

A possible way around these problems, of course, is to get a low cost hosting provider and use the Google App Engine to amplify the available bandwidth and processing power available to your web applications. Instead of thinking of the App Engine as another complete deployment platform, this just becomes a loudspeaker to help scale your tiny little app.

Here is an idea of how something like this might look like:

Building the caching part of this is not terribly difficult, Google App Engine already provides us with all the pieces we need via the MemCache Service and the URLFetch Service.

I would imagine that a typical request handler would look something like this in Groovy:

Set up your Cache

import java.util.Collections;
import javax.cache.Cache;
import javax.cache.CacheException;
import javax.cache.CacheFactory;
import javax.cache.CacheManager;

 Cache cache;

// expires cache in one hour
Map props = new Map;
props.put(GCacheFactory.EXPIRATION_DELTA, 3600);
try {
CacheFactory cacheFactory = CacheManager.getInstance().getCacheFactory();
cache = cacheFactory.createCache(props);
} catch (CacheException e) {
// ...
}

Handle Requests

processRequest(){

    // get cache ...

   def key = buildRemoteGrailsAppRequest( params )
   def page

   if( cache.containsKey( key ) ){
       page = (String) cache.get(key)}
   } else {
       page = new URL( key ).getText()
       cache.put( key, page )
   }
   render page
}

In the above code, we basically just issue a call to an externally hosted Grails server unconstrained by the file and thread restrictions of the app engine. This allows to quickly take an existing Grails application and scale it up using the App Engine. It offloads much of the page serving to the App Engine.

You could also set up  a more intelligent caching mechanism for your application that expires your caches based on a write. I would simply set this up as a onUpdate or onSave Hibernate Events listener on your externally hosted Grails application.

Given that you get ten application instances for each Google App Engine account, it becomes very viable to also offload processing intensive operations into services that are hosted on the App Engine. Say you have an App that does a lot of image manipulation and resizing. You could simply write a few simple services on the App Engine that use the supplied Image Manipulation API and store these into cache.

It helps conserve your server resources ( expensive ) and offload them onto the App Engine ( cheap ).

The App Engine affords a really neat service oriented infrastructure for Grails developers because we no longer need to consider bandwidth and server resources as an issue. By front-ending our bandwidth-intensive apps with an App-Engine Service and offloading computationally expensive operations into it, we can provide highly scalable, powerful applications at a low cost.

Of course, it requires a little bit more planning and engineering, but this type of infrastructure allows us to keep all the productivity gains of Grails without having to deal with the constrained App-Engine development environment.

What do you guys think?

How can we build user interfaces using GSP that are manageable and easy to update? Many of the books on Grails focus on the technology available in Grails, but very few give rules and techniques to make GSPs manageable.

Grails templates and the out-of-the-box functionality is great. But once you start trying to integrate layouts and designs for real world websites, you’ll find that bad coding practices can quickly lead to a chaotic and unmaintainable mess of spaghetti code.

In this post, I want to share some tips, lessons and ideas I have learned from projects worked in the last year and a half of Grails UI development.

Planning Your UI

1. Build a Skeleton with Layouts, Taglibs, Codecs and Templates.

For one of our projects, we made the mistake of giving a PSD template to an HTML/CSS contractor. He started putting things together in the way that a traditional HTML developer would, worrying more about positioning and squeezing the right graphics in place. Big chunks of code were copied and pasted into other areas so that it looked right and he would get paid. We then spent the next three weeks retrofitting his HTML mess into our GSP files and had to throw a lot of his code away.

If you’re starting a new project and are not very familiar with Grails, spend a bit of time learning and educating your team about how to properly use Sitemesh layouts, codecs, tag libraries and templates.

Once you have learned them, build a Skeleton for your site. Based on wire frames or other design artifacts, create templates and views that simply render out a string. Use layouts to organize these simple templates and views and map them to your URLMapping. Try navigating your site. This will help you identify missing functionality and better organize your project’s UI structures before changing much of your code.

2. Figure Out The Plugins You Need Early.

One of the big hurdles that get in the way of creating and maintaining Grails projects is trying to insert a plugin after some of the code and page functionality has been scripted. Different plugins will lead to different tags and UI strategies. Ask yourself the following questions:

• Are you going to use Ajax?
• Which JavaScript libraries or UI frameworks are you going to use?
• Which kind of template engine are you going to use? Sitemesh (default) or FreeMaker?
• Does your site need security and authentication?
• Do you need a blog, shopping cart solution or third party payment gateway
• Are you going to use the UI Performance Plugin?
• Are you going to use rich client technology like Flex, GWT or OpenLaszlo?
• Are you going to provide alternative views for mobile devices such as the iPhone via the iWebKit plugin?

3. Have a Strategy for Updating Semi-static Content.

There are certain parts of your website that will be updated only once in a while. These might include corporate information, about pages, contact information, etc. This content might be different than the main functionality of your website but might need frequent updates.

Depending on the complexity of your project, you might want to consider building a content management system and admin interface that is separate from your main application. This will allow you to easily maintain this type of data without having to restart or redeploy your application.

Consider hooking up to a Content Management interface like Weceem or the Grails CMS plugin. Or you might want to design your application so you’re editing this content with established platforms with beautiful editing interfaces like WordPress, and then pulling in this data via the WordPress Plugin.

When thinking about static content, also think about where and how you are going to host and update images and other media on your Grails application. Are you going to host them externally in a separate server? Would you need a plugin like static resources? Are you going to deploy a content delivery network for videos? Thinking out and planning these steps will simplify your UI development process and will prevent you from having to make changes later.

Coding Your UI

4. Build Top-Level Templates That Map To Domain Objects

Let’s assume you have a complex book listing GSP file:

Because this all renders into one page, you might start by adding each of the areas, perhaps separating each section with html markup. You’ll end up with a gsp page that looks like this:

This GSP can get very hard to manage after a while. Worst yet, it is not immediately evident when you come to the page which section actually belongs to which domain classes. You’ll end up spending a lot of time tracking each bit and section.

A better solution might be to break down the larger HTML into templates that map to the domain classes they represent. This allows you to create views used for navigation ( usually homepages and top level category pages ) and those used to display data. This helps keep views simple, allows for re-use and makes updating future pages more manageable.

So for this example, we might break down the sidebar into further smaller gsp templates, while for things that are fairly atomic like the booklist, just render the template.

( In the example above, you would probably have also separated header, footer and sidebars into a layout. )

5. Generate As Much Code As Possible.

One of the very powerful things about Grails is the ability to generate code based on domain objects. However, this mechanism is also one that is rarely used outside of the typical CRUD templates. In past projects, what I ended up doing was to edit the resulting gsp file, losing the ability to regenerate my views when I needed to change domains.

If you follow the last tip, you will see that your data views become fairly individualized, atomic containers for data. You can then use a plugin like the xTemplate plugin to define more views than the default CRUD views provided by Grails. I find that you usually want a read-only list browser, a read-only individual item browser.

This is also an excellent way to extend plugins and other functionality.

Let’s say you want to provide AJAX functionality to your domain objects and connect this into the Grails-UI plugin. Once you have built one that works, you can take this functionality and extract it so that you can use this functionality to generate more views. At the end of the day, you will be able to update individual views via the generate-views command whenever your domain object changes, and not to worry about doing this manually. Less manual labour = less mistakes.

6. Build Helper Libraries.

A trick that has served me well is to build taglibs for links and images that you are going to put on your gsp pages with a custom tag that wraps the default Grails tags. I’ve found that many websites have succint SEO and tracking requirements. By keeping an utility library that generates your links, you don’t have to worry about forgetting to add analytics or title/alt tags.

For code that you find yourself writing across projects, consider wrapping these into utility plugins that you can then share across your projects ( and contribute to the community! ).

For example, if you find yourself embedding a lot of flash files in your projects, you might want to create a tag that also writes out the alternative content and noscript tags and put it into a plugin. If your script library updates, you simply need to update your plugin instead of updating all your codebases by hand.

7. Preserve Names Across Technologies.

This one is pretty simple. Let’s assume you have a bunch of gsp files that also call css and javaScript files. Come up with a common convention to associate them. For example, you might have a set of css files for Book called book.css, book_ie6.css and book_ie7.css, and a set of JavaScripts called book.js.

If the layouts of your pages change often, consider also having css and javascript files called book_external.css and book_external.js that are served from an external source and override the files deployed with your web application. This will let you change your site without having to re-deploy, assuming that this is what is dictated by your semi-static content strategy.

If you have complex views, consider writing a helper service that has the same name as the views. For a read-only bookList, for example, build a method called getBookList() in your BookService. This prevents you from repeating this code across many controllers, and helps you keep this code maintainable. ( Also think about putting as a code template in src/templates, so you can generate helper services easily without affecting your controller ).

8. Keep Your Technologies Uncoupled.

With scriptlets and the g:if tag, it becomes very tempting to resort to all sorts of inline code generation madness to customize css and javascript views. The problem comes when you need to replace or refactor this code later on. Let’s say that you wrote a css that calculates widths according to the a value in your domain objects like this:

.book{ width: ${ book.narrow ? 40 : 120 } px }

What happens when you suddenly find out that you need to use this component somewhere else with a different margin? You would need to rewrite this section and find all the places where your code has been inlined. Instead, keep your code and style separate by strictly forcing domain objects to generate id tags in Divs, then style these divs according to your UI specifications:

<div id=" ${ book.narrow ? 'narrowBook' : 'wideBook' } " />

This way, you can simply overwrite the css file for the new page.

Similarly, it’s easy to try to insert values directly into inline JavaScript files. Just don’t. Instead, what I’ve found works is to declare variables in your GSP files,

<g:javascript>
	var bookCount = ${ Book.count() }; 
</g:javascript>

Then use them in your javascript code. The reason for this is twofold.

First, you can look at the variables you’re using and passing back and forth right away, allowing you to catch if a variable has been missed or is returning the wrong value.

Second, you can then keep the bulk of your javascript inside a separate js file. This allows you to quickly replace it with another library or test it out with fake data during development. It removes the dependency to your backend. It also enables this to be compressed via utilities like ui-performance.

9. Write Tests.

Test, we all know we should write them, but it’s easiest to forget about them when you’re working with the UI layer. After all, you’ve already written all your controller and unit tests. Why bother spending time testing out things that can be caught by a website run-through?

You will find that writing at least a minimal set of tests will enable you to keep sane once a whole whack of UI code has been written. There are a lot of plugins that can help you with catching accidental 404s and 500 errors, specially when it comes to complex HTML websites.

The list of plugins keep growing, but currently you can choose between:

• Canoo Webtest Plugin
• Selenium Plugin
• Grails Functional Testing Plugin (HtmlUnit)

Additionally, you can use external tools like JMeter, VisualVM or Perf4J to track your page performance and memory usage. Don’t forget about tools like YSlow and Google Page Speed.

At the minimum, write a set of tests that map to your URL mappings to check for 404s and 500 errors. Check your links and use a plugin like build-test-data to see that your components are working the way they are supposed to.

Anyway, these are the lessons I have learned from Grails UI development. I hope they are useful to you. What are yours?

Inspired by a recent post in the mailing list, I’ve decided to gather a list of web hosts that are Grails friendly or recommended by Developers…

• Google App Engine
• Amazon EC2
• eApps
• Morph
• A2
• Hetzner.de

Know any more? What do you use for your Grails hosting? ( List from Grails.org )

Discussions about this on the Grails Mailing List:http://www.nabble.com/Grails-Hosting%3A-Which-Company%2C-Which-Application-Server-td24860653.html

I’ve decided to share what I’ve written about Flex on Grails as a book online under the Creative Commons License.

http://sites.google.com/site/flexongrails/

Right now, only the preface and the hello world chapters are online. In the coming weeks, I’ll update the content on my hard drive, write new content and post them online. Feedback is highly appreciated.

This book will target Flex 4 and either Grails 1.2. or Grails 2.0, depending on it’s release schedule.

30 Second Elevator Pitch

This book is about combining the power of Flex 4 with the ease of use of Grails.

With Flex, we can build powerful desktop applications like Tweetdeck or incredibly useful web applications like those available at http://www.aviary.com/.

With Grails, we can harness the power of Java in a rapid development framework with the features and advantages of Ruby on Rails.

Best of all, with recent efforts like the GraniteDS plugin and the Spring-BlazeDS integration initiative, we can seamlessly integrate Flex front-ends with Grails back-ends. Development has never been so effortless.

Motivation For the Creative Commons Release.

There are just not enough people that see the potential in combining Grails and Flex.

I know that as a Java developer, I was scared away by the notion of Flex being this complex beast that forced me to work in symbols and a Flash timeline. When I started playing with it, I realized that it was more like HTML and JavaScript than this foreign Macromedia director thing I had in mind.

I can imagine the same thing is true for Grails. One of my friends described it as PHP on steroids. Java, JavaScript and ActionScript are closer in syntax to each other. The JVM and the Flash Player are similar architecturally. But not a lot of Flex developers know about Grails and not a lot Grails developers want to try Flex. We’ve been too scarred by the bad past experiences with broken Java projects or think that Flash is nothing but just a tool to build banner ads.

My experience with Grails and Flex is that it works. And it works charmingly. Most importantly, it lets old Java devs quickly move their skills to Grails and HTML/JavaScript devs quickly move their skills to Flex. It gets rid of mundane tasks like configuring hibernate XML files and having to test five different versions of each browser for each interface.

By releasing the information under the creative commons license, I hope that more people would be aware of this killer combination. Adobe has done a lot by releasing much of the Flex SDK into open source. And Grails builds on the great open source ethos. I hope that this book contributes to Flex developers being more curious about Grails, and Grails developers considering Flex as a viable alternative.

I think we are at the cusp of something wonderful. The new Data Service support introduced in Flex Builder 4 will make a lot of the pain and code generation troubles go away. We’re at a point where writing complex web services and deploying them onto new platforms like the Google App Engine can take a matter of days, not months. If I can only help one more Java dev not have to configure Hibernate manually or one more front-end dev not have to deal with IE6 stupidity, then my job is done.

A Little History on This Book

This book started as an idea on my blog post Nine Lessons from a Flex and Grails site . I was soon contacted by the good people of Apress to put together a book for their FirstPress format. I started working on it, then I fumbled. I ended up in a situation where I was sometimes working 18 hours a day to meet deadlines and was just too drained to work on anything else. I contacted the editor and told him I had to put the book on ice until I could find some time to continue working on it.

Fast forward a few months, I had moved to London to work in a very interesting startup company that does amazing stuff with visual search technology. The hours were sane and I had time to digest and process the things I had learned in the previous year. The tech community here is very inspiring. Walking to Grails talks at Skills Matter or learning about the latest Flash techniques at the London Flash Platform User Group reminded me of the excitement and wonder that these technologies could bring. I felt that I was finally in a place where I could work on the ideas of this book in peace.

All the tech pieces started to come into place too. All the floating questions when I was working on the first draft of this book solidified with more and more people interested in getting Flash/Flex and Java working together. I felt the urgency to stand on the street corner and yell : ‘hey, check out flex and grails together!’ once again.

Anyway, I’m gonna keep thinking about this book and writing content, and I hope to get ideas, suggestions and feedback from you.

There is an incredible wealth of amazing looking Flash components in the market today. Tools like Cooliris and the 3D Wall provide visually stunning components that can quickly make any website come alive. With Grails and the Google App Engine, we can build simple, secure and powerful content management engines for these components with very little effort.

In this post, I walk you through all the steps you need in hosting, deploying and securing a Cooliris CMS for the Google App Engine using Grails.

Tip: Flex is also XML friendly, so you can use this technique to add CMS-ed apps on Flex.

Videos

Videos are best viewed in HD and full screen.

1. XML-driven Flash Components, Content Management Systems and the App Engine
   
2. Building a simple CMS with Grails
   

3. Outputing XML from Grails
   

4. Integrating and connecting the Cooliris component into Grails and deployment to Google App Engine
   

5. Building a simple Image Proxy service for Flash Components to overcome Flash Security Exceptions
   

6. Access Control on the Google App Engine with Grails
   

Links:

• The Library Application: http://grailsgae.appspot.com
• Source Code on GitHub: http://github.com/tomaslin/Cooliris-CMS-for-AppEngine

Flash Components

• Flashloaded: Makers of 3dWall and other cool effects
• FlashDen.net: Component Marketplace
• Cooliris.com: Maker of the application we use.

Groovy and Grails

• Grails website
• Google App Engine Plugin
• SpringSource blog and screencast on GAE deployment
• Groovy Markup Builders
• Authentication Patch

Google App Engine

• Google App Engine for Java
• Security and Access Control

There is also a $2 Deluxe Version of these videos.

I received a free review copy of Grails 1.1 Web Application Development from Packt Press. They asked me to provide a review for it. Unfortunately, this review will be less than glowing ( sorry Packt ).

The book’s website is here, you can order from Amazon here, and there is a free sample chapter here.

It’s refreshing to see more introduction books to come into the Grails market in the last few months. When I started working with Grails a year ago, there were few references other than the Definitive Guide to Grails. Today, Apress, Manning, Packt and other individual publishers have really seen the need for good reference books. It’s also nice to see that specialized books such as Robert Fischer’s Grails Persistence with GORM and GSQL, Bashal Abdul-Jawad’s Groovy and Grails Recipes or books in other languages such as Nacho Brito’s Spanish language Manual de desarrollo web con Grails.

About the Book

This book is an introduction to web development with Grails. It competes in the same space with Apress’ Beginning Groovy and Grails and Manning’s recently published Grails in Action. I would say that it’s geared towards beginners who want to learn more about the platform, and does contain a robust example of a web application being developed – a team communication portal.

Things I liked about this book:

1. Content-wise, the book is sufficient. If you have never heard about Grails and want to sit through a good tutorial – and can get this book for cheap – it’s a welcome addition to any library. It covers all you would expect from a Grails book. Unfortunately, I think it’s done better elsewhere.
2. The example application here is relatable, in the form of a team communication project.
3. The author does cover Grails 1.1. features such as template namespaces and RESTful mappings, but unfortunately does not do so consistently across the entire book. I would have liked to see features like database password encryption mentioned in the database chapter, for example.
4. I like the reference to the rich plugin infrastructure of Grails. In the testing chapter, he mentions Marc Palmer’s excellent Functional Testing plugin, for example.
5. The file sharing ( not p2p ) chapter was very interesting. It is perhaps the one strength of this book. It’s something that is not well covered by the documentation, and the author provides a clear concise implementation here that answers a lot of the questions.

Things I didn’t like:

1. I am not sure who Jon Dickinson is, which is unfortunate. A lot of the other books in this space have been written by people who have contributed plugins or are known in the mailing lists. When I pick up Robert Fischer’s book, for example, I can associate in my mind the autobase and liquibase plugins. Glen Smith, one of the authors of Grails in Action, is the host of the grails podcast and has written groovyblogs. Peter Ledbrook, the other author of Grails in Action works for SpringSource and wrote the JSecurity plugin referenced in this book. They are people known in the community for their reputation. But I couldn’t find a blog or any info bearing his name.

   I get a sense by reading the book that Jon is more of a backend developer, dealing heavily with the business logic side of things. I think this book would have benefited from another author that deals more with front-end technogies that would have helped fleshed out the book more. It feels like some of the chapters are straight out rehashes of the user manual. Which is not a good thing. You don’t hear about things like the ui-performance plugin, xTemplates or other really nice tools that make developing frontends with Grails a joy. There is also no mention of the OpenLaszlo, Flex or GraniteDS plugins in the RIA chapter. Which is a shame.

2. The writing is a little dry. Grails in Action has a chapter called Advanced GORM Kung Fu and another one called Grails in a Hurry. The enthusiasm for the technology is apparent in that book. The writing here is very matter of fact and purposeful. But to me it’s hard to see the benefits of reading this against the excellent Grails Reference Manual.
3. The chapter organization was very strange. Groovy, the programming language used in Grails, has always been at the first or second chapters of all the other intro books. To me, it sets the stage for understanding the code in the rest of the books. Here, Groovy is introduced in chapter 4. After a long talk about setting up the application and scaffolding. It didn’t make sense to me.
4. Then there is lax terminology. To give you an example, the author lists ‘Less configuration’ as one of the advantages of Grails. I wish the author would have stuck with the more popular term  ‘Convention over Configuration’, which is an actual software pattern – so that users reading the book can take note of this and google this if they wanted.
5. The way code is presented in the book is horrid:

   code code code
   line in bold
   code code code
   In Grails in Action, there are cueballs that the authors then reference. This makes it feel more like ‘hey look at this part of code’. The way code is laid out in this book makes it feel lazy and I wish more attention would have been paid to the typesetting of this book.

6. The plugin chapter is weak compared to other books. While it’s nice to put your code into plugins, the author misses the fact that the plugin system allows for integration with other libraries, making Grails the perfect glue language. I recommend watching Russ Miles’ grails plugins podcast
7. It’s $40.

Conclusion

If this was a cheap book, I would recommend it as an extra resource to your library. It has some gems, but nothing you could not find documented on the grails.org site or the reference manual. If this book was available a year ago, it would have done really well. But today, do not waste your money on it as your primary reference. Instead, try to get a copy of either Graeme Rocher and Jeff Brown’s excellent Definitive Guide to Grails ( 2nd edition ), or Peter Ledbrook and Glen Smith’s Grails in Action.