Getting to grips with Grails, building a survey management system

Sometime in late 2012 I was discussing dissertation project ideas with my girlfriend, as she was coming up to her final year of a computing bachelors. The usual option chosen by many graduates would be to just build a website or an app, or do some form of market research. We decided to encompass all 3 to produce something that works, but ultimately something that could be of value. If I had the time, energy, and funds I’d pursue this as it has potential for a startup, but I don’t, so the important thing that I’ve taken away is the experience working with groovy, grails, and android.

The Idea…

There are 2 main business drivers behind this project. Firstly we wanted to provide a service whereby restaurant owners can register, create surveys, and make them accessible to staff such as printing QR codes onto the back of their menus. Secondly, we wanted to approach this from the end users point of view, whereby customers sitting in the restaurant could download an app for free off the public market places, scan the said QR code, and be presented with the survey the restaurant owner had created. They would fill it in via the app and submit, the restaurant owner then has immediate access to the results in the form of statistics and graphs.

Landing page for Roosearch

Landing page for Roosearch

The outcomes that we’re after:

  • Better visibility to restaurant owners on how their customers feel
  • Easy and seamless access to surveys for the customers
  • A scalable application which can handle increasing users as demand grows
  • A platform for advertising new products and features

There are 3 components to this solution:

  1. Grails web application
  2. Rest API (built into the grails application)
  3. Android app

Why Grails?

  • Develop in groovy, so very accessible to java developers.
  • Quick to prototype with “convention over configuration”
  • Views auto generated if using scaffolding.
  • Easily deployable into the cloud, package as a war and deploy to cloudbees

The Prototype…

If you’re starting out with grails, I’d highly recommend that you get a copy of IntelliJ ultimate edition (and a copy of Grails in action), the support for grails is fantastic and I found it far easier than using eclipse. Whilst there are some excellent tutorials on grails out there (the official documentation is also very good) I’ll hold off and just jump right into how the application works.

One of the awesome features of grails is that it follows the “convention over configuration”, which simply means that if you follow the convention implied by the framework, you don’t have to be concerned about configuration. You can’t escape configuration entirely, but boilerplate plumbing can be inferred by convention. An example of that is if you name your controllers like “SurveyController”, grails automatically knows its a controller for the survey class, based on naming conventions. A similar convention applies for views.

Domain model

Roosearch entity relationship diagram

Roosearch entity relationship diagram

Our data model is quite simple. We have a user, the user has some surveys, each survey has a number of questions, and each questions has a number of predefined responses. The domain classes are self explanatory, but it’s probably worth mentioning a few tweaks I made.

class User {
    String firstName
    String lastName
    String emailAddress
    String companyName
    String facebookPageLink
    String twitterHandle

    static hasMany = [surveys: Survey]
    static constraints = {
        facebookPageLink nullable: true
        twitterHandle nullable: true
    }
}

By default, all fields are mandatory, however in the above example of the User class we can override these constraints to set them as nullable. There are various other constraints that you can set, have a look at the documentation.

class Survey {

    Integer id
    String title

    static hasMany = [questions: Question]

    static mapping = {
        questions lazy: false
    }

    static constraints = {
        id()
        title()
        questions()
    }

    String toString(){
        return title
    }
}

The relationships between the classes are defined by the “static hasMany”. This basically says that one Survey has relationships to many Questions, and this relationship is identified by “questions”.

The mapping block instructs the questions to be eagerly loaded, so once a survey is loaded into memory, so are all of its questions, opposed to just the Ids which would then be loaded lazily.

It’s also useful to override the toString method on your domain objects, particularly if you have relationships as the scaffolding will create drop down lists in your views. If you don’t override toString with something sensible, you’ll just see the object hash codes instead, which isn’t very useful to the user.

Controllers

It’s the responsibility of the controllers to manipulate the underlying data model (via services for example), and respond with views to the user. You can read more about the MVC pattern here.

To get started, you could simply enable scaffolding like so.

class LoginController {

    static scaffold = true

    def index = {
        render(view: "login.gsp")
    }
}

Scaffolding is an excellent feature of grails to get you started. Grails knows the structure of your domain object, therefore it is able to dynamically create controller CRUD operations, and views to manipulate your objects. That one small line of code and you can create, updated, delete, and view your objects! Fantastic eh?!

The bad news…

Whilst scaffolding is great to get you started, the moment you want to do something out of the ordinary, or customisation on views, scaffolding becomes a bit useless, and you’ll have to implement your own controllers (and possibly views). Fortunately, grails is quite flexible so you can leave scaffolding on and just override the methods that you want to customise. As with views, you’re best off getting grails to generate them for you and then customise them, to save you having to write the entire controller/view from scratch.

The methods you can override, and there general uses are:

  • index – default action, usually just redirects to list
  • list – list all of the objects, handle pagination, filtering etc
  • create – render view to create new object
  • save – handle creation of new object, validation etc.
  • edit – render view to edit an object
  • update – handle update of object
  • delete – delete an object

You can read more about the controller actions on the grails documentation.

You can see in the show() method on the SurveyController that I’ve customised it to to add some charts into the response model. You can see how I generate the chart data by looking at the source code in github. The view can then render these as javascript charts (which I’ll come onto in a moment)

    def show(Long id) {
        def surveyInstance = Survey.get(id)
        if (!surveyInstance) {
            flash.message = message(code: 'default.not.found.message', args: [message(code: 'survey.label', default: 'Survey'), id])
            redirect(action: "list")
            return
        }

        def charts = getCharts(surveyInstance)

        [surveyInstance: surveyInstance, charts: charts]
    }

Views

Being quite fond of the default views that grails generates, and not wanting to invest a great deal of time with customisation for this prototype, I chose to generate the views and then just tweak as I needed. In reality, the only customisation I needed to do was to place a “generate QR code” link, and to insert some javacript charts for displaying survey statistics.

Having assessed HighCharts, D3, and the Google visualisation API, I opted for the latter as I felt it was far simpler to use and I didn’t have any need for the advanced features that HighCharts and D3 come with, and there was a plugin for gvisualisation.

Displaying charts was straightforward, after installing the visualisation plugin, add this snippet of code to iterate over the charts that were added to the model and display a barCoreChart.

<g:each in="${charts.values()}" var="item">
            <gvisualization:barCoreChart
                    elementId="chart-${item.question_id}"
                    title="${item.question}"
                    width="${400}" height="${240}"
                    columns="${item.column}"
                    data="${item.data}"/>

            <div id="chart-${item.question_id}" align="center"></div>
        </g:each>

This would then display something like the following, you can change various elements of the charts such as the chart type, axis labels, sizes and titles, please refer to the documentation.

Charts using Google Visualisation

Charts using Google Visualisation

QR codes

QR codes make it incredibly easy to share data to android devices, my intention was to embed a user ID in a QR code, when scanned the app can request all surveys pertinent to the user ID.

Generating QR codes is easy with the qrcode plugin. I have provided a link on the users view to generate a QR code:

<span class="property-value" aria-labelledby="qr-label">
                <g:link controller="user" action="generateQrCode" id="${userInstance.id}">Generate QR Code</g:link>
            </span>

This is bound to the generateQrCode action on the user controller, which will create a QR code from a user id and display it

    def generateQrCode(Long id){
        println "Generate QR code here..."
        String data = "$id"
        int qrSize = 500

        QRCodeRenderer qrcodeRenderer = new QRCodeRenderer()
        qrcodeRenderer.renderPng(data, qrSize, response.outputStream)
    }

As you can see, it is as simple as providing the data to be encoded, the size (x==y), and the output stream, in this case the response. When you click the link, you should see the following:

QR code generated by the qrcode plugin

QR code generated by the qrcode plugin

API

The website element is designed for the restaurant owners, the end users will be using an android app to complete surveys. Whilst I could have developed a mobile responsive page, I felt that an android app would bring a better overall experience to the user.

I have created a controller, ApiController that enables users to request surveys, and post responses.

Firstly, I created the URL mappings for this new controller

	static mappings = {
        "/api/customer/$customerid"(controller: "api", action: 'getCustomer')
        "/api/survey/$surveyid"(controller: "api", action: [GET: 'getSurvey'])
        "/api/survey"(controller: "api", action: [POST: 'surveyComplete'])

        "/$controller/$action?/$id?"{
            constraints {
                // apply constraints here
            }
        }

		"/"(controller: "home")
		"500"(view:'/error')
	}

Requests on /api/customer/$customerid, such as /api/customer/123 are routed to the getCustomer method on the api controller. The same is true for the second mapping, however the action is a GET on getSurvey (in hindsight, the first mapping should be restricted to the GET method too). The third mapping is a POST on /api/survey which will be invoked when the user has completed a survey on their device.

    def getCustomer(){
        User u = User.get(params.customerid)

        def surveysToPresent = [:]

        u.surveys.each {
            surveysToPresent << [title: it.title, id: it.id]
        }
        render(contentType: 'text/json') {[
                'company_name': u.companyName,
                'twitter' : u.twitterHandle,
                'facebook' : u.facebookPageLink,
                'surveys' : [surveysToPresent]
        ]} as JSON
    }

The getCustomer method finds the user from the customerid on the request path, retrieves the surveys and transforms them to a map containing the title and id (we don’t need the entire survey object when the user is presented with a list of surveys to select). The render statement enables us to return a json response very easily, we just return a map and grails (jackson) takes care of the json marshalling.

    def getSurvey(){
        Survey s = Survey.get(params.surveyid)

        def questionsToPresent = [:]

        questionsToPresent = s.questions.collect {
            [
                    id: it.id,
                    text: it.text,
                    responses : it.responses.collect{ resp ->
                        [id: resp.id, text: resp.text]
                    }
            ]
        }

        render(contentType: 'text/json') {[
                'id' : s.id,
                'title': s.title,
                'questions' : questionsToPresent
        ]} as JSON
    }

The getSurvey method behaves in a similar manner to getCustomer, it builds a map and renders as json.

    def surveyComplete(){
        def jsonObject = request.JSON

        Survey theSurvey = Survey.findById(jsonObject.id)

        jsonObject.responses.each{ response ->
            theSurvey.questions.find {it.id == response.question_id}.responses.find {it.id == response.response_id}.numberOfPeopleSelected++
        }
        theSurvey.save(flush: true, failOnError: true)

        render(status: 204)
    }

The surveyComplete will retrieve a survey by id, find the responses the user has provided, and increment a count. The survey is then saved and a “204 No Content” is returned.

I’ll cover how the android app consumes these services in my next post.

Deployment

As this project is just a prototype, I decided to host it on a free Cloudbees instance. The application doesn’t have any persistence layer, and all data is held in memory (which is fine for its current purpose), so when Cloudbees hibernates the instance after a period of inactivity, all user data will be lost. Deploying is simple, build the war using

grails war

Then upload the war file from the target directory to your cloud bees account, or use the command line cloud bees SDK.

View source code on Github

View live demo

(if the live demo link doesn’t work, try again in 10 minutes as the instance will be waking from hibernation)

Resources

Android; how to display a map the easy way..

I’m seeing countless questions, literally on a daily basis on StackOverflow regarding using maps on Android. To be honest I’ve never come across these problems but it seems many people have trouble using the maps in their application, so I’ll provide a clear and easy set of instructions on how to do this.

Common issues

  1. API key is incorrect
  2. You are using the standard Android emulator and not the Google APIs.
  3. You have extended Activity instead of MapActivity

The Tutorial…

The first thing you need, is an application to put your map in. For this tutorial I’m suggesting that you just create a blank android project, to get familiar with how maps work, then you can shift it into your existing application. Using IntelliJ (or Eclipse, if you must), create a new android project. I’ve called mine “MapMe”.

This will give you just one activity, such as the following :

package com.jameselsey.mapme;

import android.app.Activity;
import android.os.Bundle;

public class MapMe extends Activity
{
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
    }
}

I’d also suggest that you run this auto generated project to make sure your emulator starts up OK and you generally have a stable platform to build upon.

Next, we need to alter the emulator. The standard emulator that you get is the “android emulator”, which doesn’t allow you to use maps (at least not easily). This means that you need to create a slightly different emulator which does allow you to use maps, its quite easy.

Open up the android AVD manager (where you can create emulators / Android Virtual Devices). Click on the “available packages” option and download something called “Google APIs”, that should take a few minutes. When that is done, create a new AVD, be sure to check the “target” and make sure it is set to Google APIs. Once you’ve setup this emulator, make sure that your project is actually set to use it, by checking your run configurations and the target emulator.

Once thats done, I’d suggest you re-run the base application we’ve just created to make sure its all still OK.

OK so before we can finally jump into some code, you’ll need to go off and get an API key. This key allows you to use the Google maps API, I won’t explain how to do that here since theres already a rather good writeup from Google themselves, so go off and do that now.

Right lets hop into some code again, first thing you need to do is to add INTERNET permissions and uses-library into your AndroidManifest.xml file, mine looks as follows :

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      package="com.jameselsey.mapme"
      android:versionCode="1"
      android:versionName="1.0">
    <application android:label="MapMe" android:icon="@drawable/icon">
        <activity android:name="MapMe"
                  android:label="@string/app_name">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <uses-library android:name="com.google.android.maps" />
    </application>
    <uses-permission android:name="android.permission.INTERNET" />
</manifest> 

Next, go into main.xml under res/layout and add the following :

<com.google.android.maps.MapView
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/mapview"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:clickable="true"
    android:apiKey="YOUR_API_KEY_HERE"
/>

Now we can start doctoring our activity.

Firstly, add the following import

import com.google.android.maps.MapActivity;

And make sure that you implement the method you inherit from MapActivity :

@Override
    protected boolean isRouteDisplayed()
    {
        return false;
    }

That is pretty much it, all you need to do now is to run the application, you should have the following end result :

Complete Source

AndroidManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      package="com.jameselsey.mapme"
      android:versionCode="1"
      android:versionName="1.0">
    <application android:label="MapMe" android:icon="@drawable/icon">
        <activity android:name="MapMe"
                  android:label="@string/app_name">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <uses-library android:name="com.google.android.maps" />
    </application>
    <uses-permission android:name="android.permission.INTERNET" />
</manifest> 

res/layout/main.xml

<com.google.android.maps.MapView
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/mapview"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:clickable="true"
    android:apiKey="YOUR_API_KEY_HERE"
/>

MapMe Activity

package com.jameselsey.mapme;

import android.os.Bundle;
import com.google.android.maps.MapActivity;

public class MapMe extends MapActivity
{
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
    }

    @Override
    protected boolean isRouteDisplayed()
    {
        return false;
    }
}

Common Issues

Problems with RuntimeException : stub

java.lang.RuntimeException: stub
   at com.google.android.maps.MapView.<init> (Unknown Source)

You need to make sure that

<uses-library android:name="com.google.android.maps" />

is a child of the application tag.

Problems with importing of the com.google.* package

You may come across the following error :

java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation

If there are problems with that import (com.google.*), chances are you don’t have the maps.jar on your class path. Go back into your AVD manager, check available packages, under 3rd party libraries you should find some libraries from Google Inc., download those. If that still doesn’t solve your issue, then double check that your android “facet” is correctly using the Google APIs target.

Also, make sure that you don’t have 2 copies of the maps.jar on your class path, if you already have the target specified to Google APIs, then you don’t need to implicitly specify maps.jar on the classpath.

Maps are displaying, but all I see is grey squares…

This is most likely because you have the wrong API key, please go and re-generate that again to make sure its OK.

Further Reading

  1. Official Google Documentation
  2. MobiForge Tutorial

Tutorial; How to call Yahoo! REST web services the easy way!

I’ve got some requirements where I need to call a REST web service from an Android device, so I figured I’d get back to basics, and try to call a REST service through a simple Java application, just to see if I can get it working.

After trawling the web for what seemed like hours, I was finally able to come up with a simple yet effective solution. Consider the following

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

/**
 * This simple class demonstrates how to call a 
 * REST web service provided by Yahoo! Developer Network APIs
 * @author james.elsey
 * @date 05/October/2010
 */
public class RestTest {

	public static void main(String[] args) 
	{
		
		String yahooAppId = "sam.A2DV34EOgopFWeVeqLVnaoTVmEIVZnVtd58o5IRCsZ4kcwBTSp4gVl9E7hnnbRxDgRo-";
		String URL = "http://where.yahooapis.com/geocode?location=701+First+Ave,+Sunnyvale,+CA&appid=";

		String URLtoCall = URL + yahooAppId;
		
		try 
		{
			String responseString = httpGet(URLtoCall);
			
			// Do something with the returned response, currently just printing to console
			System.out.println(responseString);
		} 
		catch (IOException e) 
		{
			// Handle exception here, currently just printing stack trace
			e.printStackTrace();
		}

	}
	
	/**
	 * Call URL and buffer response into a String
	 * @param urlString URL to be called
	 * @return String The response XML String
	 * @throws IOException
	 */
	public static String httpGet(String urlString) throws IOException 
	{
		  URL url = new URL(urlString);
		  HttpURLConnection conn =
		      (HttpURLConnection) url.openConnection();

		  // Check for successful response code or throw error
		  if (conn.getResponseCode() != 200) 
		  {
		    throw new IOException(conn.getResponseMessage());
		  }

		  // Buffer the result into a string
		  BufferedReader buffrd = new BufferedReader(
		      new InputStreamReader(conn.getInputStream()));
		  StringBuilder sb = new StringBuilder();
		  String line;
		  while ((line = buffrd.readLine()) != null) 
		  {
		    sb.append(line);
		  }

		  buffrd.close();
		  conn.disconnect();
		  return sb.toString();
	}
	
}

This simple piece of code will create a URL to be called, call it, and print out the result which in this case is an XML String. This is a particualry basic way of consuming a REST service, and using 3rd party libraries such as Apache HTTP Commons may provide a cleaner and more effective solution

This will give you a XML String response, its up to you how you deal with the response, you can parse it using SAX for example. In my next tutorial I’ll look at parsing, and how we can intergrate this all with an Android application

I was quite impressed with the documentation and support provided by the Yahoo! Developer Network, I will certainly be using this more frequently as their web services are easier to use than Googles, which is an odd occasion