Getting to grips with Grails, building a survey management system

Sometime in late 2012 I was discussing dissertation project ideas with my girlfriend, as she was coming up to her final year of a computing bachelors. The usual option chosen by many graduates would be to just build a website or an app, or do some form of market research. We decided to encompass all 3 to produce something that works, but ultimately something that could be of value. If I had the time, energy, and funds I’d pursue this as it has potential for a startup, but I don’t, so the important thing that I’ve taken away is the experience working with groovy, grails, and android.

The Idea…

There are 2 main business drivers behind this project. Firstly we wanted to provide a service whereby restaurant owners can register, create surveys, and make them accessible to staff such as printing QR codes onto the back of their menus. Secondly, we wanted to approach this from the end users point of view, whereby customers sitting in the restaurant could download an app for free off the public market places, scan the said QR code, and be presented with the survey the restaurant owner had created. They would fill it in via the app and submit, the restaurant owner then has immediate access to the results in the form of statistics and graphs.

Landing page for Roosearch

Landing page for Roosearch

The outcomes that we’re after:

  • Better visibility to restaurant owners on how their customers feel
  • Easy and seamless access to surveys for the customers
  • A scalable application which can handle increasing users as demand grows
  • A platform for advertising new products and features

There are 3 components to this solution:

  1. Grails web application
  2. Rest API (built into the grails application)
  3. Android app

Why Grails?

  • Develop in groovy, so very accessible to java developers.
  • Quick to prototype with “convention over configuration”
  • Views auto generated if using scaffolding.
  • Easily deployable into the cloud, package as a war and deploy to cloudbees

The Prototype…

If you’re starting out with grails, I’d highly recommend that you get a copy of IntelliJ ultimate edition (and a copy of Grails in action), the support for grails is fantastic and I found it far easier than using eclipse. Whilst there are some excellent tutorials on grails out there (the official documentation is also very good) I’ll hold off and just jump right into how the application works.

One of the awesome features of grails is that it follows the “convention over configuration”, which simply means that if you follow the convention implied by the framework, you don’t have to be concerned about configuration. You can’t escape configuration entirely, but boilerplate plumbing can be inferred by convention. An example of that is if you name your controllers like “SurveyController”, grails automatically knows its a controller for the survey class, based on naming conventions. A similar convention applies for views.

Domain model

Roosearch entity relationship diagram

Roosearch entity relationship diagram

Our data model is quite simple. We have a user, the user has some surveys, each survey has a number of questions, and each questions has a number of predefined responses. The domain classes are self explanatory, but it’s probably worth mentioning a few tweaks I made.

class User {
    String firstName
    String lastName
    String emailAddress
    String companyName
    String facebookPageLink
    String twitterHandle

    static hasMany = [surveys: Survey]
    static constraints = {
        facebookPageLink nullable: true
        twitterHandle nullable: true
    }
}

By default, all fields are mandatory, however in the above example of the User class we can override these constraints to set them as nullable. There are various other constraints that you can set, have a look at the documentation.

class Survey {

    Integer id
    String title

    static hasMany = [questions: Question]

    static mapping = {
        questions lazy: false
    }

    static constraints = {
        id()
        title()
        questions()
    }

    String toString(){
        return title
    }
}

The relationships between the classes are defined by the “static hasMany”. This basically says that one Survey has relationships to many Questions, and this relationship is identified by “questions”.

The mapping block instructs the questions to be eagerly loaded, so once a survey is loaded into memory, so are all of its questions, opposed to just the Ids which would then be loaded lazily.

It’s also useful to override the toString method on your domain objects, particularly if you have relationships as the scaffolding will create drop down lists in your views. If you don’t override toString with something sensible, you’ll just see the object hash codes instead, which isn’t very useful to the user.

Controllers

It’s the responsibility of the controllers to manipulate the underlying data model (via services for example), and respond with views to the user. You can read more about the MVC pattern here.

To get started, you could simply enable scaffolding like so.

class LoginController {

    static scaffold = true

    def index = {
        render(view: "login.gsp")
    }
}

Scaffolding is an excellent feature of grails to get you started. Grails knows the structure of your domain object, therefore it is able to dynamically create controller CRUD operations, and views to manipulate your objects. That one small line of code and you can create, updated, delete, and view your objects! Fantastic eh?!

The bad news…

Whilst scaffolding is great to get you started, the moment you want to do something out of the ordinary, or customisation on views, scaffolding becomes a bit useless, and you’ll have to implement your own controllers (and possibly views). Fortunately, grails is quite flexible so you can leave scaffolding on and just override the methods that you want to customise. As with views, you’re best off getting grails to generate them for you and then customise them, to save you having to write the entire controller/view from scratch.

The methods you can override, and there general uses are:

  • index – default action, usually just redirects to list
  • list – list all of the objects, handle pagination, filtering etc
  • create – render view to create new object
  • save – handle creation of new object, validation etc.
  • edit – render view to edit an object
  • update – handle update of object
  • delete – delete an object

You can read more about the controller actions on the grails documentation.

You can see in the show() method on the SurveyController that I’ve customised it to to add some charts into the response model. You can see how I generate the chart data by looking at the source code in github. The view can then render these as javascript charts (which I’ll come onto in a moment)

    def show(Long id) {
        def surveyInstance = Survey.get(id)
        if (!surveyInstance) {
            flash.message = message(code: 'default.not.found.message', args: [message(code: 'survey.label', default: 'Survey'), id])
            redirect(action: "list")
            return
        }

        def charts = getCharts(surveyInstance)

        [surveyInstance: surveyInstance, charts: charts]
    }

Views

Being quite fond of the default views that grails generates, and not wanting to invest a great deal of time with customisation for this prototype, I chose to generate the views and then just tweak as I needed. In reality, the only customisation I needed to do was to place a “generate QR code” link, and to insert some javacript charts for displaying survey statistics.

Having assessed HighCharts, D3, and the Google visualisation API, I opted for the latter as I felt it was far simpler to use and I didn’t have any need for the advanced features that HighCharts and D3 come with, and there was a plugin for gvisualisation.

Displaying charts was straightforward, after installing the visualisation plugin, add this snippet of code to iterate over the charts that were added to the model and display a barCoreChart.

<g:each in="${charts.values()}" var="item">
            <gvisualization:barCoreChart
                    elementId="chart-${item.question_id}"
                    title="${item.question}"
                    width="${400}" height="${240}"
                    columns="${item.column}"
                    data="${item.data}"/>

            <div id="chart-${item.question_id}" align="center"></div>
        </g:each>

This would then display something like the following, you can change various elements of the charts such as the chart type, axis labels, sizes and titles, please refer to the documentation.

Charts using Google Visualisation

Charts using Google Visualisation

QR codes

QR codes make it incredibly easy to share data to android devices, my intention was to embed a user ID in a QR code, when scanned the app can request all surveys pertinent to the user ID.

Generating QR codes is easy with the qrcode plugin. I have provided a link on the users view to generate a QR code:

<span class="property-value" aria-labelledby="qr-label">
                <g:link controller="user" action="generateQrCode" id="${userInstance.id}">Generate QR Code</g:link>
            </span>

This is bound to the generateQrCode action on the user controller, which will create a QR code from a user id and display it

    def generateQrCode(Long id){
        println "Generate QR code here..."
        String data = "$id"
        int qrSize = 500

        QRCodeRenderer qrcodeRenderer = new QRCodeRenderer()
        qrcodeRenderer.renderPng(data, qrSize, response.outputStream)
    }

As you can see, it is as simple as providing the data to be encoded, the size (x==y), and the output stream, in this case the response. When you click the link, you should see the following:

QR code generated by the qrcode plugin

QR code generated by the qrcode plugin

API

The website element is designed for the restaurant owners, the end users will be using an android app to complete surveys. Whilst I could have developed a mobile responsive page, I felt that an android app would bring a better overall experience to the user.

I have created a controller, ApiController that enables users to request surveys, and post responses.

Firstly, I created the URL mappings for this new controller

	static mappings = {
        "/api/customer/$customerid"(controller: "api", action: 'getCustomer')
        "/api/survey/$surveyid"(controller: "api", action: [GET: 'getSurvey'])
        "/api/survey"(controller: "api", action: [POST: 'surveyComplete'])

        "/$controller/$action?/$id?"{
            constraints {
                // apply constraints here
            }
        }

		"/"(controller: "home")
		"500"(view:'/error')
	}

Requests on /api/customer/$customerid, such as /api/customer/123 are routed to the getCustomer method on the api controller. The same is true for the second mapping, however the action is a GET on getSurvey (in hindsight, the first mapping should be restricted to the GET method too). The third mapping is a POST on /api/survey which will be invoked when the user has completed a survey on their device.

    def getCustomer(){
        User u = User.get(params.customerid)

        def surveysToPresent = [:]

        u.surveys.each {
            surveysToPresent << [title: it.title, id: it.id]
        }
        render(contentType: 'text/json') {[
                'company_name': u.companyName,
                'twitter' : u.twitterHandle,
                'facebook' : u.facebookPageLink,
                'surveys' : [surveysToPresent]
        ]} as JSON
    }

The getCustomer method finds the user from the customerid on the request path, retrieves the surveys and transforms them to a map containing the title and id (we don’t need the entire survey object when the user is presented with a list of surveys to select). The render statement enables us to return a json response very easily, we just return a map and grails (jackson) takes care of the json marshalling.

    def getSurvey(){
        Survey s = Survey.get(params.surveyid)

        def questionsToPresent = [:]

        questionsToPresent = s.questions.collect {
            [
                    id: it.id,
                    text: it.text,
                    responses : it.responses.collect{ resp ->
                        [id: resp.id, text: resp.text]
                    }
            ]
        }

        render(contentType: 'text/json') {[
                'id' : s.id,
                'title': s.title,
                'questions' : questionsToPresent
        ]} as JSON
    }

The getSurvey method behaves in a similar manner to getCustomer, it builds a map and renders as json.

    def surveyComplete(){
        def jsonObject = request.JSON

        Survey theSurvey = Survey.findById(jsonObject.id)

        jsonObject.responses.each{ response ->
            theSurvey.questions.find {it.id == response.question_id}.responses.find {it.id == response.response_id}.numberOfPeopleSelected++
        }
        theSurvey.save(flush: true, failOnError: true)

        render(status: 204)
    }

The surveyComplete will retrieve a survey by id, find the responses the user has provided, and increment a count. The survey is then saved and a “204 No Content” is returned.

I’ll cover how the android app consumes these services in my next post.

Deployment

As this project is just a prototype, I decided to host it on a free Cloudbees instance. The application doesn’t have any persistence layer, and all data is held in memory (which is fine for its current purpose), so when Cloudbees hibernates the instance after a period of inactivity, all user data will be lost. Deploying is simple, build the war using

grails war

Then upload the war file from the target directory to your cloud bees account, or use the command line cloud bees SDK.

View source code on Github

View live demo

(if the live demo link doesn’t work, try again in 10 minutes as the instance will be waking from hibernation)

Resources

Why all Java devs should at least consider Groovy and Spock for testing

As a developer, testing is very important. Some developers have the mindset of “Meh, I write code, testing is a QAs job”, which is pretty poor. It’s much better for the developer to be test driving their code, generally if adopted well it produces better quality code, and of course the sooner issues are caught the cheaper they are to address.

Most Java developers who are following TDD probably use mockito or powermock alongside JUnit. I’ve never been much of a fan of those combinations as I believe they involve far too much boilerplate code, and test code often becomes more verbose and harder to maintain than the actual production code itself.

After being introduced to Spock, and testing using Groovy last year, I’m absolutely sold on it and have subsequently used it on several other projects.

For the purpose of this post, I’ll base it around a service class that does some things with a domain object, via a data access layer, which is something most enterprise developers can relate to.

Heres the domain class:

public class User {

    private int id;
    private String name;
    private int age;

    // Accessors omitted
}

Heres the DAO interface:

public interface UserDao {

    public User get(int id);

}

And finally the service:

public class UserService {

    private UserDao userDao;

    public UserService(UserDao userDao) {
        this.userDao = userDao;
    }

    public User findUser(int id){
        return null;
    }
}

Nothing too complex to mention here. The class that we’re going to put under test is the service. You can see that the service is dependant on a UserDao, which is passed into the constructor. This is a good design practice because you’re stating that in order to have a UserService, it must be constructed with a UserDao. This also becomes useful later when using dependency injection frameworks like Spring so you can mark them both as Components and Autowire the constructor arguments in, but alas.

Lets go ahead and create a test class for the service (command+shift+t if using IntelliJ on a mac).

class UserServiceTest extends Specification {

    UserService service
    UserDao dao = Mock(UserDao)

    def setup(){
        service = new UserService(dao)
    }

    def "it gets a user by id"(){
        given:
        def id = 1

        when:
        def result = service.findUser(id)

        then:
        1 * dao.get(id) >> new User(id:id, name:"James", age:27)
        result.id == 1
        result.name == "James"
        result.age == 27
    }
}

Here we go, right in at the deep end, let me explain what is going on here. Firstly, we’re using groovy, so although it looks like Java (I suppose it is in some respects as it compiles down to Java bytecode anyway) the syntax is a bit lighter, such as no semi-colons to terminate statements, no need for public accessor as everything is public by default, Strings for method names. If you want to learn more about groovy, check out their documentation here.

As you can see, the test class extends from spock.lang.Specification, this is a Spock base class and allows us to use the given, when and then blocks in our test.

You’ll see the subject of the test then, the service. I prefer to define this as a field and assign it in the setup, but others prefer to instantiate it in the given block of each test, I suppose this is really just a personal preference.

Creating mocks with Spock is easy, just use Mock(Class). I then pass the mocked DAO dependency into the userService in the setup method. Setup runs before each test is executed( likewise, cleanup() is run after each test completes). This is an excellent pattern for testing as you can mock out all dependencies and define their behaviour, so you’re literally just testing the service class.

A great feature of groovy is that you can use String literals to name your methods, this makes tests much easier to read and work out what it is actually testing rather than naming them as “public void testItGetsAUserById()”

Given, when, then

Spock is a behaviour driven development (BDD) testing framework, which is where it gets the given, when and then patterns from (amongst others). The easiest way I can explain it as follows:

Given some parts, when you do something, then you expect certain things to happen.

It’s probably easier to explain my test. We’re given an id of 1, you can think of this as a variable for the test. The when block is where the test starts, this is the invocation, we’re saying that when we call findUser() on the service passing in an id, we’ll get something back and assign it to the result.

The then block are your assertions, this is where you check the outcomes. The first line in the then block looks a little scary, but actually it’s very simple, lets dissect it.

1 * dao.get(id) >> new User(id:id, name:"James", age:27)

This line is setting an expectation on the mocked dao. We’re saying that we expect 1 (and only 1) invocation on the dao.get() method, that invocation must be passed id (which we defined as 1 earlier). Still with me? Good, we’re half way.

The double chevron “>>” is a spock feature, it means “then return”. So really this line reads as “we expect 1 hit on the mocked dao get(), and when we do, return a new User object”

You can also see that I’m using named parameters in the constructor of the User object, this is another neat little feature of groovy.

The rest of the then block are just assertions on the result object, not really required here as we’re doing a straight passthrough on the dao, but gives an insight as to what you’d normally want to do in more complex examples.

The implementation.

If you run the test, it’ll fail, as we haven’t implemented the service class, so lets go ahead and do that right now, its quite simple, just update the service class to the following:

public class UserService {

    private UserDao userDao;

    public UserService(UserDao userDao) {
        this.userDao = userDao;
    }

    public User findUser(int id){
        return userDao.get(id);
    }
}

Run the test again, it should pass this time.

Stepping it up

That was a reasonably simple example, lets look at creating some users. Add the following into the UserService:

public void createUser(User user){
        // check name

        // if exists, throw exception

        // if !exists, create user
    }

Then add these methods into the UserDao

public User findByName(String name);
    public void createUser(User user);

Then start with this test

def "it saves a new user"(){
        given:
        def user = new User(id: 1, name: 'James', age:27)

        when:
        service.createUser(user)

        then:
        1 * dao.findByName(user.name) >> null

        then:
        1 * dao.createUser(user)
    }

This time, we’re testing the createUser() method on the service, you’ll notice that there is nothing returned this time.

You may be asking “why are there 2 then blocks?”, if you group everything into a single then block, Spock just asserts that they all happen, it doesn’t care about ordering. If you want ordering on assertions then you need to split into separate then blocks, spock then asserts them in order. In our case, we want to firstly find by user name to see if it exists, THEN we want to create it.

Run the test, it should fail. Implement with the following and it’ll pass

    public void createUser(User user){
        User existing = userDao.findByName(user.getName());

        if(existing == null){
            userDao.createUser(user);
        }
    }

Thats great for scenarios where the user doesn’t already exist, but what if it does? Lets write so co…NO! Test first!

    def "it fails to create a user because one already exists with that name"(){
        given:
        def user = new User(id: 1, name: 'James', age:27)

        when:
        service.createUser(user)

        then:
        1 * dao.findByName(user.name) >> user

        then:
        0 * dao.createUser(user)

        then:
        def exception = thrown(RuntimeException)
        exception.message == "User with name ${user.name} already exists!"
    }

This time, when we call findByName, we want to return an existing user. Then we want 0 interactions with the createUser() mocked method.

The third then block grabs hold of the thrown exception by calling thrown() and asserts the message. Note that groovy has a neat feature called GStrings that allow you to put arguments inside quoted strings.

Run the test, it will fail. Implement with the following at it’ll pass.

public void createUser(User user){
        User existing = userDao.findByName(user.getName());

        if(existing == null){
            userDao.createUser(user);
        } else{
            throw new RuntimeException(String.format("User with name %s already exists!", user.getName()));
        }
    }

I’ll leave it there, that should give you a brief intro to Spock, there is far more that you can do with it, this is just a basic example.

Snippets of wisdom

  • Read the spock documentation!
  • You can name spock blocks such as given:”Some variables”, this is useful if its not entirely clear what your test is doing.
  • You can use _ * mock.method() when you don’t care how many times a mock is invoked.
  • You can use underscores to wildcard methods and classes in the then block, such as 0 * mock._ to indicate you expect no other calls on the mock, or 0 * _._ to indicate no calls on anything.
  • I often write the given, when and then blocks, but then I start from the when block and work outwards, sounds like an odd approach but I find it easier to work from the invocation then work out what I need (given) and then what happens(then).
  • The expect block is useful for testing simpler methods that don’t require asserting on mocks.
  • You can wildcard arguments in the then block if you don’t care what gets passed into mocks.
  • Embrace groovy closures! They can be you’re best friend in assertions!
  • You can override setupSpec and cleanupSpec if you want things to run only once for the entire spec.

Conclusion

Having used Spock (and groovy for testing) on various work and hobby projects I must admit I’ve become quite a fan. Test code is there to be an aid to the developer, not a hinderance. I find that groovy has many shortcuts (collections API to name but a few!) that make writing test code much nicer.

Check out the Github Gist here

Bootstrapping a Grails application to prepopulate data on startup

I’ve been dabbling in a little Grails recently, and I found it quite frustrating having to re-enter some sample data everytime I restarted my application, in order to have the views look meaningful.

Fortunately, theres an easy solution, just do all of your setup in the Bootstrapper class like so :

class BootStrap {

def init = { servletContext -&gt;

// Keep references on these as we'll use them for populating sample surveys
Response great = new Response(text: "Great").save()
Response average = new Response(text: "Average").save()
Response poor = new Response(text: "Poor").save()
Response tasty = new Response(text: "Tasty").save()
Response ok = new Response(text: "OK").save()
Response horrible = new Response(text: "Horrible").save()
Response friends = new Response(text: "Friends").save()
Response advert = new Response(text: "Advert").save()
Response radio = new Response(text: "Radio").save()

// These are some other responses that we'll make available in the response bank
["Excellent", "Good", "Satisfactory", "Bad", "Very bad", "true", "false", "Yes", "No", "Maybe"].each {
new Response(text: it).save()
}

Question q1 = new Question(text: "What did you think of the service?", responses: [great, average, poor]).save()
Question q2 = new Question(text: "Was the food nice?", responses: [tasty, ok, horrible]).save()
Question q3 = new Question(text: "How did you hear about our establishment?", responses: [friends, advert, radio]).save()

Survey cosmos = new Survey(title: "COSMOs customer feedback", questions: [q1, q2, q3]).save()
Survey jimmys = new Survey(title: "Jimmys Kitchen customer feedback", questions: [q1, q2, q3]).save()

new User(firstName: 'James',
lastName: 'Elsey',
emailAddress: 'james.elsey.dev@gmail.com',
companyName: 'Jimmys Kitchen',
surveys: [jimmys])
.save()

new User(firstName: 'Manabu',
lastName: 'Takano',
emailAddress: 'manny@hotmail.com',
companyName: 'COSMOs Cardiff',
surveys: [cosmos])
.save()

}
def destroy = {
}
}

Then, every time you start the application, you can be sure that it has the above data populated.

Probably not ideal for production apps, but very useful in development

Override the toString on your domain classes so they display as something useful

If you’re finding that your domain objects are not being displayed in a readable manner, chances are its because they haven’t been told to. This is often the case in drop down menus that the grails scaffolding creates. You can easily fix this by overriding the toString method on your domain class, such as the following.

class Question {

    String text

    static hasMany = [responses: Response]

    static constraints = {
        text()
        responses()
    }

    String toString(){
        return text
    }
}