Thursday, March 7, 2013

Building a Paginating Finder

My post Using Finders In Roo explored the mechanics of how to utilize a finder in a Roo generated web application.  The use case being implemented there was to allow users to search for other users in an application by filtering on a value.  The application uses a JPQL query to search for users.  That query used the like function on both the name and email address to produce a result list containing users where either the name or email contained the search value.

This filtering capability goes a long way towards assisting users in finding what they want, but there still is a gap in the functionality.  We should expect that our web application to be wildly successful and have millions of users.  Doesn't everyone's?  And with that volume of users we should also anticipate the query filter to return hundreds if not thousands of entities.  To manage this we need provide the ability to paginate the results of our filter.

The table tag that Roo generated for the application has pagination capability built in. But to use that functionality we will have to provide a count of records that match the filter and expose that value to the tag.  Also the pagination tag that Roo provides doesn't handle requests with parameters so this will need to be fixed too.  To do this we will need to;
  • Create a custom JPQL query to return the count of AppUsers that match the finders results.
  • Modify the custom find controller method we built in the last post to handle pagination.
  • Modify the pagination tag to handle request parameters correctly.


Create a Custom JPQL Query


The first step is to create the JPQL query.  Since we need this query to return a count of records that match the finder that was created in the first post (findAppUsersByNameLikeOrEmailLike) we will clone an modify that query.

Recall that Roo considers any file matching *_Roo_*.aj as being managed by the shell so the new query needs to be placed in another file or the entity definition file itself. I tend to take advantage of ITD's and the compartmentalization that they provide, so I created and AppUser_Finder.aj aspect file in the domain package.

Here is the filter count code;

package com.repik.multitenant.security.domain;

import javax.persistence.EntityManager;
import javax.persistence.TypedQuery;

privileged aspect AppUser_Finder {
    
 public static long AppUser.countAppUsersByNameLikeOrEmailLike(String name, String email) {

 if (name == null || name.length() == 0) 
  throw new IllegalArgumentException("The name argument is required");
 name = name.replace('*', '%');
 if (name.charAt(0) != '%') {
  name = "%" + name;
 }
 if (name.charAt(name.length() - 1) != '%') {
  name = name + "%";
 }

 if (email == null || email.length() == 0) 
  throw new IllegalArgumentException("The email argument is required");
 email = email.replace('*', '%');
 if (email.charAt(0) != '%') {
  email = "%" + email;
 }
 if (email.charAt(email.length() - 1) != '%') {
  email = email + "%";
 }

 EntityManager em = AppUser.entityManager();
 TypedQuery<long> q = em.createQuery("SELECT count(o) FROM AppUser AS o WHERE LOWER(o.name) LIKE LOWER(:name)  OR LOWER(o.email) LIKE LOWER(:email)", Long.class);
 q.setParameter("name", name);
 q.setParameter("email", email);
 return q.getSingleResult();
 }
}

Most of the countAppUsersByNameLikeOrEmailLike method is similar to the findAppUsersByNameLikeOrEmailLike that we cloned it from.  The major difference occurs in line 31 where the count function has been applied to the select result.  Now the query returns a count, so we need to modify the TypedQuery type to long.  Since this query returns a single result, the method should just return that rather then the TypedQuery.  So on line 34 the return statement is modified to return the single result of that query.  Doing this modification means that we also have to change the return type for the method on line 8.

Modify the Controller find Method


Now we need to modify the controller find method to handle the pagination.  There are two pagination parameters that can optionally be passed into the controller method; page and size.  The page parameter represents the page number being requested.  The size parameter represents the number of entities displayed on a page.  Our controller method will use these parameters along the finder method to fetch the AppUsers for display on the page requested.  Additionally, we will need to determine the maximum number of pages that the filter matches so the pagination logic found in the JSP can render correctly.

Here is the modified controller;

package com.repik.multitenant.security.web;

import com.repik.multitenant.security.domain.AppUser;
import org.springframework.roo.addon.web.mvc.controller.scaffold.RooWebScaffold;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;

@RequestMapping("/appusers")
@Controller
@RooWebScaffold(path = "appusers", formBackingObject = AppUser.class)
public class AppUserController {

 @RequestMapping(value="/find", produces = "text/html")
 public String find(
   @RequestParam(value = "find", required = true) String finder, 
   @RequestParam(value = "filter", required = true) String filter,
   @RequestParam(value = "page", required = false) Integer page, 
   @RequestParam(value = "size", required = false) Integer size, 
   Model uiModel) {
  if ( "findAppUsersByNameLikeOrEmailLike".equals( finder ) && ! filter.isEmpty() ) {
   int sizeNo = size == null ? 10 : size.intValue();
   uiModel.addAttribute( "size", sizeNo ) ;
   final int firstResult = page == null ? 0 : (page.intValue() - 1) * sizeNo;
   uiModel.addAttribute("appusers", AppUser.findAppUsersByNameLikeOrEmailLike(filter, filter).setFirstResult( firstResult ).setMaxResults( sizeNo ).getResultList());
   float nrOfPages = (float) AppUser.countAppUsersByNameLikeOrEmailLike( filter, filter ) / sizeNo;
   uiModel.addAttribute("maxPages", (int) ((nrOfPages > (int) nrOfPages || nrOfPages == 0.0) ? nrOfPages + 1 : nrOfPages));
   uiModel.addAttribute("filter", filter) ;
   return "appusers/list";
  }
  else
   return "redirect:/appusers" ;
 }
}

The following changes were made to the finder method from the earlier post;
  • line 19-20 additional optional request parameters were added to handle the page number and count of elements (size) on the page.
  • lines 23-26 for both the page and size attributes assign default values if missing and then add those attributes to the uiModel.
  • line 27 calculate the first record offset for the page being displayed.
  • line 28 find and add to the uiModel the AppUser records using the filter, offset and size attributes.
  • line 29 use the JPQL query from above to get a count of AppUser's that match the filter, use that to calculate the max page number.
  • line 30 add the max page number to the uiModel.


Modify the Pagination Tag to Handle Request Parameters


Within the functionality of the table tag (tags/forms/table.tagx) is the logic to embed the pagination tag (tags/util/pagination.tagx) if maxPages has been set.  In the controller code we set this attribute in the uiModel so pagination is enabled.

The problem now is that the URL's being generated are incorrect.  Specifically we have a couple of request parameters, find and filter, that are not being propagated into the URL's.  So we need to modify the pagination tag to handle this.

Adding any request parameters to URL's generated by the pagination tag requires that when building a URL we iterate though the request parameters adding those to the URL.  The exception being that if the parameter is either 'page' or 'size' we don't want to propagate those since those are the ones that are particular to the URL.  The following snippet of JSP will do this;

<c:forEach var="paramName" items="${pageContext.request.parameterNames}">
 <c:if test="${paramName ne 'page' and paramName ne 'size' }">
  <spring:param name="${paramName}" value="${param[ paramName ]}"/>
 </c:if>
</c:forEach>

Here is the complete modified pagination.tagx code;

<jsp:root xmlns:c="http://java.sun.com/jsp/jstl/core" xmlns:fn="http://java.sun.com/jsp/jstl/functions" xmlns:spring="http://www.springframework.org/tags" xmlns:jsp="http://java.sun.com/JSP/Page" version="2.0">
  <jsp:output omit-xml-declaration="yes" />

  <jsp:directive.attribute name="maxPages" type="java.lang.Integer" required="true" rtexprvalue="true" description="The maximum number of pages available (ie tableRecordCount / size)" />
  <jsp:directive.attribute name="page" type="java.lang.Integer" required="false" rtexprvalue="true" description="The current page (not required, defaults to 1)" />
  <jsp:directive.attribute name="size" type="java.lang.Integer" required="false" rtexprvalue="true" description="The number of records per page (not required, defaults to 10)" />
  <jsp:directive.attribute name="render" type="java.lang.Boolean" required="false" rtexprvalue="true" description="Indicate if the contents of this tag and all enclosed tags should be rendered (default 'true')" />

  <c:if test="${empty render or render}">

    <c:if test="${empty page || page lt 1}">
      <c:set var="page" value="1" />
    </c:if>

    <c:if test="${empty size || size lt 1}">
      <c:set var="size" value="10" />
    </c:if>

    <spring:message code="list_size" var="list_size" htmlEscape="false" />
    <c:out value="${list_size} " />

    <c:forEach var="i" begin="5" end="25" step="5">
      <c:choose>
        <c:when test="${size == i}">
          <c:out value="${i}" />
        </c:when>
        <c:otherwise>
          <spring:url value="" var="sizeUrl">
           <c:forEach var="paramName" items="${pageContext.request.parameterNames}">
            <c:if test="${paramName ne 'page' and paramName ne 'size' }">
             <spring:param name="${paramName}" value="${param[ paramName ]}"/>
            </c:if>
           </c:forEach>
            <spring:param name="page" value="1" />
            <spring:param name="size" value="${i}" />
          </spring:url>
          <a href="${sizeUrl}">${i}</a>
        </c:otherwise>
      </c:choose>
      <c:out value=" " />
    </c:forEach>
    <c:out value="| " />

    <c:if test="${page ne 1}">
      <spring:url value="" var="first">
        <c:forEach var="paramName" items="${pageContext.request.parameterNames}">
         <c:if test="${paramName ne 'page' and paramName ne 'size' }">
          <spring:param name="${paramName}" value="${param[ paramName ]}"/>
         </c:if>
        </c:forEach>
        <spring:param name="page" value="1" />
        <spring:param name="size" value="${size}" />
      </spring:url>
      <spring:url value="/resources/images/resultset_first.png" var="first_image_url" />
      <spring:message code="list_first" var="first_label" htmlEscape="false" />
      <a class="image" href="${first}" title="${fn:escapeXml(first_label)}">
        <img alt="${fn:escapeXml(first_label)}" src="${first_image_url}" />
      </a>
    </c:if>
    <c:if test="${page gt 1}">
      <spring:url value="" var="previous">
        <c:forEach var="paramName" items="${pageContext.request.parameterNames}">
         <c:if test="${paramName ne 'page' and paramName ne 'size' }">
          <spring:param name="${paramName}" value="${param[ paramName ]}"/>
         </c:if>
        </c:forEach>
        <spring:param name="page" value="${page - 1}" />
        <spring:param name="size" value="${size}" />
      </spring:url>
      <spring:url value="/resources/images/resultset_previous.png" var="previous_image_url" />
      <spring:message code="list_previous" var="previous_label" htmlEscape="false" />
      <a class="image" href="${previous}" title="${fn:escapeXml(previous_label)}">
        <img alt="${fn:escapeXml(previous_label)}" src="${previous_image_url}" />
      </a>
    </c:if>
    <c:out value=" " />
    <spring:message code="list_page" arguments="${page},${maxPages}" argumentSeparator="," />
    <c:out value=" " />
    <c:if test="${page lt maxPages}">
      <spring:url value="" var="next">
        <c:forEach var="paramName" items="${pageContext.request.parameterNames}">
         <c:if test="${paramName ne 'page' and paramName ne 'size' }">
          <spring:param name="${paramName}" value="${param[ paramName ]}"/>
         </c:if>
        </c:forEach>
        <spring:param name="page" value="${page + 1}" />
        <spring:param name="size" value="${size}" />
      </spring:url>
      <spring:url value="/resources/images/resultset_next.png" var="next_image_url" />
      <spring:message code="list_next" var="next_label" htmlEscape="false" />
      <a class="image" href="${next}" title="${fn:escapeXml(next_label)}">
        <img alt="${fn:escapeXml(next_label)}" src="${next_image_url}" />
      </a>
    </c:if>
    <c:if test="${page ne maxPages}">
      <spring:url value="" var="last">
        <c:forEach var="paramName" items="${pageContext.request.parameterNames}">
         <c:if test="${paramName ne 'page' and paramName ne 'size' }">
          <spring:param name="${paramName}" value="${param[ paramName ]}"/>
         </c:if>
        </c:forEach>
        <spring:param name="page" value="${maxPages}" />
        <spring:param name="size" value="${size}" />
      </spring:url>
      <spring:url value="/resources/images/resultset_last.png" var="last_image_url" />
      <spring:message code="list_last" var="last_label" htmlEscape="false" />
      <a class="image" href="${last}" title="${fn:escapeXml(last_label)}">
        <img alt="${fn:escapeXml(last_label)}" src="${last_image_url}" />
      </a>
    </c:if>
  </c:if>
</jsp:root>

Where the modifications are that we injected the request parameter handling snippet in at lines; 29, 46, 62, 81, and 97.

All the implementation pieces are in place now and we can support pagination of a filter.

Using Finders In Roo

The Spring Roo shell provides commands to create finder methods for entity classes.  Where finders are Roo maintained static methods wrapping JPQL queries to retrieve subsets of application entities.  Additionally as part of Spring Roo's web tier JSP implementation a filter tag is generated.  However neither the tags usage or how to use this tag with finders is in the documentation.

The out of the box implementation generated by Spring Roo provides basic listing functionality.  However often times in real life applications lists of some entities can be rather lengthy making it difficult for users to find particular entities without having to traverse multiple pages.  Applying a finder method allows the user to narrow the scope of entities being shown and subsequently finding the entity(s) of interest easier to find.

So lets first get basic filtering up and running in a web application. Setting up an using a finder involves multiple steps;
  • First we will need to create the entity finder using the Roo shell.
  • Then a new filter method to access the finder will need to be added to the controller.
  • Modify the list JSP to render the filter.
The example here will focus on the entity named AppUser.  This entity represents users in an application.  Our filter is going to address the use case where a user wants to search for other users.  The AppUser entity has two fields, name and email address, that we are going to filter on.  The user will be able to enter in a single string and the application will display users where either the name the name or email address contains that string.

First we will create the finder using the Roo shell with the command:

finder add findAppUsersByNameLikeOrEmailLike


This results in Roo generating the following code;

    public static TypedQuery AppUser.findAppUsersByNameLikeOrEmailLike(String name, String email) {
        if (name == null || name.length() == 0) throw new IllegalArgumentException("The name argument is required");
        name = name.replace('*', '%');
        if (name.charAt(0) != '%') {
            name = "%" + name;
        }
        if (name.charAt(name.length() - 1) != '%') {
            name = name + "%";
        }
        if (email == null || email.length() == 0) throw new IllegalArgumentException("The email argument is required");
        email = email.replace('*', '%');
        if (email.charAt(0) != '%') {
            email = "%" + email;
        }
        if (email.charAt(email.length() - 1) != '%') {
            email = email + "%";
        }
        EntityManager em = AppUser.entityManager();
        TypedQuery q = em.createQuery("SELECT o FROM AppUser AS o WHERE LOWER(o.name) LIKE LOWER(:name)  OR LOWER(o.email) LIKE LOWER(:email)", AppUser.class);
        q.setParameter("name", name);
        q.setParameter("email", email);
        return q;
    }

Note that Roo has done all the work of preparing the like parameters for us, substituting the '*' with '%' and placing a '%' on each end of the parameter.  Thus when a parameter such as, 'foo*bar' is supplied to the finder method the parameter provided to the query will be '%foo%bar%'.

In the JSP we will be using the find tag (tags/form/find.tagx) this tag embeds a form containing the filter parameters on that page.  Within the form is a hidden field named 'finder' that contains a symbolic name of the finder we want to use.   The Roo generated web application does not contain any special logic to auto-magically interpret the finder name and invoke that finder without having to provide hand written logic.  That logic is what we will have to provide in the controller method.  The finder name allows a controller method to support access to multiple finders.  We will not be using that functionality here but will still include the logic to test the name as a defense against wayward JSP errors else where in the application.

Finally, note that the return type is a JPA TypedQuery object.  Compare this to the findAllAppUsers method in the AppUser_Roo_Jpa_ActiveRecord.aj file, that returns a list of AppUsers.  The list JSP (/tags/form/list.tagx) expects a list of entities, so the controller method will have to invoke getResultList on the results of the entity finder and place that list in the uiModel.

Now that the finder is available we can now code the controller method.

package com.repik.multitenant.security.web;

import com.repik.multitenant.security.domain.AppUser;
import org.springframework.roo.addon.web.mvc.controller.scaffold.RooWebScaffold;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;

@RequestMapping("/appusers")
@Controller
@RooWebScaffold(path = "appusers", formBackingObject = AppUser.class)
public class AppUserController {

   @RequestMapping(value="/find", produces = "text/html")
    public String find(
         @RequestParam(value = "find", required = true) String finder,
         @RequestParam(value = "filter", required = true) String filter,  Model uiModel) {

      // test that the finder parameter is correct
      if ( "findAppUsersByNameLikeOrEmailLike".equals( finder ) ) {

         // notice that getResultList() is invoked on the finder result
         uiModel.addAttribute("appusers", AppUser.findAppUsersByNameLikeOrEmailLike(filter, filter).getResultList()) ;

         uiModel.addAttribute("filter", filter) ;
         return "appusers/list";
      }
      else
         return "redirect:/appusers" ;
    }
}

Finally we need to add the find tag to the AppUser list JSP (views/appuser/list.jspx) to generate the HTML that invokes the controller method that we just created.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<div xmlns:jsp="http://java.sun.com/JSP/Page" 
  xmlns:page="urn:jsptagdir:/WEB-INF/tags/form" 
  xmlns:table="urn:jsptagdir:/WEB-INF/tags/form/fields" version="2.0">
 <jsp:directive.page contentType="text/html;charset=UTF-8"/>
 <jsp:output omit-xml-declaration="yes"/>
    
 <page:find finderName="findAppUsersByNameLikeOrEmailLike" id="ff_com_repik_multitenant_security_domain_appuser" path="/appusers/find">
  <input data-dojo-props="trim:true" data-dojo-type="dijit/form/TextBox" id="toolbarFilter" name="filter" type="text" value="${filter}"/>
 </page:find>
 
 <page:list id="pl_com_repik_multitenant_security_domain_AppUser" items="${appusers}" z="HqueBBVm/RaI2SUiUNBFWmNCOZ0=">
  <table:table create="false" data="${appusers}" delete="false" id="l_com_repik_multitenant_security_domain_AppUser" path="/appusers" update="false" multiselect="true" z="user-managed">
   <table:column id="c_com_repik_multitenant_security_domain_AppUser_name" property="name" z="hitrYtAgDxnq3qbzqZWR5cdLT4o="/>
   <table:column id="c_com_repik_multitenant_security_domain_AppUser_email" property="email" z="gzkRNxtd6O1mO8woYZje/UYflcg="/>
  </table:table>
 </page:list>
</div>


The JSP we added was the page:find tag.  The finderName attribute value must be 'findAppUsersByNameLikeOrEmailLike' that is the value being tested in the controller method.  Within the tag itself is a input text element that the user can enter the filter text into.  Notice that a data-dojo-props attribute has been added with the value of "trim:true" and the element has been identified as a text box to Dojo using the data-dojo-type attribute.  This allows Dojo to trim the filter text for us.

 <page:find finderName="findAppUsersByNameLikeOrEmailLike" id="ff_com_repik_multitenant_security_domain_AppUser_finder" path="/appusers/find">
  <input data-dojo-props="trim:true" data-dojo-type="dijit/form/TextBox" id="toolbarFilter" name="filter" type="text" value="${filter}"/>
 </page:find>

We now completed using a finder in Spring Roo.

Sunday, March 3, 2013

Upgrading the Version Of Dojo Used By Roo

As of version 1.2.1 of Spring Roo the version of Dojo that comes packaged in the Javascript resources JAR (spring-js-resources-2.2.1-RELEASE.jar) is 1.5.  The current production version of Dojo is 1.8 with the earliest version being supported by that team is 1.6., while version 1.5 has been sunset.  

The Dojo toolkit beginning with version 1.7 adopted the use of the CommonJS Asynchronous Module Definition framework work for loading the various modules resulting in a significant rewrite of the library.  With that and other changes in architecture the Dojo the current version, 1.8, is significantly different then that of version 1.5.  While the folks involved in the Dojo project have maintained backwards compatibility to support version 1.0 architecture.  They have also made it clear that beginning with version 2.0 that support will be removed.  

So any project that involves significant modifications to the web tier should probably upgrade the version of Dojo.  Since any development that involves Dojo should be written against the new API.

Normally to upgrade Dojo one would;
  • Download the latest version of Dojo from http://dojotoolkit.org/download/. 
  • Unpack the contents of the distribution into a folder under the resources folder. 
  • Modify the mvc:resources definition in the webmvc-config to use the new distribution.
For example in the folder /src/main/resources/META-INF the dojo-release.1.8.3 distribution was added.  So the new Dojo release can be found at /META-INF/dojo-release-1.8.3 on the applications classpath.
Once the new implementation is available to the application we need to modify the webmvc-config.xml file, adding the folder containing the newly installed Dojo installation.  It is essential that the entry for the new Dojo installation is before the classpath:/META-INF/web-resources/ entry.  So the mvc:resources definition should now look something like;

<mvc:resources location="/, 
 classpath:/META-INF/dojo-release-1.8.3/, 
 classpath:/META-INF/web-resources/" 
 mapping="/resources/**">


This is when troubles in paradise begin.  The problem with this method is that the Dojo implementation is in a folder that Spring Roo is monitoring for changes.  With the Dojo implementation being rather sizable, having over eight thousand files, monitoring all these files really slows down Roo’s shell responsiveness to a crawl.  Basically rendering all the background behind the scenes magic that Roo provides non-functional.

Instead a better strategy is to package up the implementation Dojo into a JAR and then include that JAR as a dependency to the Roo project.  Now the Dojo implementation files are outside of the project folders being monitored by Roo, but available the class path.  To  upgrade Dojo using this method we need to;
  • Create a Maven project that builds a JAR file.
  • Create a META-INF folder in the project.
  • Copy the Dojo implementation into a META-INF folder.
  • Run the Maven project to package and install the resulting JAR.
  • Add a dependency to the newly created JAR to the Roo project.
  • Modify the mvc:resources definition in the webmvc-config to use the new distribution.
The maven project to build the JAR file is quite simple.  First a simple maven project is created to setup the basic maven project file structure.  
<project xmlns="http://maven.apache.org/POM/4.0.0" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>com.repik.roo</groupId>
 <artifactId>dojo-js</artifactId>
 <version>1.8.3</version>
</project>
Then in the src/main/resources a META-INF folder is created and the new Dojo installation is copied into that folder.
Running Maven on the project with the goal of install will build the JAR and place it in the local repository.   Next the dependency to the new created JAR is added to the Roo web application pom.xml;
<dependency>
 <groupId>com.repik.roo</groupId>
 <artifactId>dojo-js</artifactId>
 <version>1.8.3</version>
</dependency>

And finally to make the new implementation is available to the application we need to modify the webmvc-config.xml file as we did with the earlier scheme.  Adding the classpath folder containing the newly installed Dojo installation.  It is essential that the entry for the new Dojo installation is before the classpath:/META-INF/web-resources/ entry.  So the mvc:resources definition should now look like;
<mvc:resources location="/, 
 classpath:/META-INF/dojo-release-1.8.3/, 
 classpath:/META-INF/web-resources/" 
 mapping="/resources/**">

Now the installation is complete and the new version of Dojo is available for development in the Roo web application and the Roo shell is once again functional.