Spring Batch Tutorial (Part 1)

In this tutorial, we will create a simple Spring Batch application to demonstrate how to process a series of jobs where the primary purpose is to import a lists of comma-delimited and fixed-length records. In addition, we will add a web interface using Spring MVC to teach how to trigger jobs manually, and so that we can visually inspect the imported records. In the data layer, we will use JPA, Hibernate, and MySQL.

Table of Contents

Part 1: Introduction and Functional Specs
Part 2: Java classes
Part 3: XML configuration
Part 4: Running the Application

Dependencies

• Spring core 3.1.0.RELEASE
• Spring Batch 2.1.8.RELEASE
• See pom.xml for details

Github
To access the source code, please visit the project's Github repository (click here)

Functional Specs

Before we start, let's define the application's specs as follows:

• Import a list of comma-delimited records
• Import a list of fixed-length records
• Import a list of mixed-type records
• Jobs must be triggered using a web interface
• Display the imported records in a web interface
• Each record represents a user and its associated access levels

Here's our Use Case diagram:

[User]-(Import job1)
[User]-(Import job2) 
[User]-(Import job3) 
[User]-(View records)

The CSV Files

To visualize what we want to do, let's examine first the files that we plan to import:

User Files

user1.csv
This file contains comma-separated value (CSV) records representing User records. Each line has the following tokens: username, first name, last name, password.

user2.csv
This file contains fixed-length records representing User records. Each line has the following tokens: username(positions 1-5), first name(6-9), last name(10-16), password(17-25).

user2.csv
This file contains comma-separated value and fixed-length records representing User records. Each line has the following tokens: username, first name, last name, password.

This file contains two types of CSV-records:

• DELIMITED-RECORD-A: uses the standard comma delimiter
• DELIMITED-RECORD-B: uses | delimiter

It also contains two types of fixed-length records:

• FIXED-RECORD-A: username(16-20), first name(21-25), last name(26-31), password(32-40)
• FIXED-RECORD-B: username(16-21), first name(22-27), last name(28-33), password(35-42)

Role Files

role1.csv
This file contains comma-separated value (CSV) records representing Role records. Each line has the following tokens: username and access level.

role2.csv
This file contains fixed-length records representing Role records. Each line has the following tokens: username and access level.

role3.csv
This file contains comma-separated value (CSV) records representing Role records. Each line has the following tokens: username and access level.

By now you should have a basic idea of the file formats that we will be importing. You must realize that all we want to do is import these files and display them on a web interface.

Diagrams

Here's the Class diagram:

# Cool UML Diagram
[User|id;firstName;lastName;username;password;role{bg:orange}]1--1> [Role|id;role{bg:green}]

Here's the Activity Diagram:

(start)->import->success->(Show Success Alert)->|a|->(end),
fail->(Show Fail Alert)->|a|,
view->(Show Records)->|a|->(end)

Screenshots

Let's preview how the application will look like after it's finished. This is also a good way to clarify further the application's specs.

Entry page
The entry page is the primary page that users will see. It contains a table showing user records and four buttons for adding, editing, deleting, and reloading data. All interactions will happen in this page.

Entry page

Next

In the next section, we will write the Java classes. Click here to proceed.

---

Share the joy: blogger tutorials Social Bookmarking Blogger Widget I'm reading: Spring Batch Tutorial (Part 1) ~

Subscribe by reader Subscribe by email Share

Spring Batch Tutorial (Part 4)

Review

We have just completed our application! In the previous sections, we have discussed how to perform batch processing with Spring Batch. We have also created a Spring MVC application to act as a web interface. In this section, we will build and run the application using Maven, and demonstrate how to import the project in Eclipse.

Table of Contents

Part 1: Introduction and Functional Specs
Part 2: Java classes
Part 3: XML configuration
Part 4: Running the Application

Running the Application

Access the source code

To download the source code, please visit the project's Github repository (click here)

Preparing the data source

1. Run MySQL (install one if you don't have one yet)
2. Create a new database:
   

   spring_batch_tutorial

3. Import the following file which is included in the source code under the src/main/resources folder:
   

   schema-mysql.sql

   This script contains Spring Batch infrastructure tables which can be found in the Spring Batch core library. I have copied it here separately for easy access.
   

Building with Maven

1. Ensure Maven is installed
2. Open a command window (Windows) or a terminal (Linux/Mac)
3. Run the following command:
   

   mvn tomcat:run

4. You should see the following output:
   

   [INFO] Scanning for projects...
   [INFO] Searching repository for plugin with prefix: 'tomcat'.
   [INFO] artifact org.codehaus.mojo:tomcat-maven-plugin: checking for updates from central
   [INFO] artifact org.codehaus.mojo:tomcat-maven-plugin: checking for updates from snapshots
   [INFO] ------------------------------------------
   [INFO] Building spring-batch-tutorial Maven Webapp
   [INFO]    task-segment: [tomcat:run]
   [INFO] ------------------------------------------
   [INFO] Preparing tomcat:run
   [INFO] [apt:process {execution: default}]
   [INFO] [resources:resources {execution: default-resources}]
   [INFO] [tomcat:run {execution: default-cli}]
   [INFO] Running war on http://localhost:8080/spring-batch-tutorial
   Feb 13, 2012 9:36:54 PM org.apache.catalina.startup.Embedded start
   INFO: Starting tomcat server
   Feb 13, 2012 9:36:55 PM org.apache.catalina.core.StandardEngine start
   INFO: Starting Servlet Engine: Apache Tomcat/6.0.29
   Feb 13, 2012 9:36:55 PM org.apache.catalina.core.ApplicationContext log
   INFO: Initializing Spring root WebApplicationContext
   Feb 13, 2012 9:37:01 PM org.apache.coyote.http11.Http11Protocol init
   INFO: Initializing Coyote HTTP/1.1 on http-8080
   Feb 13, 2012 9:37:01 PM org.apache.coyote.http11.Http11Protocol start
   INFO: Starting Coyote HTTP/1.1 on http-8080
   

Note: If the project will not build due to missing repositories, please enable the repositories section in the pom.xml!

Access the Entry page

1. Follow the steps with Building with Maven
2. Open a browser
3. Enter the following URL (8080 is the default port for Tomcat):
   

   http://localhost:8080/spring-batch-tutorial/

Import the project in Eclipse

1. Ensure Maven is installed
2. Open a command window (Windows) or a terminal (Linux/Mac)
3. Run the following command:
   

   mvn eclipse:eclipse -Dwtpversion=2.0

4. You should see the following output:
   

   [INFO] Scanning for projects...
   [INFO] Searching repository for plugin with prefix: 'eclipse'.
   [INFO] org.apache.maven.plugins: checking for updates from central
   [INFO] org.apache.maven.plugins: checking for updates from snapshots
   [INFO] org.codehaus.mojo: checking for updates from central
   [INFO] org.codehaus.mojo: checking for updates from snapshots
   [INFO] artifact org.apache.maven.plugins:maven-eclipse-plugin: checking for updates from central
   [INFO] artifact org.apache.maven.plugins:maven-eclipse-plugin: checking for updates from snapshots
   [INFO] -----------------------------------------
   [INFO] Building spring-batch-tutorial Maven Webapp
   [INFO]    task-segment: [eclipse:eclipse]
   [INFO] -----------------------------------------
   [INFO] Preparing eclipse:eclipse
   [INFO] No goals needed for project - skipping
   [INFO] [eclipse:eclipse {execution: default-cli}]
   [INFO] Adding support for WTP version 2.0.
   [INFO] -----------------------------------------
   [INFO] BUILD SUCCESSFUL
   [INFO] -----------------------------------------
   

   This command will add the following files to your project:
   

   .classpath
   .project
   .settings
   target

   You may have to enable "show hidden files" in your file explorer to view them
   
5. Open Eclipse and import the project

Conclusion

That's it! We've have successfully completed our Spring Batch application and learned how to process of jobs in batches. We've also added Spring MVC support to allow jobs to be controlled online.

I hope you've enjoyed this tutorial. Don't forget to check my other tutorials at the Tutorials section.

Revision History

---

Revision	Date	Description
1	Feb 16 2012	Uploaded tutorial and Github repository

---

Share the joy: blogger tutorials Social Bookmarking Blogger Widget I'm reading: Spring Batch Tutorial (Part 4) ~

Subscribe by reader Subscribe by email Share

Spring Batch Tutorial (Part 2)

Review

In the previous section, we have laid down the functional specs of the application and examined the raw files that are to be imported. In this section, we will discuss the project's structure and write the Java classes.

Table of Contents

Part 1: Introduction and Functional Specs
Part 2: Java classes
Part 3: XML configuration
Part 4: Running the Application

Project Structure

Our application is a Maven project and therefore follows Maven structure. As we create the classes, we've organized them in logical layers: domain, repository, service, and controller.

Here's a preview of our project's structure:

The Layers

Disclaimer

I will only discuss the Spring Batch-related classes here. And I've purposely left out the unrelated classes because I have described them in detail already from my previous tutorials. See the following guides:

• Spring MVC 3.1 - Implement CRUD with Spring Data Redis (Part 3)
• Spring MVC 3.1 - Implement CRUD with Spring Data MongoDB (Part 3)
• Spring MVC 3.1, jqGrid, and Spring Data JPA Integration Guide (Part 2)

Controller Layer

The BatchJobController handles batch requests. There are three job mappings:

• /job1
• /job2
• /job3

Everytime a job is run, a new JobParameter is initialized as the job's parameter. We use the current date to be the distinguishing parameter. This means every job trigger is considered a new job.

What is a JobParameter?

"how is one JobInstance distinguished from another?" The answer is: JobParameters. JobParameters is a set of parameters used to start a batch job. They can be used for identification or even as reference data during the run:

Source: Spring Batch - Chapter 3. The Domain Language of Batch

Notice we have injected a JobLauncher. It's primary job is to start our jobs. Each job will run asynchronously (this is declared in the XML configuration).

What is a JobLauncher?

JobLauncher represents a simple interface for launching a Job with a given set of JobParameters:

Source: Spring Batch - Chapter 3. The Domain Language of Batch

Batch Layer

This layer contains various helper classes to aid us in processing batch files.

• UserFieldSetMapper - maps FieldSet result to a User object
• RoleFieldSetMapper - maps FieldSet result to a Role object. To assign the user, an extra JDBC query is performed
• MultiUserFieldSetMapper - maps FieldSet result to a User object; it removes semi-colon from the first token.
• UserItemWriter - writes a User object to the database
• RoleItemWriter - writes a Role object to the database. To assign the user, an extra JDBC query is performed

Next

In the next section, we will focus on the configuration files. Click here to proceed.

---

Share the joy: blogger tutorials Social Bookmarking Blogger Widget I'm reading: Spring Batch Tutorial (Part 2) ~

Subscribe by reader Subscribe by email Share