Skip to main content

Use JPA with MongoDb and Datanucleus



Web applications nowadays have huge demand for data processing, and they need to get them, process them, and displayed them fast. Traditional relational tables and big SQL engines fail to serve that purpose.

NoSQL movement is on the rise. There are huge numbers of alternatives to SQL like BigTable, HBase, CouchDB, Cassandra and MongoDB. They are all fast but you'll need to implement new way to work with them, since all of them have they own query languages. It would be nice that we can use them in similar or same way in our projects, after all ORM layer is there for that to decouple our objects from underlying storage.

MongoDB is one of most popular NoSQL solutions and easiest one to install. For HBase i would had to install cygwin or virtual linux, and still will be hard to confgiure it right. MongoDb is easy, it plays out of the box, just unzip it, and it still offers some nice features like MapReduce, Replication and Sharding.

Datanucleus is one of the leading ORM providers (it is old JPOX project) and they have added support and for JPA. I run in to them after they change name from JPOX when Google chose them to use it for its app engine cloud solution.

In the newest version 3.0m5 of datanucleus, they added support for JPA2 standard and with usual support for all classic RMDBS (Oracle, MySql...) they also added and NoSQL solutions like BigTable, HBase and MongoDB.

So first you will need to install MongoDB from download page, and follow quick start.

But in short you download zip, unzip it.
Create /data/db folder in your root, and you are erady.

Start MongoDB server by going to bin folder and type mongod.

Next you should start mongo shell and test is your MongoDB is working. Go to bin and start mongo.


To list dbs, use
> show dbs

To change db
> use db_name

To list collections/tables in db
> show collections

To add values in
> db.collection_name.insert( { a : 1 } )

To list all values in collection
> db.collection_name.find()

To remove all values in collection
> db.collection_name.remove()


To create small JPA and MongoDB project you can download datanucleus access platform for mongodb from here.

If you use eclipse you can find there and datanucleus plugin that will help you enchance your entities.

Create first small entity called person.

import javax.persistence.*;

@Entity
@Table(name="persons")
public class Person {
  @Id 
  @GeneratedValue(strategy=GenerationType.AUTO)
  @Column(name="_id") // comment this if you use alternative
  private String id;  // comment this if you use alternative
  // alternative to string mongo id, but then remove name '_id'
  // private Long id;
 
  private String firstName = null;
  private String lastName = null;
  private int level = 0;
 
  // getters and setters
  public String getId() {
    return id;
  }
  public void setId(String id) {
    this.id = id;
  }
  public String getFirstName() {
    return firstName;
  }
  public void setFirstName(String firstName) {
    this.firstName = firstName;
  }
  public String getLastName() {
    return lastName;
  }
  public void setLastName(String lastName) {
    this.lastName = lastName;
  }
  public int getLevel() {
    return level;
  }
  public void setLevel(int level) {
    this.level = level;
  }
 
  // to string
  @Override
  public String toString() {
    return "Person [id=" + id + ", firstName=" + firstName + ", lastName="
    + lastName + ", level=" + level + "]";
  }
 
 
}

Note, you can use and long as id but then some type of searches will be slower.
If your project needs to work and with regular SQL DBs you should use long id, otherwise if project will stay forever with MongoDB use string for id since that is natural mongo id.

Then you need to add persistance.xml to your META-INF folder and specify there connection to your mongoDB.

<?xml version="1.0" encoding="UTF-8" ?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="1.0">

    <!-- Tutorial "unit" -->
    <persistence-unit name="Tutorial">
        <properties>
            <property name="datanucleus.ConnectionURL" value="mongodb:localhost/mng1"/>
            <property name="datanucleus.storeManagerType" value="mongodb" />
            <property name="datanucleus.autoCreateSchema" value="true" />
        </properties>
    </persistence-unit>

</persistence>

Libs that you will need for this projects are:
datanucleus-core-3.0.0-m5.jar
datanucleus-mongodb-3.0.0-m5.jar
datanucleus-api-jpa-3.0.0-m5.jar
geronimo-jpa_2.0_spec-1.0.jar
jdo-api-3.1-SNAPSHOT-20110319.jar
transaction-api-1.1.jar
mongo-java-driver-2.6.3.jar
log4j-1.2.16.jar

And add simple Main class that will create entity manager and do few CRUD operations.

import java.util.List;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import javax.persistence.Query;

import test.model.Person;



public class Main {

    public static void main(String[] args) {
        System.out.println("DataNucleus Tutorial with JPA");
        System.out.println("=============================");
        EntityManagerFactory emf = Persistence.createEntityManagerFactory("Tutorial");
        
        // Create entity manager
        EntityManager em = emf.createEntityManager();
        // if you need transactions i will not use them
        //EntityTransaction tx = em.getTransaction();
        
        System.out.println("Save entities");
        System.out.println("=============================");
        Person p = new Person();
        p.setFirstName("test-name-1");
        p.setLastName("test-lastn-1");
        p.setLevel(5);
        
        em.persist(p);
        String id = p.getId();
        System.out.println("id: " + id);
        
        p = new Person();
        p.setFirstName("test-name-2");
        p.setLastName("test-lastn-2");
        p.setLevel(4);
        
        em.persist(p);
        id = p.getId();
        System.out.println("id: " + id);
        // to save for sure in db
        em.close();
        
        System.out.println("find by id");
        System.out.println("=============================");
        em = emf.createEntityManager();
        Person p2 = em.find(Person.class, id);
        System.out.println("found: " + p2.toString());
        
        
        //em = emf.createEntityManager();
        System.out.println("query");
        System.out.println("=============================");
        String qstr = "SELECT p FROM " + Person.class.getName() +
            " p WHERE p.level >= 3 ";
        System.out.println("query: " + qstr);
        Query q = em.createQuery(qstr);
        List list = (List) q.getResultList();
        for (Person per : list) {
            System.out.println("person > " + per.toString());
        }
        
        System.out.println("Remove from db");
        System.out.println("=============================");
        for (Person per : list) {
            em.remove(per);
        }
        em.close();
    }

}

You just need to compile and run your project. For compile you can use maven. pom.xml that looks like this. If you use ant or you want to do it without compile scripts note that datanucleus needs to enchance (some type of compile time weaving) entity classes before you run your project.

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>test.jpa.mongodb</groupId>
    <artifactId>datanucleus-example-jpa-mongodb</artifactId>
    <packaging>jar</packaging>
    <version>4.0</version>
    <name>DataNucleus AccessPlatform Sample for MongoDB with JPA</name>
    
    <repositories>
        <repository>
            <id>DN_M2_Repo</id>
            <name>DataNucleus Repository</name>
            <url>http://www.datanucleus.org/downloads/maven2</url>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <id>DataNucleus_2</id>
            <url>http://www.datanucleus.org/downloads/maven2/</url>
        </pluginRepository>
    </pluginRepositories>
    
    <dependencies>
        <dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-core</artifactId>
            <version>[2.9, )</version>
        </dependency>
        <dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-mongodb</artifactId>
            <version>[2.9, )</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>[1.2, 1.3)</version>
        </dependency>
        <dependency>
            <groupId>org.mongodb</groupId>
            <artifactId>mongo-java-driver</artifactId>
            <version>2.4</version>
        </dependency>
        <dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-api-jpa</artifactId>
            <version>[2.9, )</version>
        </dependency>
        <dependency>
            <groupId>javax.jdo</groupId>
            <artifactId>jdo-api</artifactId>
            <version>[3.0, )</version>
        </dependency>
        <dependency>
            <groupId>org.apache.geronimo.specs</groupId>
            <artifactId>geronimo-jpa_2.0_spec</artifactId>
            <version>1.0</version>
        </dependency>
    </dependencies>
    
    <build>
        <sourceDirectory>src/main/java</sourceDirectory>
        <outputDirectory>bin</outputDirectory>

        <resources>
            <resource>
                <directory>${basedir}/src/main/java</directory>
                <includes>
                    <include>**/persistence.xml</include>
                    <include>**/*.xml</include>
                    <include>**/*.properties</include>
                </includes>
            </resource>
        </resources>

        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.5</source>
                    <target>1.5</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.datanucleus</groupId>
                <artifactId>maven-datanucleus-plugin</artifactId>
                <version>3.0.0-m1</version>
                <configuration>
                    <log4jConfiguration>${basedir}/log4j.properties</log4jConfiguration>
                    <verbose>false</verbose>
                    <api>JPA</api>
                    <persistenceUnitName>Tutorial</persistenceUnitName>
                </configuration>
                <executions>
                    <execution>
                        <phase>compile</phase>
                        <goals>
                            <goal>enhance</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.1</version>
                <configuration>
                    <mainClass>test.service.Main</mainClass>
                    <systemProperties>
                        <systemProperty>
                            <key>log4j.configuration</key>
                            <value>file:${basedir}/log4j.properties</value>
                        </systemProperty>
                    </systemProperties>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptors>
                        <descriptor>${basedir}/assembly.xml</descriptor>
                    </descriptors>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

To compile project, just start
> mvn clean compile

And to run main
> mvn exec:java

The zip with project files can be found here, source download or here .

Tell me what do you think about article, do you need NoSQL in your project, is JPA with NoSQL good or bad thing for you?

Comments

Anonymous said…
source download link seems to be broken. can you post source code somwehere else?
Anonymous said…
Great work, Sale. :)

I've been just looking around for something similar.

Thanks,
Alex
ob1 said…
Hi there - can you offer the source code from a new location - the existing locations have expired...

Cheers, Col.
Caleb said…
Just another guy asking that you host this source code again as it has expired.
Ilja said…
I followed this tutorial and encountered with some problems. I had an exception in console:

javax.validation.ValidationException: Unable to find a default provider

I fixed this issue by adding next line to the persistence.xml:

<property name="datanucleus.validation.mode" value="none"/>
Unknown said…
Please update archive with project. Links is broken.
Darshan said…
Thank you for the information, i found the information very useful.
If anyone looking for Java training in Bangalore i suggest Apponix Technologies, they provide best Java training. For more information visit : https://www.apponix.com/Java-Institute/Java-Training-Institute-in-Bangalore.html

Popular posts from this blog

Javascript REST client

Note : I still work on text, code example should be fine. REST is the one of the most popular interfaces on the web today. One part to its success it owes to its simplicity. How number of sites that support REST grows we will need some fast and good solution to use those services on client side. Even if its possible to do it all, since REST is based on http, with old AJAX calls, it would be nice to have some little more... This is one approach that i choose. I create in javascript new object that has six methods five for REST methods, four for POST, PUT, GET and REMOVE plus one more to GET all resources, and one more to get html template to display your data. This code is based on jquery and json js libs. function RestServiceJs(newurl) { this.myurl = newurl; this.add = function(model, callback) { $.ajax({ type: 'POST', url: this.myurl, data: JSON.stringify(model), // '{"name":"' + model.name + '"}',

Gestalt Diffing algorithm in Java

What we see depends mainly on what we look for. John Lubbock We do not see things as they are, we see things as we are. Anais Nin Gestalt Diffing algorithm in Java Intro When it comes down to diffing or LCS usually story ends with Myers algorithm. It is idea to create BitMatrix (or IntMatrix) of similarities between two strings and then to approach to a problem from graph point, and find the shortest path of changes between two arrays. It sounds as a great solution, and it is, but in reality it has few drawbacks. It is algorithm that gives great results most of the time but it is a slow algorithm for some obvious cases, if strings are equal, or very similar. You create whole matrix but for your solution not all values from matrix are needed. It involves one more algorithm, finding the shortest path in it self. Then improvements of Myers algorithm came, like to check on start are strings are equals, to check for same prefix or suffix, to run snake chase from both