Showing posts with label database. Show all posts
Showing posts with label database. Show all posts

Saturday, September 12, 2020

schema vs database in MySQL and PostgreSQL


TL;DR: both schema and database mean database in MySQL.



MySQL (<— you’re here in SQL client, unless you set default schema in connection)

database1 (<— you're here if use database1;)

tables


PostgreSQL

database1

public (<— you’re here in SQL client, unless you set default search_path in connection)

tables

schema1 (<— you're here if set search_path=schema1;)

tables


Sunday, June 08, 2014

Couchbase and couchbase-client


If you got this when starting Couchbase,

Port server memcached on node 'babysitter_of_ns_1@127.0.0.1' exited with status 71. Restarting. Messages: Thu Oct 3 14:13:51.448736 EST 3: failed to ensure corefile creation

make below changes in /etc/security/limits.conf
couchbase hard nofile 10240
couchbase hard core unlimited

See http://www.couchbase.com/issues/browse/MB-4727

If you got exception about XML datatype like this, exclude org.codehaus.jettison:jettison from couchbase:couchbase-client will solve it.

13:50:00,995 ERROR [[resteasy-servlet]] Servlet.service() for servlet resteasy-servlet threw exception
java.lang.LinkageError: loader constraint violation: when resolving field "DATETIME" the class loader (instance of org/jboss/classloader/spi/base/BaseClassLoader) of the referring class, javax/xml/datatype/DatatypeConstants, and the class loader (instance of ) for the field's resolved type, javax/xml/namespace/QName, have different Class objects for that type
 at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl.(RuntimeBuiltinLeafInfoImpl.java:267)
 at com.sun.xml.bind.v2.model.impl.RuntimeTypeInfoSetImpl.(RuntimeTypeInfoSetImpl.java:65)
 at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.createTypeInfoSet(RuntimeModelBuilder.java:133)
 at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.createTypeInfoSet(RuntimeModelBuilder.java:85)
 at com.sun.xml.bind.v2.model.impl.ModelBuilder.(ModelBuilder.java:156)
 at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.(RuntimeModelBuilder.java:93)
 at com.sun.xml.bind.v2.runtime.JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:450)
 at com.sun.xml.bind.v2.runtime.JAXBContextImpl.(JAXBContextImpl.java:298)
 at com.sun.xml.bind.v2.runtime.JAXBContextImpl.(JAXBContextImpl.java:141)
 at com.sun.xml.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1157)
 at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:145)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Thursday, July 11, 2013

MongoDB-based Cache Service

I've talked about using REST web service to wrap database and provide managed repository service. This time, I'd like to discuss developing cache service with MongoDB's two convenient features.

Before I start, I want to make it clear that the cache service here is not the one that trying reduce response time to sub-ms. It's something you hesitate to call it again and want to store it somewhere. Usually it's a distributed web service (for example, Maps API from Google), or an expensive SQL statement result. You want to cache it not only because you don't want to wait for few seconds again, but also to save your usage quota, or reduce the workload of a database. In this case, you'll be happy if we can reduce the response from X seconds to X ms.

Depending on whether it's a single node or clustered environment, the size of the cached data, text or binary, there are quite a few products that can fulfill the task. But when we examine if the solution can scale up and scale out, the answer become not clear. Consider config something like Memcached in a 4 node cluster and you'll get the idea. Basically you have to explicitly tell each node, "you are in a group so you guys have a shared memory or disk".

How about share nothing? As long as a node knows the cache, it doesn't matter how many other nodes also know the cache, they can share something, with same key of course. Cache service can just become a couple of HTTP methods (POST and GET) backed by MongoDB. But why MongoDB?

One aspect of a cache is the capacity, in bytes or in number of objects. In MongoDB, you can use Capped Collections to achieve this. You can create a capped collection using
db.createCollection("mycoll", {capped:true, size:100000})
or convert a collection to capped one using
db.runCommand({"convertToCapped": "mycoll", size: 100000});

The value of size parameter is in bytes. You may not know the size of a document in the collection if you want to know how many documents can be stored in the capped collection. If you already have a amount of documents, you can run
db.mycoll.stats()
and check the avgObjSize value before converting it to capped collection. Here is an example,
{
    "ns" : "mydb.mycoll",
    "count" : 7739,
    "size" : 42885120,
    "avgObjSize" : 5541.429125209976,
    "storageSize" : 65724416,
    "numExtents" : 8,
    "nindexes" : 1,
    "lastExtentSize" : 23224320,
    "paddingFactor" : 1,
    "systemFlags" : 1,
    "userFlags" : 0,
    "totalIndexSize" : 228928,
    "indexSizes" : {
        "_id_" : 228928
    },
    "ok" : 1
}


If you run stats() on a capped collection, you'll see 2 more lines in result
    "capped" : true,
    "max" : NumberLong("9223372036854775807"),

Another feature in caching is Time To Live, which is used to specify when a cached item should be invalidated. In MongoDB, you can create index on a date field and provide expireAfterSeconds option to set the TTL of a collection.
db.mycoll.ensureIndex( { "created": 1 }, { expireAfterSeconds: 3600 } )

Note however that the background task to delete expired documents runs once every 60 seconds, so don't expect this feature working much more accurately than that. And you can't make a collection both size- and time-based (who's going to need both anyway).

So next time when you design a size-based or time-based cache, would you like to consider MongoDB?

Wednesday, April 03, 2013

Enable Full Text Search for MongoDB

If you get this in mongo console
db.coll.ensureIndex({'content':'text'})
{
    "err" : "text search not enabled",
    "code" : 16633,
    "n" : 0,
    "connectionId" : 1,
    "ok" : 1
}

, and this in mongod console
[conn1] insert test.system.indexes keyUpdates:0 exception: text search not enabled code:16633 locks(micros) w:336411 336ms

, you need to enable full text search when starting MongoDB.
mongod --setParameter textSearchEnabled=true

Try again.
db.coll.ensureIndex({'content':'text'})

What's happening in background?
[initandlisten] connection accepted from 127.0.0.1:51347 #1 (1 connection now open)
[conn1] build index test.coll { _fts: "text", _ftsx: 1 }
[conn1]     Index: (1/3) External Sort Progress: 3500/6245 56%
[conn1]     Index: (1/3) External Sort Progress: 5400/6245 86%
[conn1]  external sort used : 413 files in 25 secs
[conn1]     Index: (2/3) BTree Bottom Up Progress: 185800/2616966 7%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 401900/2616966 15%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 554200/2616966 21%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 769700/2616966 29%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 973700/2616966 37%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 1175400/2616966 44%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 1380700/2616966 52%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 1588900/2616966 60%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 1794900/2616966 68%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 1936500/2616966 73%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 2125800/2616966 81%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 2315800/2616966 88%
[conn1]     Index: (2/3) BTree Bottom Up Progress: 2528700/2616966 96%
[conn1]  done building bottom layer, going to commit
[conn1] build index done. scanned 6245 total records. 160.617 secs
[conn1] insert test.system.indexes ninserted:1 keyUpdates:0 locks(micros) w:160642416 160645ms

Check indexes.
db.coll.getIndexes()
[
    {
        "v" : 1,
        "key" : {
            "_id" : 1
        },
        "ns" : "test.coll",
        "name" : "_id_"
    },
    {
        "v" : 1,
        "key" : {
            "_fts" : "text",
            "_ftsx" : 1
        },
        "ns" : "test.coll",
        "name" : "content_text",
        "weights" : {
            "content" : 1
        },
        "default_language" : "english",
        "language_override" : "language",
        "textIndexVersion" : 1
    }
]

Check index size. It's about half the text size it indexed.
db.coll.stats()
{
    "ns" : "test.coll",
    "count" : 6245,
    "size" : 69054068,
    "avgObjSize" : 11057.496877502002,
    "storageSize" : 178520064,
    "numExtents" : 12,
    "nindexes" : 4,
    "lastExtentSize" : 49213440,
    "paddingFactor" : 1.0000000000003018,
    "systemFlags" : 0,
    "userFlags" : 1,
    "totalIndexSize" : 86314032,
    "indexSizes" : {
        "_id_" : 212576,
        "content_text" : 85381968
    },
    "ok" : 1
}

Have a test.
db.coll.runCommand("text", {search:'Hello'})
{
    "queryDebugString" : "hello||||||",
    "language" : "english",
    "results" : []
    "stats" : {
        "nscanned" : 4,
        "nscannedObjects" : 0,
        "n" : 4,
        "nfound" : 4,
        "timeMicros" : 157
    },
    "ok" : 1
}

Not bad.

Friday, June 22, 2012

Data (Repository) as a Service

In a couple of systems I'm supporting, database is used not only as persistence service, but also as single source of truth and a trusted way of inter-system communication. In Java world, JDBC is the official passport to database, no matter how many roles your database has. But my question is, is there a better way to provide data service with business values, rather than just a connection to database?

That's basically the reason I'd like to see the possibilities of Data as a Service in enterprise computing. Note that it's about software architecture and design, not infrastructure as a service, like Amazon RDS, Amazon DynamoDB, Amazon SimpleDB, Google BigQuery or OpenStack RedDwarf.

Data access layer hasn't changed much since the beginning of Java. From Java statement
DriverManager.getConnection("jdbc:oracle:thin:@//localhost:1521/myDB", "scott", "tiger");

to hibernate.properties
hibernate.connection.driver_class = org.postgresql.Driver
hibernate.connection.url = jdbc:postgresql://localhost/mydatabase
hibernate.connection.username = myuser
hibernate.connection.password = secret

to JNDI datasource
<local-tx-datasource>
  <jndi-name>TEST</jndi-name>
  <connection-url>jdbc:jtds:sqlserver://localhost:2423/TEST</connection-url>
  <driver-class>net.sourceforge.jtds.jdbc.Driver</driver-class>
  <user-name>user</user-name>
  <password>password</password>
</local-tx-datasource>

to Spring applicationContext.xml
<bean id="dataSource" destroy-method="close" class="org.apache.commons.dbcp.BasicDataSource">
  <property name="driverClassName" value="${jdbc.driverClassName}"/>
  <property name="url" value="${jdbc.url}"/>
  <property name="username" value="${jdbc.username}"/>
  <property name="password" value="${jdbc.password}"/>
</bean>

to JPA persistence.xml
<property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver" />
<property name="javax.persistence.jdbc.url" value="jdbc:mysql://localhost/test" />
<property name="javax.persistence.jdbc.user" value="user" />
<property name="javax.persistence.jdbc.password" value="password" />

As you can see, for more than a decade, we've simply given applications a bump key to database and let them do whatever they want. This reminds me of the setters in a Java Bean, or Anaemic Domain Model. Database takes no business responsibility, and totally depends on the implementation of data access layer. Thanks to generics, we have CRUD without implementing all these operations for each entity. We don't even have to know what service layer needs before providing all these operations to them. Let alone the methods defined in generic DAO that can execute arbitrary SQL statements.

So what's Data (Repository) as a Service? It's the encapsulation of a database that provides business related services (as opposed to RESTful-like SQL statements like Google BigQuery or JPA-RS) that are accessible via REST interface. Applications (as clients) define the requirements they need for data repository to implement, and utilise these services to achieve business-oriented objectives.

What's the impact to application development? Well, if you already apply multi-tired software design, the only impact happens in service layer. There is no more data access, instead there is a trustworthy, high available and fast response data service. More challenges however come from data service. Here are some points I'm having in my mind.

Transaction
Services provided by data repository are transactional. Client receives the result of a committed or roll-backed transaction but has no control over the transaction. In other words, Clients locate outside of transaction boundaries. Data repository may have operations that involve Transaction Script or even two-phase commit, but clients have no idea of these operations either.

Timeout and Asynchronous Operations
Query services may have timeout control. If underlying database fails to return resultset within specified time (defined by data repository service but may be overwritten by client), client will get notification. On the other hand, persistence services may be asynchronous. Client has several ways to get the result of a persistence request, like callback, server push or delayed polling.

Authentication
Roles can be used to provide business level access control to each individual endpoint. These controls remain consistent for multiple applications. Instead of identifying Administrator in System A, and Manager in System B, data service define roles based on business characteristics and values of data manipulations. For example, no matter which system is interacting with data repository, InvoiceViewer is not allowed to request any service that changes invoice, or anything not invoice-related, like placing an order. This is a service level control, as opposed to fine-grained (domain object level, or table level from DB point of view) access control solutions like JPA Security.

Continuous Delivery
One major change in development when applications share database is, there is no more multiple copy / version of domain objects and DAOs. Traditionally, all the applications have their own copy of mapped objects and data operations. They make their own changes and synchronise the changes made by others. Any changes to data repository service however take effect immediately to all the applications. From application point of view, there is only one up-to-date database / data service, and no more mismatch of domain object and database, or domain object / DAO versions. We may have multiple versions of data service at the same time, but the point here is isolating detailed database changes with consistent service interface, so that no development has to stop just because others made a change that has nothing to do with you.

Performance
Now we can see more applications are designed with ease of development, testability and maintainability in mind. A web application may be divided into UI and backend. This separation also addresses the diversity of user interface, like desktop and mobile. This is the performance of development and response to changes. When it comes to runtime performance, data services should be stateless and building block like, especially for query services, just like RISC's high efficiency command set, and leave the assembly tasks to applications. All application share same entity manager makes caching possible. In case extremely low latency is required, use WebSocket to avoid HTTP protocol overhead.

Entities and Value Objects
This is a side benefit of data repository service. By using JSON as data exchange, no entity will get leaked outside of transaction boundaries. By the way, there are other options in data exchange than XML and JSON, like Google uses more expressive Protocol Buffers.

Database?
By providing data services, the database used becomes a internal detail. As long as the services respond requests and return data in certain format, it doesn't matter whether the database is RDBMS or NoSQL. Data services can even provide mock data to support fast prototyping.

HATEOAS?
No, data service has no hypermedia controls (sorry Dean, ;-]). It belongs to Richardson Maturity Model Level 2 - HTTP Verbs. The main reason is that a service is not a resource. Placing an order is to persist order data, it has no possible valid subsequence states, which should live in application level.

If your database is serving SQL clients, make the change and let it provide unified and controlled service to business clients.

Thursday, May 17, 2012

One-to-one relationship using EclipseLink

Best Practice in JPA series:
Part 1 – JPA Caching in EclipseLink
Part 2 – One-to-one relationship using EclipseLink

Here are some tips on how to implement high performance one to one relationship using EclipseLink 2.3.2.

Identify owning side (side with foreign key) and inverse side (side with mappedBy attribute in @OneToOne annotation). I will use Owning entity and Inverse entity as examples in following tips.

If you define private Inverse inverse; in Owning entity and you're satisfied with inverse_id as column name for foreign key, you don't need to specify @JoinColumn(name = "inverse_id") on it. However, you need to specify @OneToOne(cascade = CascadeType.ALL) on it, so that any operations on Inverse entity can be performed from Owning entity. You also need to specify @OneToOne(mappedBy = "inverse") on private Owning owning; in Inverse entity.

To avoid N + 1 select problem, specify @BatchFetch(BatchFetchType.EXISTS) on owning property in Inverse entity. You can also use @BatchFetch(BatchFetchType.IN) or @BatchFetch(BatchFetchType.JOIN). Following SQL statements will be used respectively for better performance.

SELECT t0.* FROM INVERSE t0 WHERE EXISTS (SELECT t1.ID FROM OWNING t1 WHERE (t0.ID = t1.INVERSE_ID))

SELECT * FROM INVERSE WHERE (ID IN (?,?))

SELECT t0.* FROM INVERSE t0, OWNING t1 WHERE (t0.ID = t1.INVERSE_ID)

If both Owning entity loads Inverse entity eagerly and Inverse entity loads Owning entity eagerly, following SQL statement will still be executed N times. Otherwise N + 1 problem is solved.

SELECT * FROM OWNING WHERE (INVERSE_ID = ?)

fetch = FetchType.LAZY can be set in @OneToOne annotation on owning side and / or inverse side. But it's better to use it on inverse side, because when owning object loads inverse object eagerly (FetchType.EAGER is default in one-to-one relation) by INVERSE_ID, the result can be used to populate inverse entity cache.

If you want lazy fetch take effect outside of a Java EE 5/6 application server, VM argument -javaagent:/home/jerry/.m2/repository/org/eclipse/persistence/eclipselink/2.3.2/eclipselink-2.3.2.jar needs to be set. Note that full absolute path is used here. See Using EclipseLink JPA Weaving for more details.

Saturday, April 14, 2012

Wildcard Host doesn't work in MySQL 5.5.22

I haven't used MySQL for a while. After reinstalled my laptop with Ubuntu 12.04 Beta 2, I decided to use MySQL instead of PostgreSQL to continue my JPA study.

sudo apt-get install mysql-server

I used MySQL Administrator long time ago and now it's replaced with powerful MySQL Workbench. If you have difficulties in installing it, please follow this tutorial.

Everything seems fine, just like when I developed myTunes 6 years ago, except that % for any host doesn't work any more. So if you get

ERROR 1045 (28000): Access denied for user 'user'@'localhost' (using password: YES)

Update Host column from % to localhost for your user account and restart MySQL will solve it.

Update: you can also remove anonymous accounts by executing delete from user where user=''; (Thank you, Dan)

Friday, February 10, 2012

JPA Caching in EclipseLink 2.3.2


Best Practice in JPA series:
Part 1 – JPA Caching in EclipseLink
Part 2 – One-to-one relationship using EclipseLink


Java Persistence API 2.0 defines Level 1 (L1) Cache (Entity Cache), Level 2 (L2) Cache (Shared Entity Cache) and Query (Result) Cache. Now I can take full advantage of JPA cache, just like what I did 5, 6 years ago using Hibernate cache. Although Hibernate also has its JPA implementation, I found EclipseLink has better default settings and also easier to enable advanced features. Here are some random tips.

The shared attribute of @Cache annotation has deprecated and is replaced by isolation=CacheIsolationType.SHARED, which means sharing entity cache between EntityManager objects and allowing query cache (if enabled) to use entity cache. Even you don't set @Cache annotation on a domain object, it's enabled by default.

CacheIsolationType.PROTECTED means sharing entity cache between EntityManager objects but disallowing query cache to use entity cache, even when query cache is enabled.

CacheIsolationType.ISOLATED means not sharing entity cache between EntityManager objects and disallow query cache to use entity cache, even when query cache is enabled.

The default coordinationType=CacheCoordinationType.SEND_OBJECT_CHANGES in @Cache means any entity update to database also updates entity in cache, which is great in performance.

Set hints = { @QueryHint(name = QueryHints.QUERY_RESULTS_CACHE, value = HintValues.TRUE) } in @NamedQuery if you want to enable query cache. But unless you have a fixed domain objects, don't enable its query cache. Note that "eclipselink.query-results-cache" is not a standard JPA hint, you cannot set it to "True" for a javax.persistence.Query object.

Unless you have every domain object in entity cache, don't use hints = { @QueryHint(name = QueryHints.CACHE_USAGE, value = CacheUsage.CheckCacheOnly) } in @NamedQuery.

Set <property name="eclipselink.logging.level" value="FINE" /> in META-INFO/persistence.xml to show SQL statements.

Unless you want to change the default cache retrieve / store mode, don't need to set following properties to EntityManager object.
em.setProperty(QueryHints.CACHE_RETRIEVE_MODE, CacheRetrieveMode.USE);
em.setProperty(QueryHints.CACHE_STORE_MODE, CacheStoreMode.USE);

Even you don't set any hint in @NamedQuery, query result will be used to populate entity cache.

Only entityManager.find(entityClass, id) can use entity cache. Get entity, or get entities, by any field(s) other than id don't use entity cache.

The default sizes of entity cache and query cache are 100 each.

If you get
Exception in thread "main" java.lang.IllegalArgumentException: An exception occurred while creating a query in EntityManager:
Exception Description: Syntax error parsing the query [from Entity], line 1, column 0: unexpected token [from].
Internal Exception: NoViableAltException(33@[])
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.createQuery(EntityManagerImpl.java:1477)
change your JPA query to entityManager.createQuery("select entity from Entity entity").getResultList();

Please feel free to let me know if you have any questions regarding EclipseLink cache.

Tuesday, November 01, 2011

Funny statements regarding Database and ORM

Twitter is great to share things that's hard to misunderstand. 140 characters are just inadequate to explain your philosophy of software development, even a series of tweets. Is a meaningful tweet to you also meaningful to your thousands of followers?

I saw a series tweets from Uncle Bob Martin today and found them very confusing to me, and maybe misleading to others.

Databases are details to be hidden. They are not your central abstraction, nor are they the core of your application. Ever.

In any multi-tier software architecture, lower tier provides service to the tier that sits on it. Upper tier defines the contract for lower tier to implement, rather than vice versa. Databases, as a repository service, is by no means a central abstraction, or the core of the whole application. Otherwise, my garage is the core of my house because I store everything there.

But, databases provide an abstraction (no central here) to the tier just above the database layer. As long as the JDBC URL (from a Java developer's perspective, luckily Java is still popular in enterprise software development) is not used outside data access layer, we can say databases are hidden.

Personally, I really don't know how not to make databases hidden, how not to think databases are not central abstraction, or not the core of an application.

relational tables hold data structures, not objects. Objects are bags of behavior. Data structures are bags of data.

I think he wanted to express either relational tables hold data structures, not classes; or relational tables hold structured data, not objects. But anyway. Relational tables don't hold objects / classes is just the reason why we need ORM to map them to objects / classes, and operate hidden data though interfaces.

Any bridge is used to connect two different entities. ORM is the bridge between data and objects, or tables and classes. If the mapped objects don't provide something else convenient to application development, why do we do the mapping? From a bags of data to another bags of data? Yes, we can. But only in mapping to something like C.

The O in ORM is in error. It should be DS for Data Structure. No simple tool can map tables to objects.

Here comes the funniest part. Everyone knows ORM is first introduced in Java. That's easy to understand why ORM has a O in it. Java doesn't have struct keyword, so it's ridiculous to criticize the naming of ORM. Can't we teach Data Structure in Java? The getters and setters in mapped objects are not behavior?

Unless you apply Domain Driven Development, there are no operations other than getters and setters in mapped objects. Even if you apply DDD, the operations in mapped objects are not mapped from table, they come from business rules. For example, what, in a User table, can be mapped to a addUser() behavior in User class? Don't tell me it's mapped from a stored procedure called add_user. We're talking about tables, right? Stored procedures live inside database, but outside any tables.

If anyone really likes this kind of naming game, here is a free topic for you if you have thousands of followers like this guy. The disk in RAM disk is in error.

Have fun.

Sunday, September 25, 2011

Requirement for swap space (or virtual memory), not again

More and more users have more and more memory installed. In order to take advantage of their memory, some users move cache of Firefox or Chrome to memory from their disk, some Linux users move their temp directories to tempfs.

I mentioned MyEclipse needs swap, or virtual memory, during installation. I came across almost the same problem when I installed Oracle Database Express Edition.

This system does not meet the minimum requirements for swap space.  Based on the amount of physical memory available on the system, Oracle Database 10g Express Edition requires 1024 MB of swap space. This system has 0 MB of swap space.  Configure more swap space on the system and retry the installation.

I don't have swap partition or swap file at that time, but the installer ignored its own warning and installed itself successfully. I thank Oracle.

I repeat my point here, swap is not necessary in Linux desktop. Hope those applications that still require swap or virtual memory consider it. For those still not sure whether you can remove swap, please reduce your swappiness to 0 to see if you can live without swap.

Thursday, March 24, 2011

How to add another data source in JPA

It's quite easy to create a data source using JPA support of Spring framework. It not so difficult to add another data source to your application as well.

In META-INF/persistence.xml, define another persistence unit.

     <persistence-unit name="anotherUnit" transaction-type="RESOURCE_LOCAL">  
         <class>com.youcompany.YourClass</class>  
         <exclude-unlisted-classes>true</exclude-unlisted-classes>  
         <properties>  
             <property name="hibernate.hbm2ddl.auto" value="update" />  
             <!-- validate | update | create | create-drop -->  
         </properties>  
     </persistence-unit>  

Note that you should define all the domain classes you will be using in the defined persistence unit in <class> elements.

Define another database context xml file.
 <?xml version="1.0" encoding="UTF-8"?>  
 <beans xmlns="http://www.springframework.org/schema/beans"  
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"  
     xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p"  
     xmlns:aop="http://www.springframework.org/schema/aop"  
     xsi:schemaLocation="http://www.springframework.org/schema/beans 
     http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
     http://www.springframework.org/schema/context
     http://www.springframework.org/schema/context/spring-context-3.0.xsd
     http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
     http://www.springframework.org/schema/aop
     http://www.springframework.org/schema/aop/spring-aop-3.0.xsd">  
     <!-- holding properties for database connectivity / -->  
     <context:property-placeholder location="classpath:config.properties" />  
     <bean id="anotherDataSource" class="org.apache.commons.dbcp.BasicDataSource"  
         destroy-method="close">  
         <property name="driverClassName" value="${db.driver}" />  
         <property name="url" value="${db.url}" />  
         <property name="username" value="${db.user}" />  
         <property name="password" value="${db.pass}" />  
         <property name="validationQuery" value="${dbcp.validationQuery}" />  
         <property name="testWhileIdle" value="${dbcp.testWhileIdle}" />  
         <property name="timeBetweenEvictionRunsMillis" value="${dbcp.timeBetweenEvictionRunsMillis}" />  
         <property name="numTestsPerEvictionRun" value="${dbcp.numTestsPerEvictionRun}" />  
         <property name="minEvictableIdleTimeMillis" value="${dbcp.minEvictableIdleTimeMillis}" />  
     </bean>  
     <bean id="anotherJpaAdapter"  
         class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"  
         p:database="${db.database}" p:showSql="${db.showSql}" />  
     <bean id="anotherEntityManagerFactory"  
         class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"  
         p:dataSource-ref="anotherDataSource" p:jpaVendorAdapter-ref="anotherJpaAdapter">  
         <property name="loadTimeWeaver">  
             <bean  
                 class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver" />  
         </property>  
         <property name="persistenceUnitName" value="anotherUnit"></property>  
     </bean>  
     <bean id="anotherTxManager" class="org.springframework.orm.jpa.JpaTransactionManager"  
         p:entityManagerFactory-ref="anotherEntityManagerFactory" />  
 </beans>  

In the JPA implementation of generic DAO class, annotate the 1st persistence unit in the setter of EntityManager.

     protected EntityManager entityManager;  
     @PersistenceContext(unitName="firstUnit")  
     public void setEntityManager(EntityManager entityManager) {  
         this.entityManager = entityManager;  
     }  

Create another generic DAO class for new persistence unit. All the operations to the new domain objects should be accomplished via this new generic DAO.

If you want to access data source directly, use
     @Autowired  
     @Qualifier("anotherDataSource")  
     private DataSource dataSource;  

Don't forget to getAutoCommit and keep the status of any connection you get from data source if you need to setAutoCommit yourself, and close the connection in finally statement.

That's it.