JPA Persistence Guide (v5.2)

The simplest way of creating an EntityManagerFactory in a JavaSE environment is as follows

Here we provide the name of the persistence-unit which defines the datastore, properties, classes, meta-data etc to be used. An alternative is to specify the properties to use along with the persistence-unit name; in that case the passed properties will override any that are specified for the persistence unit itself.

If you want an application-managed EMF then you create it by injection like this, providing the name of the required persistence-unit

If you want a container-managed EM then you create it by injection like this, providing the name of the required persistence-unit

Please refer to the docs for your JavaEE server for more details.

As shown above, we create an EMF for a persistence-unit. A persistence-unit is simply a way of having independent groupings of entities, mapping info and/or jars that will be managed together. The persistence-unit is named, and the name is used for identifying it (as used above in creating the EMF). This name can also then be used when defining what classes are to be enhanced, for example.

To define a persistence-unit you first need to add a file persistence.xml to the META-INF/ directory of your application jar. This file will be used to define your persistence-unit(s). Let’s show an example

In this example we have defined 2 persistence-unit(s). The first has the name "OnlineStore" and contains 5 classes (annotated). The second has the name "Accounting" and contains a metadata file called orm.xml in a particular package (which will define the classes being part of that unit). This means that once we have defined this we can reference these persistence-unit(s) in our persistence operations. You can find the XSD for persistence.xml here.

There are several sub-elements of this persistence.xml file worth describing

An EntityManagerFactory is very configurable, and DataNucleus provides many properties to tailor its behaviour to your persistence needs.

Since the EMF has significant resources associated with it, it should always be closed when you no longer need to perform any more persistence operations. For most operations this will be when closing your application. Whenever it is you do it like this

The EntityManagerFactory has an optional cache of all objects across all _EntityManager_s. This cache is called the Level 2 (L2) cache, and JPA doesn’t define whether this should be enabled or not. With DataNucleus it defaults to enabled. The user can configure the L2 cache if they so wish; by use of the persistence property datanucleus.cache.level2.type. You set this to "type" of cache required. You currently have the following options.

The weak, soft and javax.cache caches are available in the datanucleus-core plugin. The EHCache, OSCache, Coherence, Cacheonix, and Memcache caches are available in the datanucleus-cache plugin.

In addition you can control the mode of operation of the L2 cache. You do this using the persistence property datanucleus.cache.level2.mode (or javax.persistence.sharedCache.mode). The default is UNSPECIFIED which means that DataNucleus will cache all objects of entities unless the entity is explicitly marked as not cacheable. The other options are NONE (don’t cache ever), ALL (cache all entities regardless of annotations), ENABLE_SELECTIVE (cache entities explicitly marked as cacheable), or DISABLE_SELECTIVE (cache entities unless explicitly marked as not cacheable - i.e same as our default).

Objects are placed in the L2 cache when you commit() the transaction of a EntityManager. This means that you only have datastore-persisted objects in that cache. Also, if an object is deleted during a transaction then at commit it will be removed from the L2 cache if it is present.

DataNucleus JPA allows you to generate the schema for your persistence-unit when creating an EMF. You can create, drop or drop then create the schema either directly in the datastore, or in scripts (DDL) as required. See the associated persistence properties (most of these only apply to RDBMS).

Some examples.

Example 1, to create a database using the JPA entities (metadata), and then load data

Example 2, to create DDL scripts for the JPA entities

Example 3, if you want to create the schema using your own (DDL) script, you can set

javax.persistence.schema-generation.database.action=create javax.persistence.schema-generation.create-source=script javax.persistence.schema-generation.create-script-source=META-INF/my_create_script.ddl ---

If you want to create the schema (tables + columns + constraints) during the persistence process, the property datanucleus.schema.autoCreateAll provides a way of telling DataNucleus to do this. It’s a shortcut to setting the other 3 properties to true. Thereafter, during calls to DataNucleus to persist classes or performs queries of persisted data, whenever it encounters a new class to persist that it has no information about, it will use the MetaData to check the datastore for presence of the "table", and if it doesn’t exist, will create it. In addition it will validate the correctness of the table (compared to the MetaData for the class), and any other constraints that it requires (to manage any relationships). If any constraints are missing it will create them.

DataNucleus can check any existing schema against what is implied by the MetaData.

The property datanucleus.schema.validateTables provides a way of telling DataNucleus to validate any tables that it needs against their current definition in the datastore. If the user already has a schema, and want to make sure that their tables match what DataNucleus requires (from the MetaData definition) they would set this property to true. This can be useful for example where you are trying to map to an existing schema and want to verify that you’ve got the correct MetaData definition.

The property datanucleus.schema.validateColumns provides a way of telling DataNucleus to validate any columns of the tables that it needs against their current definition in the datastore. If the user already has a schema, and want to make sure that their tables match what DataNucleus requires (from the MetaData definition) they would set this property to true. This will validate the precise column types and widths etc, including defaultability/nullability settings. Please be aware that many JDBC drivers contain bugs that return incorrect column detail information and so having this turned off is sometimes the only option (dependent on the JDBC driver quality).

The property datanucleus.schema.validateConstraints provides a way of telling DataNucleus to validate any constraints (primary keys, foreign keys, indexes) that it needs against their current definition in the datastore. If the user already has a schema, and want to make sure that their table constraints match what DataNucleus requires (from the MetaData definition) they would set this property to true.

Some datastores allow access to multiple "schemas" (such as with most RDBMS). DataNucleus will, by default, use the "default" database schema for the Connection URL and user supplied. This may cause issues where the user has been set up and in some databases (e.g Oracle) you want to write to a different schema (which that user has access to). To achieve this in DataNucleus you would set the persistence properties

This will mean that all RDBMS DDL and SQL statements will prefix table names with the necessary catalog and schema names (specify which ones your datastore supports).

The datastore will define what case of identifiers (table/column names) are accepted. By default, DataNucleus will capitalise names (assuming that the datastore supports it). You can however influence the case used for identifiers. This is specifiable with the persistence property datanucleus.identifier.case, having the following values

By default all tables are generated with columns in alphabetical order, starting with root class fields followed by subclass fields (if present in the same table) etc. This is not part of JPA but DataNucleus allows an extension to specify the relative position, such as

Note that the values of the position start at 0, and should be specified completely for all columns of all fields.

If your datastore is read-only (you can’t add/update/delete any data in it), obviously you could just configure your application to not perform these operations. An alternative is to set the EMF as read-only, by setting the persistence property datanucleus.ReadOnlyDatastore to true.

From now on, whenever you perform a persistence operation that implies a change in datastore data, the operation will throw a PersistenceException.

DataNucleus provides an additional control over the behaviour when an attempt is made to change a read-only datastore. The default behaviour is to throw an exception. You can change this using the persistence property datanucleus.readOnlyDatastoreAction with values of "EXCEPTION" (default), and "IGNORE". "IGNORE" has the effect of simply ignoring all attempted updates to readonly objects.

You can take this read-only control further and specify it just on specific classes. Like this

DataNucleus SchemaTool currently works with RDBMS, HBase, Excel, OOXML, ODF, MongoDB, Cassandra datastores and is very simple to operate. It has the following modes of operation :

In addition for RDBMS, the create/delete modes can be used by adding "-ddlFile {filename}" and this will then not create/delete the schema, but instead output the DDL for the tables/constraints into the specified file.

For the create, delete and validate modes DataNucleus SchemaTool accepts either of the following types of input.

Here we provide many different ways to invoke DataNucleus SchemaTool

As time goes by during the development of your DataNucleus JPA powered application you may need to add fields, update field mappings, or delete fields. In an ideal world the JPA provider would take care of this itself. However this is actually not part of the JPA standard and so you are reliant on what features the JPA provider possesses.

DataNucleus can cope with added fields, if you have the relevant persistence properties enabled. In this case look at datanucleus.schema.autoCreateTables, datanucleus.schema.autoCreateColumns, datanucleus.schema.autoCreateConstraints, and datanucleus.rdbms.dynamicSchemaUpdates (with this latter property of use where you have interface field(s) and a new implementation of that interface is encountered at runtime).

If you update or delete a field with an RDBMS datastore then you will need to update your schema manually. With non-RDBMS datastores deletion of fields is supported in some situations.

You should also consider making use of tools like Flyway and Liquibase since these are designed for exactly this role.

The JPA API doesn’t provide a way of accessing the schema of the datastore itself (if it has one). In the case of RDBMS it is useful to be able to find out what columns there are in a table, or what data types are supported for example. DataNucleus Access Platform provides an API for this.

The first thing to do is get your hands on the DataNucleus StoreManager and from that the StoreSchemaHandler. You do this as follows

So now we have the StoreSchemaHandler what can we do with it? Well start with the javadoc for the implementation that is used for RDBMS

You obtain an EntityManager from the EntityManagerFactory as follows

In the case of using container-managed JavaEE, you would instead obtain the EntityManager by injection

You then perform all operations that you need using this EntityManager.

If you manually created the EntityManager using createEntityManager() then you must also close it; forgetting to close it will lead to memory/resource leaks.

In general you will be performing all operations on a EntityManager within a transaction, whether your transactions are controlled by your JavaEE container, by a framework such as Spring, or by locally defined transactions. In the examples below we will omit the transaction demarcation for clarity.

The main thing that you will want to do with the data layer of a JPA-enabled application is persist your objects into the datastore. As we mentioned earlier, a EntityManagerFactory represents the datastore where the objects will be persisted. So you create a normal Java object in your application, and you then persist this as follows

This will result in the object being persisted into the datastore, though clearly it will not be persistent until you commit the transaction. The Lifecycle State of the object changes from Transient to Persistent (after persist()), to Persistent/Detached (at commit).

When you want to persist multiple objects with standard JPA you have to call persist multiple times. Fortunately DataNucleus extends this to take in a Collection or an array of entities, so you can do

As above, the objects are persisted to the datastore. The Lifecycle State of the objects change from Transient to Persistent (after persist()), to Persistent/Detached (at commit).

Once you have persisted an object, it has an "identity". This is a unique way of identifying it. When you specify the persistence for the entity you specified an id field (or fields, together with an IdClass) so you can create the identity from that. So what ? Well the identity can be used to retrieve the object again at some other part in your application. So you pass the identity into your application, and the user clicks on some button on a web page and that button corresponds to a particular object identity. You can then go back to your data layer and retrieve the object as follows

which will try to retrieve the Person object with identity of 1. If there is no Person object with that identity then it returns null.

Whilst the primary way of looking up an object is via its identity, in some cases a class has a unique key (maybe comprised of multiple field values). This is sometimes referred to as a natural id. This is not part of the JPA API, however DataNucleus makes it available. Let’s take an example

Here we have a Person class with an identity defined as a long, but also with a unique key defined as the composite of the firstName and lastName (in most societies it is possible to duplicate names amongst people, but we just take this as an example).

Now to access a Person object based on the firstName and lastName we do the following

and we retrieve the Person "George Jones".

When you need to delete an object that you had previous persisted, deleting it is simple. Firstly you need to get the object itself, and then delete it as follows

When you want to delete multiple objects with standard JPA you have to call remove multiple times. Fortunately DataNucleus extends this to take in a Collection or an array of entities, so you can do

To modify a previously persisted object you take the object and update it in your code. If the object is in "detached" state (not managed by a particular EntityManager) then when you are ready to persist the changes you do the following

If however the object was already managed at the point of updating its fields, then

when the setAddress has been called, this is intercepted by DataNucleus, and the changes will be stored for persisting. There is no need to call any EntityManager method to push the changes. This is part of the mechanism known as transparent persistence.

When you want to attach multiple modified objects with standard JPA you have to call merge multiple times. Fortunately DataNucleus extends this to take in a Collection or an array of entities, so you can do

An application that has sole access to the datastore, in general, does not need to check for updated values from the datastore. In more complicated situations the datastore may be updated by another application for example, so it may be necessary at times to check for more up-to-date values for the fields of an entity. You do that like this

This will do the following

If the object had any changes they will be thrown away by this step, and replaced by the latest datastore values.

JPA doesn’t provide a method for getting the EntityManager of an object as such. Fortunately DataNucleus provides the following

If you have an EntityManager object and want to check if it is managing a particular object you can call

When you have relationships between entities, and you persist one entity, by default the related entity will not be persisted. For each of the relation annotations @OneToOne, @OneToMany, @ManyToOne and @ManyToMany there is an attribute cascade which defaults to null but you can control what operations cascade (persist, remove, merge, detach, refresh).

Let’s use our example above, and create new Person and Account objects.

now to persist them both we have two options. Firstly with the default cascade setting

The second option is to set the metadata on Account as

now we can simply do this (since the Account has a reference to Person);

When an element is removed from a collection, or when a 1-1 relation is nulled, sometimes it is desirable to delete the other object. JPA defines a facility of removing "orphans" by specifying metadata for a 1-1 or 1-N relation. Let’s take our example. In the above relation between Account and Person if we set the "person" field to null, this should mean that the Person record is deleted. So we could change the metadata to

So from now on, if we delete the Account we delete the Person, and if we set the "person" field of Account to null then we also delete the Person.

The power of a Java persistence solution like DataNucleus is demonstrated when persisting relationships between objects. There are many types of relationships.

Each EntityManager maintains a cache of the objects that it has encountered (or have been "enlisted") during its lifetime. This is termed the Level 1 (L1) Cache. It is enabled by default and you should only ever disable it if you really know what you are doing. There are inbuilt types for the L1 Cache available for selection. DataNucleus supports the following types of L1 Cache :-

You can specify the type of L1 Cache by providing the persistence property datanucleus.cache.level1.type. You set this to the value of the type required. If you want to remove all objects from the L1 cache programmatically you should use em.clear() but bear in mind the other things that this will impact on.

Objects are placed in the L1 Cache (and updated there) during the course of the transaction. This provides rapid access to the objects in use in the users application and is used to guarantee that there is only one object with a particular identity at any one time for that EntityManager. When the EntityManager is closed the cache is cleared.

A newly created object is transient. You then persist it and it becomes persistent. You then commit the transaction and it is detached for use elsewhere in the application, in detached state. You then attach any changes back to persistence and it becomes persistent again. Finally when you delete the object from persistence and commit that transaction it is in transient state.

So a newly created object is transient. You then persist it and it becomes persistent. You then commit the transaction and it remains managed in persistent state. When you close the EntityManager it becomes detached. Finally when you delete the object from persistence and commit that transaction it is in transient state.

When you detach an object (and its graph) either explicitly (using em.detach()) or implicitly via the PersistenceContext above, you need to be careful about which fields are detached. If you detach everything then you can end up with a huge graph that could impact on the performance of your application. On the other hand you need to ensure that you have all fields that you will be needing access to whilst detached. Should you access a field that was not detached an IllegalAccessException is thrown. All fields that are loaded will be detached so make sure you either load all required when retrieving the object using Entity Graphs or you access fields whilst attached (which will load them).

JPA provides nothing to determine the lifecycle state of an object. Fortunately DataNucleus does consider this useful, so you can call the following

When an object is detached it is often useful to know which fields are loaded/dirty. You can do this with the following helper methods

If using DataNucleus JPA in a JavaSE environment the normal type of transaction is RESOURCE_LOCAL. With this type of transaction the user manages the transactions themselves, starting, committing or rolling back the transaction. With these transactions with JPA you obtain an EntityTransaction from the EntityManager, and manage it like this

In this case you will have defined your persistence-unit to be like this

or

The basic idea with Locally-Managed transactions is that you are managing the transaction start and end.

The other type of transaction with JPA is using JTA. With this type, where you have a JTA data source from which you have a UserTransaction. This UserTransaction can have resources "joined" to it. In the case of JPA, you have two scenarios. The first scenario is where you have the UserTransaction created before you create your EntityManager. The create of the EntityManager will automatically join it to the current UserTransaction, like this

so we control the transaction using the UserTransaction.

The second scenario is where the UserTransaction is started after you have the EntityManager. In this case we need to join our EntityManager to the newly created UserTransaction, like this

In the JTA case you will have defined your persistence-unit to be like this

When using a JavaEE container you are giving over control of the transactions to the container. Here you have Container-Managed Transactions. In terms of your code, you would do like the above examples except that you would OMIT the tx.begin(), tx.commit(), tx.rollback() since the JavaEE container will be doing this for you.

When you use a framework like Spring you would not need to specify the tx.begin(), tx.commit(), tx.rollback() since that would be done for you.

DataNucleus allows the ability to operate without transactions. With JPA this is enabled by default (see the 2 properties datanucleus.transaction.nontx.read, datanucleus.transaction.nontx.write set to true, the default). This means that you can read objects and make updates outside of transactions. This is effectively an "auto-commit" mode.

When using non-transactional operations, you need to pay attention to the persistence property datanucleus.transaction.nontx.atomic. If this is true then any persist/delete/update will be committed to the datastore immediately. If this is false then any persist/delete/update will be queued up until the next transaction (or em.close()) and committed with that.

DataNucleus also allows specification of the transaction isolation level, applied at the connection level, and providing a level of isolation of this process from other processed using the same database. The isolation is specified via the persistence property datanucleus.transaction.isolation. It accepts the standard JDBC values of

If the datastore doesn’t support a particular isolation level then it will silently be changed to one that is supported. As an alternative you can also specify it on a per-transaction basis as follows

Alternatively with numeric input (using numbers in parentheses above).

Obviously transactions are intended for committing changes. If you come across a situation where you don’t want to commit anything under any circumstances you can mark the transaction as "read-only" by calling

Any call to commit on the transaction will throw an exception forcing the user to roll it back.

During a transaction, depending on the configuration, operations don’t necessarily go to the datastore immediately, often waiting until commit. In some situations you need persists/updates/deletes to be in the datastore so that subsequent operations can be performed that rely on those being handled first. In this case you can flush all outstanding changes to the datastore using

You can control the flush mode for an EntityManager using

which will only flush changes at commit. This means that when a query is performed it will not see any local changes.

The default is FlushModeType.AUTO which will flush just before any query, so that the results of all queries are consistent with local changes.

A convenient vendor extension is to find which objects are waiting to be flushed at any time, like this

Occasionally you may need to persist large amounts of data in a single transaction. Since all objects need to be present in Java memory at the same time, you can get OutOfMemory errors, or your application can slow down as swapping occurs. You can alleviate this by changing how you flush/commit the persistent changes.

One way is to do it like this, where possible,

Another way, if one object is causing the persist of a huge number of related objects, is to just persist some objects without relations first, flush, and then form the relations. This then allows the above process to be utilised, manually flushing at intervals.

You can additionally consider evicting objects from the Level 1 Cache, since they will, by default, be cached until commit.

JDBC provides the ability to specify a point in a transaction and rollback to that point if required, assuming the JDBC driver supports it. DataNucleus provides this as a vendor extension, as follows

Optimistic "locking" is suitable for longer lived operations maybe where user interaction is taking place and where it would be undesirable to block access to datastore entities for the duration of the transaction. The assumption is that data altered in this transaction will not be updated by other transactions during the duration of this transaction, so the changes are not propagated to the datastore until commit()/flush(). The obvious benefit of optimistic locking is that all changes are made in a block and version checking of objects is performed before application of changes, hence this mode copes better with external processes updating the objects.

The (version of) data is checked when data is flushed (typically at commit) to ensure the integrity in this respect. The most convenient way of checking data for updates is to maintain a column on each table that handles optimistic locking data to store a version.

Rather than placing version columns on all user datastore tables, JPA allows the user to notate particular classes as requiring optimistic treatment. This is performed by specifying in MetaData (XML or annotations) the details of the field/column to use for storing the version - see versioning. With strict JPA you must have a field in your class ready to store the version. With DataNucleus we also allow a version to be stored in a surrogate column hence not requiring a field in the actual class.

In terms of the process of optimistic locking, we demonstrate this below.

We have our flush mode set to not commit until flush/commit is called (FlushModeType.AUTO). When flush is performed (either manually, via commit, or via a query requiring it) the version check(s) are performed for any modified objects to be flushed, as long as they have a version defined. Please note that for some datastores (e.g RDBMS) the version check followed by update/delete is performed in a single statement.

See also :-

Pessimistic locking isn’t the default behaviour with JPA but can be configured. It is suitable for short lived operations where no user interaction is taking place and so it is possible to block access to datastore entities for the duration of the transaction. Such locking is best employed on specific objects, rather as a global process applying to all retrieved objects.

To disable optimistic locking (or version checking) globally you would add the persistence property datanucleus.Optimistic as false. Additionally, for RDBMS, to pessimistically lock ALL retrieved objects you would also set the persistence property datanucleus.rdbms.useUpdateLock to true.

Any object that has a pessimistic lock will result in (for RDBMS) all "SELECT …​ FROM …​" retrieval statements being changed to be "SELECT …​ FROM …​ FOR UPDATE"; this will be applied only where the underlying RDBMS supports the "FOR UPDATE" syntax.

With pessimistic locking DataNucleus will grab a datastore connection at the first operation, and maintain it for the duration of the transaction. A single connection is used for the transaction (with the exception of any Value Generation operations which need datastore access, so these can use their own connection).

The JPA EntityManager allows control over locking on an object-by-object basis with several methods. For example

will retrieve the Person object with identity 1, and will lock it until the end of the transaction.

You can additionally perform an explicit lock on a specific object like this

which will lock the object from that point in the transaction.

If you wanted to lock all objects affected by a query, you can set the lock mode of the query, like this

which will add a pessimistic lock on all Person objects with surname Smith.

In terms of the process of pessimistic (datastore) locking, we demonstrate this below. Here we have disabled the default "optimistic" check process (using datanucleus.Optimistic being set to false).

So here (due to the flush mode chosen, and due to the default optimistic locking being disabled), whenever an operation is performed, DataNucleus pushes it straight to the datastore. Consequently any queries will always reflect the current state of all objects in use. This mode of operation has no version checking of objects and so, if they were updated by external processes in the meantime then, they will overwrite those changes. This is where the locking statements for particular objects is crucial, preventing them being updated externally.

One further thing to note is that you can have optimistic locking, whilst also having pessimistic locking of specific objects. You achieve this by following the optimistic locking process above, but using find and createQuery to lock specific objects using an appropriate pessimistic LockModeType.

It should be noted that DataNucleus provides two persistence properties that allow an amount of control over when flushing happens with pessimistic locking

For pessimistic/datastore transactions a connection will be obtained from the datastore when the first persistence operation is initiated. This datastore connection will be held for the duration of the transaction until such time as either commit() or rollback() are called.

For optimistic transactions the connection is only obtained when flush()/commit() is called. When flush() is called, or the transaction committed a datastore connection is finally obtained and it is held open until commit/rollback completes. When a datastore operation is required, the connection is typically released after performing that operation. So datastore connections, in general, are held for much smaller periods of time. This is complicated slightly by use of the persistence property datanucleus.IgnoreCache. When this is set to false, the connection, once obtained, is not released until the call to commit()/rollback().

When performing non-transactional operations, the default behaviour is to obtain a connection when needed, and release it after use. With RDBMS you have the option of retaining this connection ready for the next operation to save the time needed to obtain it; this is enabled by setting the persistence property datanucleus.connection.nontx.releaseAfterUse to false.

By default the connection used for transactional and non-transactional operations will be different, potentially from a different connection factory. If you set persistence property datanucleus.connection.singleConnectionPerExecutionContext to true then the connection for both transactional and non-transactional will come from the primary factory only. In addition, any connection from a transaction will not be released after commit of the transaction, and will be used thereafter for any non-transactional operations, as well as further transactions within the same EM context.

DataNucleus provides a mechanism for users to access the native connection to the datastore, so that they can perform other operations as necessary. You obtain a connection as follows

For the datastores supported by DataNucleus, the "native" object is of the following types

When you create an EntityManagerFactory using the connection URL, driver name and the username/password to use, this doesn’t necessarily pool the connections (so they would be efficiently opened/closed when needed to utilise datastore resources in an optimum way). For some of the supported datastores DataNucleus allows you to utilise a connection pool to efficiently manage the connections to the datastore. We currently provide support for the following

You need to specify the persistence property datanucleus.connectionPoolingType to be whichever of the external pooling libraries you wish to use (or "None" if you explicitly want no pooling). DataNucleus provides two sets of connections to the datastore - one for transactional usage, and one for non-transactional usage. If you want to define a different pooling for nontransactional usage then you can also specify the persistence property datanucleus.connectionPoolingType.nontx to whichever is required.

DataNucleus allows use of a data source that represents the datastore in use. With JPA you specify this typically as the JNDI name of the datasource location. This is often just a URL defining the location of the datastore, but there are in fact several ways of specifying this data source depending on the environment in which you are running.

If you specify the persistence property datanucleus.tenantId as an identifier for your user-group/application then DataNucleus will know that it needs to provide a tenancy discriminator to all primary tables of persisted classes. This discriminator is then used to separate the data of the different user-groups.

By default this will add a column TENANT_ID to each primary table, of String-based type. You can control this by specifying extension metadata for each persistable class

or using annotations

In all subsequent use of DataNucleus, any "insert" to the primary "table"(s) will also include the TENANT column value. Additionally any query will apply a WHERE clause restricting to a particular value of TENANT column.

If you have enabled multi-tenancy as above but want to disable multitenancy on a class, just specify the following metadata

or using annotations

Note that the Tenant ID can be set in one of three ways.

Now the programmer can set a different Tenant ID for each datastore access, maybe based on some session variable for example?.

JPA provides an initial entity graph, comprising the fields that will be retrieved when an object is retrieved if the user does nothing to define the required behaviour. You define this "default" by setting the fetch attribute in metadata for each field/property.

You can predefine Named Entity Graphs in metadata which can then be used at runtime when retrieving objects from the datastore (via find/query). For example, if we have the following class

and we want to have the option of the other field loaded whenever we load objects of this class, we define our annotations as

So we have defined an EntityGraph called "includeOther" that just includes the field with name other. We can retrieve this and then use it in our persistence code, as follows

Here we have made use of the EntityManager.find method and provided the property javax.persistence.loadgraph to be our EntityGraph. This means that it will fetch all fields in the default EntityGraph, plus all fields in the includeOther EntityGraph. If we had provided the property javax.persistence.fetchgraph set to our EntityGraph it would have fetched just the fields defined in that EntityGraph.

Note that you can also make use of EntityGraphs when using the JPA Query API, specifying the same properties above but as query hints.

You can define Entity Graphs at runtime, programmatically. For example, if we have the following class

and we want to have the option of the other field loaded whenever we load objects of this class, we do the following

So we have defined an EntityGraph that just includes the field with name other. We can then use this at runtime in our persistence code, as follows

Here we have made use of the EntityManager.find method and provided the property javax.persistence.loadgraph to be our EntityGraph. This means that it will fetch all fields in the default EntityGraph, plus all fields in this EntityGraph. If we had provided the property javax.persistence.fetchgraph set to our EntityGraph it would have fetched just the fields defined in that EntityGraph.

Note that you can also make use of EntityGraphs when using the JPA Query API, specifying the same properties above but as query hints, like this

An Entity itself can have several methods defined to receive events when any instances of that class pass through lifecycles changes. Let’s take an example

So in this example just before any "Account" object is persisted the validateCreate method will be called. In the same way, just after the fields of any "Account" object are loaded the adjustPreferredStatus method is called. Very simple.

You can register callbacks for the following lifecycle events

The only other rule is that any method marked to be a callback method has to take no arguments as input, and have void return.

As an alternative to having the actual callback methods in the Entity class itself you can define a separate class as an EntityListener. So lets take the example shown before and do it for an EntityListener.

So we define our "Account" entity as normal but mark it with an EntityListener, and then in the EntityListener we define the callbacks we require. As before we can define any of the 7 callbacks as we require. The only difference is that the callback method has to take an argument of type "Object" that it will be called for, and have void return.

This guide was provided by Nicolas Seyvet. It is linked to from the JBoss docs.

JBoss AS7 is a recent JavaEE server from JBoss. Despite searching in multiple locations, I could not find a comprehensive guide on how to switch from the default JBoss Hibernate JPA provider to Datanucleus. If you try this guide, please PM the author (or add a comment) and let me know how it worked out. Your feedback will be used to improve this guide. This guide is cross-referenced as part of the JBoss JPA Reference Guide.

Apache TomEE ships with OpenJPA/EclipseLink as the default JPA provider (depending on which version of TomEE), however any valid JPA provider can be used.

The basic steps are:

In a non OSGi world the persitence provider implementation is loaded using the service provider pattern. The full qualified name of the implementation is stored in a file under META-INF/services/javax.persistence.spi.PersistenceProvider (inside the jar of the implementation) and each time the persistence provider is required it gets loaded with a Class.forName using the name of the implementing class found inside the META-INF/services/javax.persistence.spi.PersistenceProvider. In the OSGi world that doesn’t work. The bundle that needs to load the persistence provider implementation cannot load META-INF/services/javax.persistence.spi.PersistenceProvider. A work around is to copy that file inside each bundle that requires access to the peristence provider. Another work around is to export the persistence provider as OSGi service. This is what the DataNucleus JPA jar does.

Further reading available on this link

Please make use of the OSGi sample. This provides a simple example that you can build and load into such as Apache Karaf to demonstrate JPA persistence. Here we attempt to highlight the key aspects specific to OSGi in this sample.

Model classes are written in the exact same way as you would for any application.

Creation of the EMF is specified in a persistence-unit as normal except that we need to provide two overriding properties

so we have provided a class loader for the OSGi context of the application, and also specified that we want to use the OSGiPluginRegistry.

All persistence and query operations using EntityManager etc thereafter are identical to what you would use in a normal JavaSE/JavaEE application.

The pom.xml also defines the imports/exports for our OSGi application bundle, so look at this if wanting guidance on what these could look like when using Maven and the "felix bundle" plugin.

If you read the file README.txt you can see basic instructions on how to deploy this application into a fresh download of Apache Karaf, and run it. It makes uses of Spring DM to start the JPA "application".

When using DataNucleus 3.x in a Virgo 3.0.x OSGi environment, which is essentially Eclipse Equinox + Spring dm Server with Spring 3.0.5.RELEASE included, the following class is working for me to use in your Spring configuration. You can use this class as a drop-in replacement for Spring’s org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean. It was inspired by the code-ish sample at HOWTO Use Datanucleus with OSGi and Spring DM.

You should perform enhancement before runtime. That is, do not use java agent since it will enhance classes at runtime, when you want responsiveness from your application.

JPA provides properties for generating the schema at startup, and DataNucleus also provides some of its own (datanucleus.schema.autoCreateAll, datanucleus.schema.autoCreateTables, datanucleus.schema.autoCreateColumns, and datanucleus.schema.autoCreateConstraints). This can cause performance issues at startup. We recommend setting these to false at runtime, and instead using SchemaTool to generate any required database schema before running DataNucleus (for RDBMS, HBase, etc).

Where you have an inheritance tree it is best to add a discriminator to the base class so that it’s simple for DataNucleus to determine the class name for a particular row. For RDBMS : this results in cleaner/simpler SQL which is faster to execute, otherwise it would be necessary to do a UNION of all possible tables. For other datastores, a discriminator stores the key information necessary to instantiate the resultant class on retrieval so ought to be more efficient also.

DataNucleus provides 3 persistence properties (datanucleus.schema.validateTables, datanucleus.schema.validateConstraints, datanucleus.schema.validateColumns) that enforce strict validation of the datastore tables against the Meta-Data defined tables. This can cause performance issues at startup. In general this should be run only at schema generation, and should be turned off for production usage. Set all of these properties to false. In addition there is a property datanucleus.rdbms.CheckExistTablesOrViews which checks whether the tables/views that the classes map onto are present in the datastore. This should be set to false if you require fast start-up. Finally, the property datanucleus.rdbms.initializeColumnInfo determines whether the default values for columns are loaded from the database. This property should be set to NONE to avoid loading database metadata.

To sum up, the optimal settings with schema creation and validation disabled are:

Creation of EntityManagerFactory objects can be expensive and should be kept to a minimum. Depending on the structure of your application, use a single factory per datastore wherever possible. Clearly if your application spans multiple servers then this may be impractical, but should be borne in mind.

You can improve startup speed by not specifying all classes in the persistence-unit so that they are discovered at runtime. Obviously this may impact on persistence operations later if classes are not known about.

Some RDBMS (such as Oracle) have trouble returning information across multiple catalogs/schemas and so, when DataNucleus starts up and tries to obtain information about the existing tables, it can take some time. This is easily remedied by specifying the catalog/schema name to be used - either for the EMF as a whole (using the persistence properties datanucleus.Catalog, datanucleus.Schema, or using the settings in persistence.xml), or for the package/class using attributes in the MetaData. This subsequently reduces the amount of information that the RDBMS needs to search through and so can give significant speed ups when you have many catalogs/schemas being managed by the RDBMS.

Clearly the structure of your application will have a major influence on how you utilise an EntityManager. A pattern that gives a clean definition of process is to use a different persistence manager for each request to the data access layer. This reduces the risk of conflicts where one thread performs an operation and this impacts on the successful completion of an operation being performed by another thread. Creation of EM’s is not an expensive process and use of multiple threads writing to the same manager should be avoided.

Make sure that you always close the EntityManager after use. It releases all resources connected to it, and failure to do so will result in memory leaks. Also note that when closing the EntityManager if you have the persistence property datanucleus.detachOnClose set to true (when in an extended PersistenceContext) this will detach all objects in the Level1 cache. Disable this if you don’t need these objects to be detached, since it can be expensive when there are many objects.

To optimise the persistence process for performance you need to analyse what operations are performed and when, to see if there are some features that you could disable to get the persistence you require and omit what is not required. If you think of a typical transaction, the following describes the process

DataNucleus, by default, will allocate connections when they are required. It then will close the connection.

In addition, when it needs to perform something via JDBC (RDBMS datastores) it will allocate a PreparedStatement, and then discard the statement after use. This can be inefficient relative to a database connection and statement pooling facility such as Apache DBCP. With Apache DBCP a Connection is allocated when required and then when it is closed the Connection isn’t actually closed but just saved in a pool for the next request that comes in for a Connection. This saves the time taken to establish a Connection and hence can give performance speed ups the order of maybe 30% or more. You can read about how to enable connection pooling with DataNucleus in the Connection Pooling Guide.

As an addendum to the above, you could also turn on caching of PreparedStatements. This can also give a performance boost, depending on your persistence code, the JDBC driver and the SQL being issued. Look at the persistence property datanucleus.connectionPool.maxStatements.

If you are retrieving an object by its identity and know that it will be present in the Level2 cache, for example, you can set the persistence property datanucleus.findObject.validateWhenCached to false and this will skip a separate call to the datastore to validate that the object exists in the datastore.

DataNucleus provides a series of value generators for generation of identity values. These can have an impact on the performance depending on the choice of generator, and also on the configuration of the generator.

The AUTO identity generator value is the recommended choice since this will allow DataNucleus to decide which identity generator is best for the datastore in use.

DataNucleus has 2 ways of handling calls to SCO Collections/Maps. The original method was to pass all calls through to the datastore. The second method (which is now the default) is to cache the collection/map elements/keys/values. This second method will read the elements/keys/values once only and thereafter use the internally cached values. This second method gives significant performance gains relative to the original method. You can configure the handling of collections/maps as follows :-

The second method also allows a finer degree of control. This allows the use of lazy loading of data, hence elements will only be loaded if they are needed. You can configure this as follows :-

Performing non-transactional reads has advantages and disadvantages in performance and data freshness in cache. The objects read are held cached by the EntityManager. The second time an application requests the same objects from the EntityManager they are retrieved from cache. The time spent reading the object from cache is minimum, but the objects may become stale and not represent the database status. If fresh values need to be loaded from the database, then the user application should first call refresh on the object.

Another disadvantage of performing non-transactional reads is that each operation realized opens a new database connection, but it can be minimized with the use of connection pools, and also on some of the datastore the (nontransactional) connection is retained.

Reading fields of unmanaged objects (outside the scope of an EntityManager) is a trivial task, but performed in a certain manner can determine the application performance. The objective here is not give you an absolute response on the subject, but point out the benefits and drawbacks for the many possible solutions.

When fetching objects you have control over what gets fetched. This can have an impact if you are then detaching those objects. With JPA the maximum fetch depth is -1 (unlimited). So with JPA you ought to set it to the extent that you want to detach, or better still make use of JPA Entity Graphs to control the specific fields to detach.

I/O consumes a huge slice of the total processing time. Therefore it is recommended to reduce or disable logging in production. To disable the logging set the DataNucleus category to OFF in the Log4j configuration. See Logging for more information.

In most applications, the performance of the persistence layer is very unlikely to be a bottleneck. More likely the design of the datastore itself, and in particular its indices are more likely to have the most impact, or alternatively network latency. That said, it is the DataNucleus projects' committed aim to provide the best performance possible, though we also want to provide functionality, so there is a compromise with respect to resource.

A benchmark is defined as "a series of persistence operations performing particular things e.g persist n objects, or retrieve n objects". If those operations are representative of your application then the benchmark is valid to you.

To find (or create) a benchmark appropriate to your project you need to determine the typical persistence operations that your application will perform. Are you interested in persisting 100 objects at once, or 1 million, for example? Then when you have a benchmark appropriate for that operation, compare the persistence solutions.

The performance tuning guide above gives a good oversight of tuning capabilities, and also refer to the following blog entry for our take on performance of DataNucleus AccessPlatform. And then the later blog entry about how to tune for bulk operations

The simplest way to monitor DataNucleus is to use its API for monitoring. Internally there are several MBeans (as used by JMX) and you can navigate to these to get the required information. To enable this set the persistence property datanucleus.enableStatistics to true. There are then two sets of statistics; one for the EMF and one for each EM. You access these as follows

The MBeans used by DataNucleus can be accessed via JMX at runtime. More about JMX here.

An MBean server is bundled with Sun/Oracle JRE since Java5, and you can easily activate DataNucleus MBeans registration by creating your EMF with the persistence property datanucleus.jmxType as default

Additionally, setting a few system properties are necessary for configuring the Sun JMX implementation. The minimum properties required are the following:

Usage example:

Once you start your application and DataNucleus is initialized you can browse DataNucleus MBeans using a tool called jconsole (jconsole is distributed with the Sun JDK) via the URL:

Note that the mode of usage is presented in this document as matter of example, and by no means we recommend to disable authentication and secured communication channels. Further details on the Sun JMX implementation and how to configure it properly can be found here.

DataNucleus MBeans are registered in a MBean Server when DataNucleus is started up (e.g. upon JPA EMF instantiation). To see the full list of DataNucleus MBeans, refer to the javadocs.

To enable management using MX4J you must specify the persistence property datanucleus.jmxType as mx4j when creating the EMF, and have the mx4j and mx4j-tools jars in the CLASSPATH.

DataNucleus uses a series of categories, and logs all messages to these categories. Currently DataNucleus uses the following

Log4J allows logging messages at various severity levels. The levels used by Log4J, and by DataNucleus’s use of Log4J are DEBUG, INFO, WARN, ERROR, FATAL. Each message is logged at a particular level to a category (as described above). The other setting is OFF which turns off a logging category; very useful in a production situation where maximum performance is required.

To enable the DataNucleus log, you need to provide a Log4J configuration file when starting up your application. This may be done for you if you are running within a JavaEE application server (check your manual for details). If you are starting your application yourself, you would set a JVM parameter as

where log4j.properties is the name of your Log4J configuration file. Please note the file: prefix to the file since a URL is expected.

The Log4J configuration file is very simple in nature, and you typically define where the log goes to (e.g to a file), and which logging level messages you want to see. Here’s an example

In this example, I am directing my log to a file (datanucleus.log). I have defined a particular "pattern" for the messages that appear in the log (to contain the date, level, category, and the message itself). In addition I have assigned a level "threshold" for each of the DataNucleus categories. So in this case I want to see all messages down to DEBUG level for the DataNucleus RDBMS persister.

java.util.logging allows logging messages at various severity levels. The levels used by java.util.logging, and by DataNucleus’s internally are fine, info, warn, severe. Each message is logged at a particular level to a category (as described above).

By default, the java.util.logging configuration is taken from a properties file <JRE_DIRECTORY>/lib/logging.properties. Modify this file and configure the categories to be logged, or use the java.util.logging.config.file system property to specify a properties file (in java.util.Properties format) where the logging configuration will be read from. Here is an example:

Please read the javadocs for java.util.logging for additional details on its configuration.

Here is a sample of the type of information you may see in the DataNucleus log when using Log4J.

So you see the time of the log message, the level of the message (DEBUG, INFO, etc), the category (DataNucleus.Datastore, etc), and the message itself. For example, if I had set the DataNucleus.Datastore.Schema to DEBUG and all other categories to INFO I would see all DDL statements sent to the database and very little else.

This guide was provided by Marco Lopes, when using DataNucleus v2.2. All of the bundles which use log4j should have org.apache.log4j in their Import-Package attribute! (use: org.apache.log4j;resolution:=optional if you don’t want to be stuck with log4j whenever you use an edited bundle in your project!).