com.sleepycat.persist
Interface EntityIndex<K,V>

All Known Implementing Classes:

PrimaryIndex, SecondaryIndex

public interface EntityIndex<K,V>

The interface for accessing keys and entities via a primary or secondary index.

EntityIndex objects are thread-safe. Multiple threads may safely call the methods of a shared EntityIndex object.

An index is conceptually a map. {key:value} mappings are stored in the index and accessed by key. In fact, for interoperability with other libraries that use the standard Java Map or SortedMap interfaces, an EntityIndex may be accessed via these standard interfaces by calling the map() or sortedMap() methods.

EntityIndex is an interface that is implemented by several classes in this package for different purposes. Depending on the context, the key type (K) and value type (V) of the index take on different meanings. The different classes that implement EntityIndex are:

• PrimaryIndex maps primary keys to entities.
• SecondaryIndex maps secondary keys to entities.
• SecondaryIndex.keysIndex maps secondary keys to primary keys.
• SecondaryIndex.subIndex(SK) maps primary keys to entities, for the subset of entities having a specified secondary key.

In all cases, the index key type (K) is a primary or secondary key class. The index value type (V) is an entity class in all cases except for a SecondaryIndex.keysIndex, when it is a primary key class.

In the following example, a Employee entity with a MANY_TO_ONE secondary key is defined.

 @Entity
 class Employee {

     @PrimaryKey
     long id;

     @SecondaryKey(relate=MANY_TO_ONE)
     String department;

     String name;

     private Employee() {}
 }

Consider that we have stored the entities below:

PrimaryIndex maps primary keys to entities:

 PrimaryIndex<Long, Employee> primaryIndex =
     store.getPrimaryIndex(Long.class, Employee.class);

SecondaryIndex maps secondary keys to entities:

 SecondaryIndex<String, Long, Employee> secondaryIndex =
     store.getSecondaryIndex(primaryIndex, String.class, "department");

SecondaryIndex.keysIndex maps secondary keys to primary keys:

 EntityIndex<String, Long> keysIndex = secondaryIndex.keysIndex();

SecondaryIndex.subIndex(SK) maps primary keys to entities, for the subset of entities having a specified secondary key:

 EntityIndex<Long, Entity> subIndex = secondaryIndex.subIndex("Engineering");

Accessing the Index

An EntityIndex provides a variety of methods for retrieving entities from an index. It also provides methods for deleting entities. However, it does not provide methods for inserting and updating. To insert and update entities, use the PrimaryIndex.put(E) family of methods in the PrimaryIndex class.

An EntityIndex supports two mechanisms for retrieving entities:

1. The get(K) method returns a single value for a given key. If there are multiple values with the same secondary key (duplicates), it returns the first entity in the duplicate set.
2. An EntityCursor can be obtained using the keys() and entities() family of methods. A cursor can be used to return all values in the index, including duplicates. A cursor can also be used to return values within a specified range of keys.

Using the example entities above, calling get(K) on the primary index will always return the employee with the given ID, or null if no such ID exists. But calling get(K) on the secondary index will retrieve the first employee in the given department, which may not be very useful:

 Employee emp = primaryIndex.get(1);      // Returns by unique ID
 emp = secondaryIndex.get("Engineering"); // Returns first in department

Using a cursor, you can iterate through all duplicates in the secondary index:

 EntityCursor<Employee> cursor = secondaryIndex.entities();
 try {
     for (Employee entity : cursor) {
         if (entity.department.equals("Engineering")) {
             // Do something with the entity...
         }
     }
 } finally {
     cursor.close();
 }

But for a large database it is much more efficient to iterate over only those entities with the secondary key you're searching for. This could be done by restricting a cursor to a range of keys:

 EntityCursor<Employee> cursor =
     secondaryIndex.entities("Engineering", true, "Engineering", true);
 try {
     for (Employee entity : cursor) {
         // Do something with the entity...
     }
 } finally {
     cursor.close();
 }

However, when you are interested only in the entities with a particular secondary key value, it is more convenient to use a sub-index:

 EntityIndex<Long, Entity> subIndex = secondaryIndex.subIndex("Engineering");
 EntityCursor<Employee> cursor = subIndex.entities();
 try {
     for (Employee entity : cursor) {
         // Do something with the entity...
     }
 } finally {
     cursor.close();
 }

In addition to being more convenient than a cursor range, a sub-index allows retrieving by primary key:

 Employee emp = subIndex.get(1);

When using a sub-index, all operations performed on the sub-index are restricted to the single key that was specified when the sub-index was created. For example, the following returns null because employee 2 is not in the Engineering department and therefore is not part of the sub-index:

 Employee emp = subIndex.get(2);

For more information on using cursors and cursor ranges, see EntityCursor.

Note that when using an index, keys and values are stored and retrieved by value not by reference. In other words, if an entity object is stored and then retrieved, or retrieved twice, each object will be a separate instance. For example, in the code below the assertion will always fail.

 MyKey key = ...;
 MyEntity entity1 = index.get(key);
 MyEntity entity2 = index.get(key);
 assert entity1 == entity2; // always fails!
 

Deleting from the Index

Any type of index may be used to delete entities with a specified key by calling delete(K). The important thing to keep in mind is that all entities with the specified key are deleted. In a primary index, at most a single entity is deleted:

 primaryIndex.delete(1); // Deletes a single employee by unique ID

But in a secondary index, multiple entities may be deleted:

 secondaryIndex.delete("Engineering"); // Deletes all Engineering employees

This begs this question: How can a single entity be deleted without knowing its primary key? The answer is to use cursors. After locating an entity using a cursor, the entity can be deleted by calling EntityCursor.delete().

Transactions

Transactions can be used to provide standard ACID (Atomicity, Consistency, Integrity and Durability) guarantees when retrieving, storing and deleting entities. This section provides a brief overview of how to use transactions with the Direct Persistence Layer. For more information on using transactions, see Writing Transactional Applications.

Transactions may be used only with a transactional EntityStore, which is one for which StoreConfig.setTransactional(true) has been called. Likewise, a transactional store may only be used with a transactional Environment, which is one for which EnvironmentConfig.setTransactional(true) has been called. For example:

 EnvironmentConfig envConfig = new EnvironmentConfig();
 envConfig.setTransactional(true);
 envConfig.setAllowCreate(true);
 Environment env = new Environment(new File("/my/data"), envConfig);

 StoreConfig storeConfig = new StoreConfig();
 storeConfig.setTransactional(true);
 storeConfig.setAllowCreate(true);
 EntityStore store = new EntityStore(env, "myStore", storeConfig);

Transactions are represented by Transaction objects, which are part of the Base API. Transactions are created using the Environment.beginTransaction method.

A transaction will include all operations for which the transaction object is passed as a method argument. All retrieval, storage and deletion methods have an optional Transaction parameter for this purpose. When a transaction is passed to a method that opens a cursor, all retrieval, storage and deletion operations performed using that cursor will be included in the transaction.

A transaction may be committed by calling Transaction.commit() or aborted by calling Transaction.abort(). For example, two employees may be deleted atomically with a transaction; other words, either both are deleted or neither is deleted:

 Transaction txn = env.beginTransaction(null, null);
 try {
     primaryIndex.delete(txn, 1);
     primaryIndex.delete(txn, 2);
     txn.commit();
     txn = null;
 } finally {
     if (txn != null) {
         txn.abort();
     }
 }

WARNING: Transactions must always be committed or aborted to prevent resource leaks which could lead to the index becoming unusable or cause an OutOfMemoryError. To ensure that a transaction is aborted in the face of exceptions, call Transaction.abort() in a finally block.

For a transactional store, storage and deletion operations are always transaction protected, whether or not a transaction is explicitly used. A null transaction argument means to perform the operation using auto-commit, or the implied thread transaction if an XAEnvironment is being used. A transaction is automatically started as part of the operation and is automatically committed if the operation completes successfully. The transaction is automatically aborted if an exception occurs during the operation, and the exception is re-thrown to the caller. For example, each employee is deleted using a an auto-commit transaction below, but it is possible that employee 1 will be deleted and employee 2 will not be deleted, if an error or crash occurs while deleting employee 2:

 primaryIndex.delete(null, 1);
 primaryIndex.delete(null, 2);

When retrieving entities, a null transaction argument means to perform the operation non-transactionally. The operation is performed outside the scope of any transaction, without providing transactional ACID guarantees. If an implied thread transaction is present (i.e. if an XAEnvironment is being used), that transaction is used. When a non-transactional store is used, transactional ACID guarantees are also not provided.

For non-transactional and auto-commit usage, overloaded signatures for retrieval, storage and deletion methods are provided to avoid having to pass a null transaction argument. For example, delete(K) may be called instead of delete(Transaction,Object). For example, the following code is equivalent to the code above where null was passed for the transaction:

 primaryIndex.delete(1);
 primaryIndex.delete(2);

For retrieval methods the overloaded signatures also include an optional LockMode parameter, and overloaded signatures for opening cursors include an optional CursorConfig parameter. These parameters are described further below in the Locking and Lock Modes section.

Transactions and Cursors

There are two special consideration when using cursors with transactions. First, for a transactional store, a non-null transaction must be passed to methods that open a cursor if that cursor will be used to delete or update entities. Cursors do not perform auto-commit when a null transaction is explicitly passed or implied by the method signature. For example, the following code will throw DatabaseException when the EntityCursor.delete() method is called:

 // Does not work with a transactional store!
 EntityCursor<Employee> cursor = primaryIndex.entities();
 try {
     for (Employee entity : cursor) {
         cursor.delete(); // Will throw DatabaseException.
     }
 } finally {
     cursor.close();
 }

Instead, the entities(Transaction,CursorConfig) signature must be used and a non-null transaction must be passed:

 EntityCursor<Employee> cursor = primaryIndex.entities(txn, null);
 try {
     for (Employee entity : cursor) {
         cursor.delete();
     }
 } finally {
     cursor.close();
 }

The second consideration is that error handling is more complex when using both transactions and cursors, for the following reasons:

1. When an exception occurs, the transaction should be aborted.
2. Cursors must be closed whether or not an exception occurs.
3. Cursors must be closed before committing or aborting the transaction.

For example:

 Transaction txn = env.beginTransaction(null, null);
 EntityCursor<Employee> cursor = null;
 try {
     cursor = primaryIndex.entities(txn, null);
     for (Employee entity : cursor) {
         cursor.delete();
     }
     cursor.close();
     cursor = null;
     txn.commit();
     txn = null;
 } finally {
     if (cursor != null) {
         cursor.close();
     }
     if (txn != null) {
         txn.abort();
     }
 }

Locking and Lock Modes

This section provides a brief overview of locking and describes how lock modes are used with the Direct Persistence Layer. For more information on locking, see Writing Transactional Applications.

When using transactions, locks are normally acquired on each entity that is retrieved or stored. The locks are used to isolate one transaction from another. Locks are normally released only when the transaction is committed or aborted.

When not using transactions, locks are also normally acquired on each entity that is retrieved or stored. However, these locks are released when the operation is complete. When using cursors, in order to provide cursor stability locks are held until the cursor is moved to a different entity or closed.

This default locking behavior provides full transactional ACID guarantees and cursor stability. However, application performance can sometimes be improved by compromising these guarantees. As described in Writing Transactional Applications, the LockMode and CursorConfig parameters are two of the mechanisms that can be used to make compromises.

For example, imagine that you need an approximate count of all entities matching certain criterion, and it is acceptable for entities to be changed by other threads or other transactions while performing this query. LockMode.READ_UNCOMMITTED can be used to perform the retrievals without acquiring any locks. This reduces memory consumption, does less processing, and improves concurrency.

 EntityCursor<Employee> cursor = primaryIndex.entities(txn, null);
 try {
     Employee entity;
     while ((entity = cursor.next(LockMode.READ_UNCOMMITTED)) != null) {
         // Examine the entity and accumulate totals...
     }
 } finally {
     cursor.close();
 }

The LockMode parameter specifies locking behavior on a per-operation basis. If null or LockMode.DEFAULT is specified, the default lock mode is used.

It is also possible to specify the default locking behavior for a cursor using CursorConfig. The example below is equivalent to the example above:

 CursorConfig config = new CursorConfig();
 config.setReadUncommitted(true);
 EntityCursor<Employee> cursor = primaryIndex.entities(txn, config);
 try {
     Employee entity;
     while ((entity = cursor.next()) != null) {
         // Examine the entity and accumulate totals...
     }
 } finally {
     cursor.close();
 }

The use of other lock modes, cursor configuration, and transaction configuration are discussed in Writing Transactional Applications.

Lock conflict handling is another important topic discussed in Writing Transactional Applications. To go along with that material, here we show a lock conflict handling loop in the context of the Direct Persistence Layer. The example below shows deleting all entities in a primary index in a single transaction. If a lock conflict occurs, the transaction is aborted and the operation is retried.

  void doTransaction(final Environment env,
                     final PrimaryIndex<Long, Employee> primaryIndex,
                     final int maxTries)
      throws DatabaseException {

      boolean success = false;
      long sleepMillis = 0;
      for (int i = 0; i < maxTries; i++) {
          // Sleep before retrying.
          if (sleepMillis != 0) {
              Thread.sleep(sleepMillis);
              sleepMillis = 0;
          }
          Transaction txn = null;
          try {
              txn = env.beginTransaction(null, null);
              final EntityCursor<Employee> cursor =
                  primaryIndex.entities(txn, null);
              try {
                  // INSERT APP-SPECIFIC CODE HERE:
                  // Perform read and write operations, for example:
                  for (Employee entity : cursor) {
                      cursor.delete();
                  }
              } finally {
                  cursor.close();
              }
              txn.commit();
              success = true;
              return;
          } catch (DeadlockException e) {
              sleepMillis = LOCK_CONFLICT_RETRY_SEC * 1000;
              continue;
          } finally {
              if (!success) {
                  if (txn != null) {
                      txn.abort();
                  }
              }
          }
      }
      // INSERT APP-SPECIFIC CODE HERE:
      // Transaction failed, despite retries.
      // Take some app-specific course of action.
  }

Low Level Access

Each Direct Persistence Layer index is associated with an underlying Database or SecondaryDatabase defined in the Base API. At this level, an index is a Btree managed by the Berkeley DB Java Edition transactional storage engine. Although you may never need to work at the Base API level, keep in mind that some types of performance tuning can be done by configuring the underlying databases. See the EntityStore class for more information on database and sequence configuration.

If you wish to access an index using the Base API, you may call the PrimaryIndex.getDatabase() or SecondaryIndex.getDatabase() method to get the underlying database. To translate between entity or key objects and DatabaseEntry objects at this level, use the bindings returned by PrimaryIndex.getEntityBinding(), PrimaryIndex.getKeyBinding(), and SecondaryIndex.getKeyBinding().

contains

boolean contains(K key)
                 throws DatabaseException

Checks for existence of a key in this index.

The operation will not be transaction protected, and LockMode.DEFAULT is used implicitly.

Parameters:

key - the key to search for.

Returns:

whether the key exists in the index.

Throws:

DatabaseException - the base class for all BDB exceptions.

contains

boolean contains(Transaction txn,
                 K key,
                 LockMode lockMode)
                 throws DatabaseException

Checks for existence of a key in this index.

Parameters:

txn - the transaction used to protect this operation, or null if the operation should not be transaction protected.

key - the key to search for.

lockMode - the lock mode to use for this operation, or null to use LockMode.DEFAULT.

Returns:

whether the key exists in the index.

Throws:

DatabaseException - the base class for all BDB exceptions.

get

V get(K key)
      throws DatabaseException

Gets an entity via a key of this index.

The operation will not be transaction protected, and LockMode.DEFAULT is used implicitly.

Parameters:

key - the key to search for.

Returns:

the value mapped to the given key, or null if the key is not present in the index.

Throws:

DatabaseException - the base class for all BDB exceptions.

get

V get(Transaction txn,
      K key,
      LockMode lockMode)
      throws DatabaseException

Gets an entity via a key of this index.

Parameters:

txn - the transaction used to protect this operation, or null if the operation should not be transaction protected.

key - the key to search for.

lockMode - the lock mode to use for this operation, or null to use LockMode.DEFAULT.

Returns:

the value mapped to the given key, or null if the key is not present in the index.

Throws:

DatabaseException - the base class for all BDB exceptions.

count

long count()
           throws DatabaseException

Returns a non-transactional count of the entities in this index.

This operation is faster than obtaining a count by scanning the index manually, and will not perturb the current contents of the cache. However, the count is not guaranteed to be accurate if there are concurrent updates. Note that this method does scan a significant portion of the index and should be considered a fairly expensive operation.

Returns:

the number of entities in this index.

Throws:

DatabaseException - the base class for all BDB exceptions.

delete

boolean delete(K key)
               throws DatabaseException

Deletes all entities with a given index key.

Auto-commit is used implicitly if the store is transactional.

Parameters:

key - the key to search for.

Returns:

whether any entities were deleted.

Throws:

DatabaseException - the base class for all BDB exceptions.

delete

boolean delete(Transaction txn,
               K key)
               throws DatabaseException

Deletes all entities with a given index key.

Parameters:

txn - the transaction used to protect this operation, null to use auto-commit, or null if the store is non-transactional.

key - the key to search for.

Returns:

whether any entities were deleted.

Throws:

DatabaseException - the base class for all BDB exceptions.

keys

EntityCursor<K> keys()
                     throws DatabaseException

Opens a cursor for traversing all keys in this index.

The operations performed with the cursor will not be transaction protected, and CursorConfig.DEFAULT is used implicitly. If the store is transactional, the cursor may not be used to update or delete entities.

Returns:

the cursor.

Throws:

DatabaseException - the base class for all BDB exceptions.

keys

EntityCursor<K> keys(Transaction txn,
                     CursorConfig config)
                     throws DatabaseException

Opens a cursor for traversing all keys in this index.

Parameters:

txn - the transaction used to protect all operations performed with the cursor, or null if the operations should not be transaction protected. If the store is non-transactional, null must be specified. For a transactional store the transaction is optional for read-only access and required for read-write access.

config - the cursor configuration that determines the default lock mode used for all cursor operations, or null to implicitly use CursorConfig.DEFAULT.

Returns:

the cursor.

Throws:

DatabaseException - the base class for all BDB exceptions.

entities

EntityCursor<V> entities()
                         throws DatabaseException

Opens a cursor for traversing all entities in this index.

The operations performed with the cursor will not be transaction protected, and CursorConfig.DEFAULT is used implicitly. If the store is transactional, the cursor may not be used to update or delete entities.

Returns:

the cursor.

Throws:

DatabaseException - the base class for all BDB exceptions.

entities

EntityCursor<V> entities(Transaction txn,
                         CursorConfig config)
                         throws DatabaseException

Opens a cursor for traversing all entities in this index.

Parameters:

txn - the transaction used to protect all operations performed with the cursor, or null if the operations should not be transaction protected. If the store is non-transactional, null must be specified. For a transactional store the transaction is optional for read-only access and required for read-write access.

config - the cursor configuration that determines the default lock mode used for all cursor operations, or null to implicitly use CursorConfig.DEFAULT.

Returns:

the cursor.

Throws:

DatabaseException - the base class for all BDB exceptions.

keys

EntityCursor<K> keys(K fromKey,
                     boolean fromInclusive,
                     K toKey,
                     boolean toInclusive)
                     throws DatabaseException

Opens a cursor for traversing keys in a key range.

The operations performed with the cursor will not be transaction protected, and CursorConfig.DEFAULT is used implicitly. If the store is transactional, the cursor may not be used to update or delete entities.

Parameters:

fromKey - is the lower bound of the key range, or null if the range has no lower bound.

fromInclusive - is true if keys greater than or equal to fromKey should be included in the key range, or false if only keys greater than fromKey should be included.

toKey - is the upper bound of the key range, or null if the range has no upper bound.

toInclusive - is true if keys less than or equal to toKey should be included in the key range, or false if only keys less than toKey should be included.

Returns:

the cursor.

Throws:

DatabaseException - the base class for all BDB exceptions.

keys

EntityCursor<K> keys(Transaction txn,
                     K fromKey,
                     boolean fromInclusive,
                     K toKey,
                     boolean toInclusive,
                     CursorConfig config)
                     throws DatabaseException

Opens a cursor for traversing keys in a key range.

Parameters:

txn - the transaction used to protect all operations performed with the cursor, or null if the operations should not be transaction protected. If the store is non-transactional, null must be specified. For a transactional store the transaction is optional for read-only access and required for read-write access.

fromKey - is the lower bound of the key range, or null if the range has no lower bound.

fromInclusive - is true if keys greater than or equal to fromKey should be included in the key range, or false if only keys greater than fromKey should be included.

toKey - is the upper bound of the key range, or null if the range has no upper bound.

toInclusive - is true if keys less than or equal to toKey should be included in the key range, or false if only keys less than toKey should be included.

config - the cursor configuration that determines the default lock mode used for all cursor operations, or null to implicitly use CursorConfig.DEFAULT.

Returns:

the cursor.

Throws:

DatabaseException - the base class for all BDB exceptions.

entities

EntityCursor<V> entities(K fromKey,
                         boolean fromInclusive,
                         K toKey,
                         boolean toInclusive)
                         throws DatabaseException

Opens a cursor for traversing entities in a key range.

The operations performed with the cursor will not be transaction protected, and CursorConfig.DEFAULT is used implicitly. If the store is transactional, the cursor may not be used to update or delete entities.

Parameters:

fromKey - is the lower bound of the key range, or null if the range has no lower bound.

fromInclusive - is true if keys greater than or equal to fromKey should be included in the key range, or false if only keys greater than fromKey should be included.

toKey - is the upper bound of the key range, or null if the range has no upper bound.

toInclusive - is true if keys less than or equal to toKey should be included in the key range, or false if only keys less than toKey should be included.

Returns:

the cursor.

Throws:

DatabaseException - the base class for all BDB exceptions.

entities

EntityCursor<V> entities(Transaction txn,
                         K fromKey,
                         boolean fromInclusive,
                         K toKey,
                         boolean toInclusive,
                         CursorConfig config)
                         throws DatabaseException

Opens a cursor for traversing entities in a key range.

Parameters:

txn - the transaction used to protect all operations performed with the cursor, or null if the operations should not be transaction protected. If the store is non-transactional, null must be specified. For a transactional store the transaction is optional for read-only access and required for read-write access.

fromKey - is the lower bound of the key range, or null if the range has no lower bound.

fromInclusive - is true if keys greater than or equal to fromKey should be included in the key range, or false if only keys greater than fromKey should be included.

toKey - is the upper bound of the key range, or null if the range has no upper bound.

toInclusive - is true if keys less than or equal to toKey should be included in the key range, or false if only keys less than toKey should be included.

config - the cursor configuration that determines the default lock mode used for all cursor operations, or null to implicitly use CursorConfig.DEFAULT.

Returns:

the cursor.

Throws:

DatabaseException - the base class for all BDB exceptions.

map

Map<K,V> map()

Returns a standard Java map based on this entity index. The StoredMap returned is defined by the Collections API. Stored collections conform to the standard Java collections framework interface.

Returns:

the map.

sortedMap

SortedMap<K,V> sortedMap()

Returns a standard Java sorted map based on this entity index. The StoredSortedMap returned is defined by the Collections API. Stored collections conform to the standard Java collections framework interface.

Returns:

the map.