casacore
Public Member Functions | Private Member Functions | List of all members
casacore::StandardStMan Class Reference

More...

#include <StandardStMan.h>

Public Member Functions

 StandardStMan (Int bucketSize=0, uInt cacheSize=1)
 Create a Standard storage manager with the given name. More...
 
 StandardStMan (const String &dataManagerName, Int bucketSize=0, uInt cacheSize=1)
 
 ~StandardStMan ()
 
- Public Member Functions inherited from casacore::SSMBase
 SSMBase (Int aBucketSize=0, uInt aCacheSize=1)
 Create a Standard storage manager with default name SSM. More...
 
 SSMBase (const String &aDataManName, Int aBucketSize=0, uInt aCacheSize=1)
 Create a Standard storage manager with the given name. More...
 
 SSMBase (const String &aDataManName, const Record &spec)
 Create a Standard storage manager with the given name. More...
 
 ~SSMBase ()
 
virtual DataManagerclone () const
 Clone this object. More...
 
virtual String dataManagerType () const
 Get the type name of the data manager (i.e. More...
 
virtual String dataManagerName () const
 Get the name given to the storage manager (in the constructor). More...
 
virtual Record dataManagerSpec () const
 Record a record containing data manager specifications. More...
 
virtual Record getProperties () const
 Get data manager properties that can be modified. More...
 
virtual void setProperties (const Record &spec)
 Modify data manager properties. More...
 
uInt getVersion () const
 Get the version of the class. More...
 
void setCacheSize (uInt aCacheSize, Bool canExceedNrBuckets=True)
 Set the cache size (in buckets). More...
 
uInt getCacheSize () const
 Get the current cache size (in buckets). More...
 
void clearCache ()
 Clear the cache used by this storage manager. More...
 
virtual void showCacheStatistics (ostream &anOs) const
 Show the statistics of all caches used. More...
 
void showIndexStatistics (ostream &anOs) const
 Show statistics of all indices used. More...
 
void showBaseStatistics (ostream &anOs) const
 Show statistics of the Base offsets/index etc. More...
 
uInt getBucketSize () const
 Get the bucket size. More...
 
rownr_t getNRow () const
 Get the number of rows in this storage manager. More...
 
virtual Bool canAddRow () const
 The storage manager can add rows. More...
 
virtual Bool canRemoveRow () const
 The storage manager can delete rows. More...
 
virtual Bool canAddColumn () const
 The storage manager can add columns. More...
 
virtual Bool canRemoveColumn () const
 The storage manager can delete columns. More...
 
SSMColumngetColumn (uInt aColNr)
 Get access to the given column. More...
 
SSMIndexgetIndex (uInt anIdxNr)
 Get access to the given Index. More...
 
void setBucketDirty ()
 Make the current bucket in the cache dirty (i.e. More...
 
StManArrayFileopenArrayFile (ByteIO::OpenOption anOpt)
 Open (if needed) the file for indirect arrays with the given mode. More...
 
char * find (rownr_t aRowNr, uInt aColNr, rownr_t &aStartRow, rownr_t &anEndRow, const String &colName)
 Find the bucket containing the column and row and return the pointer to the beginning of the column data in that bucket. More...
 
uInt getNewBucket ()
 Add a new bucket and get its bucket number. More...
 
char * getBucket (uInt aBucketNr)
 Read the bucket (if needed) and return the pointer to it. More...
 
void removeBucket (uInt aBucketNr)
 Remove a bucket from the bucket cache. More...
 
uInt getRowsPerBucket (uInt aColumn) const
 Get rows per bucket for the given column. More...
 
SSMStringHandlergetStringHandler ()
 Return a pointer to the (one and only) StringHandler object. More...
 
- Public Member Functions inherited from casacore::DataManager
 DataManager ()
 Default constructor. More...
 
virtual ~DataManager ()
 
void dataManagerInfo (Record &info) const
 Add SEQNR and SPEC (the DataManagerSpec subrecord) to the info. More...
 
virtual Bool isStorageManager () const
 Is the data manager a storage manager? The default is yes. More...
 
virtual Bool canReallocateColumns () const
 Tell if the data manager wants to reallocate the data manager column objects. More...
 
virtual DataManagerColumnreallocateColumn (DataManagerColumn *column)
 Reallocate the column object if it is part of this data manager. More...
 
uInt sequenceNr () const
 Get the (unique) sequence nr of this data manager. More...
 
uInt ncolumn () const
 Get the nr of columns in this data manager (can be zero). More...
 
Bool asBigEndian () const
 Have the data to be stored in big or little endian canonical format? More...
 
const TSMOptiontsmOption () const
 Get the TSM option. More...
 
MultiFileBasemultiFile ()
 Get the MultiFile pointer (can be 0). More...
 
String keywordName (const String &keyword) const
 Compose a keyword name from the given keyword appended with the sequence number (e.g. More...
 
String fileName () const
 Compose a unique filename from the table name and sequence number. More...
 
ByteIO::OpenOption fileOption () const
 Get the AipsIO option of the underlying file. More...
 
virtual Bool isRegular () const
 Is this a regular storage manager? It is regular if it allows addition of rows and writing data in them. More...
 
Tabletable () const
 Get the table this object is associated with. More...
 
virtual Bool canRenameColumn () const
 Does the data manager allow to rename columns? (default yes) More...
 
virtual void setMaximumCacheSize (uInt nMiB)
 Set the maximum cache size (in bytes) to be used by a storage manager. More...
 
virtual void showCacheStatistics (std::ostream &) const
 Show the data manager's IO statistics. More...
 
DataManagerColumncreateScalarColumn (const String &columnName, int dataType, const String &dataTypeId)
 Create a column in the data manager on behalf of a table column. More...
 
DataManagerColumncreateDirArrColumn (const String &columnName, int dataType, const String &dataTypeId)
 Create a direct array column. More...
 
DataManagerColumncreateIndArrColumn (const String &columnName, int dataType, const String &dataTypeId)
 Create an indirect array column. More...
 
DataManagergetClone () const
 Has the object already been cloned? More...
 
void setClone (DataManager *clone) const
 Set the pointer to the clone. More...
 

Private Member Functions

 StandardStMan (const StandardStMan &that)
 Copy constructor cannot be used. More...
 
StandardStManoperator= (const StandardStMan &that)
 Assignment cannot be used. More...
 

Additional Inherited Members

- Static Public Member Functions inherited from casacore::SSMBase
static DataManagermakeObject (const String &aDataManType, const Record &spec)
 Make the object from the type name string. More...
 
static char * readCallBack (void *anOwner, const char *aBucketStorage)
 Callbacks for BucketCache access. More...
 
static void writeCallBack (void *anOwner, char *aBucketStorage, const char *aBucket)
 
static void deleteCallBack (void *, char *aBucket)
 
static char * initCallBack (void *anOwner)
 
- Static Public Member Functions inherited from casacore::DataManager
static void registerCtor (const String &type, DataManagerCtor func)
 Register a mapping of a data manager type to its static construction function. More...
 
static DataManagerCtor getCtor (const String &dataManagerType)
 Get the "constructor" of a data manager (thread-safe). More...
 
static Bool isRegistered (const String &dataManagerType)
 Test if a data manager is registered (thread-safe). More...
 
static DataManagerunknownDataManager (const String &dataManagerType, const Record &spec)
 Serve as default function for theirRegisterMap, which catches all unknown data manager types. More...
 
- Static Public Attributes inherited from casacore::DataManager
static rownr_t MAXROWNR32
 Define the highest row number that can be represented as signed 32-bit. More...
 
- Protected Member Functions inherited from casacore::DataManager
void decrementNcolumn ()
 Decrement number of columns (in case a column is deleted). More...
 
void setEndian (Bool bigEndian)
 Tell the data manager if big or little endian format is needed. More...
 
void setTsmOption (const TSMOption &tsmOption)
 Tell the data manager which TSM option to use. More...
 
void setMultiFile (MultiFileBase *mfile)
 Tell the data manager that MultiFile can be used. More...
 
void throwDataTypeOther (const String &columnName, int dataType) const
 Throw an exception in case data type is TpOther, because the storage managers (and maybe other data managers) do not support such columns. More...
 

Detailed Description

The Standard Storage Manager

Intended use:

Public interface

Review Status

Reviewed By:
UNKNOWN
Date Reviewed:
before2004/08/25
Test programs:
tStandardStMan

Prerequisite

Etymology

StandardStMan is the data manager which stores the data in a standard way. I.e. it does not use special techniques like other storage managers do.

Synopsis

StandardStMan is meant as the storage manager to be used standardly. Other storage managers like IncrementalStMan and the TiledStMan derivatives should only be used when appropriate.
Like the other storage managers StandardStMan uses bucket-based access to its data. where a bucket contains the number of columns and rows that fit best. Variable length strings are stored in separate buckets because they do not fit in the fixed bucket layout used for the other columns. Only fixed length strings and strings <= 8 characters are stored directly. Note that, in fact, fixed length string means maximum length strings. It can be set using the setMaxLength function in class ColumnDesc or class BaseColumnDesc.

The file size is at least the size of a bucket, even if only the table contains only a few rows, thus uses only a fraction of a bucket. The default bucketsize is 32 rows. This means that if it is known in advance that the table will contain many more rows, it might make sense to construct the StandardStMan with a larger bucketsize.

StandardStMan is a robust storage manager. Care has been taken that its index cannot be corrupted in case of exceptions like device full or crash.

StandardStMan supports the following functionality:

  1. Removal of rows. This leaves some empty space in a bucket. An empty bucket will be reused.
  2. Addition of rows. This is always done in the last bucket and a new bucket is added when needed.
  3. Removal of a column. This also leaves empty space, which will be reused when a newly added column fits in it.
  4. Addition of a column. If available, empty column space is used. Otherwise it creates as many new buckets as needed.

All direct data (scalars and direct arrays) is stored in the main file. Indirect arrays (except strings) are stored in a second file. Indirect string arrays are also stored in the main file, because in that way frequently rewriting indirect strings arrays wastes far less space.

As said above all string arrays and variable length scalar strings are stored in separate string buckets.

Motivation

StManAipsIO is the standard storage manager used so far. Its major drawback is that it is memory based which makes it not usable for large tables. Furthermore it is not a very robust storage manager. When a system crashes, tables might get corrupted.
These drawbacks have been adressed in this new StandardStman. It uses a bucket-based access scheme and makes sure that its indices are stored in a way that they can hardly get corrupted.

Example

The following example shows how to create a table and how to attach the storage manager to some columns.

SetupNewTable newtab("name.data", tableDesc, Table::New);
StandardStMan stman; // define storage manager
newtab.bindColumn ("column1", stman); // bind column to st.man.
newtab.bindColumn ("column2", stman); // bind column to st.man.
Table tab(newtab); // actually create table
friend class SetupNewTable
Definition: DataManager.h:222
StandardStMan(Int bucketSize=0, uInt cacheSize=1)
Create a Standard storage manager with the given name.
@ New
create table
Definition: Table.h:176

The following example shows how to create a StandardStMan storage manager for a table with 16 rows. By giving the (expected) nr of rows to the storage manager, it can optimize its bucket size.

SetupNewTable newtab("name.data", tableDesc, Table::New);
StandardStMan stman(-16);
newtab.bindAll ("column1", stman); // bind all columns to st.man.
Table tab(newtab); // actually create table

Definition at line 149 of file StandardStMan.h.

Constructor & Destructor Documentation

◆ StandardStMan() [1/3]

casacore::StandardStMan::StandardStMan ( Int  bucketSize = 0,
uInt  cacheSize = 1 
)
explicit

Create a Standard storage manager with the given name.

If no name is used, it is set to "SSM" The name can be used to construct a

ROStandardStManAccessor object (e.g. to set the cache size).
The cache size has to be given in buckets.
The bucket size can be given in 2 ways:
- A positive number gives the bucket size in bytes. The number of rows per bucket will be calculated from it.
- A negative number gives the number of rows per bucket. The bucket size in bytes will be calculated from it. Note that in this way the maximum bucketsize is 32768 (minimum is 128).
- The default 0 means that 32 rows will be stored in a bucket.
Note that the default is only suitable for small tables. In general it makes sense to give the expected number of table rows. In that way the buckets will be small enough for small tables and not too small for large tables.

◆ StandardStMan() [2/3]

casacore::StandardStMan::StandardStMan ( const String dataManagerName,
Int  bucketSize = 0,
uInt  cacheSize = 1 
)
explicit

◆ ~StandardStMan()

casacore::StandardStMan::~StandardStMan ( )

◆ StandardStMan() [3/3]

casacore::StandardStMan::StandardStMan ( const StandardStMan that)
private

Copy constructor cannot be used.

Member Function Documentation

◆ operator=()

StandardStMan& casacore::StandardStMan::operator= ( const StandardStMan that)
private

Assignment cannot be used.


The documentation for this class was generated from the following file: