casacore
Public Member Functions | Private Member Functions | Private Attributes | List of all members
casacore::TiledLineStepper Class Reference

More...

#include <TiledLineStepper.h>

Public Member Functions

 TiledLineStepper (const IPosition &latticeShape, const IPosition &tileShape, const uInt axis)
 Construct a TiledLineStepper by specifying the Lattice shape, a tile shape and the axis along which the Vector cursor will lie (0 means the x-axis). More...
 
 TiledLineStepper (const TiledLineStepper &other)
 The copy constructor uses copy semantics. More...
 
 ~TiledLineStepper ()
 
TiledLineStepperoperator= (const TiledLineStepper &other)
 The assignment operator uses copy semantics. More...
 
virtual Bool operator++ (int)
 Increment operator (postfix or prefix version) - move the cursor forward one step. More...
 
virtual Bool operator-- (int)
 Decrement operator (postfix or prefix version) - move the cursor backwards one step. More...
 
virtual void reset ()
 Function to move the cursor to the beginning of the Lattice. More...
 
virtual Bool atStart () const
 Function which returns "True" if the cursor is at the beginning of the Lattice, otherwise, returns "False". More...
 
virtual Bool atEnd () const
 Function which returns "True" if an attempt has been made to increment the cursor beyond the end of the Lattice. More...
 
virtual uInt nsteps () const
 Function to return the number of steps (increments & decrements) taken since construction (or since last reset). More...
 
virtual IPosition position () const
 Function which returns the current position of the beginning of the cursor. More...
 
virtual IPosition endPosition () const
 Function which returns the current position of the end of the cursor. More...
 
virtual IPosition latticeShape () const
 Functions which returns the shape of the Lattice being iterated through. More...
 
virtual IPosition subLatticeShape () const
 
virtual IPosition cursorShape () const
 Function which returns the shape of the cursor. More...
 
virtual IPosition cursorAxes () const
 Function which returns the axes of the cursor. More...
 
IPosition tileShape () const
 Function which returns the shape of the "tile" the cursor will iterate through before moving onto the next tile. More...
 
virtual Bool hangOver () const
 Function which returns "True" if the increment/decrement operators have moved the cursor position such that part of the cursor beginning or end is hanging over the edge of the Lattice. More...
 
virtual void subSection (const IPosition &blc, const IPosition &trc)
 Functions to specify a "section" of the Lattice to step over. More...
 
virtual void subSection (const IPosition &blc, const IPosition &trc, const IPosition &inc)
 
virtual IPosition blc () const
 Return the bottom left hand corner (blc), top right corner (trc) or step size (increment) used by the current sub-Lattice. More...
 
virtual IPosition trc () const
 
virtual IPosition increment () const
 
virtual const IPositionaxisPath () const
 Return the axis path. More...
 
virtual LatticeNavigatorclone () const
 Function which returns a pointer to dynamic memory of an exact copy of this instance. More...
 
virtual Bool ok () const
 Function which checks the internal data of this class for correct dimensionality and consistant values. More...
 
virtual uInt calcCacheSize (const IPosition &cubeShape, const IPosition &tileShape, uInt maxCacheSize, uInt bucketSize) const
 Calculate the cache size (in tiles) for this type of access to a lattice in the given row of the tiled hypercube. More...
 
- Public Member Functions inherited from casacore::LatticeNavigator
 LatticeNavigator ()
 Default constructor. More...
 
 LatticeNavigator (const LatticeNavigator &)
 Copy constructor. More...
 
LatticeNavigatoroperator= (const LatticeNavigator &)
 Assignment. More...
 
virtual ~LatticeNavigator ()
 A virtual destructor. More...
 
Bool operator++ ()
 
Bool operator-- ()
 
virtual IPosition relativePosition () const
 
virtual IPosition relativeEndPosition () const
 
virtual IPosition hangOverBlc () const
 Functions which return the "bottom left corner" and the "top right corner" of the cursor that does not hangover. More...
 
virtual IPosition hangOverTrc () const
 

Private Member Functions

 TiledLineStepper ()
 Prevent the default constructor from being used. More...
 

Private Attributes

IPosition itsBlc
 
IPosition itsTrc
 
IPosition itsInc
 
LatticeIndexer itsSubSection
 
LatticeIndexer itsIndexer
 
LatticeIndexer itsTiler
 
IPosition itsIndexerCursorPos
 
IPosition itsTilerCursorPos
 
IPosition itsCursorShape
 
IPosition itsTileShape
 
IPosition itsAxisPath
 
uInt itsNsteps
 
uInt itsAxis
 
Bool itsEnd
 
Bool itsStart
 

Detailed Description

Step a Vector cursor optimally through a tiled Lattice.

Intended use:

Public interface

Review Status

Reviewed By:
Peter Barnes
Date Reviewed:
1999/10/30
Test programs:
tTiledLineStepper

Prerequisite

Etymology

TiledLineStepper is used to step a Vector cursor optimally through a Lattice that is tiled.

Synopsis

When you wish to traverse a Lattice (say, a PagedArray or an Image) you will usually create a LatticeIterator. Once created, you may attach a LatticeNavigator to the iterator. A TiledLineStepper, is a concrete class derived from the abstract LatticeNavigator that allows you to move a Vector cursor through the Lattice in a way that will minimize the amount of cache memory consumed.

Some Lattices (in particular PagedArrays) are stored (on disk) in tiles. For an N-dimensional Lattice a tile is an N-dimensional subsection with fewer elements along each axis. For example a Lattice of shape [512,512,4,32] may have a tile shape of [32,16,4,16], and there will be 16*32*1*2 (=1024) tiles in the entire Lattice. To allow efficient access of the data in a Lattice some tiles are cached in memory. As each tile may consume a fair bit of memory (in this example 128kBytes, assuming each element consumes 4 bytes), it is desirable to minimise the number of tiles held in the cache. But it is also desirable to minimise the number of times a tiles must be read into or written from the cache as this may require a time consuming operation like disk I/O.

Now suppose you wanted to traverse a Lattice with a Vector cursor of length 512 pixel aligned along the x-axis. Using a LatticeStepper, each Vector is retrieved from the Lattice sequentially and without any consideration of the underlying tile shape. What is the optimal cache size for the above example?

Suppose we have a cache size of 16 ie., the number of tiles along the x-axis. Then Vectors beginning at positions [0,0,0,0] to [0,15,0,0] will be stored in the cache. But the next Vector beginning at position [0,16,0,0] will flush the cache and read in another 16 tiles. This I/O takes time and will occur 16 times for each plane in the four dimensional Lattice. Further when the cursor moves to position [0,0,1,0] the 16 tiles that where initially in the cache will need to be read again. To avoid all this cache I/O it is better to have a bigger cache.

Suppose the cache size is 16*32 (=512) ie., enough tiles to contain an (x,y)-plane. Then the cache size will not be flushed until the cursor is moved to position [0,0,0,16]. Further the cache will never need to read back into memory tiles that had previously been stored in there. The cache is big enough to store tiles until they have been completely used. But this cache is 64MBytes in size, and consumes too much memory for many computers.

This where a TiledLineStepper is useful. Because it knows the shape of the tiles in the underlying Lattice it moves the cursor to return all the Vectors in the smallest possible cache of tiles before moving on to the next set of tiles. Using the above example again, the TiledLineStepper will move the beginning of the Vector cursor in the following pattern.

[0,0,0,0], [0,1,0,0], [0,2,0,0],... [0,15,0,0]
[0,0,1,0], [0,1,1,0],... [0,15,1,0],
... [0,15,3,0],
[0,0,0,1],... [0,15,3,15]

Moving the Vector cursor through all 16*4*16 (=1024 positions) can be done by caching only 16 tiles in memory (those along the x-axis). Hence the cache size need only be 2MBytes in size. Further once all 1024 vectors have been returned it is not necessary to read these 16 tiles back into memory. All the data in those tiles has already been accessed. Using a TiledLineStepper rather than a LatticeStepper has, in this example, resulted in a drop in the required cache size from 64MBytes down to 2MBytes.

In constructing a TiledLineStepper, you specify the Lattice shape, the tile shape and the axis the Vector cursor will be aligned with. Specifying an axis=0 will align the cursor with the x-axis and axis=2 will produce a cursor that is along the z-axis. The length of the cursor is always the same as the number of elements in the Lattice along the axis the cursor is aligned with.
It is possible to use the function subSection to traverse only a subsection of the lattice.

The cursor position can be incremented or decremented to retrieve the next or previous Vector in the Lattice. The position of the next Vector in the Lattice will depend on the tile shape, and is described above. Within a tile the Vector cursor will move first through the x-axis and then the y-axis (assuming we have a cursor oriented along the z-axis). In general the lower dimensions will be exhausted (within a tile) before moving the cursor through higher dimensions. This intra-tile behaviour for cursor movement extends to the inter-tile movement of the cursor between tiles.

Example

This example is of a global function that will do a 2-D inplace complex Fourier transform of an arbitrary large Lattice (which must have at least two dimensions).

A two dimensional transform is done by successive one dimensional transforms along all the rows and then all the columns in the lattice. Scoping is used to destroy iterators once they have been used. This frees up the cache memory associated with the cursor in each iterator.

void FFT2DComplex (Lattice<Complex>& cArray,
const Bool direction)
{
const uInt ndim = cArray.ndim();
AlwaysAssert(ndim > 1, AipsError);
const IPosition latticeShape = cArray.shape();
const uInt nx=latticeShape(0);
const uInt ny=latticeShape(1);
const IPosition tileShape = cArray.niceCursorShape();
{
LatticeIterator<Complex> lix(cArray, tsx);
FFTServer<Float,Complex> fftx(IPosition(1, nx));
for (lix.reset();!lix.atEnd();lix++) {
fftx.fft(lix.rwVectorCursor(), direction);
}
}
{
LatticeIterator<Complex> liy(cArray, tsy);
FFTServer<Float,Complex> ffty(IPosition(1, ny));
for (liy.reset();!liy.atEnd();liy++) {
ffty.fft(liy.rwVectorCursor(), direction);
}
}
}
#define AlwaysAssert(expr, exception)
These marcos are provided for use instead of simply using the constructors of assert_ to allow additi...
Definition: Assert.h:157
friend LatticeExprNode ndim(const LatticeExprNode &expr)
1-argument function to get the dimensionality of a lattice.
TiledLineStepper()
Prevent the default constructor from being used.
virtual IPosition latticeShape() const
Functions which returns the shape of the Lattice being iterated through.
IPosition tileShape() const
Function which returns the shape of the "tile" the cursor will iterate through before moving onto the...
LatticeExprNode ndim(const LatticeExprNode &expr)
1-argument function to get the dimensionality of a lattice.
unsigned int uInt
Definition: aipstype.h:51
bool Bool
Define the standard types used by Casacore.
Definition: aipstype.h:42

Motivation

Moving through a Lattice by equal sized chunks, and without regard to the nature of the data, is a basic and common procedure.

To Do

Definition at line 192 of file TiledLineStepper.h.

Constructor & Destructor Documentation

◆ TiledLineStepper() [1/3]

casacore::TiledLineStepper::TiledLineStepper ( const IPosition latticeShape,
const IPosition tileShape,
const uInt  axis 
)

Construct a TiledLineStepper by specifying the Lattice shape, a tile shape and the axis along which the Vector cursor will lie (0 means the x-axis).

Is is nearly always advisable to make the tileShape identical to the Lattice tileShape. This can be obtained by lat.niceCursorShape(lat.advisedMaxPixels()) where lat is a Lattice object.

◆ TiledLineStepper() [2/3]

casacore::TiledLineStepper::TiledLineStepper ( const TiledLineStepper other)

The copy constructor uses copy semantics.

◆ ~TiledLineStepper()

casacore::TiledLineStepper::~TiledLineStepper ( )

◆ TiledLineStepper() [3/3]

casacore::TiledLineStepper::TiledLineStepper ( )
private

Prevent the default constructor from being used.

Member Function Documentation

◆ atEnd()

virtual Bool casacore::TiledLineStepper::atEnd ( ) const
virtual

Function which returns "True" if an attempt has been made to increment the cursor beyond the end of the Lattice.

Implements casacore::LatticeNavigator.

◆ atStart()

virtual Bool casacore::TiledLineStepper::atStart ( ) const
virtual

Function which returns "True" if the cursor is at the beginning of the Lattice, otherwise, returns "False".

Implements casacore::LatticeNavigator.

◆ axisPath()

virtual const IPosition& casacore::TiledLineStepper::axisPath ( ) const
virtual

Return the axis path.

See LatticeStepper for a description and examples.

Implements casacore::LatticeNavigator.

◆ blc()

virtual IPosition casacore::TiledLineStepper::blc ( ) const
virtual

Return the bottom left hand corner (blc), top right corner (trc) or step size (increment) used by the current sub-Lattice.

If no sub-Lattice has been defined (with the subSection function) these functions return blc=0, trc=latticeShape-1, increment=1, ie. the entire Lattice.

Reimplemented from casacore::LatticeNavigator.

◆ calcCacheSize()

virtual uInt casacore::TiledLineStepper::calcCacheSize ( const IPosition cubeShape,
const IPosition tileShape,
uInt  maxCacheSize,
uInt  bucketSize 
) const
virtual

Calculate the cache size (in tiles) for this type of access to a lattice in the given row of the tiled hypercube.

Implements casacore::LatticeNavigator.

◆ clone()

virtual LatticeNavigator* casacore::TiledLineStepper::clone ( ) const
virtual

Function which returns a pointer to dynamic memory of an exact copy of this instance.

The pointer returned by this function must be deleted externally.

Implements casacore::LatticeNavigator.

◆ cursorAxes()

virtual IPosition casacore::TiledLineStepper::cursorAxes ( ) const
virtual

Function which returns the axes of the cursor.

Implements casacore::LatticeNavigator.

◆ cursorShape()

virtual IPosition casacore::TiledLineStepper::cursorShape ( ) const
virtual

Function which returns the shape of the cursor.

This always includes all axes (ie. it includes degenerates axes)

Implements casacore::LatticeNavigator.

◆ endPosition()

virtual IPosition casacore::TiledLineStepper::endPosition ( ) const
virtual

Function which returns the current position of the end of the cursor.

The endPosition function is relative to the origin in the main Lattice.

Implements casacore::LatticeNavigator.

◆ hangOver()

virtual Bool casacore::TiledLineStepper::hangOver ( ) const
virtual

Function which returns "True" if the increment/decrement operators have moved the cursor position such that part of the cursor beginning or end is hanging over the edge of the Lattice.

This always returns False.

Implements casacore::LatticeNavigator.

◆ increment()

virtual IPosition casacore::TiledLineStepper::increment ( ) const
virtual

Reimplemented from casacore::LatticeNavigator.

◆ latticeShape()

virtual IPosition casacore::TiledLineStepper::latticeShape ( ) const
virtual

Functions which returns the shape of the Lattice being iterated through.

latticeShape always returns the shape of the main Lattice while subLatticeShape returns the shape of any sub-Lattice defined using the subSection function.

Implements casacore::LatticeNavigator.

◆ nsteps()

virtual uInt casacore::TiledLineStepper::nsteps ( ) const
virtual

Function to return the number of steps (increments & decrements) taken since construction (or since last reset).

This is a running count of all cursor movement (operator++ or operator--), even though N-increments followed by N-decrements will always leave the cursor in the original position.

Implements casacore::LatticeNavigator.

◆ ok()

virtual Bool casacore::TiledLineStepper::ok ( ) const
virtual

Function which checks the internal data of this class for correct dimensionality and consistant values.

Returns True if everything is fine otherwise returns False

Reimplemented from casacore::LatticeNavigator.

◆ operator++()

virtual Bool casacore::TiledLineStepper::operator++ ( int  )
virtual

Increment operator (postfix or prefix version) - move the cursor forward one step.

Returns True if the cursor was moved.

Implements casacore::LatticeNavigator.

◆ operator--()

virtual Bool casacore::TiledLineStepper::operator-- ( int  )
virtual

Decrement operator (postfix or prefix version) - move the cursor backwards one step.

Returns True if the cursor was moved.

Implements casacore::LatticeNavigator.

◆ operator=()

TiledLineStepper& casacore::TiledLineStepper::operator= ( const TiledLineStepper other)

The assignment operator uses copy semantics.

◆ position()

virtual IPosition casacore::TiledLineStepper::position ( ) const
virtual

Function which returns the current position of the beginning of the cursor.

The position function is relative to the origin in the main Lattice.

Implements casacore::LatticeNavigator.

◆ reset()

virtual void casacore::TiledLineStepper::reset ( )
virtual

Function to move the cursor to the beginning of the Lattice.

Also resets the number of steps (nsteps function) to zero.

Implements casacore::LatticeNavigator.

◆ subLatticeShape()

virtual IPosition casacore::TiledLineStepper::subLatticeShape ( ) const
virtual

Reimplemented from casacore::LatticeNavigator.

◆ subSection() [1/2]

virtual void casacore::TiledLineStepper::subSection ( const IPosition blc,
const IPosition trc 
)
virtual

Functions to specify a "section" of the Lattice to step over.

A section is defined in terms of the Bottom Left Corner (blc), Top Right Corner (trc), and step size (inc), on ALL of its axes, including degenerate axes. The step size defaults to one if not specified.

Reimplemented from casacore::LatticeNavigator.

◆ subSection() [2/2]

virtual void casacore::TiledLineStepper::subSection ( const IPosition blc,
const IPosition trc,
const IPosition inc 
)
virtual

Reimplemented from casacore::LatticeNavigator.

◆ tileShape()

IPosition casacore::TiledLineStepper::tileShape ( ) const

Function which returns the shape of the "tile" the cursor will iterate through before moving onto the next tile.

THIS IS NOT THE SAME AS THE TILE SHAPE USED BY THE LATTICE. It is nearly the same except that the axis the cursor is aligned with is replaced by the shape of the Lattice on that axis. eg., If a Lattice has a shape of [512,512,4,32] and a tile shape of [32,16,4,16] then tileShape() will return [512,16,4,16] if the cursor is along the x-axis and [32,512,4,16] if the cursor is along the y-axis.

◆ trc()

virtual IPosition casacore::TiledLineStepper::trc ( ) const
virtual

Reimplemented from casacore::LatticeNavigator.

Member Data Documentation

◆ itsAxis

uInt casacore::TiledLineStepper::itsAxis
private

Definition at line 345 of file TiledLineStepper.h.

◆ itsAxisPath

IPosition casacore::TiledLineStepper::itsAxisPath
private

Definition at line 343 of file TiledLineStepper.h.

◆ itsBlc

IPosition casacore::TiledLineStepper::itsBlc
private

Definition at line 333 of file TiledLineStepper.h.

◆ itsCursorShape

IPosition casacore::TiledLineStepper::itsCursorShape
private

Definition at line 341 of file TiledLineStepper.h.

◆ itsEnd

Bool casacore::TiledLineStepper::itsEnd
private

Definition at line 346 of file TiledLineStepper.h.

◆ itsInc

IPosition casacore::TiledLineStepper::itsInc
private

Definition at line 335 of file TiledLineStepper.h.

◆ itsIndexer

LatticeIndexer casacore::TiledLineStepper::itsIndexer
private

Definition at line 337 of file TiledLineStepper.h.

◆ itsIndexerCursorPos

IPosition casacore::TiledLineStepper::itsIndexerCursorPos
private

Definition at line 339 of file TiledLineStepper.h.

◆ itsNsteps

uInt casacore::TiledLineStepper::itsNsteps
private

Definition at line 344 of file TiledLineStepper.h.

◆ itsStart

Bool casacore::TiledLineStepper::itsStart
private

Definition at line 347 of file TiledLineStepper.h.

◆ itsSubSection

LatticeIndexer casacore::TiledLineStepper::itsSubSection
private

Definition at line 336 of file TiledLineStepper.h.

◆ itsTiler

LatticeIndexer casacore::TiledLineStepper::itsTiler
private

Definition at line 338 of file TiledLineStepper.h.

◆ itsTilerCursorPos

IPosition casacore::TiledLineStepper::itsTilerCursorPos
private

Definition at line 340 of file TiledLineStepper.h.

◆ itsTileShape

IPosition casacore::TiledLineStepper::itsTileShape
private

Definition at line 342 of file TiledLineStepper.h.

◆ itsTrc

IPosition casacore::TiledLineStepper::itsTrc
private

Definition at line 334 of file TiledLineStepper.h.


The documentation for this class was generated from the following file: