casacore
|
This class is used internally by ClassicalStatistics objects. More...
#include <ClassicalQuantileComputer.h>
Public Member Functions | |
ClassicalQuantileComputer ()=delete | |
ClassicalQuantileComputer (StatisticsDataset< CASA_STATP > *dataset) | |
ClassicalQuantileComputer (const ClassicalQuantileComputer &other) | |
copy semantics More... | |
virtual | ~ClassicalQuantileComputer () |
ClassicalQuantileComputer & | operator= (const ClassicalQuantileComputer &other) |
copy semantics More... | |
virtual StatisticsAlgorithmQuantileComputer< CASA_STATP > * | clone () const |
clone this object by returning a pointer to a copy More... | |
virtual AccumType | getMedian (uInt64 mynpts, AccumType mymin, AccumType mymax, uInt binningThreshholdSizeBytes, Bool persistSortedArray, uInt nBins) |
Caller is responsible for passing correct values of mynpts, mymin, and mymax; no checking is done for correctness in this method. More... | |
virtual AccumType | getMedianAbsDevMed (uInt64 mynpts, AccumType mymin, AccumType mymax, uInt binningThreshholdSizeBytes, Bool persistSortedArray, uInt nBins) |
get the median of the absolute deviation about the median of the data. More... | |
virtual AccumType | getMedianAndQuantiles (std::map< Double, AccumType > &quantiles, const std::set< Double > &fractions, uInt64 mynpts, AccumType mymin, AccumType mymax, uInt binningThreshholdSizeBytes, Bool persistSortedArray, uInt nBins) |
If one needs to compute both the median and QuantileComputer values, it is better to call getMedianAndQuantiles() rather than getMedian() and getQuantiles() separately, as the first will scan large data sets fewer times than calling the separate methods. More... | |
virtual std::map< Double, AccumType > | getQuantiles (const std::set< Double > &fractions, uInt64 mynpts, AccumType mymin, AccumType mymax, uInt binningThreshholdSizeBytes, Bool persistSortedArray, uInt nBins) |
Get the specified Quantiles. More... | |
virtual void | reset () |
reset the private fields More... | |
![]() | |
StatisticsAlgorithmQuantileComputer ()=delete | |
virtual | ~StatisticsAlgorithmQuantileComputer () |
void | deleteSortedArray () |
delete any (partially) sorted array More... | |
void | setDataset (StatisticsDataset< CASA_STATP > *ds) |
This must be called upon the copy or assignment of the associated statistics algorithm object. More... | |
std::vector< CASA_STATP > & | _getSortedArray () |
FIXME make protected once refactor is complete. More... | |
void | _setSortedArray (const std::vector< CASA_STATP > &v) |
FIXME make protected once refactor is complete. More... | |
void | setMedian (CountedPtr< CASA_STATP > median) |
Protected Member Functions | |
virtual void | _findBins (std::vector< std::vector< uInt64 > > &binCounts, std::vector< CountedPtr< AccumType > > &sameVal, std::vector< Bool > &allSame, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const std::vector< StatsHistogram< AccumType > > &hist, const std::vector< AccumType > &maxLimit) const |
Get the counts of data within the specified histogram bins. More... | |
virtual void | _findBins (std::vector< std::vector< uInt64 > > &binCounts, std::vector< CountedPtr< AccumType > > &sameVal, std::vector< Bool > &allSame, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, const std::vector< StatsHistogram< AccumType > > &hist, const std::vector< AccumType > &maxLimit) const |
virtual void | _findBins (std::vector< std::vector< uInt64 > > &binCounts, std::vector< CountedPtr< AccumType > > &sameVal, std::vector< Bool > &allSame, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const std::vector< StatsHistogram< AccumType > > &hist, const std::vector< AccumType > &maxLimit) const |
virtual void | _findBins (std::vector< std::vector< uInt64 > > &binCounts, std::vector< CountedPtr< AccumType > > &sameVal, std::vector< Bool > &allSame, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, const std::vector< StatsHistogram< AccumType > > &hist, const std::vector< AccumType > &maxLimit) const |
virtual void | _findBins (std::vector< std::vector< uInt64 > > &binCounts, std::vector< CountedPtr< AccumType > > &sameVal, std::vector< Bool > &allSame, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const std::vector< StatsHistogram< AccumType > > &hist, const std::vector< AccumType > &maxLimit) const |
virtual void | _findBins (std::vector< std::vector< uInt64 > > &binCounts, std::vector< CountedPtr< AccumType > > &sameVal, std::vector< Bool > &allSame, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, const std::vector< StatsHistogram< AccumType > > &hist, const std::vector< AccumType > &maxLimit) const |
virtual void | _findBins (std::vector< std::vector< uInt64 > > &binCounts, std::vector< CountedPtr< AccumType > > &sameVal, std::vector< Bool > &allSame, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, const std::vector< StatsHistogram< AccumType > > &hist, const std::vector< AccumType > &maxLimit) const |
virtual void | _findBins (std::vector< std::vector< uInt64 > > &binCounts, std::vector< CountedPtr< AccumType > > &sameVal, std::vector< Bool > &allSame, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const std::vector< StatsHistogram< AccumType > > &hist, const std::vector< AccumType > &maxLimit) const |
virtual void | _populateArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const |
populate an unsorted array with valid data. More... | |
virtual void | _populateArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
ranges More... | |
virtual void | _populateArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
virtual void | _populateArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
mask and ranges More... | |
virtual void | _populateArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride) const |
weights More... | |
virtual void | _populateArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
weights and ranges More... | |
virtual void | _populateArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
weights and mask More... | |
virtual void | _populateArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
weights, mask, ranges More... | |
virtual void | _populateArrays (std::vector< std::vector< AccumType >> &arys, uInt64 ¤tCount, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const IncludeLimits &includeLimits, uInt64 maxCount) const |
Create a std::vector of unsorted arrays, one array for each bin defined by includeLimits . More... | |
virtual void | _populateArrays (std::vector< std::vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, const IncludeLimits &includeLimits, uInt64 maxCount) const |
ranges More... | |
virtual void | _populateArrays (std::vector< std::vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const IncludeLimits &includeLimits, uInt64 maxCount) const |
virtual void | _populateArrays (std::vector< std::vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, const IncludeLimits &includeLimits, uInt64 maxCount) const |
mask and ranges More... | |
virtual void | _populateArrays (std::vector< std::vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const IncludeLimits &includeLimits, uInt64 maxCount) const |
weights More... | |
virtual void | _populateArrays (std::vector< std::vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, const IncludeLimits &includeLimits, uInt64 maxCount) const |
weights and ranges More... | |
virtual void | _populateArrays (std::vector< std::vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const IncludeLimits &includeLimits, uInt64 maxCount) const |
weights and mask More... | |
virtual void | _populateArrays (std::vector< std::vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, const IncludeLimits &includeLimits, uInt64 maxCount) const |
weights, mask, ranges More... | |
virtual Bool | _populateTestArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, uInt maxElements) const |
no weights, no mask, no ranges More... | |
virtual Bool | _populateTestArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, uInt maxElements) const |
ranges More... | |
virtual Bool | _populateTestArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, uInt maxElements) const |
mask More... | |
virtual Bool | _populateTestArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, uInt maxElements) const |
mask and ranges More... | |
virtual Bool | _populateTestArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, uInt maxElements) const |
weights More... | |
virtual Bool | _populateTestArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, uInt maxElements) const |
weights and ranges More... | |
virtual Bool | _populateTestArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, uInt maxElements) const |
weights and mask More... | |
virtual Bool | _populateTestArray (std::vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, uInt maxElements) const |
weights, mask, ranges More... | |
Bool | _valuesFromSortedArray (std::map< uInt64, AccumType > &values, uInt64 mynpts, const std::set< uInt64 > &indices, uInt64 maxArraySize, Bool persistSortedArray) |
get values from sorted array if the array is small enough to be held in memory. More... | |
![]() | |
StatisticsAlgorithmQuantileComputer (StatisticsDataset< CASA_STATP > *ds) | |
ds should be the dataset object held in the StatisticsAlgorithm object. More... | |
StatisticsAlgorithmQuantileComputer (const StatisticsAlgorithmQuantileComputer &other) | |
use copy semantics. More... | |
StatisticsAlgorithmQuantileComputer & | operator= (const StatisticsAlgorithmQuantileComputer &other) |
use copy semantics. More... | |
StatisticsDataset< CASA_STATP > * | _getDataset () |
CountedPtr< CASA_STATP > | _getMedian () const |
CountedPtr< CASA_STATP > | _getMedianAbsDevMedian () const |
void | _setMedianAbsDevMedian (CountedPtr< CASA_STATP > medAbsDevMed) |
Private Types | |
using | LimitPair = std::pair< AccumType, AccumType > |
using | LimitPairVectorIter = typename std::vector< LimitPair >::const_iterator |
using | IndexValueMap = typename std::map< uInt64, AccumType > |
using | IndexSet = std::set< uInt64 > |
Private Member Functions | |
std::vector< std::vector< uInt64 > > | _binCounts (std::vector< CountedPtr< AccumType > > &sameVal, const std::vector< StatsHistogram< AccumType > > &hist) |
tally the number of data points that fall into each bin provided by hist . More... | |
void | _computeBins (std::vector< std::vector< uInt64 >> &bins, std::vector< CountedPtr< AccumType >> &sameVal, std::vector< Bool > &allSame, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 count, const std::vector< StatsHistogram< AccumType >> &hist, const std::vector< AccumType > &maxLimit, const typename StatisticsDataset< CASA_STATP >::ChunkData &chunk) |
void | _computeDataArray (std::vector< AccumType > &ary, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 dataCount, const typename StatisticsDataset< CASA_STATP >::ChunkData &chunk) |
void | _computeDataArrays (std::vector< std::vector< AccumType > > &arys, uInt64 ¤tCount, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 dataCount, const IncludeLimits &includeLimits, uInt64 maxCount, const typename StatisticsDataset< CASA_STATP >::ChunkData &chunk) |
void | _createDataArray (std::vector< AccumType > &array) |
Create an unsorted array of the complete data set. More... | |
void | _createDataArrays (std::vector< std::vector< AccumType >> &arrays, const IncludeLimits &includeLimits, uInt64 maxCount) |
std::vector< IndexValueMap > | _dataFromMultipleBins (const std::vector< StatsHistogram< AccumType > > &hist, uInt64 maxArraySize, const std::vector< IndexSet > &dataIndices, uInt nBins) |
extract data from multiple histograms given by hist . More... | |
std::vector< IndexValueMap > | _dataFromSingleBins (const std::vector< uInt64 > &binNpts, uInt64 maxArraySize, const IncludeLimits &binLimits, const std::vector< IndexSet > &dataIndices, uInt nBins) |
IndexValueMap | _indicesToValues (uInt64 mynpts, AccumType mymin, AccumType mymax, uInt64 maxArraySize, const IndexSet &dataIndices, Bool persistSortedArray, uInt nBins) |
get the values for the specified indices in the sorted array of all good data More... | |
Static Private Member Functions | |
static IndexSet | _medianIndices (uInt64 mynpts) |
get the index (for odd npts) or indices (for even npts) of the median of the sorted array. More... | |
Private Attributes | |
Bool | _doMedAbsDevMed |
AccumType | _myMedian |
for use in often repeatedly run macros More... | |
This class is used internally by ClassicalStatistics objects.
It should never be explicitly instantiated by an API developer. See the documentation of StatisticsAlgorithm for details regarding QuantileComputer classes.
Definition at line 49 of file ClassicalQuantileComputer.h.
|
private |
Definition at line 55 of file ClassicalQuantileComputer.h.
|
private |
Definition at line 54 of file ClassicalQuantileComputer.h.
|
private |
Definition at line 52 of file ClassicalQuantileComputer.h.
|
private |
Definition at line 53 of file ClassicalQuantileComputer.h.
|
delete |
casacore::ClassicalQuantileComputer< AccumType, DataIterator, MaskIterator, WeightsIterator >::ClassicalQuantileComputer | ( | StatisticsDataset< CASA_STATP > * | dataset | ) |
casacore::ClassicalQuantileComputer< AccumType, DataIterator, MaskIterator, WeightsIterator >::ClassicalQuantileComputer | ( | const ClassicalQuantileComputer< AccumType, DataIterator, MaskIterator, WeightsIterator > & | other | ) |
copy semantics
|
virtual |
|
private |
tally the number of data points that fall into each bin provided by hist
.
Any points that are less than hist.minLimit or greater than hist.minLimit + hist.nBins*hist.binWidth are not included in the counts. A data point that falls exactly on a bin boundary is considered to be in the higher index bin. sameVal
will be non-null if all the good values in the histogram range are the same. In that case, the value held will be the value of each of those data points.
|
private |
|
private |
|
private |
|
private |
Create an unsorted array of the complete data set.
If includeLimits
is specified, only points within those limits (including min but excluding max, as per definition of bins), are included.
|
private |
|
private |
extract data from multiple histograms given by hist
.
dataIndices
represent the indices of the sorted arrays of values to extract. There should be exactly one set of data indices to extract for each supplied histogram. The data indices are relative to the minimum value of the minimum bin in their respective histograms. The ordering of the maps in the returned std::vector represent the ordering of histograms in hist
. hist
should contain non-overlapping histograms and the histograms should be specified in ascending order.
|
private |
|
protectedvirtual |
|
protectedvirtual |
|
protectedvirtual |
|
protectedvirtual |
|
protectedvirtual |
|
protectedvirtual |
|
protectedvirtual |
|
protectedvirtual |
Get the counts of data within the specified histogram bins.
The number of arrays within binCounts will be equal to the number of histograms in hist
. Each array within binCounts
will have the same number of elements as the number of bins in its corresponding histogram in hist
.
|
private |
get the values for the specified indices in the sorted array of all good data
|
staticprivate |
get the index (for odd npts) or indices (for even npts) of the median of the sorted array.
|
protectedvirtual |
weights and mask
|
protectedvirtual |
weights, mask, ranges
|
protectedvirtual |
weights
|
protectedvirtual |
weights and ranges
|
protectedvirtual |
populate an unsorted array with valid data.
no weights, no mask, no ranges
|
protectedvirtual |
ranges
|
protectedvirtual |
|
protectedvirtual |
mask and ranges
|
protectedvirtual |
weights, mask, ranges
|
protectedvirtual |
weights and mask
|
protectedvirtual |
weights and ranges
|
protectedvirtual |
weights
|
protectedvirtual |
ranges
|
protectedvirtual |
mask and ranges
|
protectedvirtual |
|
protectedvirtual |
Create a std::vector of unsorted arrays, one array for each bin defined by includeLimits
.
includeLimits
should be non-overlapping and should be given in ascending order (the algorithm used assumes this). Once the sum of the lengths of all arrays equals maxCount
the method will return with no further processing. no weights, no mask, no ranges
|
protectedvirtual |
weights, mask, ranges
|
protectedvirtual |
weights and mask
|
protectedvirtual |
weights
|
protectedvirtual |
weights and ranges
|
protectedvirtual |
ranges
|
protectedvirtual |
mask and ranges
|
protectedvirtual |
mask
|
protectedvirtual |
no weights, no mask, no ranges
|
protected |
get values from sorted array if the array is small enough to be held in memory.
Note that this is the array containing all good data, not data in just a single bin representing a subset of good data. Returns True if the data were successfully retrieved. If True is returned, the values map will contain a map of index to value. It is the caller's responsibility to check that mynpts
is not 0; no checking is done here.
|
virtual |
clone this object by returning a pointer to a copy
Implements casacore::StatisticsAlgorithmQuantileComputer< CASA_STATP >.
Reimplemented in casacore::HingesFencesQuantileComputer< AccumType, DataIterator, MaskIterator, WeightsIterator >, casacore::ConstrainedRangeQuantileComputer< AccumType, DataIterator, MaskIterator, WeightsIterator >, and casacore::ConstrainedRangeQuantileComputer< CASA_STATP >.
|
virtual |
Caller is responsible for passing correct values of mynpts, mymin, and mymax; no checking is done for correctness in this method.
Reimplemented in casacore::ConstrainedRangeQuantileComputer< CASA_STATP >.
|
virtual |
get the median of the absolute deviation about the median of the data.
Reimplemented in casacore::ConstrainedRangeQuantileComputer< CASA_STATP >.
|
virtual |
If one needs to compute both the median and QuantileComputer values, it is better to call getMedianAndQuantiles() rather than getMedian() and getQuantiles() separately, as the first will scan large data sets fewer times than calling the separate methods.
The return value is the median; the quantiles are returned in the quantiles
map. Values in the fractions
set represent the locations in the CDF and should be between 0 and 1, exclusive.
|
virtual |
Get the specified Quantiles.
fractions
must be between 0 and 1, noninclusive.
ClassicalQuantileComputer& casacore::ClassicalQuantileComputer< AccumType, DataIterator, MaskIterator, WeightsIterator >::operator= | ( | const ClassicalQuantileComputer< AccumType, DataIterator, MaskIterator, WeightsIterator > & | other | ) |
copy semantics
|
virtual |
reset the private fields
Reimplemented from casacore::StatisticsAlgorithmQuantileComputer< CASA_STATP >.
Reimplemented in casacore::HingesFencesQuantileComputer< AccumType, DataIterator, MaskIterator, WeightsIterator >.
|
private |
Definition at line 399 of file ClassicalQuantileComputer.h.
|
private |
for use in often repeatedly run macros
Definition at line 401 of file ClassicalQuantileComputer.h.