org.HdrHistogram

Class ConcurrentHistogram

java.lang.Object

org.HdrHistogram.EncodableHistogram

org.HdrHistogram.AbstractHistogramBase

org.HdrHistogram.AbstractHistogram

org.HdrHistogram.Histogram

org.HdrHistogram.ConcurrentHistogram

All Implemented Interfaces:

java.io.Serializable, ValueRecorder

Direct Known Subclasses:

Recorder.InternalConcurrentHistogram

public class ConcurrentHistogram
extends Histogram

An integer values High Dynamic Range (HDR) Histogram that supports safe concurrent recording operations.

A ConcurrentHistogram guarantees lossless recording of values into the histogram even when the histogram is updated by multiple threads, and supports auto-resize and shift operations that may result from or occur concurrently with other recording operations.

It is important to note that concurrent recording, auto-sizing, and value shifting are the only thread-safe behaviors provided by ConcurrentHistogram, and that it is not otherwise synchronized. Specifically, ConcurrentHistogram provides no implicit synchronization that would prevent the contents of the histogram from changing during queries, iterations, copies, or addition operations on the histogram. Callers wishing to make potentially concurrent, multi-threaded updates that would safely work in the presence of queries, copies, or additions of histogram objects should either take care to externally synchronize and/or order their access, use the SynchronizedHistogram variant, or (recommended) use Recorder or SingleWriterRecorder which are intended for this purpose.

Auto-resizing: When constructed with no specified value range range (or when auto-resize is turned on with AbstractHistogram.setAutoResize(boolean)) a Histogram will auto-resize its dynamic range to include recorded values as they are encountered. Note that recording calls that cause auto-resizing may take longer to execute, as resizing incurs allocation and copying of internal data structures.

See package description for org.HdrHistogram for details.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	ConcurrentHistogram.AtomicLongArrayWithNormalizingOffset

Nested classes/interfaces inherited from class org.HdrHistogram.AbstractHistogram

AbstractHistogram.AllValues, AbstractHistogram.LinearBucketValues, AbstractHistogram.LogarithmicBucketValues, AbstractHistogram.Percentiles, AbstractHistogram.RecordedValues

Field Summary

Fields

Modifier and Type	Field and Description
(package private) ConcurrentHistogram.AtomicLongArrayWithNormalizingOffset	activeCounts
(package private) ConcurrentHistogram.AtomicLongArrayWithNormalizingOffset	inactiveCounts
(package private) long	totalCount
(package private) static java.util.concurrent.atomic.AtomicLongFieldUpdater<ConcurrentHistogram>	totalCountUpdater
(package private) WriterReaderPhaser	wrp

Fields inherited from class org.HdrHistogram.Histogram

counts, normalizingIndexOffset

Fields inherited from class org.HdrHistogram.AbstractHistogram

leadingZeroCountBase, maxValue, minNonZeroValue, subBucketHalfCount, subBucketHalfCountMagnitude, subBucketMask, unitMagnitude, unitMagnitudeMask

Fields inherited from class org.HdrHistogram.AbstractHistogramBase

autoResize, bucketCount, constructionIdentityCount, countsArrayLength, doubleToIntegerValueConversionRatio, endTimeStampMsec, highestTrackableValue, identity, integerToDoubleValueConversionRatio, intermediateUncompressedByteArray, intermediateUncompressedByteBuffer, lowestDiscernibleValue, numberOfSignificantValueDigits, percentileIterator, recordedValuesIterator, startTimeStampMsec, subBucketCount, tag, wordSizeInBytes

Constructor Summary

Constructors

Constructor and Description
ConcurrentHistogram(AbstractHistogram source) Construct a histogram with the same range settings as a given source histogram, duplicating the source's start/end timestamps (but NOT it's contents)
ConcurrentHistogram(int numberOfSignificantValueDigits) Construct an auto-resizing ConcurrentHistogram with a lowest discernible value of 1 and an auto-adjusting highestTrackableValue.
ConcurrentHistogram(long highestTrackableValue, int numberOfSignificantValueDigits) Construct a ConcurrentHistogram given the Highest value to be tracked and a number of significant decimal digits.
ConcurrentHistogram(long lowestDiscernibleValue, long highestTrackableValue, int numberOfSignificantValueDigits) Construct a ConcurrentHistogram given the Lowest and Highest values to be tracked and a number of significant decimal digits.

Method Summary

Modifier and Type	Method and Description
(package private) int	_getEstimatedFootprintInBytes()
(package private) void	addToCountAtIndex(int index, long value)
(package private) void	addToTotalCount(long value)
(package private) void	clearCounts()
ConcurrentHistogram	copy() Create a copy of this histogram, complete with data and everything.
ConcurrentHistogram	copyCorrectedForCoordinatedOmission(long expectedIntervalBetweenValueSamples) Get a copy of this histogram, corrected for coordinated omission.
private void	copyInactiveCountsContentsOnResize(ConcurrentHistogram.AtomicLongArrayWithNormalizingOffset oldInactiveCounts, int countsDelta)
static ConcurrentHistogram	decodeFromByteBuffer(java.nio.ByteBuffer buffer, long minBarForHighestTrackableValue) Construct a new histogram by decoding it from a ByteBuffer.
static ConcurrentHistogram	decodeFromCompressedByteBuffer(java.nio.ByteBuffer buffer, long minBarForHighestTrackableValue) Construct a new histogram by decoding it from a compressed form in a ByteBuffer.
(package private) void	fillBufferFromCountsArray(java.nio.ByteBuffer buffer)
(package private) void	fillCountsArrayFromBuffer(java.nio.ByteBuffer buffer, int length)
(package private) long	getCountAtIndex(int index)
(package private) long	getCountAtNormalizedIndex(int index)
(package private) int	getNormalizingIndexOffset()
long	getTotalCount() Get the total count of all recorded values in the histogram
(package private) void	incrementCountAtIndex(int index)
(package private) void	incrementTotalCount()
private void	readObject(java.io.ObjectInputStream o)
(package private) void	recordConvertedDoubleValue(double value)
void	recordConvertedDoubleValueWithCount(double value, long count)
(package private) void	resize(long newHighestTrackableValue)
void	setAutoResize(boolean autoResize) Control whether or not the histogram can auto-resize and auto-adjust it's highestTrackableValue
(package private) void	setCountAtIndex(int index, long value)
(package private) void	setCountAtNormalizedIndex(int index, long value)
(package private) void	setIntegerToDoubleValueConversionRatio(double integerToDoubleValueConversionRatio)
(package private) void	setNormalizingIndexOffset(int normalizingIndexOffset)
private void	setNormalizingIndexOffset(int normalizingIndexOffset, int shiftedAmount, boolean lowestHalfBucketPopulated, double newIntegerToDoubleValueConversionRatio)
(package private) void	setTotalCount(long totalCount)
private void	shiftLowestInactiveHalfBucketContentsLeft(int shiftAmount, int preShiftZeroIndex)
(package private) void	shiftNormalizingIndexByOffset(int offsetToAdd, boolean lowestHalfBucketPopulated, double newIntegerToDoubleValueConversionRatio)

Methods inherited from class org.HdrHistogram.AbstractHistogram

add, addWhileCorrectingForCoordinatedOmission, allValues, copyInto, copyIntoCorrectedForCoordinatedOmission, countsArrayIndex, decodeFromByteBuffer, decodeFromCompressedByteBuffer, determineArrayLengthNeeded, encodeIntoByteBuffer, encodeIntoCompressedByteBuffer, encodeIntoCompressedByteBuffer, equals, establishInternalTackingValues, establishInternalTackingValues, establishSize, getBucketIndex, getBucketsNeededToCoverValue, getCountAtValue, getCountBetweenValues, getEndTimeStamp, getEstimatedFootprintInBytes, getHighestTrackableValue, getLengthForNumberOfBuckets, getLowestDiscernibleValue, getMaxValue, getMaxValueAsDouble, getMean, getMinNonZeroValue, getMinValue, getNeededByteBufferCapacity, getNeededByteBufferCapacity, getNeededPayloadByteBufferCapacity, getNeededV0PayloadByteBufferCapacity, getNumberOfSignificantValueDigits, getPercentileAtOrBelowValue, getStartTimeStamp, getStdDeviation, getSubBucketIndex, getTag, getValueAtPercentile, hashCode, highestEquivalentValue, isAutoResize, linearBucketValues, logarithmicBucketValues, lowestEquivalentValue, medianEquivalentValue, nextNonEquivalentValue, nonConcurrentNormalizingIndexShift, normalizeIndex, numberOfSubbuckets, outputPercentileDistribution, outputPercentileDistribution, outputPercentileDistribution, percentiles, recordedValues, recordValue, recordValue, recordValueWithCount, recordValueWithExpectedInterval, reset, setEndTimeStamp, setStartTimeStamp, setTag, shiftValuesLeft, shiftValuesLeft, shiftValuesRight, shiftValuesRight, sizeOfEquivalentValueRange, subtract, supportsAutoResize, updateMinAndMax, valueFromIndex, valuesAreEquivalent

Methods inherited from class org.HdrHistogram.AbstractHistogramBase

getDoubleToIntegerValueConversionRatio, getIntegerToDoubleValueConversionRatio, nonConcurrentSetIntegerToDoubleValueConversionRatio

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

totalCountUpdater

static final java.util.concurrent.atomic.AtomicLongFieldUpdater<ConcurrentHistogram> totalCountUpdater

totalCount

volatile long totalCount

activeCounts

volatile ConcurrentHistogram.AtomicLongArrayWithNormalizingOffset activeCounts

inactiveCounts

volatile ConcurrentHistogram.AtomicLongArrayWithNormalizingOffset inactiveCounts

wrp

transient WriterReaderPhaser wrp

Constructor Detail

ConcurrentHistogram

public ConcurrentHistogram(int numberOfSignificantValueDigits)

Construct an auto-resizing ConcurrentHistogram with a lowest discernible value of 1 and an auto-adjusting highestTrackableValue. Can auto-resize up to track values up to (Long.MAX_VALUE / 2).

Parameters:

numberOfSignificantValueDigits - Specifies the precision to use. This is the number of significant decimal digits to which the histogram will maintain value resolution and separation. Must be a non-negative integer between 0 and 5.

ConcurrentHistogram

public ConcurrentHistogram(long highestTrackableValue,
                           int numberOfSignificantValueDigits)

Construct a ConcurrentHistogram given the Highest value to be tracked and a number of significant decimal digits. The histogram will be constructed to implicitly track (distinguish from 0) values as low as 1.

Parameters:

highestTrackableValue - The highest value to be tracked by the histogram. Must be a positive integer that is >= 2.

numberOfSignificantValueDigits - Specifies the precision to use. This is the number of significant decimal digits to which the histogram will maintain value resolution and separation. Must be a non-negative integer between 0 and 5.

ConcurrentHistogram

public ConcurrentHistogram(long lowestDiscernibleValue,
                           long highestTrackableValue,
                           int numberOfSignificantValueDigits)

Construct a ConcurrentHistogram given the Lowest and Highest values to be tracked and a number of significant decimal digits. Providing a lowestDiscernibleValue is useful is situations where the units used for the histogram's values are much smaller that the minimal accuracy required. E.g. when tracking time values stated in nanosecond units, where the minimal accuracy required is a microsecond, the proper value for lowestDiscernibleValue would be 1000.

Parameters:

lowestDiscernibleValue - The lowest value that can be tracked (distinguished from 0) by the histogram. Must be a positive integer that is >= 1. May be internally rounded down to nearest power of 2.

highestTrackableValue - The highest value to be tracked by the histogram. Must be a positive integer that is >= (2 * lowestDiscernibleValue).

numberOfSignificantValueDigits - Specifies the precision to use. This is the number of significant decimal digits to which the histogram will maintain value resolution and separation. Must be a non-negative integer between 0 and 5.

ConcurrentHistogram

public ConcurrentHistogram(AbstractHistogram source)

Construct a histogram with the same range settings as a given source histogram, duplicating the source's start/end timestamps (but NOT it's contents)

Parameters:

source - The source histogram to duplicate

Method Detail

setIntegerToDoubleValueConversionRatio

void setIntegerToDoubleValueConversionRatio(double integerToDoubleValueConversionRatio)

Overrides:

setIntegerToDoubleValueConversionRatio in class Histogram

getCountAtIndex

long getCountAtIndex(int index)

Overrides:

getCountAtIndex in class Histogram

getCountAtNormalizedIndex

long getCountAtNormalizedIndex(int index)

Overrides:

getCountAtNormalizedIndex in class Histogram

incrementCountAtIndex

void incrementCountAtIndex(int index)

Overrides:

incrementCountAtIndex in class Histogram

addToCountAtIndex

void addToCountAtIndex(int index,
                       long value)

Overrides:

addToCountAtIndex in class Histogram

setCountAtIndex

void setCountAtIndex(int index,
                     long value)

Overrides:

setCountAtIndex in class Histogram

setCountAtNormalizedIndex

void setCountAtNormalizedIndex(int index,
                               long value)

Overrides:

setCountAtNormalizedIndex in class Histogram

recordConvertedDoubleValue

void recordConvertedDoubleValue(double value)

Overrides:

recordConvertedDoubleValue in class AbstractHistogram

recordConvertedDoubleValueWithCount

public void recordConvertedDoubleValueWithCount(double value,
                                                long count)
                                         throws java.lang.ArrayIndexOutOfBoundsException

Overrides:

recordConvertedDoubleValueWithCount in class AbstractHistogram

Throws:

java.lang.ArrayIndexOutOfBoundsException

getNormalizingIndexOffset

int getNormalizingIndexOffset()

Overrides:

getNormalizingIndexOffset in class Histogram

setNormalizingIndexOffset

void setNormalizingIndexOffset(int normalizingIndexOffset)

Overrides:

setNormalizingIndexOffset in class Histogram

setNormalizingIndexOffset

private void setNormalizingIndexOffset(int normalizingIndexOffset,
                                       int shiftedAmount,
                                       boolean lowestHalfBucketPopulated,
                                       double newIntegerToDoubleValueConversionRatio)

shiftLowestInactiveHalfBucketContentsLeft

private void shiftLowestInactiveHalfBucketContentsLeft(int shiftAmount,
                                                       int preShiftZeroIndex)

shiftNormalizingIndexByOffset

void shiftNormalizingIndexByOffset(int offsetToAdd,
                                   boolean lowestHalfBucketPopulated,
                                   double newIntegerToDoubleValueConversionRatio)

Overrides:

shiftNormalizingIndexByOffset in class Histogram

resize

void resize(long newHighestTrackableValue)

Overrides:

resize in class Histogram

copyInactiveCountsContentsOnResize

private void copyInactiveCountsContentsOnResize(ConcurrentHistogram.AtomicLongArrayWithNormalizingOffset oldInactiveCounts,
                                                int countsDelta)

setAutoResize

public void setAutoResize(boolean autoResize)

Description copied from class: AbstractHistogram

Control whether or not the histogram can auto-resize and auto-adjust it's highestTrackableValue

Overrides:

setAutoResize in class AbstractHistogram

Parameters:

autoResize - autoResize setting

clearCounts

void clearCounts()

Overrides:

clearCounts in class Histogram

copy

public ConcurrentHistogram copy()

Description copied from class: AbstractHistogram

Create a copy of this histogram, complete with data and everything.

Overrides:

copy in class Histogram

Returns:

A distinct copy of this histogram.

copyCorrectedForCoordinatedOmission

public ConcurrentHistogram copyCorrectedForCoordinatedOmission(long expectedIntervalBetweenValueSamples)

Description copied from class: AbstractHistogram

Get a copy of this histogram, corrected for coordinated omission.

To compensate for the loss of sampled values when a recorded value is larger than the expected interval between value samples, the new histogram will include an auto-generated additional series of decreasingly-smaller (down to the expectedIntervalBetweenValueSamples) value records for each count found in the current histogram that is larger than the expectedIntervalBetweenValueSamples. Note: This is a post-correction method, as opposed to the at-recording correction method provided by recordValueWithExpectedInterval. The two methods are mutually exclusive, and only one of the two should be be used on a given data set to correct for the same coordinated omission issue. by

See notes in the description of the Histogram calls for an illustration of why this corrective behavior is important.

Overrides:

copyCorrectedForCoordinatedOmission in class Histogram

Parameters:

expectedIntervalBetweenValueSamples - If expectedIntervalBetweenValueSamples is larger than 0, add auto-generated value records as appropriate if value is larger than expectedIntervalBetweenValueSamples

Returns:

a copy of this histogram, corrected for coordinated omission.

getTotalCount

public long getTotalCount()

Description copied from class: AbstractHistogram

Get the total count of all recorded values in the histogram

Overrides:

getTotalCount in class Histogram

Returns:

the total count of all recorded values in the histogram

setTotalCount

void setTotalCount(long totalCount)

Overrides:

setTotalCount in class Histogram

incrementTotalCount

void incrementTotalCount()

Overrides:

incrementTotalCount in class Histogram

addToTotalCount

void addToTotalCount(long value)

Overrides:

addToTotalCount in class Histogram

_getEstimatedFootprintInBytes

int _getEstimatedFootprintInBytes()

Overrides:

_getEstimatedFootprintInBytes in class Histogram

decodeFromByteBuffer

public static ConcurrentHistogram decodeFromByteBuffer(java.nio.ByteBuffer buffer,
                                                       long minBarForHighestTrackableValue)

Construct a new histogram by decoding it from a ByteBuffer.

Parameters:

buffer - The buffer to decode from

minBarForHighestTrackableValue - Force highestTrackableValue to be set at least this high

Returns:

The newly constructed histogram

decodeFromCompressedByteBuffer

public static ConcurrentHistogram decodeFromCompressedByteBuffer(java.nio.ByteBuffer buffer,
                                                                 long minBarForHighestTrackableValue)
                                                          throws java.util.zip.DataFormatException

Construct a new histogram by decoding it from a compressed form in a ByteBuffer.

Parameters:

buffer - The buffer to decode from

minBarForHighestTrackableValue - Force highestTrackableValue to be set at least this high

Returns:

The newly constructed histogram

Throws:

java.util.zip.DataFormatException - on error parsing/decompressing the buffer

readObject

private void readObject(java.io.ObjectInputStream o)
                 throws java.io.IOException,
                        java.lang.ClassNotFoundException

Throws:

java.io.IOException

java.lang.ClassNotFoundException

fillCountsArrayFromBuffer

void fillCountsArrayFromBuffer(java.nio.ByteBuffer buffer,
                               int length)

Overrides:

fillCountsArrayFromBuffer in class Histogram

fillBufferFromCountsArray

void fillBufferFromCountsArray(java.nio.ByteBuffer buffer)

Overrides:

fillBufferFromCountsArray in class AbstractHistogram