Field3D
|
#include <SparseFile.h>
Public Types | |
typedef std::vector< Sparse::SparseBlock< Data_T > * > | BlockPtrs |
typedef Reference * | Ptr |
Public Member Functions | |
float | averageLoads () const |
Returns the average number of loads per accessed block in this file, for cache statistics. More... | |
int | blockSize (int blockIdx) const |
Returns the number of bytes used by the data in the block. More... | |
void | closeFile () |
Closes the file. Will be re-opened as needed. More... | |
void | decBlockRef (int blockIdx) |
Decrement reference count on a block. More... | |
bool | fileIsOpen () |
Checks if the file used by this reference is open already. More... | |
void | incBlockRef (int blockIdx) |
Increment reference count on a block, indicates the block is currently in use, so prevents it from being unloaded. More... | |
void | loadBlock (int blockIdx) |
Loads the block with the given index into memory. We don't pass in a reference to where the data should go since this is already know in the blocks data member. More... | |
int | numLoadedBlocks () const |
Returns the total number of blocks that are currently loaded, for statistics. More... | |
void | openFile () |
Opens the file. This is done just before the first request to loadBlock. This is delayed so that the original file open has closed the file and doesn't cause any Hdf5 hiccups. More... | |
void | resetCacheStatistics () |
Resets counts of total block loads. More... | |
void | setNumBlocks (int numBlocks) |
Sets the number of blocks used by the SparseField we're supporting. More... | |
int | totalLoadedBlocks () const |
Returns the total number of blocks that were ever loaded (max 1 per block, not the number of blocks), for statistics. More... | |
int | totalLoads () const |
Returns the total number of loads of the blocks of this file, for cache statistics. More... | |
void | unloadBlock (int blockIdx) |
Unloads the block with the given index from memory. More... | |
~Reference () | |
Destructor. More... | |
Static Public Member Functions | |
static Ptr | create (const std::string a_filename, const std::string a_layerPath) |
Returns a shared pointer to a reference. Preferred way of creating References. More... | |
Public Attributes | |
std::vector< int > | blockLoaded |
Whether each block is loaded. We don't use bools since vector<bool> is weird. More... | |
boost::mutex * | blockMutex |
Allocated array of mutexes, one per block, to lock each block individually, for guaranteeing thread-safe updates of the ref counts. More... | |
BlockPtrs | blocks |
Pointers to each block. This is so we can go in and manipulate them as we please. More... | |
std::vector< bool > | blockUsed |
Flags of whether the blocks have been accessed since they were last considered for deallocation by the Second-chance/Clock caching system. More... | |
std::vector< int > | fileBlockIndices |
Index in file for each block. More... | |
std::string | filename |
std::string | layerPath |
std::vector< int > | loadCounts |
Per-block counts of the number of times each block has been loaded, for cache statistics. More... | |
int | occupiedBlocks |
std::vector< int > | refCounts |
Per-block counts of the number of current references to the blocks. If a block's ref count is non-zero, then the block shouldn't be unloaded. More... | |
int | valuesPerBlock |
Private Types | |
typedef boost::mutex | Mutex |
Private Member Functions | |
Reference & | operator= (const Reference &o) |
Assignment operator. Clears ref counts and rebuilds mutex array. More... | |
Reference (const std::string filename, const std::string layerPath) | |
Constructor. Requires the filename and layer path of the field to be known. More... | |
Reference (const Reference &o) | |
Copy constructor. Clears ref counts and rebuilds mutex array. More... | |
Private Attributes | |
hid_t | m_fileHandle |
Holds the Hdf5 handle to the file. More... | |
Hdf5Util::H5ScopedGopen | m_layerGroup |
Hold the group containing the data open for the duration of the Reference's existence. More... | |
Mutex | m_mutex |
Mutex to prevent two threads from modifying conflicting data. More... | |
size_t | m_numActiveBlocks |
Number of currently active blocks. More... | |
SparseDataReader< Data_T > * | m_reader |
Pointer to the reader object. NULL at construction time. Created in openFile(). More... | |
Handles information about and block loading for a single SparseField as referenced on disk
Definition at line 96 of file SparseFile.h.
typedef Reference* SparseFile::Reference< Data_T >::Ptr |
Definition at line 107 of file SparseFile.h.
typedef std::vector<Sparse::SparseBlock<Data_T>*> SparseFile::Reference< Data_T >::BlockPtrs |
Definition at line 109 of file SparseFile.h.
|
private |
Definition at line 193 of file SparseFile.h.
SparseFile::Reference< Data_T >::~Reference | ( | ) |
Destructor.
Definition at line 533 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockMutex, SparseFile::Reference< Data_T >::closeFile(), and SparseFile::Reference< Data_T >::m_reader.
|
private |
Constructor. Requires the filename and layer path of the field to be known.
Definition at line 520 of file SparseFile.h.
Referenced by SparseFile::Reference< Data_T >::create().
|
private |
Copy constructor. Clears ref counts and rebuilds mutex array.
Definition at line 547 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockMutex, and SparseFile::Reference< Data_T >::m_reader.
|
static |
Returns a shared pointer to a reference. Preferred way of creating References.
Definition at line 604 of file SparseFile.h.
References SparseFile::Reference< Data_T >::Reference().
Referenced by SparseFileManager::getNextId().
bool SparseFile::Reference< Data_T >::fileIsOpen | ( | ) |
Checks if the file used by this reference is open already.
Definition at line 613 of file SparseFile.h.
References SparseFile::Reference< Data_T >::m_fileHandle.
Referenced by SparseFileManager::activateBlock(), and SparseFile::Reference< Data_T >::openFile().
void SparseFile::Reference< Data_T >::setNumBlocks | ( | int | numBlocks | ) |
Sets the number of blocks used by the SparseField we're supporting.
Definition at line 621 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockLoaded, SparseFile::Reference< Data_T >::blockMutex, SparseFile::Reference< Data_T >::blocks, SparseFile::Reference< Data_T >::blockUsed, SparseFile::Reference< Data_T >::fileBlockIndices, SparseFile::Reference< Data_T >::loadCounts, SparseFile::Reference< Data_T >::m_mutex, and SparseFile::Reference< Data_T >::refCounts.
void SparseFile::Reference< Data_T >::openFile | ( | ) |
Opens the file. This is done just before the first request to loadBlock. This is delayed so that the original file open has closed the file and doesn't cause any Hdf5 hiccups.
Definition at line 639 of file SparseFile.h.
References SparseFile::Reference< Data_T >::fileIsOpen(), SparseFile::Reference< Data_T >::filename, g_hdf5Mutex, Hdf5Util::H5Base::id(), SparseFile::Reference< Data_T >::layerPath, SparseFile::Reference< Data_T >::m_fileHandle, SparseFile::Reference< Data_T >::m_layerGroup, SparseFile::Reference< Data_T >::m_mutex, SparseFile::Reference< Data_T >::m_reader, SparseFile::Reference< Data_T >::occupiedBlocks, Hdf5Util::H5ScopedGopen::open(), Msg::print(), Msg::SevWarning, and SparseFile::Reference< Data_T >::valuesPerBlock.
Referenced by SparseFileManager::activateBlock().
void SparseFile::Reference< Data_T >::closeFile | ( | ) |
Closes the file. Will be re-opened as needed.
Definition at line 681 of file SparseFile.h.
References SparseFile::Reference< Data_T >::filename, SparseFile::Reference< Data_T >::m_fileHandle, and Msg::print().
Referenced by SparseFile::Reference< Data_T >::unloadBlock(), and SparseFile::Reference< Data_T >::~Reference().
void SparseFile::Reference< Data_T >::loadBlock | ( | int | blockIdx | ) |
Loads the block with the given index into memory. We don't pass in a reference to where the data should go since this is already know in the blocks data member.
Definition at line 693 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockLoaded, SparseFile::Reference< Data_T >::blocks, SparseFile::Reference< Data_T >::fileBlockIndices, SparseFile::Reference< Data_T >::m_mutex, SparseFile::Reference< Data_T >::m_numActiveBlocks, SparseFile::Reference< Data_T >::m_reader, and SparseFile::Reference< Data_T >::valuesPerBlock.
Referenced by SparseFileManager::activateBlock().
void SparseFile::Reference< Data_T >::unloadBlock | ( | int | blockIdx | ) |
Unloads the block with the given index from memory.
Definition at line 712 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockLoaded, SparseFile::Reference< Data_T >::blocks, SparseFile::Reference< Data_T >::closeFile(), and SparseFile::Reference< Data_T >::m_numActiveBlocks.
Referenced by SparseFileManager::deallocateBlock().
void SparseFile::Reference< Data_T >::incBlockRef | ( | int | blockIdx | ) |
Increment reference count on a block, indicates the block is currently in use, so prevents it from being unloaded.
Definition at line 731 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockMutex, and SparseFile::Reference< Data_T >::refCounts.
Referenced by SparseFileManager::incBlockRef().
void SparseFile::Reference< Data_T >::decBlockRef | ( | int | blockIdx | ) |
Decrement reference count on a block.
Definition at line 740 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockMutex, and SparseFile::Reference< Data_T >::refCounts.
Referenced by SparseFileManager::decBlockRef().
int SparseFile::Reference< Data_T >::blockSize | ( | int | blockIdx | ) | const |
Returns the number of bytes used by the data in the block.
Definition at line 749 of file SparseFile.h.
References SparseFile::Reference< Data_T >::valuesPerBlock.
Referenced by SparseFileManager::activateBlock(), SparseFileManager::deallocateBlock(), and SparseFileManager::removeFieldFromCache().
int SparseFile::Reference< Data_T >::totalLoads | ( | ) | const |
Returns the total number of loads of the blocks of this file, for cache statistics.
Definition at line 757 of file SparseFile.h.
References SparseFile::Reference< Data_T >::loadCounts.
int SparseFile::Reference< Data_T >::numLoadedBlocks | ( | ) | const |
Returns the total number of blocks that are currently loaded, for statistics.
Definition at line 771 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockLoaded.
int SparseFile::Reference< Data_T >::totalLoadedBlocks | ( | ) | const |
Returns the total number of blocks that were ever loaded (max 1 per block, not the number of blocks), for statistics.
Definition at line 786 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockLoaded, and SparseFile::Reference< Data_T >::loadCounts.
float SparseFile::Reference< Data_T >::averageLoads | ( | ) | const |
Returns the average number of loads per accessed block in this file, for cache statistics.
Definition at line 811 of file SparseFile.h.
References SparseFile::Reference< Data_T >::loadCounts.
void SparseFile::Reference< Data_T >::resetCacheStatistics | ( | ) |
Resets counts of total block loads.
Definition at line 829 of file SparseFile.h.
References SparseFile::Reference< Data_T >::loadCounts.
|
private |
Assignment operator. Clears ref counts and rebuilds mutex array.
Definition at line 558 of file SparseFile.h.
References SparseFile::Reference< Data_T >::blockLoaded, SparseFile::Reference< Data_T >::blockMutex, SparseFile::Reference< Data_T >::blocks, SparseFile::Reference< Data_T >::blockUsed, SparseFile::Reference< Data_T >::fileBlockIndices, SparseFile::Reference< Data_T >::filename, SparseFile::Reference< Data_T >::layerPath, SparseFile::Reference< Data_T >::loadCounts, SparseFile::Reference< Data_T >::m_fileHandle, SparseFile::Reference< Data_T >::m_layerGroup, SparseFile::Reference< Data_T >::m_reader, SparseFile::Reference< Data_T >::occupiedBlocks, Hdf5Util::H5ScopedGopen::open(), SparseFile::Reference< Data_T >::refCounts, and SparseFile::Reference< Data_T >::valuesPerBlock.
std::string SparseFile::Reference< Data_T >::filename |
Definition at line 113 of file SparseFile.h.
Referenced by SparseFile::Reference< Data_T >::closeFile(), SparseField< Data_T >::copySparseField(), SparseFile::Reference< Data_T >::openFile(), and SparseFile::Reference< Data_T >::operator=().
std::string SparseFile::Reference< Data_T >::layerPath |
Definition at line 114 of file SparseFile.h.
Referenced by SparseField< Data_T >::copySparseField(), SparseFile::Reference< Data_T >::openFile(), and SparseFile::Reference< Data_T >::operator=().
int SparseFile::Reference< Data_T >::valuesPerBlock |
int SparseFile::Reference< Data_T >::occupiedBlocks |
Definition at line 116 of file SparseFile.h.
Referenced by SparseField< Data_T >::copySparseField(), SparseFile::Reference< Data_T >::openFile(), and SparseFile::Reference< Data_T >::operator=().
std::vector<int> SparseFile::Reference< Data_T >::fileBlockIndices |
Index in file for each block.
Definition at line 119 of file SparseFile.h.
Referenced by SparseFileManager::activateBlock(), SparseFileManager::decBlockRef(), SparseFileManager::incBlockRef(), SparseFile::Reference< Data_T >::loadBlock(), SparseFile::Reference< Data_T >::operator=(), SparseFileManager::removeFieldFromCache(), SparseFile::Reference< Data_T >::setNumBlocks(), and SparseField< Data_T >::setupReferenceBlocks().
std::vector<int> SparseFile::Reference< Data_T >::blockLoaded |
Whether each block is loaded. We don't use bools since vector<bool> is weird.
Definition at line 122 of file SparseFile.h.
Referenced by SparseFileManager::activateBlock(), SparseFile::Reference< Data_T >::loadBlock(), SparseFile::Reference< Data_T >::numLoadedBlocks(), SparseFile::Reference< Data_T >::operator=(), SparseFileManager::removeFieldFromCache(), SparseFile::Reference< Data_T >::setNumBlocks(), SparseFile::Reference< Data_T >::totalLoadedBlocks(), and SparseFile::Reference< Data_T >::unloadBlock().
BlockPtrs SparseFile::Reference< Data_T >::blocks |
Pointers to each block. This is so we can go in and manipulate them as we please.
Definition at line 125 of file SparseFile.h.
Referenced by SparseFile::Reference< Data_T >::loadBlock(), SparseFile::Reference< Data_T >::operator=(), SparseFileManager::removeFieldFromCache(), SparseFile::Reference< Data_T >::setNumBlocks(), SparseField< Data_T >::setupReferenceBlocks(), and SparseFile::Reference< Data_T >::unloadBlock().
std::vector<bool> SparseFile::Reference< Data_T >::blockUsed |
Flags of whether the blocks have been accessed since they were last considered for deallocation by the Second-chance/Clock caching system.
Definition at line 129 of file SparseFile.h.
Referenced by SparseFileManager::activateBlock(), SparseFileManager::deallocateBlock(), SparseFile::Reference< Data_T >::operator=(), SparseFileManager::removeFieldFromCache(), and SparseFile::Reference< Data_T >::setNumBlocks().
std::vector<int> SparseFile::Reference< Data_T >::loadCounts |
Per-block counts of the number of times each block has been loaded, for cache statistics.
Definition at line 132 of file SparseFile.h.
Referenced by SparseFileManager::activateBlock(), SparseFile::Reference< Data_T >::averageLoads(), SparseFile::Reference< Data_T >::operator=(), SparseFileManager::removeFieldFromCache(), SparseFile::Reference< Data_T >::resetCacheStatistics(), SparseFile::Reference< Data_T >::setNumBlocks(), SparseFile::Reference< Data_T >::totalLoadedBlocks(), and SparseFile::Reference< Data_T >::totalLoads().
std::vector<int> SparseFile::Reference< Data_T >::refCounts |
Per-block counts of the number of current references to the blocks. If a block's ref count is non-zero, then the block shouldn't be unloaded.
Definition at line 136 of file SparseFile.h.
Referenced by SparseFileManager::deallocateBlock(), SparseFile::Reference< Data_T >::decBlockRef(), SparseFile::Reference< Data_T >::incBlockRef(), SparseFile::Reference< Data_T >::operator=(), SparseFileManager::removeFieldFromCache(), and SparseFile::Reference< Data_T >::setNumBlocks().
boost::mutex* SparseFile::Reference< Data_T >::blockMutex |
Allocated array of mutexes, one per block, to lock each block individually, for guaranteeing thread-safe updates of the ref counts.
Definition at line 140 of file SparseFile.h.
Referenced by SparseFileManager::activateBlock(), SparseFileManager::deallocateBlock(), SparseFile::Reference< Data_T >::decBlockRef(), SparseFile::Reference< Data_T >::incBlockRef(), SparseFile::Reference< Data_T >::operator=(), SparseFile::Reference< Data_T >::Reference(), SparseFileManager::removeFieldFromCache(), SparseFile::Reference< Data_T >::setNumBlocks(), and SparseFile::Reference< Data_T >::~Reference().
|
private |
Holds the Hdf5 handle to the file.
Definition at line 209 of file SparseFile.h.
Referenced by SparseFile::Reference< Data_T >::closeFile(), SparseFile::Reference< Data_T >::fileIsOpen(), SparseFile::Reference< Data_T >::openFile(), and SparseFile::Reference< Data_T >::operator=().
|
private |
Hold the group containing the data open for the duration of the Reference's existence.
Definition at line 213 of file SparseFile.h.
Referenced by SparseFile::Reference< Data_T >::openFile(), and SparseFile::Reference< Data_T >::operator=().
|
private |
Pointer to the reader object. NULL at construction time. Created in openFile().
Definition at line 217 of file SparseFile.h.
Referenced by SparseFile::Reference< Data_T >::loadBlock(), SparseFile::Reference< Data_T >::openFile(), SparseFile::Reference< Data_T >::operator=(), SparseFile::Reference< Data_T >::Reference(), and SparseFile::Reference< Data_T >::~Reference().
|
private |
Mutex to prevent two threads from modifying conflicting data.
Definition at line 221 of file SparseFile.h.
Referenced by SparseFileManager::activateBlock(), SparseFile::FileReferences::append(), SparseFileManager::getNextId(), SparseFile::Reference< Data_T >::loadBlock(), SparseFile::FileReferences::numRefs(), SparseFile::Reference< Data_T >::openFile(), SparseFileManager::removeFieldFromCache(), and SparseFile::Reference< Data_T >::setNumBlocks().
|
private |
Number of currently active blocks.
Definition at line 224 of file SparseFile.h.
Referenced by SparseFile::Reference< Data_T >::loadBlock(), and SparseFile::Reference< Data_T >::unloadBlock().