Field3D
Field3DInputFile Class Reference

Provides reading of .f3d (internally, hdf5) files.Refer to using_files for examples of how to use this in your code. More...

#include <Field3DFile.h>

Inheritance diagram for Field3DInputFile:
Field3DFileBase

Public Member Functions

bool open (const std::string &filename)
 Opens the given file. More...
 
bool readGroupMembership (GroupMembershipMap &gpMembershipMap)
 Read the group membership for the partitions. More...
 
Constructors & destructor
 Field3DInputFile ()
 
virtual ~Field3DInputFile ()
 
Reading layers from disk
template<class Data_T >
Field< Data_T >::Vec readScalarLayers (const std::string &layerName=std::string("")) const
 Retrieves all the layers of scalar type and maintains their on-disk data types. More...
 
template<class Data_T >
Field< Data_T >::Vec readScalarLayers (const std::string &partitionName, const std::string &layerName) const
 This one allows the allows the partitionName to be passed in. More...
 
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Vec readVectorLayers (const std::string &layerName=std::string("")) const
 Retrieves all the layers of vector type and maintains their on-disk data types. More...
 
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Vec readVectorLayers (const std::string &partitionName, const std::string &layerName) const
 This version allows you to pass in the partition name. More...
 
template<template< typename T > class Field_T, class Data_T >
Field_T< Data_T >::Vec readScalarLayersAs (const std::string &layerName=std::string("")) const
 Retrieves all layers for all partitions. Converts it to the given template type if needed. More...
 
template<template< typename T > class Field_T, class Data_T >
Field_T< Data_T >::Vec readScalarLayersAs (const std::string &partitionName, const std::string &layerName) const
 Retrieves a layers given their and its parent partition's name. Converts it to the given template type if needed. More...
 
template<template< typename T > class Field_T, class Data_T >
Field_T< FIELD3D_VEC3_T< Data_T > >::Vec readVectorLayersAs (const std::string &layerName=std::string("")) const
 Retrieves a layers for all partitions. Converts it to the given template type if needed. More...
 
template<template< typename T > class Field_T, class Data_T >
Field_T< FIELD3D_VEC3_T< Data_T > >::Vec readVectorLayersAs (const std::string &partitionName, const std::string &layerName) const
 Retrieves a layers given their and its parent partition's name. Converts it to the given template type if needed. More...
 
Reading proxy data from disk
template<class Data_T >
EmptyField< Data_T >::Vec readProxyLayer (const std::string &partitionName, const std::string &layerName, bool isVectorLayer) const
 Retrieves a proxy version (EmptyField) of each layer . More...
 
template<class Data_T >
EmptyField< Data_T >::Ptr readProxyLayer (hid_t location, const std::string &name, const std::string &attribute, FieldMapping::Ptr mapping) const
 Retrieves a proxy version (EmptyField) from a given HDF5 location. More...
 
template<class Data_T >
EmptyField< Data_T >::Vec readProxyScalarLayers (const std::string &name=std::string("")) const
 Retrieves a proxy version (EmptyField) of each scalar layer. More...
 
template<class Data_T >
EmptyField< Data_T >::Vec readProxyVectorLayers (const std::string &name=std::string("")) const
 Retrieves a proxy version (EmptyField) of each vector layer. More...
 
Internal utility methods
herr_t parsePartition (hid_t loc_id, const std::string partitionName)
 Gets called from parsePartitions. Not intended for any other use. More...
 
herr_t parseLayer (hid_t loc_id, const std::string &partitionName, const std::string &layerName)
 Gets called from parsePartitions. Not intended for any other use. More...
 
- Public Member Functions inherited from Field3DFileBase
void clear ()
 Clear the data structures and close the file. More...
 
bool close ()
 Closes the file. No need to call this unless you specifically want to close the file early. It will close once the File object goes out of scope. More...
 
FieldMetadata< Field3DFileBase > & metadata ()
 accessor to the m_metadata class More...
 
const FieldMetadata< Field3DFileBase > & metadata () const
 Read only access to the m_metadata class. More...
 
virtual void metadataHasChanged (const std::string &)
 This function should implemented by concrete classes to get the callback when metadata changes. More...
 
 Field3DFileBase ()
 
virtual ~Field3DFileBase ()=0
 Pure virtual destructor to ensure we never instantiate this class. More...
 
void getPartitionNames (std::vector< std::string > &names) const
 Gets the names of all the partitions in the file. More...
 
void getScalarLayerNames (std::vector< std::string > &names, const std::string &partitionName) const
 Gets the names of all the scalar layers in a given partition. More...
 
void getVectorLayerNames (std::vector< std::string > &names, const std::string &partitionName) const
 Gets the names of all the vector layers in a given partition. More...
 
File::Partition::Ptr getPartition (const std::string &partitionName) const
 Returns a pointer to the given partition. More...
 
std::string intPartitionName (const std::string &partitionName, const std::string &layerName, FieldRes::Ptr field)
 Returns a unique partition name given the requested name. This ensures that partitions with matching mappings get the same name but each subsequent differing mapping gets a new, separate name. More...
 
std::string removeUniqueId (const std::string &partitionName) const
 Strips any unique identifiers from the partition name and returns the original name. More...
 
void addGroupMembership (const GroupMembershipMap &groupMembers)
 Add to the group membership. More...
 
void printHierarchy () const
 

Private Member Functions

template<class Data_T >
Field< Data_T >::Ptr readLayer (const std::string &intPartitionName, const std::string &layerName, bool isVectorLayer) const
 This call does the actual reading of a layer. Notice that it expects a unique -internal- partition name. More...
 
bool readMetadata (hid_t metadata_id, FieldBase::Ptr field) const
 Read metadata for this layer. More...
 
bool readMetadata (hid_t metadata_id)
 Read global metadata for this file. More...
 
bool readPartitionAndLayerInfo ()
 Sets up all the partitions and layers, but does not load any data. More...
 
template<class Data_T >
Field< Data_T >::Ptr readScalarLayer (const std::string &intPartitionName, const std::string &layerName) const
 Retrieves a single layer given its and its parent partition's name. Maintains the on-disk data types. More...
 
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Ptr readVectorLayer (const std::string &intPartitionName, const std::string &layerName) const
 Retrieves a single layer given its and its parent partition's name. Maintains the on-disk data types. More...
 

Private Attributes

std::string m_filename
 Filename, only to be set by open(). More...
 

Additional Inherited Members

- Public Types inherited from Field3DFileBase
typedef std::map< std::string, std::string > GroupMembershipMap
 
- Protected Types inherited from Field3DFileBase
typedef std::map< std::string, int > PartitionCountMap
 
typedef std::vector< File::Partition::PtrPartitionList
 
- Protected Member Functions inherited from Field3DFileBase
void closeInternal ()
 Closes the file if open. More...
 
File::Partition::Ptr partition (const std::string &partitionName)
 Returns a pointer to the given partition. More...
 
File::Partition::Ptr partition (const std::string &partitionName) const
 Returns a pointer to the given partition. More...
 
void getIntPartitionNames (std::vector< std::string > &names) const
 Gets the names of all the -internal- partitions in the file. More...
 
void getIntScalarLayerNames (std::vector< std::string > &names, const std::string &intPartitionName) const
 Gets the names of all the scalar layers in a given partition, but assumes that partition name is the -internal- partition name. More...
 
void getIntVectorLayerNames (std::vector< std::string > &names, const std::string &intPartitionName) const
 Gets the names of all the vector layers in a given partition, but assumes that partition name is the -internal- partition name. More...
 
int numIntPartitions (const std::string &partitionName) const
 Returns the number of internal partitions for a given partition name. More...
 
std::string makeIntPartitionName (const std::string &partitionsName, int i) const
 Makes an internal partition name given the external partition name. Effectively just tacks on .X to the name, where X is the number. More...
 
- Protected Attributes inherited from Field3DFileBase
hid_t m_file
 The hdf5 id of the current file. Will be -1 if no file is open. More...
 
GroupMembershipMap m_groupMembership
 Keeps track of group membership for each layer of partition name. The key is the "group" and the value is a space separated list of "partitionName.0:Layer1 partitionName.1:Layer0 ...". More...
 
std::vector< LayerInfom_layerInfo
 This stores layer info. More...
 
FieldMetadata< Field3DFileBasem_metadata
 metadata More...
 
PartitionCountMap m_partitionCount
 Contains a counter for each partition name. This is used to keep multiple fields with the same name unique in the file. More...
 
std::vector< std::string > m_partitionNames
 This stores partition names. More...
 
PartitionList m_partitions
 Vector of partitions. More...
 

Detailed Description

Provides reading of .f3d (internally, hdf5) files.

Refer to using_files for examples of how to use this in your code.

Note
We distinguish between scalar and vector layers even though both are templated. A scalarField<float> layer is interchangeable with a scalarField<double> (conceptually) but not with a scalar<V3f>, and thus not with vectorField<float>.

Definition at line 434 of file Field3DFile.h.

Constructor & Destructor Documentation

Field3DInputFile::Field3DInputFile ( )

Definition at line 556 of file Field3DFile.cpp.

557 {
558  // Empty
559 }
Field3DInputFile::~Field3DInputFile ( )
virtual

Definition at line 563 of file Field3DFile.cpp.

References Field3DFileBase::clear().

564 {
565  clear();
566 }
void clear()
Clear the data structures and close the file.

Member Function Documentation

template<class Data_T >
Field< Data_T >::Vec Field3DInputFile::readScalarLayers ( const std::string &  layerName = std::string("")) const

Retrieves all the layers of scalar type and maintains their on-disk data types.

Parameters
layerNameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 907 of file Field3DFile.h.

908 {
909  using namespace std;
910 
911  typedef typename Field<Data_T>::Ptr FieldPtr;
912  typedef typename Field<Data_T>::Vec FieldList;
913 
914  FieldList ret;
915  std::vector<std::string> parts;
916  getIntPartitionNames(parts);
917 
918  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
919  std::vector<std::string> layers;
920  getIntScalarLayerNames(layers, *p);
921  for (vector<string>::iterator l = layers.begin(); l != layers.end(); ++l) {
922  // Only read if it matches the name
923  if ((name.length() == 0) || (*l == name)) {
924  FieldPtr mf = readScalarLayer<Data_T>(*p, *l);
925  if (mf) {
926  ret.push_back(mf);
927  }
928  }
929  }
930  }
931 
932  return ret;
933 }
void getIntScalarLayerNames(std::vector< std::string > &names, const std::string &intPartitionName) const
Gets the names of all the scalar layers in a given partition, but assumes that partition name is the ...
void getIntPartitionNames(std::vector< std::string > &names) const
Gets the names of all the -internal- partitions in the file.
std::vector< Ptr > Vec
This is a convenience typedef for the list that Field3DInputFile::readScalarLayers() and Field3DInput...
Definition: Field.h:408
boost::intrusive_ptr< Field > Ptr
Definition: Field.h:400
template<class Data_T >
Field< Data_T >::Vec Field3DInputFile::readScalarLayers ( const std::string &  partitionName,
const std::string &  layerName 
) const

This one allows the allows the partitionName to be passed in.

Definition at line 939 of file Field3DFile.h.

941 {
942  using namespace std;
943 
944  typedef typename Field<Data_T>::Ptr FieldPtr;
945  typedef typename Field<Data_T>::Vec FieldList;
946 
947  FieldList ret;
948 
949  if ((layerName.length() == 0) || (partitionName.length() == 0))
950  return ret;
951 
952  std::vector<std::string> parts;
953  getIntPartitionNames(parts);
954 
955  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
956  std::vector<std::string> layers;
957  getIntScalarLayerNames(layers, *p);
958  if (removeUniqueId(*p) == partitionName) {
959  for (vector<string>::iterator l = layers.begin();
960  l != layers.end(); ++l) {
961  // Only read if it matches the name
962  if (*l == layerName) {
963  FieldPtr mf = readScalarLayer<Data_T>(*p, *l);
964  if (mf)
965  ret.push_back(mf);
966  }
967  }
968  }
969  }
970 
971  return ret;
972 }
void getIntScalarLayerNames(std::vector< std::string > &names, const std::string &intPartitionName) const
Gets the names of all the scalar layers in a given partition, but assumes that partition name is the ...
void getIntPartitionNames(std::vector< std::string > &names) const
Gets the names of all the -internal- partitions in the file.
std::string removeUniqueId(const std::string &partitionName) const
Strips any unique identifiers from the partition name and returns the original name.
std::vector< Ptr > Vec
This is a convenience typedef for the list that Field3DInputFile::readScalarLayers() and Field3DInput...
Definition: Field.h:408
boost::intrusive_ptr< Field > Ptr
Definition: Field.h:400
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Vec Field3DInputFile::readVectorLayers ( const std::string &  layerName = std::string("")) const

Retrieves all the layers of vector type and maintains their on-disk data types.

Parameters
layerNameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 978 of file Field3DFile.h.

979 {
980  using namespace std;
981 
982  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Ptr FieldPtr;
983  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Vec FieldList;
984 
985  FieldList ret;
986 
987  std::vector<std::string> parts;
988  getIntPartitionNames(parts);
989 
990  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
991  std::vector<std::string> layers;
992  getIntVectorLayerNames(layers, *p);
993  for (vector<string>::iterator l = layers.begin(); l != layers.end(); ++l) {
994  // Only read if it matches the name
995  if ((name.length() == 0) || (*l == name)) {
996  FieldPtr mf = readVectorLayer<Data_T>(*p, *l);
997  if (mf)
998  ret.push_back(mf);
999  }
1000  }
1001  }
1002 
1003  return ret;
1004 }
void getIntVectorLayerNames(std::vector< std::string > &names, const std::string &intPartitionName) const
Gets the names of all the vector layers in a given partition, but assumes that partition name is the ...
Definition: Field.h:394
void getIntPartitionNames(std::vector< std::string > &names) const
Gets the names of all the -internal- partitions in the file.
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Vec Field3DInputFile::readVectorLayers ( const std::string &  partitionName,
const std::string &  layerName 
) const

This version allows you to pass in the partition name.

Definition at line 1010 of file Field3DFile.h.

1012 {
1013  using namespace std;
1014 
1015  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Ptr FieldPtr;
1016  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Vec FieldList;
1017 
1018  FieldList ret;
1019 
1020  if ((layerName.length() == 0) || (partitionName.length() == 0))
1021  return ret;
1022 
1023  std::vector<std::string> parts;
1024  getIntPartitionNames(parts);
1025 
1026  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
1027  std::vector<std::string> layers;
1028  getIntVectorLayerNames(layers, *p);
1029  if (removeUniqueId(*p) == partitionName) {
1030  for (vector<string>::iterator l = layers.begin();
1031  l != layers.end(); ++l) {
1032  // Only read if it matches the name
1033  if (*l == layerName) {
1034  FieldPtr mf = readVectorLayer<Data_T>(*p, *l);
1035  if (mf)
1036  ret.push_back(mf);
1037  }
1038  }
1039  }
1040  }
1041 
1042  return ret;
1043 }
void getIntVectorLayerNames(std::vector< std::string > &names, const std::string &intPartitionName) const
Gets the names of all the vector layers in a given partition, but assumes that partition name is the ...
Definition: Field.h:394
void getIntPartitionNames(std::vector< std::string > &names) const
Gets the names of all the -internal- partitions in the file.
std::string removeUniqueId(const std::string &partitionName) const
Strips any unique identifiers from the partition name and returns the original name.
template<template< typename T > class Field_T, class Data_T >
Field_T<Data_T>::Vec Field3DInputFile::readScalarLayersAs ( const std::string &  layerName = std::string("")) const
inline

Retrieves all layers for all partitions. Converts it to the given template type if needed.

Definition at line 485 of file Field3DFile.h.

References field_dynamic_cast().

486  {
487  typedef typename Field<Data_T>::Vec FieldList;
488  typedef typename Field_T<Data_T>::Vec TypedFieldList;
489 
490  // First, read the layers as-is
491  FieldList originals;
492  originals = readScalarLayers<Data_T>(layerName);
493 
494  // Loop over fields, converting if needed
495  TypedFieldList output;
496  typename FieldList::iterator i = originals.begin();
497  for (; i != originals.end(); ++i) {
498  typename Field_T<Data_T>::Ptr targetField;
499  targetField = field_dynamic_cast<Field_T<Data_T> >(*i);
500  if (targetField) {
501  output.push_back(targetField);
502  } else {
503  typename Field_T<Data_T>::Ptr newTarget(new Field_T<Data_T>);
504  newTarget->name = (*i)->name;
505  newTarget->attribute = (*i)->attribute;
506  newTarget->copyMetadata(**i);
507  newTarget->copyFrom(*i);
508  output.push_back(newTarget);
509  }
510  }
511 
512  return output;
513  }
Field_T::Ptr field_dynamic_cast(RefBase::Ptr field)
Dynamic cast that uses string-comparison in order to be safe even after an object crosses a shared li...
Definition: RefCount.h:256
std::vector< Ptr > Vec
This is a convenience typedef for the list that Field3DInputFile::readScalarLayers() and Field3DInput...
Definition: Field.h:408
template<template< typename T > class Field_T, class Data_T >
Field_T<Data_T>::Vec Field3DInputFile::readScalarLayersAs ( const std::string &  partitionName,
const std::string &  layerName 
) const
inline

Retrieves a layers given their and its parent partition's name. Converts it to the given template type if needed.

Definition at line 519 of file Field3DFile.h.

References field_dynamic_cast().

521  {
522  typedef typename Field<Data_T>::Vec FieldList;
523  typedef typename Field_T<Data_T>::Vec TypedFieldList;
524 
525  // First, read the layers as-is
526  FieldList originals;
527  originals = readScalarLayers<Data_T>(partitionName, layerName);
528 
529  // Loop over fields, converting if needed
530  TypedFieldList output;
531  typename FieldList::iterator i = originals.begin();
532  for (; i != originals.end(); ++i) {
533  typename Field_T<Data_T>::Ptr targetField;
534  targetField = field_dynamic_cast<Field_T<Data_T> >(*i);
535  if (targetField) {
536  output.push_back(targetField);
537  } else {
538  typename Field_T<Data_T>::Ptr newTarget(new Field_T<Data_T>);
539  newTarget->name = (*i)->name;
540  newTarget->attribute = (*i)->attribute;
541  newTarget->copyMetadata(**i);
542  newTarget->copyFrom(*i);
543  output.push_back(newTarget);
544  }
545  }
546 
547  return output;
548  }
Field_T::Ptr field_dynamic_cast(RefBase::Ptr field)
Dynamic cast that uses string-comparison in order to be safe even after an object crosses a shared li...
Definition: RefCount.h:256
std::vector< Ptr > Vec
This is a convenience typedef for the list that Field3DInputFile::readScalarLayers() and Field3DInput...
Definition: Field.h:408
template<template< typename T > class Field_T, class Data_T >
Field_T<FIELD3D_VEC3_T<Data_T> >::Vec Field3DInputFile::readVectorLayersAs ( const std::string &  layerName = std::string("")) const
inline

Retrieves a layers for all partitions. Converts it to the given template type if needed.

Definition at line 554 of file Field3DFile.h.

References field_dynamic_cast().

555  {
556  typedef Field_T<FIELD3D_VEC3_T<Data_T> > TypedVField;
557  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Vec FieldList;
558  typedef typename Field_T<FIELD3D_VEC3_T<Data_T> >::Vec TypedFieldList;
559 
560  // First, read the layers as-is
561  FieldList originals;
562  originals = readVectorLayers<Data_T>(layerName);
563 
564  // Loop over fields, converting if needed
565  TypedFieldList output;
566  typename FieldList::iterator i = originals.begin();
567  for (; i != originals.end(); ++i) {
568  typename TypedVField::Ptr targetField;
569  targetField = field_dynamic_cast<TypedVField>(*i);
570  if (targetField) {
571  output.push_back(targetField);
572  } else {
573  typename TypedVField::Ptr newTarget(new TypedVField);
574  newTarget->name = (*i)->name;
575  newTarget->attribute = (*i)->attribute;
576  newTarget->copyMetadata(**i);
577  newTarget->copyFrom(*i);
578  output.push_back(newTarget);
579  }
580  }
581 
582  return output;
583  }
Field_T::Ptr field_dynamic_cast(RefBase::Ptr field)
Dynamic cast that uses string-comparison in order to be safe even after an object crosses a shared li...
Definition: RefCount.h:256
Definition: Field.h:394
template<template< typename T > class Field_T, class Data_T >
Field_T<FIELD3D_VEC3_T<Data_T> >::Vec Field3DInputFile::readVectorLayersAs ( const std::string &  partitionName,
const std::string &  layerName 
) const
inline

Retrieves a layers given their and its parent partition's name. Converts it to the given template type if needed.

Definition at line 589 of file Field3DFile.h.

References field_dynamic_cast(), and File::Layer::name.

591  {
592  typedef Field_T<FIELD3D_VEC3_T<Data_T> > TypedVField;
593  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Vec FieldList;
594  typedef typename Field_T<FIELD3D_VEC3_T<Data_T> >::Vec TypedFieldList;
595 
596  // First, read the layers as-is
597  FieldList originals;
598  originals = readVectorLayers<Data_T>(partitionName, layerName);
599 
600  // Loop over fields, converting if needed
601  TypedFieldList output;
602  typename FieldList::iterator i = originals.begin();
603  for (; i != originals.end(); ++i) {
604  typename TypedVField::Ptr targetField;
605  targetField = field_dynamic_cast<TypedVField>(*i);
606  if (targetField) {
607  output.push_back(targetField);
608  } else {
609  typename TypedVField::Ptr newTarget(new TypedVField);
610  newTarget->name = (*i)->name;
611  newTarget->attribute = (*i)->attribute;
612  newTarget->copyMetadata(**i);
613  newTarget->copyFrom(*i);
614  output.push_back(newTarget);
615  }
616  }
617 
618  return output;
619  }
Field_T::Ptr field_dynamic_cast(RefBase::Ptr field)
Dynamic cast that uses string-comparison in order to be safe even after an object crosses a shared li...
Definition: RefCount.h:256
Definition: Field.h:394
template<class Data_T >
EmptyField< Data_T >::Vec Field3DInputFile::readProxyLayer ( const std::string &  partitionName,
const std::string &  layerName,
bool  isVectorLayer 
) const

Retrieves a proxy version (EmptyField) of each layer .

Note
Although the call is templated, all fields are read, regardless of bit depth.
Parameters
nameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 1154 of file Field3DFile.h.

References g_hdf5Mutex, Hdf5Util::H5Base::id(), FieldBase::metadata(), File::Layer::name, File::Layer::parent, Msg::print(), Hdf5Util::readAttribute(), FieldMetadata< CallBack_T >::setIntMetadata(), and Msg::SevWarning.

1157 {
1158  using namespace boost;
1159  using namespace std;
1160  using namespace Hdf5Util;
1161 
1162  GlobalLock lock(g_hdf5Mutex);
1163 
1164  // Instantiate a null pointer for easier code reading
1165  typename EmptyField<Data_T>::Vec emptyList, output;
1166 
1167  if ((layerName.length() == 0) || (partitionName.length() == 0))
1168  return emptyList;
1169 
1170  std::vector<std::string> parts, layers;
1171  getIntPartitionNames(parts);
1172 
1173  bool foundPartition = false;
1174 
1175  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
1176  if (removeUniqueId(*p) == partitionName) {
1177  foundPartition = true;
1178  if (isVectorLayer) {
1179  getIntVectorLayerNames(layers, *p);
1180  } else {
1181  getIntScalarLayerNames(layers, *p);
1182  }
1183  for (vector<string>::iterator l = layers.begin();
1184  l != layers.end(); ++l) {
1185  if (*l == layerName) {
1186  // Find the partition
1187  File::Partition::Ptr part = partition(*p);
1188  if (!part) {
1189  Msg::print(Msg::SevWarning, "Couldn't find partition: " + *p);
1190  return emptyList;
1191  }
1192  // Find the layer
1193  const File::Layer *layer;
1194  if (isVectorLayer)
1195  layer = part->vectorLayer(layerName);
1196  else
1197  layer = part->scalarLayer(layerName);
1198  if (!layer) {
1199  Msg::print(Msg::SevWarning, "Couldn't find layer: " + layerName);
1200  return emptyList;
1201  }
1202  // Open the layer group
1203  string layerPath = layer->parent + "/" + layer->name;
1204  H5ScopedGopen layerGroup(m_file, layerPath.c_str());
1205  if (layerGroup.id() < 0) {
1206  Msg::print(Msg::SevWarning, "Couldn't find layer group "
1207  + layerName + " in .f3d file ");
1208  return emptyList;
1209  }
1210 
1211  // Make the proxy representation
1212  typename EmptyField<Data_T>::Ptr field =
1213  readProxyLayer<Data_T>(layerGroup, partitionName, layerName,
1214  part->mapping);
1215 
1216  // Read MIPField's number of mip levels
1217  int numLevels = 0;
1218  H5ScopedGopen mipGroup(layerGroup, "mip_levels");
1219  if (mipGroup.id() >= 0)
1220  readAttribute(mipGroup, "levels", 1, numLevels);
1221  field->metadata().setIntMetadata("mip_levels", numLevels);
1222 
1223  // Add field to output
1224  output.push_back(field);
1225  }
1226  }
1227  }
1228  }
1229 
1230  if (!foundPartition) {
1231  Msg::print(Msg::SevWarning, "Couldn't find partition: " + partitionName);
1232  return emptyList;
1233  }
1234 
1235  return output;
1236 }
Contains utility functions and classes for Hdf5 files.
Definition: Hdf5Util.h:86
FieldMetadata< FieldBase > & metadata()
accessor to the m_metadata class
Definition: Field.h:155
boost::intrusive_ptr< Partition > Ptr
Definition: Field3DFile.h:152
void getIntVectorLayerNames(std::vector< std::string > &names, const std::string &intPartitionName) const
Gets the names of all the vector layers in a given partition, but assumes that partition name is the ...
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity. ...
Definition: Log.cpp:66
File::Partition::Ptr partition(const std::string &partitionName)
Returns a pointer to the given partition.
void getIntScalarLayerNames(std::vector< std::string > &names, const std::string &intPartitionName) const
Gets the names of all the scalar layers in a given partition, but assumes that partition name is the ...
void getIntPartitionNames(std::vector< std::string > &names) const
Gets the names of all the -internal- partitions in the file.
std::string removeUniqueId(const std::string &partitionName) const
Strips any unique identifiers from the partition name and returns the original name.
std::vector< Ptr > Vec
Definition: EmptyField.h:94
std::string name
The name of the layer (always available)
Definition: Field3DFile.h:124
boost::intrusive_ptr< EmptyField > Ptr
Definition: EmptyField.h:93
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
void setIntMetadata(const std::string &name, const int val)
Set the a int value for the given metadata name.
hid_t m_file
The hdf5 id of the current file. Will be -1 if no file is open.
Definition: Field3DFile.h:387
Scoped object - opens a group on creation and closes it on destruction.
Definition: Hdf5Util.h:194
std::string parent
The name of the parent partition. We need this in order to open its group.
Definition: Field3DFile.h:127
template<class Data_T >
EmptyField< Data_T >::Ptr Field3DInputFile::readProxyLayer ( hid_t  location,
const std::string &  name,
const std::string &  attribute,
FieldMapping::Ptr  mapping 
) const

Retrieves a proxy version (EmptyField) from a given HDF5 location.

Note
Although the call is templated, all fields are read, regardless of bit depth.
Parameters
locationHDF5 file location

Definition at line 1242 of file Field3DFile.h.

References FieldBase::attribute, g_hdf5Mutex, Hdf5Util::H5Base::id(), File::Layer::name, FieldBase::name, Hdf5Util::readAttribute(), FieldRes::setMapping(), and ResizableField< Data_T >::setSize().

1246 {
1247  using namespace boost;
1248  using namespace std;
1249  using namespace Hdf5Util;
1250 
1251  typename EmptyField<Data_T>::Ptr null;
1252 
1253  GlobalLock lock(g_hdf5Mutex);
1254 
1255  // Read the extents and data window
1256  Box3i extents, dataW;
1257  if (!readAttribute(location, "extents", 6, extents.min.x)) {
1258  return null;
1259  }
1260  if (!readAttribute(location, "data_window", 6, dataW.min.x)) {
1261  return null;
1262  }
1263 
1264  // Construct the field and load the data
1265  typename EmptyField<Data_T>::Ptr field(new EmptyField<Data_T>);
1266  field->setSize(extents, dataW);
1267 
1268  // Read the metadata
1269  H5ScopedGopen metadataGroup(location, "metadata");
1270  if (metadataGroup.id() > 0) {
1271  readMetadata(metadataGroup.id(), field);
1272  }
1273 
1274  // Set field properties
1275  field->name = name;
1276  field->attribute = attribute;
1277  field->setMapping(mapping);
1278 
1279  return field;
1280 }
Imath::Box3i Box3i
Definition: SpiMathLib.h:77
Contains utility functions and classes for Hdf5 files.
Definition: Hdf5Util.h:86
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
This subclass of Field does not store any data.
Definition: EmptyField.h:86
boost::intrusive_ptr< EmptyField > Ptr
Definition: EmptyField.h:93
bool readMetadata(hid_t metadata_id, FieldBase::Ptr field) const
Read metadata for this layer.
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
Scoped object - opens a group on creation and closes it on destruction.
Definition: Hdf5Util.h:194
template<class Data_T >
EmptyField< Data_T >::Vec Field3DInputFile::readProxyScalarLayers ( const std::string &  name = std::string("")) const

Retrieves a proxy version (EmptyField) of each scalar layer.

Note
Although the call is templated, all fields are read, regardless of bit depth.
Parameters
nameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 1286 of file Field3DFile.h.

1287 {
1288  using namespace std;
1289 
1290  typedef typename EmptyField<Data_T>::Ptr FieldPtr;
1291  typedef std::vector<FieldPtr> FieldList;
1292 
1293  FieldList ret;
1294 
1295  std::vector<std::string> parts;
1296  getPartitionNames(parts);
1297 
1298  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
1299  std::vector<std::string> layers;
1300  getScalarLayerNames(layers, *p);
1301  for (vector<string>::iterator l = layers.begin(); l != layers.end(); ++l) {
1302  // Only read if it matches the name
1303  if ((name.length() == 0) || (*l == name)) {
1304  FieldList f = readProxyLayer<Data_T>(*p, *l, false);
1305  for (typename FieldList::iterator i = f.begin(); i != f.end(); ++i) {
1306  if (*i) {
1307  ret.push_back(*i);
1308  }
1309  }
1310  }
1311  }
1312  }
1313 
1314  return ret;
1315 }
boost::intrusive_ptr< EmptyField > Ptr
Definition: EmptyField.h:93
void getPartitionNames(std::vector< std::string > &names) const
Gets the names of all the partitions in the file.
void getScalarLayerNames(std::vector< std::string > &names, const std::string &partitionName) const
Gets the names of all the scalar layers in a given partition.
template<class Data_T >
EmptyField< Data_T >::Vec Field3DInputFile::readProxyVectorLayers ( const std::string &  name = std::string("")) const

Retrieves a proxy version (EmptyField) of each vector layer.

Note
Although the call is templated, all fields are read, regardless of bit depth.
Parameters
nameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 1321 of file Field3DFile.h.

1322 {
1323  using namespace std;
1324 
1325  typedef typename EmptyField<Data_T>::Ptr FieldPtr;
1326  typedef std::vector<FieldPtr> FieldList;
1327 
1328  FieldList ret;
1329 
1330  std::vector<std::string> parts;
1331  getPartitionNames(parts);
1332 
1333  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
1334  std::vector<std::string> layers;
1335  getVectorLayerNames(layers, *p);
1336  for (vector<string>::iterator l = layers.begin(); l != layers.end(); ++l) {
1337  // Only read if it matches the name
1338  if ((name.length() == 0) || (*l == name)) {
1339  FieldList f = readProxyLayer<Data_T>(*p, *l, true);
1340  for (typename FieldList::iterator i = f.begin(); i != f.end(); ++i) {
1341  if (*i) {
1342  ret.push_back(*i);
1343  }
1344  }
1345  }
1346  }
1347  }
1348 
1349  return ret;
1350 }
boost::intrusive_ptr< EmptyField > Ptr
Definition: EmptyField.h:93
void getPartitionNames(std::vector< std::string > &names) const
Gets the names of all the partitions in the file.
void getVectorLayerNames(std::vector< std::string > &names, const std::string &partitionName) const
Gets the names of all the vector layers in a given partition.
bool Field3DInputFile::open ( const std::string &  filename)

Opens the given file.

Returns
Whether successful

Definition at line 570 of file Field3DFile.cpp.

References Field3DFileBase::clear(), Field3DFileBase::close(), g_hdf5Mutex, Hdf5Util::H5Base::id(), Field3DFileBase::m_file, Msg::print(), Hdf5Util::readAttribute(), Msg::SevWarning, and Exc::Exception::what().

Referenced by FieldGroup< BaseTypeList_T, Dims_T >::load().

571 {
572  GlobalLock lock(g_hdf5Mutex);
573 
574  clear();
575 
576  bool success = true;
577 
578  m_filename = filename;
579 
580  try {
581 
582  string version;
583 
584  // Throws exceptions if the file doesn't exist.
585  // This was added because H5Fopen prints out a lot of junk
586  // to the terminal.
587  checkFile(filename);
588 
589  m_file = H5Fopen(filename.c_str(), H5F_ACC_RDONLY, H5P_DEFAULT);
590 
591  if (m_file < 0)
592  throw NoSuchFileException(filename);
593 
594  int fileVersion[3];
595  try {
596  if (!readAttribute(m_file, k_versionAttrName, 3, fileVersion[0])) {
597  //Msg::print(Msg::SevWarning, "Missing version_number attribute");
598  } else {
599  if (!isSupportedFileVersion(fileVersion, k_minFileVersion)) {
600  stringstream versionStr;
601  versionStr << fileVersion[0] << "."
602  << fileVersion[1] << "."
603  << fileVersion[2];
604  throw UnsupportedVersionException(versionStr.str());
605  }
606  }
607  }
608  catch (MissingAttributeException &) {
609  //Msg::print(Msg::SevWarning, "Missing version_number attribute");
610  }
611 
612  try {
613  if (H5Lexists(m_file, "field3d_global_metadata", H5P_DEFAULT)) {
614  // read the metadata
615  H5ScopedGopen metadataGroup(m_file, "field3d_global_metadata");
616  if (metadataGroup.id() > 0) {
617  readMetadata(metadataGroup.id());
618  }
619  }
620  }
621  catch (...) {
623  "Unknown error when reading file metadata ");
624  //throw BadFileHierarchyException(filename);
625  }
626 
627  try {
628  if (!readPartitionAndLayerInfo()) {
629  success = false;
630  }
631  }
632  catch (MissingGroupException &e) {
633  Msg::print(Msg::SevWarning, "Missing group: " + string(e.what()));
634  throw BadFileHierarchyException(filename);
635  }
636  catch (ReadMappingException &e) {
637  Msg::print(Msg::SevWarning, "Couldn't read mapping for partition: "
638  + string(e.what()));
639  throw BadFileHierarchyException(filename);
640  }
641  catch (Exception &e) {
642  Msg::print(Msg::SevWarning, "Unknown error when reading file hierarchy: "
643  + string(e.what()));
644  throw BadFileHierarchyException(filename);
645  }
646  catch (...) {
648  "Unknown error when reading file hierarchy. ");
649  throw BadFileHierarchyException(filename);
650  }
651 
652  }
653  catch (NoSuchFileException &e) {
654  Msg::print(Msg::SevWarning, "Couldn't open file: "
655  + string(e.what()) );
656  success = false;
657  }
658  catch (MissingAttributeException &e) {
660  "In file: " + filename + " - "
661  + string(e.what()) );
662  success = false;
663  }
664  catch (UnsupportedVersionException &e) {
666  "In file: " + filename + " - File version can not be read: "
667  + string(e.what()));
668  success = false;
669  }
670  catch (BadFileHierarchyException &) {
672  "In file: " + filename + " - Bad file hierarchy. ");
673  success = false;
674  }
675  catch (...) {
677  "In file: " + filename + " Unknown exception ");
678  success = false;
679  }
680 
681  if (!success)
682  close();
683 
684  return success;
685 }
virtual const char * what() const
Definition: Exception.h:90
std::string m_filename
Filename, only to be set by open().
Definition: Field3DFile.h:730
void clear()
Clear the data structures and close the file.
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity. ...
Definition: Log.cpp:66
bool readPartitionAndLayerInfo()
Sets up all the partitions and layers, but does not load any data.
bool close()
Closes the file. No need to call this unless you specifically want to close the file early...
bool readMetadata(hid_t metadata_id, FieldBase::Ptr field) const
Read metadata for this layer.
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
hid_t m_file
The hdf5 id of the current file. Will be -1 if no file is open.
Definition: Field3DFile.h:387
Scoped object - opens a group on creation and closes it on destruction.
Definition: Hdf5Util.h:194
herr_t Field3DInputFile::parsePartition ( hid_t  loc_id,
const std::string  partitionName 
)

Gets called from parsePartitions. Not intended for any other use.

Definition at line 783 of file Field3DFile.cpp.

References Field3DFileBase::m_partitionNames.

Referenced by InputFile::parsePartitions().

785 {
786  // Add the partition ---
787 
788  m_partitionNames.push_back(string(itemName));
789  return 0;
790 }
std::vector< std::string > m_partitionNames
This stores partition names.
Definition: Field3DFile.h:391
herr_t Field3DInputFile::parseLayer ( hid_t  layerGroup,
const std::string &  partitionName,
const std::string &  layerName 
)

Gets called from parsePartitions. Not intended for any other use.

Note
Don't throw exceptions into the hdf5 lib.
Todo:
Set some sort of flag if we fail during this call. We can't throw exceptions inside hdf5.

Definition at line 797 of file Field3DFile.cpp.

References Field3DFileBase::m_layerInfo, Msg::print(), Hdf5Util::readAttribute(), readMetadata(), and Msg::SevWarning.

Referenced by InputFile::parseLayers().

800 {
801  int components;
802  if (!readAttribute(layerGroup, string("components"), 1, components)) {
803  Msg::print(Msg::SevWarning, "Couldn't read components attribute for layer "
804  + partitionName + "/" + layerName);
805  return 0;
806  }
807 
808  LayerInfo linfo(partitionName,layerName,components);
809 
810  m_layerInfo.push_back(linfo);
811 
812  return 0;
813 }
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity. ...
Definition: Log.cpp:66
std::vector< LayerInfo > m_layerInfo
This stores layer info.
Definition: Field3DFile.h:384
bool Field3DInputFile::readGroupMembership ( GroupMembershipMap gpMembershipMap)

Read the group membership for the partitions.

Definition at line 1040 of file Field3DFile.cpp.

References g_hdf5Mutex, Hdf5Util::H5Base::id(), Field3DFileBase::m_file, Field3DFileBase::m_groupMembership, Msg::print(), Hdf5Util::readAttribute(), Field3DFileBase::removeUniqueId(), and Msg::SevWarning.

Referenced by readMetadata().

1041 {
1042  GlobalLock lock(g_hdf5Mutex);
1043 
1044  if (!H5Lexists(m_file, "field3d_group_membership", H5P_DEFAULT)) {
1045  return false;
1046  }
1047 
1048  H5ScopedGopen memberGroup(m_file, "field3d_group_membership");
1049  if (memberGroup < 0) {
1050  return false;
1051  }
1052 
1053  typedef boost::tokenizer<boost::char_separator<char> > Tok;
1054 
1055  hsize_t num_attrs = H5Aget_num_attrs(memberGroup);
1056  if (num_attrs > 0) {
1057 
1058  for (hsize_t idx=0; idx < num_attrs ; ++idx) {
1059  H5ScopedAopenIdx attrIdx(memberGroup, idx);
1060  size_t len = H5Aget_name(attrIdx.id(), 0, NULL);
1061  if (len>0) {
1062  char *name = new char[len+1];
1063  if (H5Aget_name(attrIdx.id(), len+1, name) > 0) {
1064 
1065  if (string(name) == "is_field3d_group_membership")
1066  continue;
1067 
1068  H5ScopedAopen attr(memberGroup, name, H5P_DEFAULT);
1069  H5ScopedAget_space attrSpace(attr);
1070  H5ScopedAget_type attrType(attr);
1071  H5T_class_t typeClass = H5Tget_class(attrType);
1072 
1073  if (typeClass == H5T_STRING) {
1074  string value;
1075  if (!readAttribute(memberGroup, name, value)) {
1077  "Failed to read group membership data "
1078  + string(name));
1079  continue;
1080  }
1081 
1082  {
1083  boost::char_separator<char> sep(" :");
1084  Tok tok(value, sep);
1085  string new_value;
1086  for(Tok::iterator beg=tok.begin(); beg!=tok.end();){
1087 
1088  string fieldgroup = *beg; ++beg;
1089  fieldgroup = removeUniqueId(fieldgroup) + ":" + *beg; ++beg;
1090  new_value += fieldgroup + " ";
1091  }
1092 
1093  m_groupMembership[name] = value;
1094  gpMembershipMap[name] = new_value;
1095  }
1096  }
1097  }
1098  }
1099  }
1100  }
1101 
1102  return true;
1103 }
Scoped object - opens an attribute data type on creation and closes it on destruction.
Definition: Hdf5Util.h:310
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity. ...
Definition: Log.cpp:66
std::string removeUniqueId(const std::string &partitionName) const
Strips any unique identifiers from the partition name and returns the original name.
Scoped object - Opens attribute by name and closes it on destruction.
Definition: Hdf5Util.h:113
Scoped object - Opens attribute by index and closes it on destruction.
Definition: Hdf5Util.h:142
Scoped object - opens an attribute data space on creation and closes it on destruction.
Definition: Hdf5Util.h:287
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
hid_t m_file
The hdf5 id of the current file. Will be -1 if no file is open.
Definition: Field3DFile.h:387
GroupMembershipMap m_groupMembership
Keeps track of group membership for each layer of partition name. The key is the "group" and the valu...
Definition: Field3DFile.h:400
Scoped object - opens a group on creation and closes it on destruction.
Definition: Hdf5Util.h:194
template<class Data_T >
Field< Data_T >::Ptr Field3DInputFile::readScalarLayer ( const std::string &  intPartitionName,
const std::string &  layerName 
) const
private

Retrieves a single layer given its and its parent partition's name. Maintains the on-disk data types.

Definition at line 1356 of file Field3DFile.h.

1358 {
1359  return readLayer<Data_T>(intPartitionName, layerName, false);
1360 }
std::string intPartitionName(const std::string &partitionName, const std::string &layerName, FieldRes::Ptr field)
Returns a unique partition name given the requested name. This ensures that partitions with matching ...
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Ptr Field3DInputFile::readVectorLayer ( const std::string &  intPartitionName,
const std::string &  layerName 
) const
private

Retrieves a single layer given its and its parent partition's name. Maintains the on-disk data types.

Definition at line 1366 of file Field3DFile.h.

1368 {
1369  return readLayer<FIELD3D_VEC3_T<Data_T> >(intPartitionName, layerName, true);
1370 }
std::string intPartitionName(const std::string &partitionName, const std::string &layerName, FieldRes::Ptr field)
Returns a unique partition name given the requested name. This ensures that partitions with matching ...
template<class Data_T >
Field< Data_T >::Ptr Field3DInputFile::readLayer ( const std::string &  intPartitionName,
const std::string &  layerName,
bool  isVectorLayer 
) const
private

This call does the actual reading of a layer. Notice that it expects a unique -internal- partition name.

Definition at line 1049 of file Field3DFile.h.

References FieldBase::attribute, FieldCache< Data_T >::cacheField(), g_hdf5Mutex, FieldCache< Data_T >::getCachedField(), File::Layer::name, FieldBase::name, File::Layer::parent, Msg::print(), Hdf5Util::readAttribute(), FieldRes::setMapping(), Msg::SevWarning, and FieldCache< Data_T >::singleton().

1052 {
1053  using namespace boost;
1054  using namespace std;
1055  using namespace Hdf5Util;
1056 
1057  typedef typename Field<Data_T>::Ptr FieldPtr;
1058 
1059  GlobalLock lock(g_hdf5Mutex);
1060 
1061  // Instantiate a null pointer for easier code reading
1062  FieldPtr nullPtr;
1063 
1064  // Find the partition
1066  if (!part) {
1067  Msg::print(Msg::SevWarning, "Couldn't find partition: " + intPartitionName);
1068  return nullPtr;
1069  }
1070 
1071  // Find the layer in the partition
1072  const File::Layer *l;
1073  if (isVectorLayer)
1074  l = part->vectorLayer(layerName);
1075  else
1076  l = part->scalarLayer(layerName);
1077  if (!l) {
1078  Msg::print(Msg::SevWarning, "Couldn't find layer: " + layerName );
1079  return nullPtr;
1080  }
1081 
1082  // Open the layer group
1083  string layerPath = l->parent + "/" + l->name;
1084  H5ScopedGopen layerGroup(m_file, layerPath.c_str());
1085 
1086  if (layerGroup.id() < 0) {
1087  Msg::print(Msg::SevWarning, "Couldn't find layer group " + layerName
1088  + " in .f3d file ");
1089  return nullPtr;
1090  }
1091 
1092  // Get the class name
1093  string className;
1094  if (!readAttribute(layerGroup.id(), "class_name", className)) {
1095  Msg::print(Msg::SevWarning, "Couldn't find class_name attrib in layer " +
1096  layerName);
1097  return nullPtr;
1098  }
1099 
1100  // Check the cache
1101 
1103  FieldPtr cachedField = cache.getCachedField(m_filename, layerPath);
1104 
1105  if (cachedField) {
1106  return cachedField;
1107  }
1108 
1109  // Construct the field and load the data
1110 
1111  // Unlock the g_hdf5Mutex while calling readField() so that other threads
1112  // have a chance to pre-empt loading in between (prevents deadlocks with
1113  // sparse reader)
1114  lock.unlock();
1115 
1116  typename Field<Data_T>::Ptr field;
1117  field = readField<Data_T>(className, layerGroup.id(), m_filename, layerPath);
1118 
1119  if (!field) {
1120 #if 0 // This isn't really an error
1121  Msg::print(Msg::SevWarning, "Couldn't read the layer data of layer: "
1122  + layerName);
1123 #endif
1124  return nullPtr;
1125  }
1126 
1127  // Now we need to use Hdf5 again, so re-aquire the lock.
1128  lock.lock();
1129 
1130  // read the metadata
1131  string metadataPath = layerPath + "/metadata";
1132  H5ScopedGopen metadataGroup(m_file, metadataPath.c_str());
1133  if (metadataGroup.id() > 0) {
1134  readMetadata(metadataGroup.id(), field);
1135  }
1136 
1137  // Set the name of the field so it's possible to re-create the file
1139  field->attribute = layerName;
1140  field->setMapping(part->mapping);
1141 
1142  // Cache the field for future use
1143  if (field) {
1144  cache.cacheField(field, m_filename, layerPath);
1145  }
1146 
1147  return field;
1148 }
Contains utility functions and classes for Hdf5 files.
Definition: Hdf5Util.h:86
boost::intrusive_ptr< Partition > Ptr
Definition: Field3DFile.h:152
std::string m_filename
Filename, only to be set by open().
Definition: Field3DFile.h:730
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity. ...
Definition: Log.cpp:66
FieldPtr getCachedField(const std::string &filename, const std::string &layerPath)
Checks the cache for a previously loaded field.
Definition: FieldCache.h:155
File::Partition::Ptr partition(const std::string &partitionName)
Returns a pointer to the given partition.
std::string intPartitionName(const std::string &partitionName, const std::string &layerName, FieldRes::Ptr field)
Returns a unique partition name given the requested name. This ensures that partitions with matching ...
std::string removeUniqueId(const std::string &partitionName) const
Strips any unique identifiers from the partition name and returns the original name.
void cacheField(FieldPtr field, const std::string &filename, const std::string &layerPath)
Adds the given field to the cache.
Definition: FieldCache.h:176
std::string name
The name of the layer (always available)
Definition: Field3DFile.h:124
void setMapping(FieldMapping::Ptr mapping)
Sets the field&#39;s mapping.
Definition: Field.h:352
static FieldCache & singleton()
Returns a reference to the FieldCache singleton.
Definition: FieldCache.h:142
bool readMetadata(hid_t metadata_id, FieldBase::Ptr field) const
Read metadata for this layer.
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
std::string attribute
Optional name of the attribute the field represents.
Definition: Field.h:178
hid_t m_file
The hdf5 id of the current file. Will be -1 if no file is open.
Definition: Field3DFile.h:387
Scoped object - opens a group on creation and closes it on destruction.
Definition: Hdf5Util.h:194
std::string parent
The name of the parent partition. We need this in order to open its group.
Definition: Field3DFile.h:127
boost::intrusive_ptr< Field > Ptr
Definition: Field.h:400
std::string name
Optional name of the field.
Definition: Field.h:176
bool Field3DInputFile::readPartitionAndLayerInfo ( )
private

Sets up all the partitions and layers, but does not load any data.

Definition at line 689 of file Field3DFile.cpp.

References InputFile::ParseLayersInfo::file, g_hdf5Mutex, Hdf5Util::H5Base::id(), Field3DFileBase::m_file, Field3DFileBase::m_layerInfo, Field3DFileBase::m_partitionNames, Field3DFileBase::m_partitions, File::Layer::name, File::Layer::parent, InputFile::parseLayers(), InputFile::parsePartitions(), Field3DFileBase::partition(), InputFile::ParseLayersInfo::partitionName, Msg::print(), readFieldMapping(), and Msg::SevWarning.

690 {
691  using namespace InputFile;
692 
693  GlobalLock lock(g_hdf5Mutex);
694 
695  // First, find the partitions ---
696 
697  herr_t status;
698  status = H5Literate(m_file, H5_INDEX_NAME, H5_ITER_NATIVE, NULL,
699  &parsePartitions, this);
700 
701  // Get the partition names to store
702  m_partitions.clear();
703 
704  for (size_t i=0; i < m_partitionNames.size(); i++) {
705  Partition::Ptr part(new Partition);
706  part->name = m_partitionNames[i];
707  m_partitions.push_back(part);
708  }
709 
710  // For each partition, find its mapping ---
711 
712  for (PartitionList::iterator i = m_partitions.begin();
713  i != m_partitions.end(); ++i) {
714 
715  // Open the partition
716  H5ScopedGopen partitionGroup(m_file, (**i).name);
717 
718  string mappingPath = "/" + (**i).name + "/" + k_mappingStr;
719 
720  // Open up the mapping group
721  H5ScopedGopen mappingGroup(m_file, mappingPath);
722  if (mappingGroup.id() < 0)
723  throw MissingGroupException((**i).name + "/" + k_mappingStr);
724 
725  // Try to build a mapping from it
726  FieldMapping::Ptr mapping;
727 
728  mapping = readFieldMapping(mappingGroup.id());
729  if (!mapping) {
730  Msg::print(Msg::SevWarning, "Got a null pointer when reading mapping");
731  throw ReadMappingException((**i).name);
732  }
733 
734  // Attach the mapping to the partition
735  (**i).mapping = mapping;
736 
737  }
738 
739  // ... And then find its layers ---
740 
741  for (PartitionList::const_iterator i = m_partitions.begin();
742  i != m_partitions.end(); ++i) {
743 
744  // Open the partition
745  H5ScopedGopen partitionGroup(m_file, (**i).name);
746 
747  // Set up the info struct for the callback
748  ParseLayersInfo info;
749  info.file = this;
750  info.partitionName = (**i).name;
751 
752  m_layerInfo.clear();
753 
754  status = H5Literate(partitionGroup.id(), H5_INDEX_NAME, H5_ITER_NATIVE,
755  NULL, &parseLayers, &info);
756 
757  //set the layer information on the partitions here
758 
759  for (std::vector<LayerInfo>::iterator i = m_layerInfo.begin();
760  i != m_layerInfo.end(); i++) {
761 
762  std::string parent = i->parentName;
763 
764  Partition::Ptr part = partition(parent);
765 
766  Layer layer;
767  layer.name = i->name;
768  layer.parent = i->parentName;
769  if (i->components == 1) {
770  part->addScalarLayer(layer);
771  } else if (i->components == 3) {
772  part->addVectorLayer(layer);
773  }
774  }
775 
776  }
777 
778  return true;
779 }
FIELD3D_API herr_t parseLayers(hid_t loc_id, const char *partitionName, const H5L_info_t *linfo, void *opdata)
Gets called from readPartitionAndLayerInfo to check each group found under the root of the file...
boost::intrusive_ptr< Partition > Ptr
Definition: Field3DFile.h:152
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity. ...
Definition: Log.cpp:66
File::Partition::Ptr partition(const std::string &partitionName)
Returns a pointer to the given partition.
std::vector< std::string > m_partitionNames
This stores partition names.
Definition: Field3DFile.h:391
boost::intrusive_ptr< FieldMapping > Ptr
Definition: FieldMapping.h:92
FieldMapping::Ptr readFieldMapping(hid_t mappingGroup)
This function creates a FieldMappingIO instance based on className read from mappingGroup location wh...
std::string name
The name of the layer (always available)
Definition: Field3DFile.h:124
PartitionList m_partitions
Vector of partitions.
Definition: Field3DFile.h:389
Namespace for file input specifics.
Definition: Field3DFile.h:876
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
std::vector< LayerInfo > m_layerInfo
This stores layer info.
Definition: Field3DFile.h:384
struct used to pass the class and partition info back to the parseLayers() callback ...
Definition: Field3DFile.h:881
FIELD3D_API herr_t parsePartitions(hid_t loc_id, const char *partitionName, const H5L_info_t *linfo, void *opdata)
Gets called from readPartitionAndLayerInfo to check each group found under the root of the file...
hid_t m_file
The hdf5 id of the current file. Will be -1 if no file is open.
Definition: Field3DFile.h:387
Field3DInputFile * file
Definition: Field3DFile.h:883
Scoped object - opens a group on creation and closes it on destruction.
Definition: Hdf5Util.h:194
std::string parent
The name of the parent partition. We need this in order to open its group.
Definition: Field3DFile.h:127
bool Field3DInputFile::readMetadata ( hid_t  metadata_id,
FieldBase::Ptr  field 
) const
private

Read metadata for this layer.

Todo:
Replace char* with std::string

Definition at line 820 of file Field3DFile.cpp.

References g_hdf5Mutex, Hdf5Util::H5Base::id(), Msg::print(), Hdf5Util::readAttribute(), and Msg::SevWarning.

Referenced by parseLayer().

821 {
822  GlobalLock lock(g_hdf5Mutex);
823 
824  hsize_t num_attrs = H5Aget_num_attrs(metadata_id);
825 
826  if (num_attrs > 0) {
827  for (hsize_t idx=0; idx < num_attrs ; ++idx) {
828  H5ScopedAopenIdx attrIdx(metadata_id, idx);
829  size_t len = H5Aget_name(attrIdx.id(), 0, NULL);
830  if (len > 0) {
831  char *name = new char[len+1];
832  if (H5Aget_name(attrIdx.id(), len+1, name) > 0) {
833  H5ScopedAopen attr(metadata_id, name, H5P_DEFAULT);
834  H5ScopedAget_space attrSpace(attr);
835  H5ScopedAget_type attrType(attr);
836  H5T_class_t typeClass = H5Tget_class(attrType);
837 
838  if (typeClass == H5T_STRING) {
839  string value;
840  if (!readAttribute(metadata_id, name, value)) {
842  "Failed to read metadata " + string(name));
843  if (name) {
844  delete[] name;
845  }
846  continue;
847  }
848  field->metadata().setStrMetadata(name, value);
849 
850  }
851  else {
852 
853  if (H5Sget_simple_extent_ndims(attrSpace) != 1) {
854  Msg::print(Msg::SevWarning, "Bad attribute rank for attribute "
855  + string(name));
856  if (name) {
857  delete[] name;
858  }
859  continue;
860  }
861 
862  hsize_t dims[1];
863  H5Sget_simple_extent_dims(attrSpace, dims, NULL);
864 
865  if (typeClass == H5T_INTEGER) {
866  if (dims[0] == 1){
867  int value;
868  if (!readAttribute(metadata_id, name, dims[0], value))
869  Msg::print(Msg::SevWarning, "Failed to read metadata "
870  + string(name));
871  field->metadata().setIntMetadata(name, value);
872  }
873  else if (dims[0] == 3){
874  V3i value;
875  if (!readAttribute(metadata_id, name, dims[0], value.x))
876  Msg::print(Msg::SevWarning, "Failed to read metadata " +
877  string(name) );
878  field->metadata().setVecIntMetadata(name, value);
879  }
880  else {
882  "Attribute of size " +
883  boost::lexical_cast<std::string>(dims[0])
884  + " is not valid for metadata");
885  }
886  }
887  else if (typeClass == H5T_FLOAT) {
888  if (dims[0] == 1){
889  float value;
890  if (!readAttribute(metadata_id, name, dims[0], value))
891  Msg::print(Msg::SevWarning, "Failed to read metadata " +
892  string(name) );
893 
894  field->metadata().setFloatMetadata(name, value);
895  }
896  else if (dims[0] == 3){
897  V3f value;
898  if (!readAttribute(metadata_id, name, dims[0], value.x))
899  Msg::print(Msg::SevWarning, "Failed to read metadata "+
900  string(name) );
901  field->metadata().setVecFloatMetadata(name, value);
902  }
903  else {
904  Msg::print(Msg::SevWarning, "Attribute of size " +
905  boost::lexical_cast<std::string>(dims[0]) +
906  " is not valid for metadata");
907  }
908  }
909  else {
910  Msg::print(Msg::SevWarning, "Attribute '" + string(name) +
911  + "' has unsupported data type for metadata");
912 
913  }
914  }
915  }
916  if (name) {
917  delete[] name;
918  }
919  }
920  }
921  }
922 
923  return true;
924 }
Scoped object - opens an attribute data type on creation and closes it on destruction.
Definition: Hdf5Util.h:310
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity. ...
Definition: Log.cpp:66
Scoped object - Opens attribute by name and closes it on destruction.
Definition: Hdf5Util.h:113
Imath::V3i V3i
Definition: SpiMathLib.h:71
Scoped object - Opens attribute by index and closes it on destruction.
Definition: Hdf5Util.h:142
Imath::V3f V3f
Definition: SpiMathLib.h:73
Scoped object - opens an attribute data space on creation and closes it on destruction.
Definition: Hdf5Util.h:287
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
bool Field3DInputFile::readMetadata ( hid_t  metadata_id)
private

Read global metadata for this file.

Todo:
Replace char* with std::string

Definition at line 930 of file Field3DFile.cpp.

References g_hdf5Mutex, Hdf5Util::H5Base::id(), Field3DFileBase::metadata(), Msg::print(), Hdf5Util::readAttribute(), readGroupMembership(), FieldMetadata< CallBack_T >::setFloatMetadata(), FieldMetadata< CallBack_T >::setIntMetadata(), FieldMetadata< CallBack_T >::setStrMetadata(), FieldMetadata< CallBack_T >::setVecFloatMetadata(), FieldMetadata< CallBack_T >::setVecIntMetadata(), and Msg::SevWarning.

931 {
932  GlobalLock lock(g_hdf5Mutex);
933 
934  hsize_t num_attrs = H5Aget_num_attrs(metadata_id);
935 
936  if (num_attrs > 0) {
937  for (hsize_t idx=0; idx < num_attrs ; ++idx) {
938  H5ScopedAopenIdx attrIdx(metadata_id, idx);
939  size_t len = H5Aget_name(attrIdx.id(), 0, NULL);
940  if (len > 0) {
941  char *name = new char[len+1];
942  if (H5Aget_name(attrIdx.id(), len+1, name) > 0) {
943  H5ScopedAopen attr(metadata_id, name, H5P_DEFAULT);
944  H5ScopedAget_space attrSpace(attr);
945  H5ScopedAget_type attrType(attr);
946  H5T_class_t typeClass = H5Tget_class(attrType);
947 
948  if (typeClass == H5T_STRING) {
949  string value;
950  if (!readAttribute(metadata_id, name, value)) {
952  "Failed to read metadata " + string(name));
953  if (name) {
954  delete[] name;
955  }
956  continue;
957  }
958  metadata().setStrMetadata(name, value);
959 
960  }
961  else {
962 
963  if (H5Sget_simple_extent_ndims(attrSpace) != 1) {
964  Msg::print(Msg::SevWarning, "Bad attribute rank for attribute "
965  + string(name));
966  if (name) {
967  delete[] name;
968  }
969  continue;
970  }
971 
972  hsize_t dims[1];
973  H5Sget_simple_extent_dims(attrSpace, dims, NULL);
974 
975  if (typeClass == H5T_INTEGER) {
976  if (dims[0] == 1){
977  int value;
978  if (!readAttribute(metadata_id, name, dims[0], value))
979  Msg::print(Msg::SevWarning, "Failed to read metadata "
980  + string(name));
981  metadata().setIntMetadata(name, value);
982  }
983  else if (dims[0] == 3){
984  V3i value;
985  if (!readAttribute(metadata_id, name, dims[0], value.x))
986  Msg::print(Msg::SevWarning, "Failed to read metadata " +
987  string(name) );
988  metadata().setVecIntMetadata(name, value);
989  }
990  else {
992  "Attribute of size " +
993  boost::lexical_cast<std::string>(dims[0])
994  + " is not valid for metadata");
995  }
996  }
997  else if (typeClass == H5T_FLOAT) {
998  if (dims[0] == 1){
999  float value;
1000  if (!readAttribute(metadata_id, name, dims[0], value))
1001  Msg::print(Msg::SevWarning, "Failed to read metadata " +
1002  string(name) );
1003 
1004  metadata().setFloatMetadata(name, value);
1005  }
1006  else if (dims[0] == 3){
1007  V3f value;
1008  if (!readAttribute(metadata_id, name, dims[0], value.x))
1009  Msg::print(Msg::SevWarning, "Failed to read metadata "+
1010  string(name) );
1011  metadata().setVecFloatMetadata(name, value);
1012  }
1013  else {
1014  Msg::print(Msg::SevWarning, "Attribute of size " +
1015  boost::lexical_cast<std::string>(dims[0]) +
1016  " is not valid for metadata");
1017  }
1018  }
1019  else {
1020  Msg::print(Msg::SevWarning, "Attribute '" + string(name) +
1021  + "' has unsupported data type for metadata");
1022 
1023  }
1024  }
1025  }
1026  if (name) {
1027  delete[] name;
1028  }
1029  }
1030  }
1031  }
1032 
1033  return true;
1034 }
Scoped object - opens an attribute data type on creation and closes it on destruction.
Definition: Hdf5Util.h:310
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
void setFloatMetadata(const std::string &name, const float val)
Set the a float value for the given metadata name.
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity. ...
Definition: Log.cpp:66
void setVecIntMetadata(const std::string &name, const V3i &val)
Set the a V3i value for the given metadata name.
Scoped object - Opens attribute by name and closes it on destruction.
Definition: Hdf5Util.h:113
Imath::V3i V3i
Definition: SpiMathLib.h:71
void setVecFloatMetadata(const std::string &name, const V3f &val)
Set the a V3f value for the given metadata name.
Scoped object - Opens attribute by index and closes it on destruction.
Definition: Hdf5Util.h:142
void setStrMetadata(const std::string &name, const std::string &val)
Set the a string value for the given metadata name.
Imath::V3f V3f
Definition: SpiMathLib.h:73
Scoped object - opens an attribute data space on creation and closes it on destruction.
Definition: Hdf5Util.h:287
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
FieldMetadata< Field3DFileBase > & metadata()
accessor to the m_metadata class
Definition: Field3DFile.h:318
void setIntMetadata(const std::string &name, const int val)
Set the a int value for the given metadata name.

Member Data Documentation

std::string Field3DInputFile::m_filename
private

Filename, only to be set by open().

Definition at line 730 of file Field3DFile.h.


The documentation for this class was generated from the following files: