Here are the low level routines used in the CRIS/SIS data retrieval libraries. This documentation is primarily intended for those who need to port and/or rewrite parts of the libraries. See notes on setting up the data retrieval libraries for instructions on setting up the source, makefiles, and directory structure involved in creating the libraries.
getBeginEndSecsOfStructPriorTo
void getBeginEndSecsOfStructPriorTo(uint32 *startAndStop,
uint32 secondsSince1996, char *dataFilePath, char *structName);
This routine is neccessary for handling those data structures which
do not have a constant time period of validity (e.g. diagnostic events).
When the user asks for such a data struct associated with a time
secondsSince1996, we must step back through the data file from
secondsSince1996 until we find a non-zero struct. Once found,
we compare secondsSince1996 with the start and stop times of the
struct to see if secondsSince1996 took place while this struct was
valid. This stepping back may proceed back across a day boundary, but
it is assumed that on not finding a non-zero struct throughout the
entire previous day, no such struct will be found which is valid for
the time (secondsSince1996) requested.
If a struct is found which is valid at the time secondsSince1996, the start and stop times of the struct are returned as startAndStop[0] and startAndStop[1] respectively. Else they are both set to zero.
Input: startAndStop is expected to be a 2 element uint32 array. secondsSince1996 is the time requested by the user. dataFilePath is a string which contains the path to the data file we wish to search. structName is the name of the structures contained in the data file.
Output: Nothing is explicitly returned, but getBeginEndSecsOfStructPriorTo will set the two values of the uint32 array startAndStop.
Note: This is one of the more complex routines in the library mostly due to an attempt to optimize the search for the appropriate struct. The routine makes a guess at where the struct it wants will be in the file, jumps in and searches from there. A simpler, but slower implementation could just iterate through all the structs in the file until it finds what it is looking for...
getDataFileSuffixFromStructName
void getDataFileSuffixFromStructName(char *suffixBuffer, char *structName);The data files in the CRIS/SIS archive have suffixes associated with them, such as "hp" (high priority rates), "evt" (event cycles), etc. getDataFileSuffixFromStructName looks at the name of the structure we are dealing with and determines the suffix string associated with it.
This routine is useful in generating data file paths to the desired archive files.
Input: suffixBuffer is expected to be a char array at least 5 elements long (I made it 10 for the hell of it.) structName is the full name of the structure type involved (e.g., "L1CrisLowPriorityRate").
Output: suffixBuffer is set to the string associated with the archive file for the structure type structName.
getDaysStructs
void *getDaysStructs(int day1996, char *structName);getDaysStructs is used to retrieve all the data archived in a single data file (the data is archived in day-long files, according to the start time stamp on the data). getDaysStructs determines the memory needed to store the days worth of data, allocates it, reads the non-zero structs from the file into memory, terminates the array of structs with a zeroed struct, chops the memory allocation down to size (since any zeroed structs in the file were not copied into the array), and returns a pointer (void *) to the caller. The caller must handle deallocation of this memory when it is through with it.
Input: day1996 is the "day 1996" where day1996 = 1 on 1/1/96. structName is the full name of the structure for the data type we are requesting (e.g., "L1CrisLowPriorityRate").
Output: A pointer (void *) to the beginning of an array of structs of type structName. This pointer must be cast to the type structName by the caller.
getDaysTimeStamps
uint32 *getDaysTimeStamps(int day1996, char *structName);getDaysTimeStamps uses getDaysStructs to read all the data in one archive file and returns the start (and stop, where appropriate) time stamps from each of the non-zero data structs in the file. getDaysTimeStamps is used primarily as a diagnostic tool. getDaysTimeStamps allocates the memory required to create an array of uint32 time stamps. This caller must handle deallocation of this memory when it is through with it.
Input: day1996 is the "day 1996" where day1996 = 1 on 1/1/96. structName is the full name of the structure for the data type we are requesting (e.g., "L1CrisLowPriorityRate").
Output: A pointer (uint32 *) to an array of time stamps. Element 0 is the start time of the first struct, and element 1 is the stop time for the same struct. Element 2 is the start time of the next struct, and element 3 is the stop time. This pattern continues for the rest of the non-zero structs found for that day and data type. The array is terminated by a single element equal to zero (while looking down a list of start times in the array, it would be the only start time equal to zero.) For those structs which do not have a stop time stamp associated with them, the stop time is set to 1 (to avoid confusing them with the terminal zero value of the array.)
getFilesStructCount
int getFilesStructCount(char *dataFilePath, int structSize, int hasHeader);getFilesStructCount is used to determine the number of data structs of size structSize in the file pointed at by dataFilePath. It does this merely by looking at the size of the file, subtracting out the number of bytes in the standard CRIS/SIS data archive header if the flag hasHeader is set (it usually will be), and then dividing by structSize. If the divide does not return an integral number, getFilesStructCount will exit with an error message.
Input: The string specifying the pathname of the data file dataFilePath, the size of the structures the file should be made up of, structSize, and the flag hasHeader which indicates whether the file should have a standard CRIS/SIS data archive header in it as well as the data.
Output: The number of data structures in the data file. Beware that this is a "raw" number, which does not indicate whether there are zeroed (no data in the struct) structures in the file or not.
getInstrumentIDFromStructName
char getInstrumentIDFromStructName(char *structName);getInstrumentIDFromStructName merely indicates whether we are dealing with CRIS or with SIS data.
Input: structName is the full name of the structure for the data type we are requesting (e.g., "L1CrisLowPriorityRate").
Output: Either 'c' for CRIS or 's' for SIS.
getStopTimeStamp
uint32 getStopTimeStamp(void *pStruct, char *structName);getStopTimeStamp extracts the stop time stamp from the struct of type structName. If there is no stop time stamp field in the struct, a zero is returned.
Input: A pointer (void *) to the beginning of the structure, pStruct, and structName, the full name of the struct type we are dealing with.
Output: The stop time stamp associated with the structure, or zero if the struct type has no stop time field.
getStruct
void *getStruct(uint32 secondsSince1996, char *structName);getStruct retrieves any data structure of type structName from the archives which was valid at time secondsSince1996. Beware that despite the singular nature of the name, a pointer to an array of structs of type structName is always returned. This array is always terminated by a zeroed struct of type structName. The reason for this is that, due to some anomoly with the clock perhaps, more than 1 data struct may be stamped as valid during a particular interval, so there is a small chance of getting more than a single "valid" data struct back. There is a much better chance of the array only containing the terminal zeroed struct. In such a case, either no data exists during the interval or it has not yet been archived.
getStruct is really just a special case of getStructs, where both secondsSince1996Start and secondsSince1996Stop are set to the same value, secondsSince1996.
Memory has been allocated inside the call to getStruct for the array of structures returned, and this memory needs to be freed by the caller.
Input: We are interested in finding a data structure of the type structName which is valid at the time secondsSince1996. structName is the full name of the structure for the data type we are requesting (e.g., "L1CrisLowPriorityRate").
Output: A pointer (void *) to an array of structures of type structName. The calling routine must cast the pointer to type structName.
getStructSizeFromName
int getStructSizeFromName(char *structName);getStructSizeFromName merely returns the size (in bytes) of the data struct structName.
Input: structName is the full name of the struct type (e.g., "L1CrisLowPriorityRate").
Output: The size of the struct type structName in bytes.
getStructs
void *getStructs(uint32 secondsSince1996Start, uint32 secondsSince1996Stop, char *structName);getStructs retrieves any data structures of type structName from the archives which was valid between the start time, secondsSince1996Start, and the stop time, secondsSince1996Stop (But see the note below for an important clarification of exactly what this means.) A pointer (void *) to an array of structs of type structName is returned. This array is always terminated by a zeroed struct of type structName.
The start and stop times used in getStructs can be across an arbitrary time interval. This may cause problems if users abuse it. In other words, getStructs can be called on to allocate an arbitrary amount of memory for the struct array it will be creating. Users should be warned about this (though they will likely be accessing getStructs via a more user friendly wrapper like getGroupHiRates_s or the like.)
Memory has been allocated in the call to getStructs for the array of structures returned, and this memory needs to be freed by the caller at some point.
Input: The time interval over which to retrieve valid data structs is times greater than or equal to secondsSince1996Start up, but less than the start time stamp on the struct which is valid at secondsSince1996Stop. (Confused? See note below.) structName is the full name of the structure for the data type we are requesting (e.g., "L1CrisLowPriorityRate").
Output: A pointer (void *) to the array of structs of type structName. The calling routine must cast the pointer to type structName.
Note: A structure which is valid at the time secondsSince1996Start will always be returned by this call, but the structure valid at secondsSince1996Stop will not be returned unless it is the same structure which is valid at secondsSince1996Start. Any and all structures valid at times between secondsSince1996Start and secondsSince1996Stop will be returned (which seems too obvious to state.) The reason that the last structure (the one valid at secondsSince1996Stop) is not returned by this call is to prevent seemingly contiguous calls to getStructs from returning overlapping data. For this reason, any series of calls to getStructs for contiguous data must use the prior secondsSince1996Stop as the secondsSince1996Start for the new call. Otherwise there will be a 1 second gap in each call's time intervals.
getSubsetOfDaysStructs
void *getSubsetOfDaysStructs(char *path, char *structName,
uint32 getStampsAfter, uint32 getStampsUpTo);
getSubsetOfDaysStructs is the real data retrieval engine. All calls to
retrieve data from the archive go through this routine at some point.
getSubsetOfDaysStructs will look at 1 file pointed at by path, and return a pointer to an array of structs of type structName which have time stamps greater than getStampsAfter and less than or equal to getStampsUpTo. It does not care about times of validity of data structs, but only what the start time stamp of a struct is. Times of validity must be dealt with before the call to getSubsetOfDaysStructs in order to just hand it the values getStampsAfter and getStampsUpTo.
getSubsetOfDaysStructs allocates enough memory to read the entire day's file, copies in the structs it wants, adds the terminal zeroed struct, chops the memory allocation down to size, and then returns. This memory needs to be deallocated by the caller at some point.
Input: path is the string giving the location of the data file in the file system. structName is the full name of the structure for the data type we are requesting (e.g., "L1CrisLowPriorityRate"). The times getStampsAfter and getStampsUpTo refer to the time interval in which data will be culled from the file, dependent on their start time stamps.
Output: A pointer (void *) to the array of structs of type structName. The calling routine must cast the pointer to type structName.
getTimeStamp
uint32 getTimeStamp(void *pStruct, char *structName);getTimeStamp merely returns the start time stamp of a data struct of type structName.
Input: A pointer (void *) to the start of the data struct, pStruct, and structName, the full name of the structure (e.g., "L1CrisLowPriorityRate").
Output: The start time stamp of the data struct.
getWorkingStartTime
uint32 getWorkingStartTime(uint32 secondsSince1996, char *structName);getWorkingStartTime finds the earliest time a start time stamp could have in order for a data struct of type structName to be valid at the time secondsSince1996. Generally, this is an easy process of just subtracting the period associated with data structs of type structName (and adding 1 second), but in cases where the data is not valid in periodic intervals, stop times of a previous struct must be consulted instead to find this working start time. (The routine getBeginEndSecsOfStructPriorTo is utilized in these cases.)
The idea here is to find begin times which can be passed to the routine getSubsetOfDaysStructs such that getSubsetOfDaysStructs only needs to concern itself with a data struct's time stamp instead of worrying about the time interval over which the data was valid. That concern is instead dealt with here in getWorkingStartTime.
Input: secondsSince1996 is the time at which the caller wishes to find a data struct which is valid. structName is the full name of the structurefor the data type the caller is interested in.
Output: The earliest time stamp of which the data struct which is valid at secondsSince1996 could possibly have. (This corresponds to 1 second past the latest time stamp that a data struct prior to the valid data struct could possibly have, and as such, no possible overlap in retrieving the correct data exists...)