).
Input:
- pTimeStruct is a pointer to a tm struct (as defined in ).
See code sample for fields of interest and possible values. Be aware
that tm_year is the year since 1900, and most fields have a zero least
value. The exception is the month-day, tm_mday.
Output:
- A long, unsigned int, the number of seconds-since-1996 (which corresponds
to the values of the spacecraft clock's time stamps on all data.
Get the number of seconds since 1996 from the day-of-1996
uint32 getSecondsSince1996FromDay1996(int day1996);
getSecondsSince1996FromDay1996 returns the seconds since 1996
corresponding to the zeroth second of the day day1996. Be aware
that day1996 = 1 is defined as 1/1/96.
Input:
- day1996 is the day-of-1996 of interest.
Output:
- A long, unsigned int, the number of seconds since 1996 (which corresponds
to the values of the spacecraft clock's time stamps on all data.
General Archive Information
End-user routines to get general information about the archive and archive files.
Determine the date of the first data archived for an instrument and data type
void getFirstDateOfArchivedData(char* structName, struct tm *pTimeStruct);
getFirstDateOfArchivedData is used to determine what the first day of
a particular type of data is in the archive for a particular instrument
(CRIS or SIS).
Input:
- structName is a pointer to a string with the name of the data type
you are interested in.
- pTimeStruct is pointer to a struct tm to be filled in by
getFirstDateOfArchivedData.
Output:
- The struct tm pointed to by pTimeStruct is filled with the values
corresponding to the first date the data is available for.
Determine the date of the last data archived for an instrument and data type
void getLastDateOfArchivedData(char* structName, struct tm *pTimeStruct);
getLastDateOfArchivedData is used to determine what the first last of
a particular type of data is in the archive for a particular instrument
(CRIS or SIS).
Input:
- structName is the name of the data type name you are interested in.
- pTimeStruct is a pointer to a struct tm to be filled in by
getLastDateOfArchivedData.
Output:
- The struct tm pointed to by pTimeStruct is filled with the values
corresponding to the last date the data is available for.
Get a list of the time stamps of a particular data type for a given day
uint32 *getTimeStampsOfDaysStructs(int day1996, char *structName);
Get a list of the time stamps of a particular data type for a given
day. The time stamps returned are those starting at the data valid
at second zero of the day and ending just before the data which is
valid at second zero of the next day. Any intervals of missing data
for the day return values of -1 for their time stamps. Beware that
both start and stop time stamps are returned. For periodic data types,
this means all of the stop time stamps will be set to -1 (because
they have no stop time stamps.)
You need to free the memory allocated to the time stamp list when you
are finished using it.
Input:
- day1996 is the day-of-1996 of interest.
- structName is the full name of the data type of interest.
Output:
- An array of start and stop time stamps, terminated by a zero value.
The array is arranged such that startStamp[0] is followed by stopStamp[0],
followed by startStamp[1], and so forth.
Event Processing Tools
End-user routines specific to the processing of event data.
Get a list of times which can be used to iteratively retrieve a day's
worth of event data
uint32 *getRequestTimesForDaysEvtCycles(int day1996, char instrumentID);
Get a list of times which can be used to iteratively retrieve a day's
worth of event data with no possibility of getting overlapping data
when using this routine to retrieve data over many days. These times
*are not* data time stamps because actually getting the time stamps
would be just as time consuming as a full read of the data.
You need to free the memory allocated to this list when you are
finished using it.
Input:
- day1996 is the day-of-1996.
- instrumentID is the id of the instrument of interest: 'c' for CRIS
and 's' for SIS.
Output:
- An array of times (in seconds-since-1996), terminated by a zero value.
Low Level Time Stamp Routines
Low level routines for time stamp determination.
Get the start time stamp of a data struct
uint32 getTimeStamp(void *pStruct, char *structName);
getTimeStamp merely returns the start time stamp of a data struct of type
structName.
Input:
- pStruct is a pointer (void *) to the start of the data struct.
- structName is the full name of the structure (e.g., "L1CrisLowPriorityRate").
Output:
- The start time stamp of the data struct.
Get a struct's stop time stamp (if it has one)
uint32 getStopTimeStamp(void *pStruct, char *structName);
getStopTimeStamp extracts the stop time stamp from the struct of type
structName. If there is no stop time stamp field in the struct, a zero is
returned.
Input:
- pStruct is a pointer (void *) to the beginning of a structure.
- structName is the full name of the struct type we are dealing with.
Output:
- The stop time stamp associated with the structure,
- or zero if the struct type has no stop time field.
Low Level Archive Information Tools
Low level archive data retrieval utilities.
Get the time stamps of the data in one archive file (WARNING: See below.)
uint32 *getArchiveFilesTimeStamps(int day1996, char *structName);
getArchiveFilesTimeStamps creates a list of the time stamps of a
particular type of data in one daily archive file. WARNING: This routine
*does not* return what is referred to elsewhere as a day's set of data.
This routine should be used to check what time stamps are on data in an
archive file for debugging purposes only. Do not process data using this
routine.
You must free the memory associated with the time stamp list when you
are finished using it.
Input:
- day1996 is the day-of-1996 of interest.
- structName is the name of the data structure you are interested in.
Output:
- The list of time stamps (in seconds-since-1996) on the data in the
archive file. Time stamps equal to 0 in the data will be set to -1
to differentiate them from the terminal value in the list, which is
0.
Construct the path to a data archive file
void generateDataFilePath(char *pathBuffer, struct tm *timeStruct,
char instrumentID, const char *dataFileSuffix);
generateDataFilePath constructs the path to a particular archive data file.
Input:
- pathBuffer is a char array which must be passed to generateDataFilePath
in which to place the generated path. I recommend at least a 50 character
long array.
- timeStruct is the date in struct tm format (see ).
- instrumentID is a char, 'c' for CRIS, 's' for SIS.
- dataFileSuffix is a pointer to a string, the archive data file suffix
corresponding to the data type involved (bp0, ct, de, do0, do1, evt, hp,
hsk, lp, sub, or sum)
Output:
- The generated path is placed into pathBuffer.
Get the suffix which marks a data type in the archive files
void getDataFileSuffixFromStructName(char *suffixBuffer, char *structName);
The data files in the CRIS/SIS archive have suffixes associated with them,
such as "hp" (high priority rates), "evt" (event cycles), etc.
getDataFileSuffixFromStructName looks at the name of the structure
we are dealing with and determines the suffix string associated with it.
This routine is useful in generating data file paths to the desired archive
files.
Input:
- suffixBuffer is expected to be a char array at least 5 elements long (I
made it 10 for the heck of it.)
- structName is the full name of the structure type involved (e.g.,
"L1CrisLowPriorityRate").
Output:
- suffixBuffer is set to the string associated with the archive file for the
structure type structName.
Get the data type's validity period per data struct
int getDataTypePeriodFromStructName(char *structName);
Most of the data types in the CRIS/SIS archive are periodic in the time
interval they cover. getDataTypePeriodFromStructName takes the
name of a data type and returns this period.
Input:
- structName is the full name of the structure type involved (e.g.,
"L1CrisLowPriorityRate").
Output:
- The time period that a data struct of the type given by
structName is valid for.
- If the data type is non-periodic, a zero is returned.
Get the data type's instrument ID ('c' for CRIS, 's' for SIS)
char getInstrumentIDFromStructName(char *structName);
Get an ID for the instrument associated with the data type
given by structName.
Input:
- structName is the name of the data type of interest.
Output:
- Either 'c' for CRIS or 's' for SIS.
Get the number of structs in a particular archive file
int getFilesStructCount(char *dataFilePath, int structSize, int hasHeader);
getFilesStructCount is used to determine the number of data structs of
size structSize in the file pointed at by dataFilePath. It does this merely
by looking at the size of the file, subtracting out the number of bytes in
the standard CRIS/SIS data archive header if the flag hasHeader is
set (it usually will be), and then dividing by structSize. If the divide
does not return an integral number, getFilesStructCount will exit with
an error message.
Input:
- dataFilePath is the string specifying the pathname of the data file
dataFilePath.
- structSize is the size of the structures the file should be made up of.
- hasHeader is a flag indicating whether the file should have a standard
CRIS/SIS data archive header in it in addition to the data.
Output:
- The number of data structures in the data file. Beware that
this is a "raw" number, which does not indicate whether there are
zeroed (no data in the struct) structures in the file or not.
Get the size (in bytes) of a type of data struct
int getStructSizeFromName(char *structName);
getStructSizeFromName merely returns the size (in bytes) of the data
struct structName.
Input:
- structName is the full name of the struct type (e.g.,
"L1CrisLowPriorityRate").
Output:
- The size of the struct type structName in bytes.
Check that a data type archive file suffix is valid
char *validateDataFileSuffix(char *suffix);
Check that a data type archive file suffix is valid. If it is not
valid, but is recognizable, the suffix is fixed to the valid
version. If it is not recognizable, the process exits.
Input:
- suffix is a pointer to the suffix string.
Output:
- A pointer to a valid suffix string.
- If no valid suffix can be determined, the process terminates.
Check that an instrument ID is valid
char validateInstrumentID(char id);
Check that an instrument ID is valid. If it is not
valid, but is recognizable, the ID is fixed to the valid
version. If it is not recognizable, the process exits.
Input:
Output:
- The valid ID character. That is, a 'c' (CRIS) or an 's' (SIS).
- If the correct ID cannot be determined, the process exits.
Low Level Data Retrieval Routines
Low level archive data retrieval engines and low level wrappers.
Get a set of data structs between a start and a stop time
void *getStructs(uint32 secondsSince1996Start,
uint32 secondsSince1996Stop, char *structName);
getStructs retrieves any data structures of type structName from the
archives which was valid between the start time,
secondsSince1996Start, and the stop time, secondsSince1996Stop
(But see the note below for an important clarification of exactly what
this means.) A pointer (void *) to an array of structs of type
structName is returned. This array is always terminated by a zeroed
struct of type structName.
The start and stop times used in getStructs can be across an arbitrary
time interval. This may cause problems if users abuse it. In other
words, getStructs can be called on to allocate an arbitrary amount of
memory for the struct array it will be creating. Users should be
warned about this (though they will likely be accessing getStructs via
a more user friendly wrapper like getGroupHiRates_s or the like.)
Memory has been allocated in the call to getStructs for the array of
structures returned, and this memory needs to be freed by the caller at
some point.
Note: A structure which is valid at the time secondsSince1996Start
will always be returned by this call, but the structure valid at
secondsSince1996Stop will not be returned unless it is the same
structure which is valid at secondsSince1996Start. Any and all
structures valid at times between secondsSince1996Start and
secondsSince1996Stop will be returned (which seems too obvious to
state.) The reason that the last structure (the one valid at
secondsSince1996Stop) is not returned by this call is to prevent
seemingly contiguous calls to getStructs from returning overlapping
data. For this reason, any series of calls to getStructs for contiguous
data must use the prior secondsSince1996Stop as the
secondsSince1996Start for the new call. Otherwise there will be a 1
second gap in each call's time intervals.
Input:
- secondsSince1996Start is the start second on which to look for valid
data. See note above for specifics.
- secondsSince1996Stop is the stop second for the data retrieval. See
note above for specifics.
- structName is the full name of the structure for the data type we are
requesting (e.g., "L1CrisLowPriorityRate").
Output:
- A pointer (void *) to the array of structs of type structName.
The calling routine must cast the pointer to type structName.
Get a data struct valid at a particular second-since-1996
void *getStruct(uint32 secondsSince1996, char *structName);
getStruct retrieves any data structure of type structName from the
archives which was valid at time secondsSince1996. Beware that
despite the singular nature of the name, a pointer to an array of
structs of type structName is always returned. This array is always
terminated by a zeroed struct of type structName. The reason for this
is that, due to some anomoly with the clock perhaps, more than 1 data
struct may be stamped as valid during a particular interval, so there is
a small chance of getting more than a single "valid" data struct back.
There is a much better chance of the array only containing the
terminal zeroed struct. In such a case, either no data exists during the
interval or it has not yet been archived.
-
getStruct is really just a special case of getStructs, where both
secondsSince1996Start and secondsSince1996Stop are set to the
same value, secondsSince1996.
-
Memory has been allocated inside the call to getStruct for the array of
structures returned, and this memory needs to be freed by the caller.
Input:
- secondsSince1996 is the second-since-1996 we are looking for valid data.
- structName is the full name of the structure for the data type we are
requesting (e.g., "L1CrisLowPriorityRate").
Output:
- A pointer (void *) to an array of structures of type
structName. The calling routine must cast the pointer to type
structName.
A get a day's worth of data structs
void *getDaysStructs(int day1996, char *structName);
A get a day's worth of structs, starting with the data struct which
is valid at second zero of the day (which is usually archived in the
file of the previous day), and ending just prior to the data struct
which is valid at second zero of the next day (which is usually the
last struct in this day's archive file.)
getDaysStructs determines the memory needed to
store the days worth of data, allocates it, reads the non-zero structs
from the file into memory, terminates the array of structs with a
zeroed struct, chops the memory allocation down to size (since any
zeroed structs in the file were not copied into the array), and returns a
pointer (void *) to the caller. The caller must handle deallocation of
this memory when it is through with it.
Input:
- day1996 is the "day 1996" where day1996 = 1 on 1/1/96.
- structName is the full name of the structure for the data type we are
requesting (e.g., "L1CrisLowPriorityRate").
Output:
- A pointer (void *) to the beginning of an array of structs of
type structName. This pointer must be cast to the type structName by
the caller.
Low Level File I/O and System Utilities
Low level file I/O routines.
Check on the existence of a file
int fileExists(char *path);
fileExists is used primarily by the put data routines to check if an
archive data file exists already or needs to be created and initialized.
Returns 1 (true) if the file does exist, and 0 (false) if not. If the
file is actually a directory, 0 is returned.
Input:
- A pointer to the file's path string.
Output:
- 1 if the file exists.
- 0 if the file does not exist (or if the file is actually a directory.)
Check on the existence of a directory
int directoryExists(char *path);
directoryExists is used primarily by the put data routines to check if an
archive directory already exists (before createDirectory is called).
Returns 1 (true) if the directory does exist, and 0 (false) if not.
Input:
- A pointer to the directory's path string.
Output:
- 1 if the directory exists.
- 0 if the file does not exist (or if the directory is actually a file.)
Create a new directory
int createDirectory(char *path);
createDirectory is used primarily by the put data routines to create
new locations for archiving the data. If createDirectory fails to create
the directory, the program will exit.1
Input:
- A pointer to the directory's path string.
Output:
- 1 if successful.
- Program exits if unsuccessful.
Open a file (or create it if it doesn't exist) with read/write capability
int openOrCreateFileToReadWrite(char *path);
openOrCreateFileToReadWrite is a low level routine used internally in
the CRIS/SIS routines. It is used mostly by the put routines to write
data to an archive file. If the file does not exist, it is created,
initialized, and re-opened with read and write capability.
openOrCreateFileToReadWrite also does some error checking and
exits if the file cannot be created or opened as expected.
Input:
- path is a pointer to a string containing the path to the file.
Output:
- 1 if successful.
- The run terminates if openOrCreateFileToReadWrite is unsuccessful.
Read from a file
int readFromFile(int fd, char *buffer, int bytesToRead);
readFromFile is a low level routine used internally in the CRIS/SIS
routines. Primarily it just adds a little error checking to the usual read
calls. If the routine does not read the expected number of bytes into
the buffer, it will exit.
Input:
- fd is the file descriptor for the already opened file.
- buffer is the user allocated buffer to read into from the file.
- bytesToRead is the number of bytes to read from file (cannot be
larger than the buffer size.)
Output:
- buffer is written to and 1 is returned.
- If readFromFile is unsuccessful, the program terminates.
Write to a file
int writeToFile(int fd, char *buffer, int bytesToWrite);
writeToFile is a low level routine used internally in the CRIS/SIS
routines. Primarily it just adds a little error checking to the usual write
calls. If writeToFile does not write the expected number of bytes from
the buffer into the file, it will exit.
Input:
- fd is the file descriptor for the already opened file.
- buffer is the user allocated buffer containing the data to write
into the file.
- bytesToWrite is the number of bytes to write into the file (cannot
be larger than the buffer size.)
Output:
- A file is written to and 1 is returned.
- If the write is unsuccessful, the program exits.
Low Level Data Archival Routines
Low level data archival routines. Available for C only.
Initialize a day's data file for a particular data type
void createInitializedFile(int fd, char dataBuffer[], int bufferSize,
int dataTypeID);
createInitializedFile initializes a day's data file for a particular data
type. It writes the 32 byte header and zeros the rest of the file. The file
is made large enough on initialization to contain all of the given type of
data for the full day. createInitializedFile assumes that the data type
has a periodicity of 256 seconds per data struct. Data types with
different periododicities use the routine
createInitializedFileWithIntervalN. Non-periodic data types use
neither.
Input:
- fd is the file descriptor of the just created, but uninitialized, file.
- bufferSize is equal to the size of a single struct of the data type
which is to be archived in this file.
- dataBuffer is a char array of size bufferSize.
- dataTypeID is the ID number assigned to the data type to be
archived in this file. These IDs are defined in the header file
FileTypes.h.
Output:
- None. (A file is written to disk.)
Initialize aday's data file for a data type with a period other than 256 secs
void createInitializedFileWithIntervalN(int fd, char dataBuffer[],
int bufferSize, int dataTypeID, int timeInterval);
See notes for routine createInitializedFile.
Input:
- fd is the file descriptor of the just created, but uninitialized, file.
- bufferSize is equal to the size of a single struct of the data type
which is to be archived in this file.
- dataBuffer is a char array of size bufferSize.
- dataTypeID is the ID number assigned to the data type to be
archived in this file. These IDs are defined in the header file
FileTypes.h.
- timeInterval is the period of time (in seconds) for which the
data structs in question cover. (createInitializedFile assumes
this value to be 256.)
Output:
- None. (A file is written to disk.)
Generate a data file path and create directories as needed for storage
void determineDataFilePathAndCreateDirsAsNeeded(uint32 secondsSince1996,
const char *dataFileSuffix, char *path,
int *pIntervalNumber, char instrumentID);
determineDataFilePathAndCreateDirsAsNeeded determines the
path to the archive file of a particular data type from a particular day.
If parts of this path do not exist yet, this routine attempts to create
them before handing back the file's (or file-to-be's) path.
Input:
- secondsSince1996 is the number of seconds since 1/1/96
00:00:00. All data structs are time tagged with such a number.
- dataFileSuffix is the 2 or 3 character string which is appended
to all data files containing a particular type of data. E.g., the
"hp" at the end of the file sis_prelim.1997.10.31.hp is the
marker used for all high priority rate data.
- path is a pointer to a char array of sufficient size to hold the file
path.
- pIntervalNumber is a pointer to an int.
- instrumentID is either 'c' or 's' to denote CRIS or SIS
respectively.
Output:
- The path of the data file is placed in the location pointed to by
the char* path (which is passed to the routine as an argument).
- The time slot in the data file associated with the value of the
secondsSince1996 argument is returned in the location pointed
to by the int* pIntervalNumber (which is passed to the routine
as an argument).
Reorder data structs in a file
int sortTrackers(const void *pVoid1, const void *pVoid2);
sortTrackers is called by qsort in a few routines to reorder data structs
in their files. This routine is used only for the reordering of
non-periodic data types (i.e., those with start and stop time stamps) in
their files when a struct needs to be written to an archive file
somewhere before the end of the file in order to preserve a
chronological ordering of the structs in the file.
Input:
- pVoid1 is a void pointer to the first struct.
- pVoid2 is a void pointer to the second struct.
Output:
- An integer is returned signifying the chronological ordering (by
start time stamps) of the two structs being tested.
Write a data struct into the proper position in its archive file
int writeDataStructToFile(int fd, int intervalNumber, char dataBuffer[],
int bufferSize, char *pDataStruct, char *path);
writeDataStructToFile is used to write a data struct into the proper
position in its archive file.
First this routine checks what has been written into the data file at the
interval where the passed data struct is to be written. If the interval is
all zeros (done at file initialization), the struct is written and a status
of 0 is returned.
If the interval is found to contain data already, both the passed struct
and the one already in the file are written out to an auxiliary file (same
file name with ".overlap" suffix appended), the interval is filled with
char -1s, and a status of 1 is returned.
If the interval is filled with char -1s, then the passed struct is written
to the auxiliary file and a status of 2 is returned.
Finally, if the passed data struct is identical to the data already in the
interval, nothing is written, and a status of 3 is returned.
Input:
- fd is the file descriptor for the already opened-for-write data file.
- intervalNumber is the number of the slot within the data file
where the data will be (or attempted to be) written.
- dataBuffer is a char array of size equal to the struct to be written.
- bufferSize is the size of the struct to be written.
- pDataStruct is a pointer to the struct to be written.
- The path of the data file is passed in case an "overlap" file path
needs to be constructed to write overlap data (see above.)
Output:
- An integer status is returned. See description above for details.
Write data archive file's header
void writeHeader(int fd, int dataTypeID);
writeHeader is used for writing the 32 byte header present at the
beginning of every data file in the archive. See
header description.
Input:
- fd is the file descriptor for the already opened-for-write data file.
- dataTypeID is an ID for the data type to be written to the file.
These IDs are defined in the C header file FileTypes.h.
Output:
- None. (A 32 byte header is written at the beginning of a data file.)