CRIS/SIS: Data Archive File Details

All Data Archive Files

A block of data is handed to the putData routines as a structure, but it is then written to file as a byte array. Later, the getData routines will reverse the process by reading a specified time slot contents as a byte array into memory allocated to a struct. This may cause problems if different architectures are used to do the putData than are used to do the getData.

Each data file also has a 32 byte header.

Periodic Data

When a day's data file is first created for data structs delivered from the spacecraft at constant intervals, the data file is initialized with space for data covering the entire day.

For example, L1CrisLowPriorityRate structs cover a period of 256 seconds, so a daily archive file for CRIS low priority data will have 338 "time slots" to cover the day's 86400 seconds. (338 * 256 = 86528 = 86400 + 128). Note: The 128 second overlap into the next day is ignored, as any data falling therein will be placed into the following day's data.

[Important Notes: Users of the data retrieval libraries do not need to know anything at all about these "time slots" or any of the archival mechanics. Users merely ask for a block of data which was valid at a particular second, and the routines retrieve the data if such exists in the archive.

Also, if you are interested in how the data is retrieved from an archive, be careful to note that the "time slots" of an archive file are used by the retrieval routines merely as rough guides to where the data should be. The retrieval routines will step forward or back from that point, looking at actual data time stamps in order to retrieve the correct data.]

Now, assuming an L1CrisLowPriorityRate struct is of size 784 bytes, one day's CRIS low priority rate file will be of size 32 (header) + (784 * 338) = 265024 bytes. Under normal operating conditions, the spacecraft will return no more than 338 CRIS low priority rate data blocks, each with a time stamp differing by 256 seconds (or more, if there is any data dropout for whatever reason). The data having a time stamp falling between 0 and 255 seconds into the day in question will be written into slot 0 (slots are numbered 0 - 337), the data with a time stamp falling between 256 and 511 seconds into the day will be written into slot 1, and so forth. Into the last slot will be written the data with a time stamp between 86272 and 86399 seconds into the day. Any time stamp falling beyond that range falls in the next day and will be placed in the next day's data file.


In the illustration above, we see a typical archival of data. This data has a periodicity of 256 seconds. We can see that day 642 has been broken up into "time slots" of 256 second intervals. Shown here is part of slot 9, slots 10 - 12, and part of slot 13. Along the top, are listed the beginning seconds for each of these slot's intervals.

In red, we see the blocks of data with their respective time stamps (the actual time stamps in the data structs are in seconds-since-1996, but here have been translated into day-of-1996, second-of-day for clarity.) The blue arrow pointing down at the beginning of each of the data blocks points into the slot in the archive file into which that particular data has been archived. Note that the slot a particular data block will be archived in is solely determined by the start time stamp on the block.

As can be seen in the illustration above, data stamped as day 642, sec 2495 is archived in day 642's slot 9, data stamped as day 642, second 2751 is archived in day 642's slot 10, etc.


Some confusion may arise about what occurs at the end/start of a day, especially due to the fact that the last slot of a day generally covers only a partial period. The illustrations below address these questions.

In the above illustration, we see the end of day 642 and the start of day 643 for data with, again, a periodicity of 256 seconds. Shown are day 642's slots 335 to 337 and day 643's slots 0 and 1.

Note that slot 337 (day 642) is only half as wide as the other slots. This is due to the fact that since all slots start at a second-of-day = n * 256 (and thus slot 337 starts at second-of-day = 337 * 256 = 86272), and the last second of the day is second 86399, slot 337 time interval terminates at the day's end instead of after a full period.

Because slot 337 is not a full period wide, it will be a common (and valid) occurance for no data to be archived in slot 337. Such a case is illustrated in the image below.


Non-Periodic Data

Archive files for data types not recieved on a periodic basis are added to as each block of data comes in. Each block will have a start and stop time stamp. The contents of the file are shuffled about in order that the new block can be placed into the file ordered by start time stamps. I.e., these files are not constant in size, neither do they utilize any concept of "time slots" as the periodic data structs do.


bruce