wiki:TimeState

Binary file (*.tmst) holding detailed Time State information.

  • The file resides in ~/ultrascan/results/<runid> after data import.
  • The name there is "<runid>.time_state.tmst".
  • It is binary in content.
  • The format is self-defining so that it is extensible. The definitions are held in a separate corresponding XML file (*/<runid>.time_state.xml).
  • The file initially resides in ~/ultrascan/imports/<imp_runid> if it has been generated along with *.mwrs or similar files.
  • If the file does not exist in */imports/*, it is generated on import.
  • The general format of contents is as follows.
    • File Header
    • Time State Data Records
  • Integer values are in big-endian format (2-byte or 4-byte).
  • Float values are in big-endian Intel float (IEEE-754) format (4-byte).
  • For completeness, there exists additional possible data types.
    • 8-byte float, which is essentially a double in big-endian byte order.
    • Character string of arbitrary (1- to 127-byte) length.
    • Neither of these will likely be used initially.
  • The Header section contains the following.
    • 4-byte magic number ("USTS") as file type identifier.
    • 1-byte major version (initially "1").
    • 1-byte minor version (initially "0").
  • The separate sister XML file defines the following.
    • Number of times ("time_count", the number of data records).
    • Boolean flag (1|0) if a constant time increment exists ("constant_incr").
    • Time increment in seconds ("time_increment"; for example, 1.0).
    • First time ("first_time", most likely 0.0).
    • Data record field definitions, given as "key" and "format" pairs.
      • The key values field-identifying strings (e.g., "Time", "Omega2t", ...)
      • The format values are 2- to 4-byte strings that give type and length ( "I4", "F4", "I2", "I1", "F8", "Cnnn" [e.g., "C12"] )
    • The key values given must, of course, be unique.
  • The Data section contains the following.
    • A set of contiguous data records, "time_count" of them.
    • Each record is a set of contiguous binary values.
    • The count of values in each record equals the number of keys given.
    • Binary values in a record are in the order that keys were specified.
    • The length of each value is as specified in its format string (e.g., 4 bytes for "F4", 2 bytes for "I2", 13 bytes for "C13").
  • The Import version of this file will often have a constant time increment and a large number of data records equal to the duration of the experiment in seconds divided by the increment (e.g., 1). This is the real purpose of the object/file: to hold state information for high-resolution time sampling.
  • Absent an import time state pair of files, one might be generated in order to hold values for scan times, such as raw rotor speed ("RawSpeed"). In this case, the number of data records is equal to the number of AUC scans and not at a constant time increment (constant_incr="0").
  • The pair of files (.tmst and .xml) must always exist in the same directory. Their base names would only differ in extension. For example,
          CdTe_10j_U.time_state.tmst
          CdTe_10j_U.time_state.xml
    
  • An API (utils library class) for creating and reading TMST objects has been constructed.

XML that defines the contents of a sibling TMST binary file.

A sample XML, meant to illustrate the range of possible attributes.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE US_TimeState>
<US_TimeState version="1.0">
  <file time_count="30321" constant_incr="1" time_increment="1" first_time="0"/>
  <value key="Time" format="I4"/>
  <value key="Omega2t" format="F4"/>
  <value key="OnScan" format="I1"/>
  <value key="Scan" format="I2"/>
  <value key="Omega2tE" format="F8"/>
  <value key="Comments" format="C12"/>
</US_TimeState>

Options for <file> attributes.

  • "time_count" is given if known and is the total number of data records.
  • "constant_incr" is a boolean ("1" or "0") and specifies whether or not data records are for times at a constant increment. If "0", records must have a "Time" attribute.
  • "time_increment" is given if constant_incr=1, or defaults to 1.
  • "first_time" is given if constant_incr=1, or defaults to 0.
  • all time values are in seconds.

Options for <value> attributes.

  • "key" is a character string identifying the value.
  • "format" identifies the storage format for the value, where
    • "I4" means 4-byte integer,
    • "I2" means 2-byte integer,
    • "I1" means 1-byte integer (most often a boolean, either "1" or "0"),
    • "F4" means 4-byte IEEE-754 float,
    • "F8" means 8-byte IEEE-754 double,
    • "Cnnn" means character string, where the following numeric gives the length and must be a positive value.
  • All given "key" strings must be unique.

Storage implied by <value> attributes.

  • Each data record is a binary blob with values stored contiguously in the order that value keys are given.
  • For the sample above with formats of I4,F4,I1,I2,F8,C12; the total length of each record is 31 bytes (4 + 4 + 1 + 2 + 8 + 12).
  • The storage for character values is the given length (12 in our example), but when specified they may be of any length.
  • Given character values are padded or truncated so all are stored in the same sized space in the records.
  • I values are stored in big-endian format.
  • F values are stored in big-endian format.
  • C values are stored as 8-bit ASCII.

API fetch/store implications.

  • All "I*" values are (full) integer externally, regardless of storage length.
  • All "F*" values are double externally, regardless of storage length.
  • An XML file in a directory is always associated with a binary TMST file in that same directory which has the same base name, with ".xml" changed to ".tmst" for the sibling file.

An imported XML with true time state details.

It is likely to be something be like the following.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE US_TimeState>
<US_TimeState version="1.0">
  <file time_count="30321" constant_incr="1" time_increment="1" first_time="0"/>
  <value key="Time" format="F4"/>
  <value key="Omega2t" format="F4"/>
  <value key="RawSpeed" format="I4"/>
  <value key="Temperature" format="F4"/>
  <value key="Vacuum" format="F4"/>
</US_TimeState>

An XML where the import set does not exist.

One would be generated in us_convert to supply raw rotor speed at each scan and would likely be similar to the following.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE US_TimeState>
<US_TimeState version="1.0">
  <file time_count="130" constant_incr="0"/>
  <value key="Time" format="F4"/>
  <value key="RawSpeed" format="I4"/>
  <value key="Scan" format="I2"/>
</US_TimeState>
Last modified 5 years ago Last modified on Mar 8, 2014 1:34:56 PM