XTF Export Abstract
On this page:
Abstract
The XTF export module can be used to created XTF files from recorded Qinsy databases. The XTF file format (Extended Triton Format), as founded by website of Triton Imaging, contains binary records for Position, Attitude, Sidescan Sonar, Single Beam Echosounder, Multi-Transducer and Multibeam Echosounder data. XTF Records can roughly be divided into two categories: original raw data packets and native data packets.
The original raw packets are packets that are typically send "over the wire" by a sonar system to the acquisition software and are dumped into the database as is. The design and format description of the packet is the responsibility of the sonar manufacturer and are typically not described in the XTF format description, only the ID of the packet is described there.
The native packets are typically not sent out by a sonar or any other hardware but defined by Triton (the designer of XTF) or 3rd party vendors (e.g. QPS) for the storage of generic data like positions, attitudes, sidescan or even echosounder records. The definitions of these native packets are contained in the XTF format description.
By default the original raw packets are not stored by Qinsy, this needs to be activated for each driver in DbSetup. The native XTF packets can be exported directly from the data packets inside the Qinsy db, for those it is not necessary to activate the storage.
How to Start XTFExport
The export of XTF data can only be carried out on Qinsy databases that contain recorded data. The export is started from Raw Data Manager application. This will show you a list of recorded databases with raw data. Select one or more databases, making sure that all the databases originate from the same template or else the export may give unexpected results. Next select the Export option, this launches a dialog with export options, then select the option "Raw Data - XTF". Press OK, this will launch the XTF Export application. Note that when you selected a very large number of database the U/I of XTFExport is a bit slow to react on start-up because it is scanning the databases.
Wizard Setup pages description
The text below will guide you through the XTF Export wizard pages:
Source data
On this page an overview is shown of the selected databases and of the time span that the data needs to be exported. The data can be exported completely or only a part of the database file based on a start and end time. The base date, which is used for the internal XTF relative time stamping mechanism, needs to be set manually and needs to be earlier (but not more than +/- 25 days earlier) as the first recorded observation in the database.
XTF Files
In case multiple databases are selected to export, you can choose whether the data is to be stored in one single XTF file or as a separate XTF file for each database. In both options the XTF files can grow very large so a limit can be set to the XTF file size. The data will be automatically split into smaller files. If this option is selected then you can set on overlap period (in seconds) between the split files to ensure all data is stored and no data is lost. The records within the overlap period are exported to the previous XTF file as well as to the next XTF file.
System Selection - Multibeam Echosounder
Select for which multibeam system(s) the data records should be exported to the XTF format.You can only select the systems here that are defined in the selected database. Also select to which XTF record type the system data should be exported: QPS Multibeam Record type 28 or Original Raw Record. The first can always be selected because these records are generated from the generic Qinsy database record. But the second option can only be selected when the original raw records are contained in the database and this is only the case when the correct driver setup was used while online. If no raw data is encountered in the database(s) then the original raw option will be "greyed out ".
The option "Include Intensity beam time series (Original raw record)" will be enabled when this data type (e.g. Reson Snippets or Kongsberg Seabed Imagery) was recorded in the Database. Select this option when the intensity beam time series raw record should be exported to XTF. This option is "greyed out" when no such data is encountered in the database(s).
The "Include Secondary system" option should be checked when a second system needs to be exported, typically this will be the second head of a dual head multibeam system but you are free to choose any other system. The options work the same as for the first echosounder.
System Selection - Sidescan Sonar and Echosounders
Select which Sidescan Sonar system / Single Beam Echosounder / Multi-transducer Echosounder data should be exported to the XTF file.
Sidescan
For the sidescan system the XTF record type can be selected, either XTF Sidescan record or Original Raw Record. The first can always be selected because these records are generated from the generic Qinsy database sidescan record. But the second option can only be selected when the original raw records are contained in the database and this is only the case when the correct driver driver setup was used while online. Note that for pre-April 2013 Qinsy versions the raw sidescan data was typically used to store Intensity Beam Time Series records (e.g. Reson Snippets). If those databases are encountered and the snippet records need to be exported then select the original raw record option.
Singlebeam
Select one or more single beam systems.
Single Beam Echosounder data is exported from the Qinsy data records directly and can hence always be exported. The records in the XTF file that the data is written to are of type QPS Single Beam Data (type 26).
Multi-transducer
Select one or more Multi-transducer Echosounders to export. This data is exported fro the Qinsy data records directly and can hence always be exported . The records in the XTF file that the data is written to are of type QPS Multitransducer Data (type 27).
System Selection - Sensors
Every bathymetry or sidescan record in the XTF file is preceded by an extensive ping header record of 256 bytes with meta data. This ping header contains not only the size and Id of the packet but also some meta data valid for the time in the record. This meta data includes an atittude and position but also fields for ROV depth, altitude, layback, speed, sound speed etc. In order to fill those fields you must specify from which system the data should be taken. Note that many of these fields in the header are optional and as such are filled with zero's by XTFExport. If no system is selected then the fields will be filled with zero's too.
Be careful with the Sound Velocity option, this will replace the sound speed in the ping header which by default will originate from the sonar and as such represents the value used for the beam forming. So this option is more useful for sidescan than for multibeam echosounder records.
Attitude (and position) is stored in two different ways in XTF, firstly interpolated as part of the meta data header that precedes a sidescan or echosounder ping packet, secondly as individual XTF Attitude records.
The export of the individual attitude records can be controlled with the option "Include attitude data records". When this option is enabled then the selected Pitch Roll Heave Sensor and Gyro system data are exported to individual XTF Attitude records (type 3). If this option is disabled then the individual records are not exported but the system data will still be used to fill the ping headers. If you wish also to disable this then make sure to select <None> for the Gyro and Motion system, this will fill the appropriate XTF ping header fields with zero's.
In case the "Include attitude data records" option is enabled then every Pitch Roll Heave system update as found in the Qinsy database will be exported ot the XTF. The gyro value will be interpolated to the observation time of the attitude.
System Selection - Positioning
You must always select a proper positioning, either Raw or Result. The positioning records can be derived from the position data records in the database or from a node result (calculated) position. In case the results data (calculated position) is exported, the user should select a computation and node and on which datum this position should be exported. Note that the result data file must exist if you wish to use results. When the result user interface is "greyed out" it means that no result file is found, if required then replaying the database will re-create the result file.
Positions are stored in two different ways in XTF, first interpolated as part of the meta data header that precedes a sidescan or echosounder ping packet, secondly as individual XTF Position records (type 107 default or optional type 100).
The export of the individual position records can be controlled with the option "Include positioning data records". When this option is enabled then the selected Position system data are exported to individual XTF position records.
Older version of XTFExport (pre-April 2013) always exported to the somewhat obscure Reson Position packet (type 100) but since this has been taken out of the XTF format description it was decided to change this to the better known Position Raw Navigation packet (type 107). So by default the newer XTF Export versions will export to this new type 107. If you still would like to export to the older Reson type a registry key can be activated in order to force this.
Additional Parameters
These options can not be derived from the Qinsy database and need to entered by the user.
See also the XTF format description on the website of Triton Imaging
- The XTF sonar Type is part of the XTF file Header and can optionally be filled out to inform 3rd party software what data is contained in the created XTF file.
- The manual Sonar Attitude is filled in the SensorPrimaryAltitude field in the ping header (legacy).
- The TypeOfChannel is stored in the CHANINFO structure that is part of the XTF file Header. In most cases the default will do fine.
- [Sidescan Only] Maximum number of samples per channel can be used if the 3rd party software is not capable of receiving a varying number of samples per ping and per channel. XtfExport will (smartly) reduce the number of samples to the figure set here.
Select zero to disable the sample reduction.
Additionally some extra options are given.
- Set zero latency in Reson Seabat 81XX original raw packets
Some 3rd part vendors require the latency fields in the raw Reson packets of an 81XX system (e.g. 8101, 8125) to be set at zero because otherwise the wrong ping time is calculated.
Next press the Finish button to execute the export. The progress dialog will show an overview of the activity of the export process.
After the progress is complete you can optionally view the last created XTF file with the Qinsy XTF Viewer after closing the export application.
The settings, system settings etc., are stored in the registry when the application is closed and will be reloaded next time the XTF Export is started.