FMGT - Supported Data Formats

FMGT is currently designed to process backscatter, beam average or sidescan data from several data formats.  Some file types contain everything that FMGT needs to do its work: georeferenced bathymetry data (the soundings, with appropriate corrections applied), and the backscatter imagery data.  Files that provide all of this can be imported into the project directly.  There are file formats where one or the other is present, in these cases, FMGT will allow you to pair the bathymetry in one file with the backscatter data in another file.  These two categories of supported files and pairings are discussed further below.

Direct Support

 Certain file types provide everything that FMGT needs to process data.

• GSF – GSF files that contain beam averaged or beam time series data can normally be mosaicked in FMGT.  The sensor ID must be correctly configured, which allows the GSF I/O library to read/write the sensor specific sub-records that encode the sonar settings.  GSF supports encoding of snippet time-series as well as beam average relative amplitude and calibrated amplitude.  Note that the GSF format does not offer representation for every make/model of multibeam system.  Well supported systems include Kongsberg, Reson, R2Sonic and Norbit.  For these makes of sonar, GSF provides an adequate storage mechanism for the data required for backscatter processing.  Data from other makes of multibeam systems may be recorded in a GSF file, however, the format will not provide a mechanism to encode the sonar settings and there will be limited capability to make backscatter corrections for these types of systems.  It will still be possible to generate backscatter imagery products in FMGT, however, the results will likely be contaminated with artifacts.

• ALL – The ALL format is the native format that Kongsberg uses to record data from their systems.  It contains the raw measurements as well as the sounding georeferenced locations as computed by Kongsberg's real-time data integration engine.  FMGT supports the vast majority of sonar models that can be represented in the ALL format, including modern and older models. Please see the table at the end of this page for further details.

• KMALL - The KMALL format is the new Kongsberg format for their systems. It contains the raw measurements as well as the sounding georeferenced locations as computed by Kongsberg's real-time data integration engine.  FMGT supports the majority of sonar models that can be represented in the KMALL format. Please see the table at the end of this page for further details.
• XTF – XTF files that contain sidescan data can be mosaicked. There is minimal support for sidescan processing as FMGT's primary focus is multibeam time series or beam average data.
  
  

Bathymetry and Backscatter File Pairing

Users can pair bathymetry and backscatter files using a multi-stage import and conversion wizard.  For these cases, the two data streams are combined into a new GSF file that then follows the standard GSF processing flow.  The bathymetry data are used to fill the new GSF with the bathymetric solutions.   Any data cleaning that has been done to the incoming bathymetry data will be preserved in the new GSF.  The companion backscatter file provides the backscatter data and the sonar settings necessary for processing the backscatter, any bathymetry information in the backscatter files is ignored in favor of the information in the bathymetry file that the user has provided.  In all cases of pairing, the new GSF file that is created is the one that is actually imported into the FMGT project for processing and the raw bathymetry and backscatter files are no longer needed by FMGT.  The GSF files that are created by the pairing process are stored in the FMGT project's Output/GSF folder.

Supported Pairing Combinations

The table below lists the supported bathymetry and backscatter formats and gives an indication of which file types can be paired.  If you see a pairing that is not supported that you'd be interested in having as a capability, please login to jira.qps.nl and create a feature request ticket.

Further discussion on the nuances of the particular bathymetry and backscatter file types is provided in the sections below the table.

1: Dual head pairing supported

Bathymetry Sources

Bathymetry files contain georeferenced sounding footprint locations as well as beam flags to indicate whether a particular beam has been accepted or rejected.  These are the only bits of information that FMGT will use from these files, all other information in these files is ignored as it is not necessary for processing in FMGT.

QPD - For QPS QINSy and/or Qimera users.  This file can only be paired with a raw .db file, this being the raw file format recorded by QPS QINSy. FMGT provides the ability to import db/qpd pairs directly from QINSy if the user is satisfied with the data quality results coming directly from the online computations in QINSy and there is little or no data editing required. QPS clients who also have Qimera can use Qimera to process their .db files and can export a GSF from Qimera to be used directly in FMGT.  Qimera allows for bathymetric post-processing corrections to be adjusted, as well as data editing capabilities to remove poor quality bathymetric solutions.  The preferred QPS workflow for backscatter processing is to acquire in QINSy then to review data in Qimera for data cleaning purposes, and finally to export GSF files from Qimera for use in FMGT.  Alternately, the db/qpd files in the Qimera project can be imported via pairing in FMGT though it is easier to export from Qimera (a single-click versus a multi-stage import pairing wizard in FMGT).

GSF - GSF is an exchange format that is commonly available as an export option from many acquisition and post-processing software packages.  In most cases, it only contains the bathymetry data and it must be combined with the original raw data file to get the backscatter and sensor configuration information necessary for backscatter processing.

HDCS - Caris HIPS HDCS format is a proprietary and closed file format that can be accessed via the HIPS I/O library.  Using HIPS I/O requires a valid Caris license to run.  In FMGT, this is only required during the GSF creation process.  Please note that for the moment, the R2SC/R2S pairings with this format are only supporting overall single head. Dual head is supported for pairings with ALL / KMALL files, S7K/7K, and XTF files containing Reson or R2Sonic.  Once the files have been paired to create a GSF, FMGT does not require HIPS I/O and the Caris license is not required for further processing of files that are converted and imported into the project.

Backscatter Sources

DB - QPS QINSy software receives raw sonar datagrams which are then used to compute the sounding footprint locations.  The results are stored in a QPD file, but the raw observations are stored in the DB file.  The content of the raw observations is driven primarily by the multibeam driver setup and the QINSy operator must ensure that the driver is correctly configured to record the appropriate raw sensor datagrams to support backscatter processing.  This is detailed in the setup of the various QINSy driver manuals for multibeam systems that are supported for processing in FMGT.

ALL & KMALL - Kongsberg .all and .kmall formats contain georeferenced bathymetry and backcatter, however, they do not provide a mechanism to encode user accept/reject flags for soundings.  In some cases, some bathymetric data cleaning is required and the .all / .kmall file must be paired with the output of the bathymetric data cleaning.  During pairing, the bathymetry and cleaning from the bathymetry file completely overrides the values that are stored in the original .all / .kmall file.

S7K/7K - Reson S7K files contain raw sonar measurements in the form of ranges/angles for bathymetry and snippets for backscatter.  The S7K format is also used by Norbit to encode information from their systems.  FMGT supports the raw snippet format and the newer "Normalized Backscatter" format.  When both types of records are found, the "Normalized Backscatter" record is used to populate the GSF file that results from the pairing process.  When recorded in Reson software, such as PDS2000 or 7kCenter, these files can be used directly for pairing with a bathymetry data file.  When recording in Hypack, the raw sonar datagrams are stored in a companion file that sits alongside the HYPACK HSX and RAW files with a file extension of 7K.  The 7K file is a slight variant of the S7K format, during pairing and import, FMGT will convert this file to a standard S7K format.  The user has the option to save this S7K file as well, if they desire.  Note that the contents of the 7K file are controlled by the HYPACK driver settings and users should verify that the appropriate datagrams are being recorded as soon as possible during acquisition.

R2SC/R2S - Though R2Sonic does not officially support a file format, the data format for their systems is openly available and some users choose to record that raw datastream to a file using custom recording software.  These files consist of a direct concatenation of the received binary file packets into a file on disk.  Given that there is no standard, QPS has designated the file extension .R2SC, following along from the HYPACK convention to record R2Sonic data in R2S files.  When recorded directly from the ethernet stream of data from the sonar, these files can be used directly for pairing with a bathymetry data file.  When recording in Hypack, the raw sonar datagrams are stored in a companion file that sits alongside the HSX and RAW files with a file extension of R2S.  This particular file is a slight variant of the R2Sonic raw binary format, during pairing and import, FMGT will convert this to a R2SC file which is consistent with the raw binary data stream being recorded directly to disk.  The user has the option to save this R2SC file as well, if they desire.  Note that the contents of the R2S file are controlled by the HYPACK driver settings and users should verify that the appropriate datagrams are being recorded as soon as possible during acquisition.

GSF - GSF is an interchange format which may contain the snippet data, even though this is not often the case.

XTF - XTF is an interchange format which may contain the raw sonar observations in the native binary representation for a variety of makes and models of sonars.  Currently, the only sonars that are supported via this route are Reson and R2Sonic. For specific XTF/HDCS pairings note that this is only supported for Reson 7028 packets.

Supported Systems for .ALL formats

FMGT supports the .ALL formats from Kongsberg from the following systems:

The following systems are not supported:  EM950, EM1000, EM12, EM100. If you have questions, please contact support@qps.nl

Supported Systems for .KMALL formats

FMGT supports the .KMALL formats from Kongsberg from the following systems (as of the FMGT 7.11.0 release):

Other

As of FMGT 7.11.0, Edgetech 6205 data is supported from DB/QPD pair or GSF.