Qimera Metadata Extraction Process
Qimera - Metadata Extraction Process
When data is loaded into Qimera, a metadata extraction process runs to intelligently strip information out of the file to optimize the Dynamic Workflow engine of Qimera, thus reducing the amount of work you have to do to process your data. The metadata extraction has some variability based on the file type and the basics are listed below.
When a file is loaded into Qimera, it is given a unique identifier and the filename and identifier are used to create a specific folder inside the project ~/Metadata folder. For example, if the first file loaded is named 0000_20140321_034136_ICEBEAM.all, the associated folder will be named 0000_20140321_034136_ICEBEAM_00001. If the next line is named 0001_20140321_035137_ICEBEAM.all, its folder will be named 0001_20140321_035137_ICEBEAM_00002 and so on. WARNING: The contents of these folders are not meant to be edited by the user and could have unintended consequences! So please don't move, delete, rename or otherwise edit any of these files.
Raw Sonar Files
The following are the tasks performed on raw sonar files during the metadata extraction process.
File Indexing
All raw sonar files supported by Qimera are packetized in nature and proceed linearly in time. Qimera creates an index file that allows for fast non-linear access to every distinct packet type inside the file. Qimera's optimized I/O engine maximizes the read performance of raw sonar files on all platforms (Windows, Mac, Linux).
Coordinate System Extraction
When loading QINSy (.db) files, the intended project coordinate system will be extracted and if the file is the first in the project and Auto was selected during project creation, Qimera will set the project coordinate system to this coordinate system. Qimera will also select the input navigation datum information to construct the coordinate system of the incoming navigation. For other file formats, if the Auto option was selected, Qimera will choose the nearest UTM zone as the project coordinate system upon import of the first raw sonar file into the project. This data is persisted in the project.xml file.
Vessel Information Extraction
Vessel file information such as sensor, linear and angular offsets is extracted and stored as .vessel files in the line metadata folder. If the vessel name is new, it will be added to the overall vessel list in the Qimera project.
Computation Settings Extraction
Certain raw sonar file types may have redundant time series data streams. The computation settings stores the choices made by the acquisition system as to which of these are the primary, secondary, etc. sources to use during processing. These file types are stored as.echo_##files in the line metadata folder. The "##" in the file name is the system id.
Bathymetry Start/End Time Extraction
Qimera stores the bathymetry's first and last packet times in the project.xml file.
Sound Velocity Profile Extraction
Qimera extracts and stores all sound velocity profiles in a generic binary format with the .bsvp_## extension in the line metadata folder. The "##" is the SVP number in the file.
Time Series Data Extraction
Qimera extracts the following time series data to the listed file extensions when available. This data is extracted and stored at full resolution in the line metadata folder. The "##" in each file name is the system id.
- Position (.position_##)
- Position Standard Deviation (.positionsd_##)
- Motion (.motion_##)
- Motion Standard Deviation (.motionsd_##)
- Heading (.heading_##)
- Heading Standard Deviation (.headingsd_##)
- Height (.height_##)
- Height Standard Deviation (.heightsd_##)
- Surface Sound Speed (.soundspeed_##)
- Pressure (.pressure_##)
- Depth (.depth_##)
- Tide (.tide_##)
- Dynamic Squat (.squat_##)
Processing Settings Extraction
This file will maintain all the processing settings that the user selects when for the file using the Processing Settings dialog. It is created during metadata extraction and intelligent options are made based on the metadata extracted from the file.
Time Series Solution Creation
Since Qimera supports multiply redundant sources of data for the same time series attribute, the solution files (timeseriesname.sol) maintains the priority order of the particular attributes.
QPD Matching
A description of this process can be found here.
WCD Matching
A description of this process can be found here.
Track Line Creation
If Qimera was able to match a QPD file to the raw sonar file (.db), it will create a track line based on the port-side sonar transducer location. If a QPD does not exist, the track line is created from the primary position data time series. The track line object is in the Fledermaus SD (.sd) format.
Binary Bathymetry Conversion
When Qimera loads raw sonar data in ASCII format such as that found in Hypack (.hsx) files, it will automatically convert the bathymetry data to an internal binary format for faster multi-core processing during sound velocity correction.
Processed Point Files
Processed Point Files extract a .metadata.xml file which contains information required by Qimera to work with the file. Qimera will also create a track line or extents object in the Fledermaus SD (.sd) format.
Navigation Files
Navigation files follow a similar process as raw sonar files. The following steps are performed and described above.
- File Indexing
- Time Series Data Extraction
- Track Line Creation
Tide Files
Tide files have the Time Series Data Extraction step described above performed on their data.
SVP Files
SVP Files are imported and automatically converted to the Qimera .bsvp format.
Return to: Dynamic Workflow
Return to: Qimera Table of Contents