BHD - Storage Button

Storage

The following provides a brief overview of data Storage.
Data is captured in various formats.
Various options are available for automatic selection and storage.
Not all of these data types are pertinent to dredging operations, e.g. XTF.

Storage

Database (*.db)

File in which the raw system data is stored as well as the settings as defined in the Database Setup program.
Can be replayed to generate new results.

Sounding Grid (*.grd)

Corrected elevation/depth data stored in grid cells.
Level of correction depends on whether you choose to apply: motion, tide, draft, and sound velocity profile.

Option to store attribute data for each cell: mean, maximum, minimum, hit count, 95% confidence level, date, and quality flag.

DTM File (*.qpd etc.)

Corrected elevation/depth data stored as points (X/Y/Z) in various formats including *.qpd, *.pts (various formats).

Each stored point can carry multiple flags based on various criteria:

outside limits for depth, range, sector, intensity, height, polygon; heave above; TPE exceeds; exclude beams; despike; data reduction.

Storage Control

Various automated options including: splitting of data files, line selection, line heading, data recording, line selection and headings of lines.

Log File (*.txt)

Log data to a separate ASCII file.
Only available when a Generic ASCII data logger system was created in the Database Setup program.

Much more detailed descriptions are available here:

In the Controller's Help pages
In the Knowledge Base under Session Setup - Storage.

Database (DB File)

Storage parameters for raw data are displayed on this page.

Raw data will always be stored during recording; it is not possible to switch this off.

Click on Database.

Make selections and data entries for:

Filename
Use Linename	Select whether or not to use the line name as storage name for the raw data. (Option disabled if Use Formatting is enabled.)
Use Prefix number	A prefix number (0001) is added to the storage name. Every time that a recording is stopped, the prefix number increases by one.
Use Formatting	When enabled enter a custom storage file name. The codes as listed in the Formatting pane below are replaced by certain values. (Option disabled if Use Linename is enabled.)
Filename	This line shows the file name as it will be stored. The raw data is stored as a database with the extension .db in the Database folder of the project. Note that a suffix number (0001) will always be added, incrementing by 1 to denote which file in a batch it is.
Formatting
Format specifiers	Use these codes to define the line name. Separate them with spaces and dashes to make the file name easier to read. The format specifiers are only used when the Use Formatting check box is ticked.

Return to top of page.

Return to BHD - Session Settings.

Sounding Grid (GRD)

A sounding grid file must be created before any gridded depths can be stored.
This can be done either in the Processing Manager program, in the Sounding Grid Utility or directly in the Session Setup by clicking on the "New" button.
Any grid file used must be copied to the folder “\Griddata” of the current QINSy project.

This section assumes a sounding grid has been made and any design and/or pre-dredging depth data has been imported.
Please refer to Sounding Grid Preparation and Importing Design Model for further details.

Click on Sounding Grid.

Make selections and enter parameter values as follows:

File
Format	First select a "*.grd - Sounding Grid" format from the pull down menu.
Filename	Press the browse button and select the appropriate grid file in the standard MSWindows selection window.
Overwrite Mode	This parameter is only shown when the Date attribute has been enabled during grid creation for one or more of the layers. By default new depth data is automatically merged with existing data. In dredging operations overwriting of existing data is often what is needed to monitor progress. Age - Newer data will overwrite older data if the age of the older data is greater than the Age time entered. Used when using a Multibeam on a cutter dredger. When swinging in a cut the data is averaged if the data is not older than the threshold age set. This can be useful if you want to redo a survey line and only want to overwrite the sounding grid data of the previous attempt. Time (older than) - Newer data will overwrite older data if the time tag of the older data is before the time and date entered.
New	Refer to Sounding Grid Preparation. In our example a new grid is created with layers for: Dredge Bathymetry - initially populated with the InSurvey DTM and then updated during backhoe operations. Dredge Head Height - used when the bucket is being employed as an 'echsounder' to check dredged depths. InSurvey - populated with the InSurvey DTM and used for comparison with the Dredge Bathymetry layer. Design - populated with a design model DTM and used for comparisons with other layers. Post Survey - DTM from final survey and used for comparisons with other layers, e.g. for volume calculations.
Clear Layers	If necessary erase all data from one or more selected layer(s) in the currently selected sounding grid file.
Import	If necessary import data (points file or another grid file) into alayer the currently selected sounding grid file. Refer to Sounding Grid Preparation for details.

Select a storage layer for each system. This can be a different layer or the same layer.

Return to top of page.

Return to BHD - Session Settings.

DTM File

DTM data is motion and heading corrected geo-referenced footprint data, i.e. deepest bucket penetration.
File contents reflect the amount of dredging done.
The DTM data can be stored in several formats, ready to import into external processing software (Validator, Qimera, Fledermaus), or other third party programs (Terramodel, CARIS, Cyclone, etc.) for volume computations.

Depending on the situation, the active Sounding Grid layer may be sufficient for volume calculations, without resort to the DTM surface created with these files.

Click on DTM File.

Make selections and enter parameter values as follows:

File
Format	Select the format to use for storage of DTM data. Refer to Storage - DTM File for format descriptions. The proprietary .qpd - QINSy Processing is the recommended format because it can be imported into the Validator, Qimera or Fledermaus for further processing, analyzing and/or publishing. Because QPD files cannot be appended to, this format is not available when replaying recorded databases in batch in combination with the file Mode set to User-defined name*.
Mode	The mode determines the name of the storage file: Use Database storage name - By Default and Recommended. The DTM file will have the same storage name as the raw data file. This is highly recommended, since some processing tools require a DTM file to have the same name as the corresponding database. User-defined name - User is able to enter a different file name. This is not recommended.
Filename	A file name preview.
FlaggedFP (footprint)	If you chose Format '.qpd - QINSy Processing' this extra line appears. Select one of three options: All footprints are stored, flagged footprints are inactive (Recommended) -* The most used option. Data that did not pass the online filters has been flagged, but not deleted. So during processing the flagged data can be easily restored if needed. All footprints are stored, all footprints are active - Normal, full QPD. Flagged footprints are not stored - Reduced QPD. The file is the same QPD format as both options above, but does not contain footprints that have been flagged by the online filters. The main benefit will be the smaller file size of the .qpd files. However, if you need to restore flagged data you will need to completely replay the database and create a new .qpd file with different online filter settings. Because QPD files cannot be appended to, this format is not available when replaying recorded databases in batch in combination with the file Mode set to User-defined name.

Make selections as follows:

Systems
Use System	Activate the Dredging system(s) to use in the generation of the DTM file.
Layer name	Not applicable.

Return to top of page.

Return to BHD - Session Settings.

Control

Use these options to keep control of the recording and storage of data.

Backhoe dredging is more likely to be controlled by the Eventing System which can be configured to start, pause and stop recording.
Still it may be necessary/useful to set up some up some of the controls here, especially storage database splitting options.

Click on Control

Database files can grow very large when collecting large amounts of data (e.g. multibeam or laser).

It is a good idea to split these files to a size that can be more easily handled by the processing software.
When the limit is reached during recording a new database is automatically opened and logging switches to the new database.
The first database is then closed, ensuring no loss of data.
The first database has "- 0001.db" in the name, the second "- 0002.db", etc.

Make selections and enter parameter values as follows:

Note: For Manual splitting press F11 during the recording session. Updating continuously, DB and DTM file sizes are displayed in the Controller.

Automate selection of the next line using these options:

Automatic Line Selection

Selection Method

Disabled - Lines are not selected automatically.
Planning - Line selection adheres to the order in which they were organized in Line Planning.
Dynamic - The program determines whichever line is closest to the vessel, selecting that line.

Line Bearing

Planning - Uses the heading of lines or sections of routes as they were defined in the Line Database file (from the first co-ordinate to the second, etc.).
Planned heading can be reversed if needed using the right mouse menu.
During the survey the line bearing can be reversed (with recording off) using function key (F7).
Heading based- The heading of the line or route is automatically determined using the heading of the object.
Closest distance to SOL or EOL- The heading of the line or route is automatically determined from the steered node location:
A) The planned heading will be used when the steered node is before the SOL.
B) The planned heading will be reversed when the steered node is after the EOL.
C) The planned heading will be unchanged when the steered node is in the line (between SOL and EOL).

Automatically start and stop recording based on these options.

Automatic Recording Options
Auto-Recording Method	Disabled - When Disabled use the recording button in the Controller dialog. Fix based - Start recording based upon the current fix number. The start and end of line fix numbers for every line are defined on the Fixing page, and it is possible to set a fix number here so that recording starts early, and stops late. Distance based - Start recording based upon the distance to / from the start and end of the selected line. MP based- Start recording based upon the distances along the main line. MP (meter point, a value in meters) values are normally used with Lines, KP (kilometer point, a value in kilometers) values are often used with Routes (pipelines). This entry requires MP values, e.g. for a segment of a route/pipeline between KP 1.900 and 2.200, enter 1900 and 2200. A reversed route will not affect the auto-recording, but only the offtrack distance value. Time based - Enter the date and time at which recording should take place.
Offtrack Threshold	If the recording vessel strays more than the distance entered here, recording is paused until the vessel re-enters the Offtrack corridor. Option appears after enabling the 'Start at' line.
Wait before next line	To accommodate the helmsman's steering time enter how long the software should wait before selecting the next line. Option appears after enabling the 'Stop at' line.
Start at Distance to SOL	Enter how far before the Start of Line recording should take place. Leave the value at zero when you want recording to start at the beginning of the line. Enter a positive value to start recording before the start of line. Enter a negative value to start recording after the start of line.
Stop at Distance from EOL	Enter how far after the End of Line recording should take place. Leave the value at zero when you want recording to stop at the end of the line. Enter a positive value to stop after the end of line. Enter a negative value to stop before the end of line.
Start/Stop at MP	Enter how far along the main line recording should start and stop.
Start/Stop Date	Enter a date manually or select one from the drop down menu.
Start/Stop Time (Local)	Enter a time manually or change it with the up/down arrows in the drop down menu.

In the Alert Display storage can be automatically paused when an alert arises.
For example, storage is paused if the positioning system quality is outside a specified limit, or if data from a particular sensor fails.
Once the alert is OK again, storage / recording can be set to automatically resume.

Create a back-up of the raw data file or move the original file to another location by selecting from these options:.

Return to top of page.

Return to BHD - Session Settings.

Log File

Storing to Log File is only available when a Generic ASCII Data Logger system was created in the template database (Database Setup program).
To set up this output system please refer to https://qpssoftware.atlassian.net/wiki/spaces/qinsy/pages/625378486 and Generic Layout Editor and Generic Layout Editor Recorded Webinar.

Log Files can only be written during Online data acquisition and not when Replaying *.DB files.

The file can contain, amongst numerous other items, settings of all used systems and nodes, raw or corrected observations, and all results data.
These items are stored in a XML file located in the \Project Folder\Settings\LogFile folder.
A complete definition of every item is provided in Generic Layout Editor.
All selected items are arranged in any user-defined format desired.

Click on Log File.

Enable one or more layouts with which to write the Log Files.

The XML files listed are those made using the Generic Layout Editor to define the data content and format, and resident in the \Project Folder\Settings\LogFile folder.

Make selections and enter parameter values for:

LogFiles Properties
Folder	Location of the file highlighted in the top pane.
Log File	Use the browse to open User Defined Format Specifiers dialog in which the log file name is defined. In the example #S translates to 'Storage Database Filename', i.e. the same name as the *.DB file name. When the same logfile name is selected for multiple layouts, all data are stored in one and the same file. The properties are already set in the Generic Layout Editor program but can be changed here as well.
When	Always - Log continuously. Recording - Only log settings when recording survey data. Started/Stopped by Eventing System - when an Eventing System is employed, specific events can be used to start and stop log file recording.
Transfer Method	None - none of the log files will be copied and/or moved to another folder. Create Copy After Logging - a Transfer Folder line is displayed. Use the browse button to find the folder to which the copied file is saved. Move After Logging - a Transfer Folder line is displayed. Use the browse button to find the folder to which the original file is moved.
Trigger	Data logging is based on one of these trigger methods: Time based - Every time there is an update in data, this will be logged. Fix based - Logging takes place with every fix. System based - Logging takes place when an update from the selected system is received. Event based - Logging takes place when a certain event occurs.
System	When data logging is 'System Based', an additional selection line appears in which the triggering system is chosen from a list of all the systems defined in the template database.
Update Rate	Choose how often data are logged to file:
Events	When data logging is ' Event based' choose which events to use as logging triggers.

Return to top of page.

Return to BHD - Session Settings.