Qimera 2.2.0 - Improvements

On this page:

Grid 5 Stability, Scalability & Improvements

This release focused much effort on improving stability and scalability of the newer format version we use for Dynamic Surfaces.  From a user point of view, you will find more stability in version 2.2 since we have addressed some problems that escaped our attention with the Qimera 2.0 release.  Thank you to all users who reported these to us and for giving us the opportunity to investigate further and make the software stronger.  We appreciate your help.

From a stability standpoint, there have been two major improvements.  Firstly, there is much better control over simultaneous reading/writing of the surface, particularly when the grid tiles are streaming from disk while edits are being made.  The work we have done for this results in a more stable user experience as users edit data and update the grid while simultaneously panning and/or zooming to other areas on the surface.  Secondly, we have completely rebuilt the Surface Edit Overview dock.

Surface Edit Overview

In our internal investigations from user feedback, we found that many instabilities came from the use of the Surface Edit Overview window, these were due to competing access to the Dynamic Surface and dual rendering of the same surface.  As you know, this dock gives an overview of a Dynamic Surface while you simultaneously view the same surface in the main Scene.  In response, we have rebuilt the Surface Edit Overview from the ground up to address issues with the previous architecture to avoid the challenges from the previous solution.

In this rebuild, we have also focused in improving load time and responsiveness while in use.  We have tested with a 100 GB Dynamic Surface and we are happy to report that the load time has reduced from 36 seconds down to 6 seconds on our testing hardware.  With the same 100 GB surface, we have also tested that the zoom/pan and redraw responsiveness is as good as it is with smaller data sets.  Since this was a fresh rebuild, we also took the opportunity to ensure that the Overview works and behaves consistently with all mouse modes and keyboard modifiers from the Scene and other plotting tools.  Additionally, we have made improvements to ensure that the Overview inherits the viewed depth layer, color map and color map range from the Scene when the Slice Editor is launched.

Shading Improvements

Many enhancements have been made to the shadows that are produced when operating in Cast Shadows mode.  The look and feel of the shading is now more consistent with differing grid cell sizes and also there is improved consistency between the two different shading modes (Hill Shading vs. Hill Shading + Cast Shadows).  Previously these two modes had differences, with these differences also varying with grid cell size.

We have also adjusted the default shading parameters: Specular was 0.106 but is now 0.1 and Vertical Scale is now 2x, whereas previously it was 5x.  You will need to reset your Shared Preferences to the new default to activate them, otherwise your previous defaults will be honored.  These new defaults are now consistent between Qinsy 9, Qimera 2 and Fledermaus 8.  A Dynamic Surface that is produced online in Qinsy should match ones made in Qimera more closely, along with Fledermaus.

Additionally, shading improvements have been made on exported geotiffs in that they now match what is currently displayed in the Scene for a "what you see is what you get" experience.  Previously, the geotiff exports always had cast shadows, whether the user had them enabled in the Scene or not.  Also, there were slight variations in brightness and general shading with geotiff exports due to different shading algorithms being used in the Scene and in the geotiff exporter.  These have been aligned.

The recalculation of shadows during editing operations in tools, like the Slice Editor, has been enhanced to recalculate the shading in the entire selection area.  In previous releases, it would only locally recompute the shading in the grid cells that were actively regridded, thus a spike with a long shadow would leave a long shadow behind even after the spike was edited.  In this update, the entire selection area is re-shaded when you edit.  Additionally, the tool to recompute surface illumination under the Dynamic Surface menu is aware of user selections on the surface (e.g. rectangular selection).  If any shading errors occur after editing, the user can isolate the area and have it recompute shading for just that area.  Previously, this operation would update the entire surface.  During all of the shading improvements, we have fixed a problem that was highlighted in the Qimera 2.1 release notes as a known limitation: You can now select multiple surfaces and adjust the shading parameters and every selected surface will recompute its shading.

General Improvements

Below is a list of general improvements that have been made regarding Dynamic Surfaces.

1. The automated Update Timer has been removed from the Dynamic Surface.  It would check the modification times of QPDs that are located outside the project, typical of Production Line Processing workflows, and would signal that the surface required updating.  This was removed since it would cause a race condition between ongoing grid updates in the main project and the update timer, which would then cause unnecessarily long grid times.  In place of a timer, the modification times of QPDs are checked at the time a project is reloaded only, or when a user right-clicks a Dynamic Surface and chooses "Update from Edits".  When doing the latter operation, a dialog will warn how much of the surface will be updated, allowing the user to proceed or delay until a later time.
2. The bounds, or extents, of a grid are now displayed as edge-to-edge in the Properties tab.  Previously, they were reported as center-to-center, thus under reporting the true geographic extent of the grid by a half cell size.
3. When creating Dynamic or Static surfaces, the default file name is changed to match what is done in Qinsy: "Dynamic Surface".  Previously, Qimera was not adding a space between the words.
4. The behavior of the user preference for default viewing layer for a Dynamic Surface has been changed to be only applied to a newly created surface.  Previously, this was applying to all surfaces each time a project was reloaded.

Geodetics

Several changes were made to the New Project dialog and the Geodetic configuration dialog based on user feedback on the new interfaces that were launched with Qimera version 2.0.0.  Terminology has been improved in various dialogs, with the primary aim being to enhance clarity and reduce use of acronyms.  Care has been taken to use terminology that is most commonly used and in some areas tool-tips have been added with further explanation. The behavior and layout of various dialogs has been improved for a cleaner and simpler user experience and workflow.  All changes combined give an improved user experience. In addition a number of bug fixes have been made.

Starting with the project creation dialog, the previously used acronym CRS has been dropped in favor of full text.

Moving next to the Geodetic Configuration dialog, there are several minor improvements as well.  In particular, it is much clearer now that the Guided Setup and Advanced Setup options are mutually exclusive workflows: the Detailed Overview tab has been removed in favor of having to specifically click a button to switch to an advanced setup.  Acronyms have been removed and clearer terminology has been used to make it clear what is required from the user.  A help button has been added that will launch a web browser with the manual for the Geodetic User Interface.

The Advanced Configuration has improved as well, largely with a review of terminology to ensure consistency and clarity, and also removing acronyms in favor of full text to help users with limited geodetic knowledge.  Action buttons are at the top left to guide the user to the logical first step to take when coming to this mode.

Lastly, the layout of the Test dialog has been improved as well to make it easier to compare geodetic test points with the current configuration, transform, vertical offsets, etc.  We have used standard geodetic terms like "Ellipsoidal Height", "Geoid Height" and "Orthometric Height", along with the usual symbolic terms for these (h, N and H, respectively).  As with the other dialogs, acronyms have been removed as much as possible.  The vertical layout makes it easy to visually scan and compare between Input and Output systems, the previous layout of this dialog made it difficult to visually compare results between the Input and Output systems.

Filter Speeds & Improvements

We have improved the processing speed of some of the more recently released filters.  From testing of a standard "single day survey" data set:

• Cluster filter - Improvement of ~35%
• Intensity & TPU filters - Improvement of ~10x

Also, the filter selection dialog was getting quite messy so we've renamed some of the filters for consistency and clarity.

We have added a text entry filter field where you can start typing to quickly isolate the filter you want.  The image sequence below shows this in action.

Plotting & Editing Tools

In preparation for the eventual integration of the Fledermaus 8 visualization engine, we have rebuilt the plotting architecture of many of the plotting tools in Qimera.  These are listed below, there are no user facing changes but we wanted to make all users aware that these tools have new architecture.

• Patch test
• Wobble tool
• Time-series Editor
• Multiplot tool
• SVP Editor
• Cross-check tool
• Profile tool
• Coverage/Extinction plot

In future releases, we will move to more complex tools such as the Swath Editor and Slice Editor.  As part of the rebuild of the above tools, we did extensive testing and we have fixed several bugs and made general improvements.  There are improvements made in other tools not included in the list above.  These are all summarized below.

• Slice Editor
  • Improved responsiveness of editing tools, particularly the polygon selection tool
  • Auto-zoom behaviour in 3D Editor improved to adjust to vertical bounds of a new selection
  • New keyboard shortcut that applies the currently active filter from the filter toolbar: Shift + f
• SVP Editor
  • You can now delete multiple row entries at the same time, select the rows and hit the delete key.
  • While editing in the plot, the Control key will now allow you to quickly change from reject to accept, and vice versa.  This is in alignment with how editing works in the Slice Editor and Swath Editor.
• Swath Editor
  • Improved text for water column viewer, it now indicates clearly when a file does not contain WC data, or if a ping is missing WC data.
  • Ctrl-Z keyboard shortcut will now undo selected soundings in the water column viewer, previously this was only possible with the undo button on the toolbar.
  • We are limiting visualization of MBE or SBE or laser and you can no longer view multiple sensors simultaneously (use Slice Editor or 3D Editor for this).  The reason for this is that systems do not transmit synchronously, thus the concept of a ping by ping display breaks down.  Further more, laser scanner scanning patterns are very different from multibeam, thus a Swath Editor is not always a good way to visualize the data.
    
• Multi-plot
  • You can double-click to add/remove items in the configuration dialog where you choose which time-series to plot.
  • Line thickness settings are remembered between sessions.
  • Home button behavior has been improved to correctly reset the view extents to the entire range of the loaded time-series.
  • The entire time-series is drawn now at full resolution.  This will show any spikes.  Previously, the data was decimated to speed up draw times, and the full resolution was only shown when zoomed in.
• Time-series Editor
  • The entire time-series is drawn now at full resolution.  This will show any spikes.  Previously, the data was decimated to speed up draw times, and the full resolution was only shown when zoomed in.
• ENC Editor
  • The TECSOU attribute for newly created S57 objects now defaults to '3' to indicate that it came from a multibeam echosounder.  Previously, it defaulted to '1' to denote that it came from a single beam echosounder.
• Geo Picking dock
  • A button was added to the Geo Pick dock to promote discoverability on what this dock is used for.  Previously, the user needed to know to choose the "Geo Pick" button from the main Scene toolbar.
  • The "Auto Add Points" option has been removed.  All Geo Picks are now added to the Geo Pick table.
• Profiling Tool
  • Line thickness settings are remembered between sessions.
• Patch Test & Wobble Tool
  • Rejected soundings are disabled for viewing by default

Imports & Exports

ASCII export of raw sonar files

The layout of the exporter dialog has been improved to have all user selection elements visible at the top level.  This was done to reduce human error during export since the settings that were previously only visible when launching a secondary dialog were sometimes forgotten by users.  Also, there is a new tickbox that allows users to have the dialog remember their last export settings.  This covers the majority of the settings on the dialog with the exception of the selected systems (which can vary between different raw sonar files), geodetic settings and whether or not to export a merged file or single files.

Imports

Several changes and improvements have been made to various imports capabilities in this release.  These are summarized below.

• GSF
  • We have cleaned up the handling of GSF beam flags so that they are treated in the same way regardless of import as raw sonar file or as processed points.  This was done inconsistently before as these used different historic code paths.
    

    • Filtered flags in the GSF have filtered flags set in the QPD

    • Manual flags in the GSF have manual flags set in the QPD

  • The automatic Direct to QPD conversion has been disabled for GSF import.
    • The Direct to QPD process takes the GSF point solutions and re-references all point solutions to the sonar head, instead of the reference points, since this is the convention in the QPD.
    • This process is prone to incorrect results depending on the presence or absence of lever arm information between the sonar head and the reference point in the GSF, this being primarily a factor of the source of the GSF file itself.
    • Users bringing in GSF for raw sonar processing will be prompted to process the raw observations.
    • For users who wish to just do point editing, the option to import GSF as a Processed Point file provides an alternate workflow.
    • The capability to convert a GSF directly to QPD can still be triggered from manually from the Tools menu.
    • We have done this due to user feedback where the Direct to QPD results were nonsensical, but that would then be reasonable if the user decided to raytrace the solutions.
• LAS
  • Large LAS files that are imported will give users a warning about maximum file sizes supported by the QPD format in Qimera.
  • Previously, Qimera would attempt to convert the .las/.laz files into QPD, resulting in a crash or hanging software.
  • In the future, we will be working on methods to allow for import of larger .las/.laz files.

GSF Export

Several changes and improvements have been made to GSF export capabilities in this release.  These are summarized below.

• We have cleaned up the handling of beam flags to improve workflows going to Teledyne-Caris HIPS.
  • Previously, beams that were rejected with a filter (as opposed to being rejected manually) were lost entirely upon import into HIPS and conversion to HDCS format.
  • This was due to a misunderstanding in how HIPS treats beam flags when coming in from GSF files.
  • We were able to clarify this with the cooperation of Teledyne-Caris and also Leidos, the maintainers of the GSF format.
• Setting a vertical separation model or a fixed vertical offset will now indicate that a GSF has had a vertical corrector applied in the book keeping that is done internally in the GSF.
  • The separation model value and/or fixed vertical offset will appear as a tide corrector when these exported GSFs are imported back into Qimera.
  • Previously, these offsets were not applied correctly for GSFs that were exported from Qimera and then re-imported into another project.
• Extra motion sensor data is written out at the end of the file in order to cover the entire transmit and receive cycle for deep water systems.
  • Since Qimera 2.0, the motion sensor data was capped at +/- 5 seconds from the beginning or ending of the time span associated with the multibeam sensor.
  • In this release, we extend the time cap at the end of the file so that deep water systems have additional motion sensor data.
• GSF export from KMALL files now includes the beam sector ID for each receiver beam.
  • Previously, this was omitted, and this would have affected re-processing capabilities if the GSF was re-imported for raw sonar processing.
  • It would not have affected the footprint positions or depths in the GSF itself.
• GSF export now has an additional option to include "Interpolated Soundings", this is for Kongsberg .all and .kmall files only.
  • It is meant primarily for .all/.kmall users who want to do backscatter processing in FMGT.
  • The GSF format does not have a natural way to highlight Interpolated Soundings as such and thus the FMGT option to include Interpolated Soundings has no effect (this is typically used to increase backscatter coverage at the edges of the swath).
  • This is a compromise mechanism for users who want to do this and it works by just flagging all Interpolated Soundings as Accepted, they are flagged as Rejected by default in Qimera.
  • This option will enable all of the Interpolated Soundings to be accepted during export to GSF, but it preserves the flagging logic in the originating QPD.

Removed Functionality

We have retired some older functionality for various reasons.  We deemed these to be low risk removals, please get in touch with us if this impacts your workflows and we can help to find alternate solutions.

• Export Processed Points
  • This functionality was under the Exports menu for raw sonar files.  Since it provided export formats for very old formats which are no longer in use much, it was decided to remove it.  It has been replaced by the Generic ASCII Exporter.
• Export Fledermaus Scene
  • We have removed the ability to export Fledermaus 7 .scene files.  Fledermaus 7 does not support Version 5 Dynamic Surfaces which was released in Qimera 2.0.  Fledermaus 8 can load the project directly without the need to create a Scene.

Miscellaneous

1. Project creation
   1. Added warnings to avoid users accidentally creating projects within projects.
   2. New setting that allows users to choose to have a fixed default project creation location.
      1. This was previously the behavior with Qimera 1, but it was removed with Qimera 2 when the new common New Project dialog was made in common with Qinsy, Qimera and Fledermaus.
      2. We've brought back the ability to set a default project directory.
      3. For now the default behavior is to change the new project start up location to be the most recent directory in which a project was created.
      4. We may consider changing this in the future based on user feedback.
2. Processing
   1. Correct handling of dual-head Reson S7K data, previously the head tilt angles reversed.  Note that this was JUST with Reson .S7K file format, data logged in Qinsy was not affected by this.  This would have affected Norbit sensors as well.
   2. POSMV TrueHeave files handle the GPS week rollover now in the event that data is logged through the week rollover.
      
   3. Water column processing is now available for new Kongsberg KMALL when logged in Qinsy .db format, previously was only available when logged in Kongsberg .kmall format.
   4. Job Activity Log - Added date to the text entries, was previously just time
3. Scroll Selection
   1. Using Ctrl key and then left click now locks the scroll box to the closest line at the CLICKED position.  Previously it would lock onto the selected line but then would place the box at the very start of the line.
   2. In the upgraded Surface Edit Overview, users can now lock the scroll box to any graphical line in the display, including track lines and imported CAD lines.