Qimera 1.5.0 Release

Highlights on this page

Thanks for your feedback

Introduction

4 April 2017, QPS is pleased to release Qimera 1.5, the fifth full feature release since its initial launch 22 months ago.  As always, our primary mission with Qimera is to make hydrographic data processing intuitive and as simple as possible, all the while offering powerful capabilities to those that need them without cluttering the workflow for those that don't.

For new capability, we look this time to provide some workflows that allow multiple users to contribute to an overall processing effort.  With Qimera 1.5, we introduce two new ways to share work among many people.

• Cooperative Cleaning: Allow multiple users to collaboratively clean large data projects while maintaining data integrity.  This is done by splitting up a master project into several smaller sub projects.  The master project maintains full data processing capability while cleaning sub projects only allow data editing.  Data cleaning staff work in their sub projects and their cleaning efforts are checked back in to the main project once complete.  In the meantime, the main project may have had reprocessing done, like SVP or SBET application, and the cleaning from the cleaning projects is copied right in gracefully without affecting reprocessing efforts done in the main project.
• Production Line Processing: Allow staged processing to break apart processing and cleaning tasks to spread the work among individuals much in the same way that an assembly line works in a factory.  This is perfect for keeping up with daily “pulses” of data during a survey.  The mathematical processing for a day’s worth of data is handled in a processing project by the surveyor, this can even be configured with Qimera Live to streamline operations even more.  The processed QPD sounding output from the surveyor's project is then aggregated into a master delivery project where results can be compared against previous days and further cleaning can be done.  This is done repeatedly, with a day of work being mathematically processed in a processing project and the output being added to a steadily growing main project.  A link is maintained between the processing projects and the master project such that the processing project can be re-opened for further corrections if problems are identified.  When it’s all done, the combined output in the master delivery project is encapsulated for delivery.

Also with this release, we make yet another step forward on our roadmap to realign and adjust the software products from the QINSy and Fledermaus families.

• We follow up on feedback from our Qloud users, who have had a nearly complete functional replacement since Qimera 1.3.  We've hosted several workshops with many clients and have focused on some of the common bits of feedback that we've heard across the board.  In this release, Qloud users will find a more responsive Slice Editor, and they'll find polygon creation, preservation, import and export.  These polygons can then be used for spatial filtering within the application and you can also import CAD polygons and use them the same way.  Also for Qloud users, we have a new Point Cleaning work area preset that greatly simplifies the display and only shows the widgets that are relevant for point cleaning.  Lastly, you can now export a Dynamic Surface to a new layer in a Sounding Grid file, preserving workflows that once existed between Qloud and other QPS applications like the Sounding Grid Utility.
• We are building up yet more support for the sensor modalities that are supported in QINSy, this time you'll find Laser support.  In previous releases, Qimera only allowed for point editing of laser data, now users can do full sensor processing of QINSy laser data.  You can re-process for sensor offsets and alignment angles and you can apply SBET, tide, etc.  Calibration and wobble analysis will come in a later release.
  
• We have migrated the Geoid Models that were previously bundled with Qimera into a separate installer.  The separate installer is also used by QINSy, thus the QINSy/Qimera experience is improved in that you only need to download, install and manage a single set of geodetic resources.  As a result, the Qimera installer is now much smaller, resulting in a quicker download which is important for those working offshore.

Read on for details below and, as always, please get in touch with us via the online support system if you have any questions about how the new features may fit into your existing workflows.  There are several features to highlight in this release, as you'll see below.  Many of the other new features are driven by client feedback and feature requests and for these we thank our clients for getting in touch with us with their great ideas!

What's New

New Capabilities & Workflows

Multiple User Workflow Enhancements

Cooperative Cleaning

This method allows the senior data processor or Surveyor-in-charge to maintain full control of the sensor and geodetic information while farming out the data cleaning to junior and/or multiple data processors. This is done by creating sub projects from the main project, these sub projects have their own Dynamic Surface and QPDs to work on so there is no resource conflict between the main project and the sub projects.  Once cleaning is done in the sub projects, Qimera then seamlessly re-integrates the data back to the master project for final quality control and product creation by the lead surveyor.  The graphic below explains the basic workflow concepts.

Benefits of this are:

• Multiple users can be tasked with working on the main bottleneck of multibeam sonar processing which is data cleaning, thereby rapidly reducing the time it takes to clean and validate multibeam survey data.
• All creation and re-integration of sub-projects is controlled by Qimera. There is no manual copy and paste operations, thus minimizing the effects of human error.
• Allows focused cleaning of large area surveys by multiple users while maintaining control by the senior data processor.
• Sub projects only require a Qimera Clean license and therefore will reduce the total cost of ownership of the software for a data processing team.
• Data cleaning can be performed by less experienced personnel again reducing project costs.
• Sub projects contain QPD files only and are thus limited to cleaning only.  This reduces the risk of junior personnel accidentally applying correctors or making configuration changes.
• Sub projects can be moved off the network to a local machine to overcome slow network bandwidth, while maintaining the integrity of the data by keeping the master copy.

The image below shows the main project being cut into a series of nine sub-projects with the new "Create Cleaning Projects" dialog.  The user specifies the cleaning project base name, location for output and the number of rows and tiles to cut the project into.  Alternately, the main project surveyor can also designate a particular area for a cleaning project using the rectangular selection tool and the dialog to create a cleaning project will use this area instead.

The image below shows the data from the point of view of the person cleaning one of the sub projects.  They see the QPDs from the main project that overlap their work area and they see the outline of the entire project as well as the outline of their sub area.  Since it is only QPDs that are brought into a cleaning project, the only actions that are permissible are data cleaning.  Compartmentalizing the data into cleaning projects limits the risk of accidental re-processing from inexperienced or curious data processors who like to explore what all the buttons do.

Once the data cleaner's work is done, the main project is opened up by the lead surveyor and they can import the cleaning from the cleaning project.  Only the editing that was done inside the cleaning area limit rectangle is brought back into the main project, further protecting the main project from overzealous cleaning that may have been done beyond the data cleaner's work area.

Production Line Processing

This method is used where full processing capability is required for multiple users but the final result needs to be checked and final products created from it. Production line cleaning merges multiple Qimera projects into one single project where data cleaning and QC can be performed, this is done by importing the QPD results from the processing projects into a new and final project where the results are staged for delivery.  The graphic below explains the basic workflow concepts.

Benefits of this are:

• For a Hydrographic office, a hydrographic survey can be portioned up between the bathymetry appraisal team for full validation by each person, but then pulled together at the end for final QC and then chart production requirements.
• For Pipeline and cable surveys where the data can be divided into an alignment sheet per data processor. All of the separate projects can be brought together into one for the senior data processor to quality control the data and create the final product.
• Processing can be divided offshore into one processor per day.  The mathematical processing is done in the daily project and is limited to the data and ancillary information to deal with the single day only.  This reduces the complexity of managing several tens or hundreds of additional ancillary files that may accrue over the span of a project.  The daily processing project only needs the ancillary data for that day, making this experience much less prone to error.
• All merging operations are performed by the software NOT by humans, therefore eliminating error.

The series of images below show two separate sets of data being processed to get from raw sensor data to georeferenced soundings.  The third image shows the QPD results from the first two small projects being merged into an assembly project, with the Slice Editor being used to verify the line up between the two different data sets.  It's important to note that the QPDs are shared between the processing project and the main project and it is up to you to ensure that you do not attempt to write to QPDs at the same time.  The workflows for which this is meant to be used largely protect against this since data should not be submitted for inclusion in the master project until the mathematical processing is done, i.e the mathematical processing project is done and is closed.  The main processor then pulls the data into the main project with the understanding that the mathematical processing project will not be re-opened unless they surveyor is directed to do so, which should only be necessary if problems occur.  A little bit of human coordination goes a long way in orchestrating this and avoids complex resource locking schemes in other software packages which sometimes lock users out of their data.  Humans can be smart and can coordinate, we've made just enough functionality to let you coordinate and work together.

Snapshot To Scene

Do a quick snapshot of soundings in any sounding editor to quickly capture point clouds into the Scene to stage for further analysis or simply for marking up areas of interest.  A snapshot can be done from the Swath Editor, the Slice Editor and the 3D Editor.  This is a great way to mark up areas of interest that need further investigation but you can also use this to easily isolate points of interest for 4D Scene building for Fledermaus visualization, analysis or movie making.  The points are saved in Fledermaus .sd format for direct use in Qimera or Fledermaus, but they can also be exported as ASCII XYZ for quick load into other software applications.  The sequence of images below shows the Slice Editor being used to isolate the soundings collected over a wreck.  

By switching to the Slice Editor's "Select and Edit" mode of operation, you can lasso select the exact points you want to extract, see below.  Clicking the camera icon in the Slice Editor gives a snapshot of the selected soundings in the Scene by adding a Fledermaus .sd file into the project.  

The points in the snapshot .sd maintain the full metadata of the soundings, including TPU and intensity; these can then be used to color the .sd points or to control their size.

Laser from QINSy .db

Qimera 1.5 now supports full sensor processing of laser data from QINSy .db files.  You can import .db files with laser, you can process them to compute footprints, you can adjust the sensor location and alignment angles in the vessel configuration, you can apply post-processed navigation like SBET, etc.  We will examine augmenting the Patch Test tool for laser in a later release, along with the Wobble Analysis tool.

Laser data can be added to Dynamic Surfaces and the Slice Editor and 3D Editor can be used to validate points and remove outliers.  The image below shows the Thames River Bridge with the point cloud being drawn in the 3D Editor.  Combining this with the Snapshot to Scene capability lets you quickly grab sections of laser data for engineering analysis.

Qimera Live Improvements

Qimera Live ingests new files and automatically processes them and adds them to your grid.  At the end of the day, close the lid on your laptop and go home with a project that is ready to go.  This is built-in to Qimera and is available at no extra cost.  Previous versions of Qimera Live required the input of at least one raw sonar file and the creation of an initial Dynamic Surface.  With the 1.5 version, you can now start with an empty project and get right to work onboard with live data import without having to import a file or create a Dynamic Surface.  With this new capability, you can start with an empty project and have it hook directly up to the data directory.  Qimera Live now provides a simple one-click start up that lets you process your data on the ship in near real-time.  Not only is it one-click, there is no process to design or configure, it just works.  For incoming data files that do not support vessel configuration data (e.g. S7K or JSF), the capability delivered in Qimera 1.4 lets you specify a vessel configuration to use on startup.

Polygon draw, save and re-use for spatially bounded filtering & clipping

Many Qloud users missed the ability to draw polygons and to be able to preserve them for use within the same project, or export them to use in other projects.  These polygons were often used for spatial filtering, often times being used to clip the outer limit of a survey area to a specified boundary.  Now Qimera will let you create and preserve polygons using the spatial selection tools that are already in Qimera (the rectangle, polygon and lasso area selection tools).  Under the Layer menu, users will now find a new entry that lets you promote your spatial selections into line objects: "Create Line Object from Current Selection".  The promoted line objects will appear in the Sd Objects node of the Layers widget where you access your maps.

A polygon line object can also be created by importing a CAD file.  These line objects, along with those created from a user selection, can now be used to create spatial selections for use in filtering or clipping operations.  Right-click on any Sd line object and choose "Use Line as Selection".  This will switch you to the lasso mouse mode and will create a lasso selection for you based on the points in your line object.  This spatial selection can now be used for any of the usual operations in Qimera, like limiting the area for applying a filter or launching the Slice Editor for a particular location.  The Sd line objects that are created can be used from one Qimera project to another since the .sd file format is supported natively in Qimera and also in Fledermaus.  You can also export these line objects as CAD files or as XYZ points.

Coverage Extinction Plot

There is a new analysis plotting tool that lets you assess the swath coverage capabilities of your multibeam system.  Data to fuel this type of plot usually consists of a set of survey lines running from the shallowest to deepest depth range of the system.  As the depth increases, the outermost beams have a harder time picking up a detection and the coverage decreases until finally reaching a point where even the nadir beams can no longer detect the bottom.  Having this type of capability is very handy during Sea Acceptance Trials since the surveyor can assess the system's swath coverage performance relative to the manufacturer's claimed specifications to ensure compliance.  You don't want to take delivery of a system that isn't performing to specification, and this tool will help you assess compliance.  It also helps raise attention to noise limitation issues due to mechanical, acoustic or electrical sources of interference.  For example, repeating these types of field trials before and after visits to ship yards can also catch problems early on since often times there are mechanical and electrical changes made during dry dock periods, and sometimes even modifications to the hull which can create flow noise problems.  Many examples of this type of plot being used to assess and diagnose overall multibeam system health can be seen in the technical reports produced by the Multibeam Advisory Committee (MAC) on their website, http://mac.unols.org/reports.

 The image below repeats the analysis of a Kongsberg EM710 data set done by the MAC during a sea trial aboard R/V Falkor (look for the 2014 (Spring) report on this at the link above, specifically page 18).  The graph indicates that the EM710 achieved maximum swath width at a depth of about 500-600 m, which is within expectation.  Full extinction occurred near 2,000 m, which was also expected.  As with all Qimera plotting widgets, the graph can be saved as a report quality PNG file with full markup of axes, etc.

Feature Set List

We've made a number of other features, tweaks and improvements.   These are listed below, along with all the features explored above, with a bit of explanation.

• New Tools and Capabilities
  • Cooperative Cleaning.  You can split up large processing projects into smaller cleaning projects to allow multiple users to contribute in data validation efforts while the lead processor does mathematical processing in the same project.
  • Production Line Processing.  You can now collate data from several projects together into a master project for final validation and product preparation.  Mathematical processing is done in the smaller projects, along with data validation if desired.  The final validation and junction analysis in areas of overlap between projects is done in the assembly project.  If problems are found, the surveyor can be directed to re-open their processing project to make corrections, the QPDs that become corrected are then detected in the main project and the mapping products will update the next time the master project is opened.
  • Snapshot To Scene.  You can now quickly isolate soundings of interest and have them included in the Scene as a set of points objects.  This can be used to mark up areas of interest that require further investigation, perhaps even for identifying features of note.  The .sd point files that are created can be used directly in Fledermaus, but they can also be exported quickly as ASCII text files for use in other software.
  • R2Sonic multi-spectral is now supported in that there is a new custom filter available via the Filter Operation Toolbar that lets you specify the frequency that you'd like to preserve in the QPDs. All other soundings remain from pings with other frequencies, however they are disabled via the filter flag.  You can now choose your desired frequency for bathymetric processing.  This augments the multi-spectral backscatter capabilities delivered in FMGT in Fledermaus version 7.7.0.
• QPS Seamless Workflow Improvements
  • Laser from QINSy .db.  You can now perform raw sensor processing for laser sensors acquired with QINSy.  This includes processing directly from QINSy .db files, you can also adjust sensor location and alignment angles and apply post-processed navigation like SBET.  Patch test and wobble analysis support for laser will come in a later release.
  • Qimera can now be launched from the QINSy Console with your QINSy project "ready to go".  Consult the QINSy 8.16 release notes for more information.
  • Geoid Model files are now fully shared with QINSy using a common installer for these files.  QINSy and Qimera now share a common Geoid Model installation location and the Qimera installer has reduced accordingly in size.
  • Ability to export to a new layer in Sounding Grid file.  For QINSy users who were used to sliding a new layer of a grid into an existing Sounding Grid, you can now do this from Qimera.
• Imports and Exports
  • Qimera Live Improvements.  You can start now with a completely empty project.  Previously you had to import at least one line and create one Dynamic Surface.  Now you can start it up from scratch from a completely empty project.
  • R2Sonic raw format.  Qimera now supports import of raw R2Sonic data records that are sometimes recorded using custom data loggers on AUVs where a traditional acquisition system is not used.  Following the convention we established in FMGT in Fledermaus version 7.7, files with the extension .R2SC can now be imported and combined with navigation, motion and SVP.
  • Export track to ASCII now supports specification of the outgoing geodetic coordinate frame.  Previosly, you could only export in the project's projected units.
• CAD Format Support and Capabilities
  
  • Polygon draw, save and re-use for spatially bounded filtering & clipping.  You can draw polygons, have them preserved as line objects (.sd format) in the Scene.  These can then be converted back into a selection and used for spatial filtering.  The same capability allows imported CAD objects to be used for spatially bounded filtering & clipping.  Drawn polygons that are promoted to line objects can also be exported for use in other software applications.
  • Allow for selecting multiple QGF files during CAD import.
• Data Validation
  
  • Workspace preset for just point cleaning.   We did this in response to Qloud user feedback, there is a preset that minimizes the number of widgets and tools shown to users when they are only interested in editing soundings.
  • Performance improvements for Slice Editor.  Based on feedback from our Qloud users, we made several efficiency improvements to the Slice Editor for increased responsiveness for the rectangular selection mode.  We will be porting these advances to the Scroll select mode in a follow up release.
  • The "suspect" filter has been ported from Fledermaus to Qimera for FMHydro users.  You can now flag soundings as "suspect" when they deviate from the Dynamic Surface depth by a prescribed distance.
• Analysis
  
  • Coverage Extinction Plot.  You can now plot the outer soundings for a specified set of survey lines to establish the coverage capabilities of your multibeam system.  This is handy for assessing coverage capabilities as a function of water depth to compare against the manufacturer's specifications.
  • More control over how patch test offsets are applied.  For systems with separate transmit and receiver locations, you can now choose to have the patch test offsets be applied to both the transmitter and receiver.  This is primarily for Kongsberg EM2040 systems with separate transmit and receiver locations, as requested by our colleagues at Kongsberg Maritime.  Previously, Qimera followed the QINSy convention of applying the roll offset to the receiver only and the pitch/heading offsets to the transmitter only.  The patch test tool will now detect when you have separate transmitter and receiver locations and will ask you for your preference.
  • Plot transducer tracks in Overview Window for patch test.  When you run the patch test tool, the trackline drawn in the 4D Scene now reflects the position of the sonar head(s) to better establish which soundings are immediately below the transducer for pitch alignment.
• Misc
  • Plot SVP locations in Scene.  If SVP profiles have their position specified, they will be plotted in the Scene to aid in quality assurance of SVP metadata and also in application of SVP.
  • Escape key shortcut to return to Explore mode has been disabled.  The Escape key previously returned Qimera to "Explore" mode, which frustrated many non-Fledermaus users.
  • Ability to organize sources and layers alphabetically.  You can re-organize items alphabetically using the regrouping capability that has been available since Qimera was launched.

Bug Fixes

We typically deal with bug fixes in our maintenance releases.  Here is a list of those that we've fixed since our last maintenance release (Qimera version 1.4.4).

• Fixed a bug where the 'Nearest in Distance' setting for SVP processing was not transitioning midway between casts.
• Reverting a Vessel Configuration when an SBET was applied was causing a crash. The crash and the underlying cause have been fixed.
• The Override Configuration dropdown is now correctly populated with the selected configuration.
• A CUBE update bug was inadvertently introduce in Qimera 1.4.2. This has been fixed.
• QINSy .db files that contain subbottom sounder systems will now import successfully in Qimera.
• Water Column Tool
  • Under certain circumstances, saving a fan snapshot in the Water Column Tool was causing a crash. This is now fixed.
  • Fixed a crash that sometimes happened when applying sidelobe suppression.
  • Improved support for certain R2Sonic data where the dynamic range of the water column imagery was not being converted properly.
  • Certain QINSy .db files containing single beam systems were prevented from carrying out water column point extraction. This will now work correctly.

Installation, Upgrade and Ongoing Support Information for Existing Users

Users with existing Qimera 1.4 projects will be able to open them in Qimera 1.5 but the projects will become upgraded to version 1.5 and will not be able to be re-opened in Qimera 1.4.  A small number of the capabilities in this release require upgrades to some of the file formats in a typical Qimera project.

The Geoid Model files, which change infrequently, have been removed from the Qimera installer and are now provided as a separate installer.  This is the same installer that QINSy uses as of the 8.15 release (released December, 2016).  The Qimera installer is now much smaller, making for a quicker download.  New users will need to download and install the Geoid Models.  Users with pre-existing Qimera installations can install the new 1.5 version and it will continue to search for Geoid Models in the locations used by previous Qimera installations, but only if Qimera 1.5 is installed as an upgrade to a previously installed version.  If users instead decide to install Qimera 1.5 alongside a previous installation by choosing a separate installation directory, the previously used Geoid Models will not be used when running Qimera 1.5.  If a user decides to install the standalone Geoid Models, they will be installed in the shared common location with QINSy.  The previously installed Geoid Models will not be removed unless a user uninstalls Qimera.  The new Geoid Model installer file will vary between Windows, Mac and Linux operating systems in behavior only, e.g. Windows runs an installer but Linux will be a zip file that must be manually unzipped in the correct location.  Refer to the installation instructions for the Geoid Model download page for more information.

The Qimera 1.1 release had introduced new beam flagging capabilities that required a QPD upgrade.  Users with older Qimera 1.0 projects will need to upgrade their QPDs when moving to Qimera 1.5. You will be prompted to upgrade the QPDs in your project when it is first opened.

During the development phase for Qimera 1.5, we paid close attention to incoming bug reports from our clients and we've streamlined our release process to get fixes out to clients in a timely manner, as often as required.  This new, streamlined process enabled delivery of twenty-four maintenance releases since the launch of Qimera 1.0 in June of 2015.  We will continue to provide this quick turnaround level of service to our customers with Qimera 1.5.  If you find a bug that stops your workflow, please contact us and we'll attempt to get a fix to you as quickly as possible.

When we first released Qimera, we purposely kept it lean and mean with the intent of focusing on client feedback to help direct us on how best to hone existing capabilities and also to help us decide which new functionality clients need most.  We've done the same with Qimera 1.1 through to Qimera 1.5, so please keep ideas and feedback coming, it helps us to make Qimera better for all users!  Please feel free to contact us via the online support system for feedback, feature requests and bug reports.

Documentation

• The Reference Manual has been updated for all features in this release
  
• We're continuing to update other Knowledge Base articles.  The online versions always have the freshest content.
• Several new "How To" style documents are available to give some guidance on how to use some of the new features in Qimera 1.5.  The online versions are always most up-to-date.

Limitations & Known Issues

Qimera should not open QINSy projects while QINSy is running, acquiring data or generating Dynamic Surfaces.  Data loss may occur if this warning is not heeded.  Qimera is currently meant to only open QINSy projects after they are acquired.  If you need to use Qimera with QINSy in a real-time environment, it is recommended that you configure QINSy to copy .db files to a backup directory at the end of each line.  You should then create a new Qimera project and use Qimera Live (available as of Qimera 1.1) to monitor the backup directory, import the data, process it and add it to a Dynamic Surface.

If you are a QINSy and Qimera user, you should not copy the QINSy project to another location on the same computer as the original QINSy project.  We are currently improving how file paths to various resource files and project file assets are managed between our applications, in the meantime, if you want to process a copy of your QINSy project in Qimera, you should copy it onto another computer.  This is a known limitation that we are actively working on to fix and we will let you know when this limitation no longer applies.  Of course, if you are willing to work directly on the QINSy project once QINSy is offline, then this is the preferred option and you can ignore this warning.