News

Laramie, WY,  February 9, 2024 – Lifeboat LLC, developers of innovative features that are critical to the HDF5 user community, has won a competitive grant to develop software to protect the confidentiality and integrity of data stored in HDF5.

The US Department of Energy has announced Lifeboat’s receipt of a competitive SBIR grant to develop software to support encryption in HDF5.

Data has become one of the most valuable assets.  Protecting data from unauthorized access is one of the highest priorities for all organizations that collect, store and process sensitive data, including government, financial, medical and pharmaceutical companies and research institutions including HPC centers all over the world. Recent NIST “High-Performance Computing (HPC) Security” report identified data integrity and confidentiality in “Access”, “High-Performance Computing”, and “Data Storage” areas as the major security concerns that have to be addressed by the HPC centers. Clearly, the identified problems of storing, moving and accessing sensitive data are universal.

Many data management solutions, for example, commercial Oracle database, MongoDB database, and TileDB, and universal data management platforms, Google and Amazon Cloud, offer different data protection solutions. Unfortunately, HDF5, being the de-facto standard for managing, sharing and archiving data, is not suited for storing sensitive data due to its open file format specification. The content of an HDF5 file can often be analyzed, viewed and modified with the open source and commercial tools readily available on the systems or simple scripts written in high-level languages, for example, Python or Julia.

Open file format and easy access to data creates a risk to storing sensitive data in HDF5.  Encrypting the whole HDF5 file is one of the possible mitigation solutions when data is transferred and/or stored. Unfortunately, the file has to be decrypted before an HDF5 application can read or write data to it, leaving data insecure for the time required to run the application. 

Through this grant, Lifeboat will extend HDF5 to support data encryption while preserving all existing HDF5 functionality such as efficient sequential and parallel I/O, data compression, data portability, etc. “We are very excited about this opportunity: we will not only address a long-standing request from our users to give them a mechanism for protecting sensitive HDF5 data, but we also give them peace of mind that the data cannot be easily compromised when it is moved, stored and accessed (just don’t keep the keys with the data!). Data is always secure.”, says Elena Pourmal, CEO, Lifeboat, LLC.. 

Please don’t hesitate to contact us if you are interested in the HDF5 encryption feature and would like to learn more.