Reference data

One of Hermine’s two main purposes is to facilitate Open Source licenses analysis. As a tool it aims at automating the application of this analysis to your SBOMs, as a community it aims at building a common language and understanding of licenses and their obligations within Hermine’s user community. This is materialized by Hermine Community Data, which is built on Hermine-data repository.

The reference Data is to display this community data and allow you to manage the interactions between your data and community data.

Hermine Reference Data is made of 2 top-level types of object :

  • Licenses

  • Compliance actions

General principles

The community data is stored as individual json files in their respective directories (generics and licenses) on Hermine-data repository.

For licenses, the files are names {SPDX ID}.json.

For generics, the title of the obligation is used and filtered to be suitable for a file name (no special characters, no spaces, etc.) : {Filtered title}.json.

This allows to easily follow the changes to the objects and have readable diffs, leveraging the native capabilities of git and its tooling ecosystem.

For each release of the data, a global shared.json file, containing all the licenses and generics obligations, is generated by the to_dist.py script, which is launched by the GitLab CI.

Importing Community Data into your Hermine Server

  1. Get the latest release of the data : https://gitlab.com/hermine-project/hermine-data/-/releases/permalink/latest/downloads/shared.json

  2. Load it as your reference data

    • If you have installed Hermine manually

# in your poetry environment
python hermine/manage.py init_shared_data /path/to/shared.json

- If you have installed Hermine via docker compose

Copy `shared.json` into the `docker` folder of the Hermine code, so that

your next image build picks it up. To apply it without rebuilding, copy it into the running container as well, at the path the entrypoint reads:

docker cp shared.json <django container>:/opt/hermine/shared.json
docker restart <django container>

  1. Restart your server

    Reference data is read once at startup and cached for the lifetime of the process. Loading it on a running server is not enough: the workers keep answering from the reference they loaded when they started, and nothing in the interface indicates that the figures shown are stale.

Note

This is not the same thing as Licenses > Import, which also accepts a shared.json file. That form writes every object straight into your own database, in one go, with no comparison and no choice. Loading the file as reference data leaves your licenses untouched and lets you decide, on the Shared reference data page, what you want to take from the community.

Interacting with reference data in your Hermine instance

Summary of reference data

Once you have imported reference data, if your user has the appropriate rights, you will have a new Shared reference data menu entry in the Legal section, leading to the /shared/ page, that presents a summary of the differences between local and reference data.

In the licenses and generics pages

In the licenses and generics pages, a column Differences with reference shows the status of each license ; values can be Local only,No differences or See differences with a link to the page showing the differences.

Page showing the differences

The page lists the differences, showing local and reference value, with a button to update your local value.

Taking data from the reference

The Shared reference data page offers several actions, and they do not carry the same risk.

Copying the licenses that exist in the reference and not in your instance, or taking a single license or a single obligation, only adds what you do not have.

Sync everything overwrites. It replaces every field of every license present on both sides, deletes its obligations and recreates them from the reference. On an instance where licenses have been analyzed locally, that analysis is lost: fields your team filled in, obligations your team wrote. Use it on a fresh instance, or on one whose license data has never been edited. Anywhere else, take what is missing and review the differences one by one.

In the license and compliance action pages

In each license or compliance action page, if the license or compliance action identifier is present in the reference data, an information box will be displayed if the local data for this object differs from the one in Reference Data, with a link to display the differences.

Exporting data to contribute to Hermine data

To export license data, go to /licenses/ and choose “Export all licenses and compliance actions as an archive (for contributing to hermine-data)” in the Import/export menu. Decompress the archive in a temp folder.

Clone the repository at https://gitlab.com/hermine-project/hermine-data , create a branch, copy the files you want to summit from the downloaded archive into the licences and generics folder.

Git add, commit, push and create a Merge request.