Data import and export

Export data from and import data to your Vault

note

If you're using the hosted version of Vault, contact us if you need an export of your data.

Overview

Vault enables you to export data from an instance and import it into another. You can use this capability when moving your Vault to a different region or cloud provider or migrating to a Vault that uses a different KMS (such as moving from a local seed KMS to an AWS or GCP KMS.)

You could also use import to bring in a large number of records from an external system to Vault by building the export file manually.

Export data

Before exporting, set the PVAULT_KMS_EXPORT_URI or PVAULT_KMS_EXPORT_SEED key management service environment variables to define your export encryption key. These keys in the source Vault must match the keys in the destination Vault.

These keys are mandatory for the export process and are used to encrypt the exported data. This is a safety mechanism to ensure that the data can't be readable outside of the Vault. All property values of the exported objects are encrypted except for the id property which is not sensitive and is used to identify the object.

This mechanism ensures that only someone with access to the KMS can decrypt the zip data. Without access, the zip data can't be decrypted. Note that the export_key entry in the info.json is also encrypted with the KMS and can't be used to decrypt the data.

See the property encryption guide to harden access to your KMS.

Once all keys are set, export your data using the export CLI command. The export includes all collection schemas, all collection object data, the IAM configuration, all data types, and all bundles. You can also include archived objects or exclude any items from the export.

note

The export does not include tokens. Talk to us if you require this functionality.

Import data

To import data, ensure that the destination Vault has been set up with the same KMS used for the export in the source Vault. The encrypted key is stored in the ZIP, so you don't need to provide it again.

If you're importing from another system, you have to create the ZIP file (see Export file specifications). When doing this, the only item required in info.json is the Vault version, such as:

{
  "vault": {
    "vault_version": "1.6.0.1-gb6ce68acc"
  }
}

The rest of the parameters are included by the export as extra details, but aren't used during the import. Note that such a configuration doesn't have the export_key included and as such the system expects to receive clear (unencrypted) text data.

note

You can only import data into the same version of Vault as the data was exported from.

You import the exported zip file using the import CLI command by providing the location of the zip file.

Export file specifications

The export file is a zip archive with this content:

🗄️ vault_export_{timestamp}.zip
    📂 bundles
        📄 {bundle-name}.js // Using standard bundle JS format
        📄 ...
    📂 data
        📄 {collection-name}.json // See Data entries.
        📄 ...
    📂 schemas
        📄 {collection-name}.json // Using the collection JSON format used in the collections API
        📄 ...
    📄 data_types.json // Array of data type models, same as the data types API
    📄 iam.toml // Standard IAM toml format
    📄 info.json // See info.json.

The only mandatory file in the zip is info.json.

For example, a zip containing only info.json and data/customers.json imports the objects of the customers collection (assuming the customers collection is defined in Vault, it fails if customers collection is not defined.)

Data entries

Data entries are formatted as JSON lines (each line is in JSON format). Each line includes a JSON object with an objects key containing an array of objects. Each object in the array follows the standard object format as in the object APIs. Each line has no limit to the number of objects in it, as long as each line doesn't exceed 1MB of data. The file must end with an empty newline.

{"objects": [{...}, {...}, ...]}
{"objects": [{...}, {...}, ...]}
...

info.json

info.json is required and formatted like this:

{
  "vault": {
    "vault_version": "1.6.0.1-gb6ce68acc", // The Vault version this zip is compatible with (required for import). If the zip is built manually, set to the Vault version you are importing to.
    "cli_version": "v1.6.0-114-gb6ce68acc", // CLI version used for the export (irrelevant for import)
    "vault_id": "191994678324457472", // Vault ID the data exported from (irrelevant for import)
    "generation_number": 8, // Last generation number at the time of the export (irrelevant for import)
    "export_key": "..." // The export key used to encrypt the exported data (optional.) If not provided, the import expects data to be unencrypted, such as for manual migration of data to Vault.
  },
  "export": {
    "export_time": "2023-06-14T20:18:19.204464Z", // Time of export (irrelevant for import)
    "export_params": { // Filters used during the export (irrelevant for import)
      "collections": null,
      "export_archived": false,
      "export_schemas": true,
      "export_iam": true,
      "export_bundles": true,
      "export_data_types": true,
      "export_objects": true,
      "export_tokens": false
    }
  }
}