Skip to main content

Data import and export

Export data from and import data to your Vault​

note

If you're using the hosted version of Vault, contact us if you need an export of your data.

Overview​

Vault enables you to export data from an instance and import it into another. You can use this capability when moving your Vault to a different region or cloud provider or migrating to a Vault that uses a different KMS (such as moving from a local seed KMS to an AWS or GCP KMS.)

You could also use import to bring in a large number of records from an external system to Vault by building the export file manually.

Export data​

Before exporting, set the PVAULT_KMS_EXPORT_URI or PVAULT_KMS_EXPORT_SEED key management service environment variables to define your export encryption key. The same export key configured in the source Vault must be set in the destination Vault.

These keys are mandatory for the export process and are used to encrypt the exported data. This is a safety mechanism to ensure that the data can't be readable outside of the Vault. All property values of the exported objects and tokens are encrypted except for the id property which is not sensitive and is used to identify the object.

This mechanism ensures that only someone with access to the KMS can decrypt the zip data. Without access, the zip data can't be decrypted.

See the property encryption guide to harden access to your KMS.

Once all keys are set, export your data using the export CLI command. The export includes all collection schemas, all collection object data, token data, the IAM configuration, all data types, and all bundles. You can also include archived objects or exclude any items from the export.

Import data​

To import data, ensure that the destination Vault has been set up with the same PVAULT_KMS_EXPORT_URI or PVAULT_KMS_EXPORT_SEED key management service environment variables used for the export in the source Vault.

If you're importing from another system, you have to create the ZIP file (see Export file specifications). When doing this, the only item required in info.json is the Vault version, such as:

{
  "vault": {
    "vault_version": "1.6.0.1-gb6ce68acc"
  }
}

The rest of the parameters are included by the export as extra details, but aren't used during the import. Note that such a configuration doesn't have the export_key included and as such the system expects to receive clear (unencrypted) text data.

note

It is highly recommended to import data into the same version of Vault as the data was exported from to avoid any compatibility issues. If you're importing into a different version use the --force flag to bypass the version check.

You import the exported zip file using the import CLI command by providing the location of the zip file.

Export file specifications​

The export file is a zip archive with this content:

🗄️ vault_export_{timestamp}.zip
    📂 bundles
        📄 {bundle-name}.js // Using standard bundle JS format
        📄 ...
    📂 schemas
        📄 {collection-name}.json // Using the collection JSON format used in the collections API
        📄 ...
    📂 objects
        📄 {collection-name}.json // See object details.
        📄 ...
    📂 tokens
        📄 {collection-name}.json // See token details.
        📄 ...
    📄 data_types.json // Array of data type models, same as the data types API
    📄 iam.toml // Standard IAM toml format
    📄 info.json // See info.json.

The only mandatory file in the zip is info.json.

For example, a zip containing only info.json and objects/customers.json imports the objects of the customers collection (assuming the customers collection is defined in Vault, it fails if customers collection is not defined.)

Object entries​

Object entries are formatted as JSON lines (each line is in JSON format). Each line includes a JSON object with an objects key containing an array of objects. Each object in the array follows the standard object format as in the object APIs. Each line has no limit to the number of objects in it, as long as each line doesn't exceed 1MB of data. The file must end with an empty newline.

{"objects": [{...}, {...}, ...]}
{"objects": [{...}, {...}, ...]}
...

Token entries​

Token entries are formatted as JSON lines (each line is in JSON format). Each line includes a JSON object with an tokens key containing an array of tokens. Each token in the array contains the following entries: "token_id", "token_type", "props", "fields", "object_id", "scope", "expiration", "tags", "created_at", "modified_at", and "fptemplate". Each line has no limit to the number of tokens in it, as long as each line doesn't exceed 1MB of data. The file must end with an empty newline.

{"tokens": [{...}, {...}, ...]}
{"tokens": [{...}, {...}, ...]}
...

info.json​

info.json is required and formatted like this:

{
  "vault": {
    "vault_version": "1.6.0.1-gb6ce68acc", // The Vault version this zip is compatible with (required for import). If the zip is built manually, set to the Vault version you are importing to.
    "cli_version": "v1.6.0-114-gb6ce68acc", // CLI version used for the export (irrelevant for import)
    "vault_id": "191994678324457472", // Vault ID the data exported from (irrelevant for import)
    "generation_number": 8, // Last generation number at the time of the export (irrelevant for import)
    "export_key": "..." // The export key used to encrypt the exported data (optional.) If not provided, the import expects data to be unencrypted, such as for manual migration of data to Vault.
  },
  "export": {
    "export_time": "2023-06-14T20:18:19.204464Z", // Time of export (irrelevant for import)
    "export_params": { // Filters used during the export (irrelevant for import)
      "collections": null,
      "export_archived": false,
      "export_schemas": true,
      "export_iam": true,
      "export_bundles": true,
      "export_data_types": true,
      "export_objects": true,
      "export_tokens": false
    }
  }
}

Limitations​

  1. Data is always exported in an encrypted form. This is a failsafe added to prevent an inadvertent data leakage.
  2. Objects that were encrypted by the source Vault using the Encrypt API will not be able to get decrypted on the destination Vault.
  3. Exporting pci_oneway tokens are not supported. Exporting it would break the one way requirement.
  4. Importing pci_oneway tokens and deterministic tokens are not supported.
  5. Import of pci tokens is supported only if the token data is not present in the destination Vault.
  6. The source Vault and destination Vault must be of the same Vault release.

Exporting non-encrypted copy of your data​

Since data is only exported encrypted, to get a non-encrypted copy of your data requires running an additional tool provided by Piiano to transform the encrypted data to clear text. That tool will require your encryption key.

On this page