Custom data types

Discover how to add custom data types to Vault and how they work with policies

Vault includes a set of built-in data types. Vault also enables you to define custom data types. These custom data types can be assigned to properties and used just like the built-in types.

Custom data types are based on the built-in data types with the addition of JavaScript code to define validators, normalizers, and transformers for the data type. This JavaScript code is specified to Vault in bundles.

note

You can not base a custom type on another custom type (only on the built-in types).

This page describes how to define custom data types dynamically or using a configuration file, and how policies control access to custom data types.

Define custom data types dynamically

You can define custom data types dynamically using the data type REST API operations and CLI commands and specify the JavaScript code used in those data types with the bundle REST API operations and CLI commands.

Data types configuration file

Specifying bundles and data types with the API or CLI means you need to run code to insert custom data types after Vault loads. However, you can load custom data types and their associated bundles when you start Vault. The file is loaded the first time Vault start and reloaded on subsequent starts if the PVAULT_SERVICE_UPDATE_SCHEMA_ON_START environment variable is true.

note

If you're using the hosted version of Vault, contact us if you wish set Vault to load the configuration file on each start.

The data types configuration file is called pvault.types.toml and stored in the directory /etc/pvault/conf.d. If the file is missing, Vault skips the data type loading process.

The types configuration file contains custom data type definitions and bundle declarations. Each bundle declaration includes the name of the file that contains the bundle content. These files are loaded from /etc/pvault/conf.d/pvault.bundles.

If the format of the types configuration file is invalid, any bundle files are missing, or the addition of the bundles or data types fails, Vault does not load.

Types configuration file format

The types configuration file is a TOML file with two sections.

– [types] contains the definitions of the custom data types. – [bundles] contains the list of bundles.

This is an example of a valid pvault.types.toml file:

[types]
    [types.AgeYears]
        description = "Age in years"
        based_on = "INTEGER"
        validator = "sample.between_0_120"
        normalizer = "sample.identity"
        transformers = ["sample.modulo_10"]

    [types.CaseInsensitiveName]
        description = "Case insensitive name"
        based_on = "NAME"
        normalizer = "sample.lower_case"

    [types.Interval]
        description = "Interval"
        based_on = "STRING"
        validator = "sample.interval_HH_MM_SS"
        transformers = ["sample.mask_last_4", "sample.truncate_last_4"]

    [types.HTTPS_URL]
        description = "HTTPS URL"
        based_on = "URL"
        validator = "sample.is_https"

    [types.DOB_with_age]
        description = "Date of birth with age"
        based_on = "DATE_OF_BIRTH"
        transformers = ["sample.age"]

    [types.EMAIL_protected]
        description = "Protected email"
        based_on = "EMAIL"
        transformers = [
            "sample.consented_email",
            "sample.redacted_email",
            "sample.allow_by_country",
            "idor.idor_protected"
        ]

[bundles]
    [bundles.sample]
        description = "This bundle contains some sample transformers, normalizers and validators"
        file_name = "sample.js"

    [bundles.idor]
        description = "This bundle contains the IDOR transformer and all its dependencies"
        file_name = "idor.js"

This example file:

Defines the custom data types AgeYears, CaseInsensitiveName, Interval, HTTPS_URL, DOB_with_age, and EMAIL_protected.
Declares the bundles sample and idor.

Type functions, if specified in the definition of a data type, use the qualified name format <bundle-name>.<function-name>, where:

<bundle-name> is the name of the bundle where the type function is defined and exported.
<function-name> is the exported name of the function.

See Exporting a type function for more information.

All the bundles referenced by data types in the [types] section must be declared in the [bundles] section.

The declaration of each bundle specifies a file name for the bundle. A file with this name must be in the /etc/pvault/conf.d/pvault.bundles directory.

Types configuration file validation

Vault does not start if:

The file does not comply with the TOML format.
A type refers to one or more validators, normalizers, or transformers that are not defined.
A type has the same name as a built-in type.
A custom type is used in a property of a collection but is not defined.
Any of the referenced Javascript code does not compile.