Create a custom data type

Learn how to add a bundle and definition of a custom data type to Vault

To create a custom data type, you:

1. Create self-contained JavaScript code that defines the data type's validators, normalizers, and transformers. Then save that code as a bundle in Vault.
   note

   You can use code from one or more bundles when defining a custom data type. However, all the code must be self-contained within a bundle; you can't references code in one bundle from code in another. A bundle can contain one or more validators, normalizers, and transformers.

2. Create the definition of your custom data type and save it in Vault.

You can then add properties using your custom data type to collections and save and retrieve data using the custom data type.

In this guide, you create a custom data type for the United Kingdom National Insurance number. The custom data type includes a validator to ensure the input is in the correct format and a normalizer to remove hyphens and spaces.

Create the bundles

A bundle is a self-contained JavaScript file that defines a data type's validators, normalizers, and transformers. See the bundles reference for more information on defining your JavaScript and creating a bundle.

You can load bundles using the add bundle CLI command and add bundle API operation or save it to the /etc/pvault/conf.d/pvault.bundles directory, which is loaded when Vault starts.

Define the JavaScript bundle

This bundle contains JavaScript for validating and normalizing the National Insurance number.

// export the validator for use by Vault
exports.validateNINumber =  {
  type: "validator",
  description: "United Kingdom National Insurance number validator.",
  handler: validateNINumber
}

// validate the expression then check for invalid prefixes
function validateNINumber(ninoparam) {
  // first normalize the input
  nino=normalizeNINumber(ninoparam);
  // Regular expression to match the NI number format
  const regex = /^[A-CEGHJ-PR-TW-Z]{2}\d{6}[A-D]$/;

  // Test the input against the regular expression
  if (!regex.test(nino)) {
    return false;
  }

  // Further check for disallowed prefixes.
  const prefix = nino.substring(0, 2);
  const invalidPrefixes = ["BG", "GB", "NK", "KN", "TN", "NT", "ZZ"];

  if (invalidPrefixes.includes(prefix)) {
    return false;
  }
  return true;
}

// export the normalizing function for use by Vault
exports.normalizeNINumber =  {
  type: "normalizer",
  description: "United Kingdom National Insurance number normalizer.",
  handler: normalizeNINumber
}

// normalize the value by removing spaces and hyphens
function normalizeNINumber(nino) {
  // Use regular expression to replace hyphens and white spaces with an empty string
  return nino.replace(/[-\s]/g, '');
}

Save your file to a suitable location with the name UKNI-bundle.js.

Add the bundles to Vault

You add the bundle to Vault from the saved file using the CLI like this:

pvault bundle add --name UKNI_bundle --description "UK NI no. validator" --bundle-code @UKNI-bundle.js

You get a response similar to this:

------------------------------------------------------------------------------------------------------------------------------+
|     description     |         creation_time         |    name     |     normalizers     |     validators     | transformers |
+---------------------+-------------------------------+-------------+---------------------+--------------------+--------------+
| UK NI no. validator | Sat, 07 Sep 2024 17:43:42 UTC | UKNI_bundle | [normalizeNINumber] | [validateNINumber] | []           |
+---------------------+-------------------------------+-------------+---------------------+--------------------+--------------+

Create the date type

A custom data type definition defines the built-in data type it is based on, the custom types name and description, and references the functions you defined in bundles to implement the custom data types validators, normalizers, transformers, and default transformer. See the custom data types reference for more information on defining your custom data type.

You load custom data types using the Add data type CLI command or Add data type API operation. You can also load custom data types using a configuration file which is loaded when Vault starts.

Construct the JSON

The CLI command and API operation specify the custom data type using a JSON format. This example creates a custom data type called UKNI, based on the built-in string data type and uses the validator end normalizer from the bundle.

{
  "name": "UKNI",
  "base_type_name": "STRING",
  "description": "United Kingdom National Insurance number.",
  "validator": "UKNI_bundle.validateNINumber",
  "normalizer": "UKNI_bundle.normalizeNINumber"
}

Define data type to vault

You add the custom data type to Vault using the CLI like this:

pvault datatype add --datatype-json='
{
  "name": "UKNI",
  "base_type_name": "STRING",
  "description": "United Kingdom National Insurance number.",
  "validator": "UKNI_bundle.validateNINumber",
  "normalizer": "UKNI_bundle.normalizeNINumber"
}'

You get a response similar to this:

+--------------------------------+-----------+-------------------------------+------+----------------+-------------------------------+---------------------+--------------+
|          description           | validator |         creation_time         | name | base_type_name |          normalizer           | default_transformer | transformers |
+--------------------------------+-----------+-------------------------------+------+----------------+-------------------------------+---------------------+--------------+
| United Kingdom National        |           | Sun, 08 Sep 2024 09:34:32 UTC | UKNI | STRING         | UKNI_bundle.normalizeNINumber |                     | []           |
| Insurance number.              |           |                               |      |                |                               |                     |              |
+--------------------------------+-----------+-------------------------------+------+----------------+-------------------------------+---------------------+--------------+

Using your custom data type

You can now use your custom data type as you would use a built-in data type.

Add property to collection

pvault collection add --collection-pvschema "
customers PERSONS (
  name NAME,
  ukni UKNI UNIQUE
);"

Store objects using custom data type

pvault object add --collection customers --fields '{ "name":"John Doe", "ukni":"ZC443091B" }'

You get a response similar to this:

+--------------------------------------+
|                 id                   |
+--------------------------------------+
| ab3d5be0-ffde-4a8c-983a-f79dd5d34e17 |
+--------------------------------------+

Retrieve custom data property

pvault object get \
        --id ab3d5be0-ffde-4a8c-983a-f79dd5d34e17 \
        --props name,ukni \
        --collection customers

You get a response similar to this:

+----------+-----------+
|   name   |   ukni    |
+----------+-----------+
| John Doe | ZC443091B |
+----------+-----------+

Adding an incompatible input, such as using one of the illegal prefixes ('GB'):

pvault object add --collection customers --fields '{ "name":"John Doe", "ukni":"GB453091B" }'

Should fail with:

ERR Error code: PV3217, Status code: 400, Message: Custom validation failed., Context: map[data_type_name:UKNI property_name:ukni]

Adding non-normalized value:

pvault object add --collection customers --fields '{ "name":"Jane Doe", "ukni":"ZC-55-3 0 9 1 B" }'

Should normalize it such as:

pvault object list -c customers -a

Displaying 2 results.
+--------------------------------------+----------+-----------+
|                  id                  |   name   |   ukni    |
+--------------------------------------+----------+-----------+
| ab3d5be0-ffde-4a8c-983a-f79dd5d34e17 | John Doe | ZC443091B |
| 983e3546-7d99-48b6-828f-3214ee1907ab | Jane Doe | ZC553091B |
+--------------------------------------+----------+-----------+

caution

The normalizer is not called before the validator because it may not know how to normalize illegal input. In cases where it is safe to do so, such as in this example, you can call it manually from the validator code.