Custom JavaScript format-preserving templates

Learn how to add format-preserving templates to Vault using JavaScript that enable you to customize the IDs generated for tokens.

By default, Vault uses the UUID format for token IDs. Vault also provides a built-in format preserving template, primary_account_number, for PAN (credit card number format). To use other formats, you add format-preserving templates.

Format-preserving tokens are helpful where you want to ensure tokenized data is validated by downstream systems or retains elements used for other purposes. Take phone numbers. These may be validated when stored, and a system could use the numbers' area codes for analytics. Standard tokenization would generate a UUID that wouldn't validate and wouldn't preserve the area code. Using a custom format-preserving template, you can tokenize phone numbers and generate token IDs that have the same format as the original data, making them valid input for existing systems. You also can keep the area-code in the token ID so as not to lose the analytics capability.

To learn more about format-preserving templates and built-in templates, see Create format preserving tokens.

Add a format-preserving template

Custom format-preserving templates are JavaScript functions loaded into Vault using bundles. They are used to control the ID of tokens created using the Tokenize and Rotate operations. In the function code, you can use a random seed, the source object fields, and other details to format the token ID.

See the Bundles reference documentation for information on how to load the job script code for a custom format-preserving template into Vault. The remainder of this page describes managing and applying custom format-preserving templates.

Use a format-preserving template

You use a custom format-preserving template in the Tokenize and Rotate operations. In the call, you set fptemplate to name of the template in the format of <bundle-name>.<function-name>. The fpprops input, which is mapped to propNames argument of the handler, is not required for custom templates.

The custom format-preserving template handler then accepts the context argument and returns a string (the ID). See Bundle handlers for the full handler specification and details of global functions.

Format-preserving tokens comply with the predictability rules of token types. For example, if tokenizing with a pci token type, a format-preserving template generates the same ID for each instance of the same value. If the same token ID should be used, the template's JavaScript function will not be called. You can assume that if the JavaScript function is being called, a new token ID should be generated.

warning

If the function is changed or deleted (by updating or deleting the bundle), calls to Tokenize and Rotate tokens operation may produce unexpected ID formats or fail.

Token ID rotation using a format-preserving template

When using the Rotate API, the template's JavaScript function is called with the following fields of the input:

• rotate: Set to true.
• rotatedTokenId: the original token ID. This field is used only if this is a rotate-tokens operation.

See Bundle handlers for more details on the functions's arguments.

Example: Format-preserving template for a phone number

This template generates a token ID with the format of a US phone number, keeping the first three digits (area code, NPA) the same as the original input. The function assumes that one of the tokenized fields is a property called "phone_number" of data type PHONE_NUMBER.

exports.us_phone_number = {
  type: 'format_preserving_template',
  description: 'Generate an ID with a US phone number format.',
  async handler({ seed, fields }) {
    // Validate the input
    if (
      typeof fields !== 'object' ||
      !('phone_number' in fields) ||
      typeof fields.phone_number !== 'string'
    ) {
      throw new Error('Invalid input');
    }

    const phoneNumber = fields.phone_number;

    // Skip the prefix "+1", and preserve the area code (NPA).
    const npa = phoneNumber.substring(2, 5);

    // Parse the seed and use it to create a 7 digit number.
    const randomDigits = atob(seed)
      .split('')
      .map(b => b.charCodeAt(0))
      .map((byte) => byte % 10) // Convert each byte to one digit (0-9)
      .slice(0, 7) // Take the first 7 digits
      .join('');

    return '+1' + npa + randomDigits;
  }
}

A token using the format-preserving template can be created like this:

pvault token create -c buyers --type randomized --object-id cc9a39c5-4734-4786-b317-e16705d5128f --fptemplate bundle1.us_phone_number

You get a response similar to this:

+--------------+
|   token_id   |
+--------------+
| +11234567890 |
+--------------+

Example: Format-preserving template for an email

Tokenizing an email address that passes validation can be done without using a format-preserving template. To do this you take a token ID (which by default is in UUID format) and suffix it with @email.com to get, for example, 01943fcf-9d31-7dbb-912a-ed5dc41d162b@email.com. You save this suffixed ID in your system. Then, before detokenizing, strip off the @email.com.

However, using a format-preserving template means that you get a token ID that you don't have to manipulate. You can achieve this with a function like this:

function base64ToHex(str) {
  const raw = atob(str);
  let result = '';
  for (let i = 0; i < raw.length; i++) {
    const hex = raw.charCodeAt(i).toString(16);
    result += (hex.length === 2 ? hex : '0' + hex);
  }
  return result;
}

exports.email = {
  type: 'format_preserving_template',
  description: 'Generate an ID in email format.',
  async handler({ seed }) {
    return base64ToHex(seed) + '@email.com';
  }
}

Or, to preserve the email domain after tokenization:

function base64ToHex(str) {
  const raw = atob(str);
  let result = '';
  for (let i = 0; i < raw.length; i++) {
    const hex = raw.charCodeAt(i).toString(16);
    result += (hex.length === 2 ? hex : '0' + hex);
  }
  return result;
}

exports.email = {
  type: 'format_preserving_template',
  description: 'Generate an ID in email format.',
  async handler({ seed, fields }) {
    // Validate the input
    if (
      typeof fields !== 'object' ||
      !('email' in fields) ||
      typeof fields.email !== 'string'
    ) {
      throw new Error('Invalid input');
    }

    const email = fields.email;

    // Preserve the email domain.
    const domain = email.split('@')[1];

    return base64ToHex(seed) + '@' + domain;
  }
}

In both cases, a token with the format-preserving ID is created like this:

pvault token create -c buyers --type randomized --object-id cc9a39c5-4734-4786-b317-e16705d5128f --fptemplate bundle1.email

You get a response similar to this:

+-----------------------------------------------------------+
|                         token_id                          |
+-----------------------------------------------------------+
| 0fffd453f1d646b401f0fffd0fffd62490fffd5c120fffd@email.com |
+-----------------------------------------------------------+