Skip to main content

Detokenization

Detokenization refers to the process by which non-sensitive token identifiers are replaced with the original token data represented by those tokens. Basis Theory supports detokenization through the use of expressions within our serverless Reactor platform and the Proxy.

Detokenization is performed whenever a detokenization expression is identified within a request. In their simplest form, these are Liquid objects of the form {{ token: <tokenId> }}. This expression will be replaced with the token with ID <tokenId>.

Any string value contained in a detokenization expression is expected to represent the id of a token within your tenant. If no such token is found with that identifier, the request will be rejected with a 400 error.

If a token is found with the given identifier, but the calling application is missing permission to use that token, then the request will be rejected with a 403 error. See the following links for more information about the permissions required to use tokens in a Reactor or Proxy.

Token data can also be transformed before including it in a request by applying filters within an expression. In general, filters are applied during detokenization using one of the following syntaxes:

  • Full Token Expressions
    • {{ token: <tokenId> | <filter1> | <filter2> | ... }}
    • Filters able to compute on any token property
  • Data only Expressions
    • {{ <tokenId> | <filter1> | <filter2> | ... }}
    • Filters only able to compute on data
  • Token Intent Expressions
    • {{ token_intent: <tokenIntentId> | <filter1> | <filter2> | ... }}
    • Filters able to compute on any token intent property

Check out the detokenization examples below to see some examples of filters in action.

Detokenizing Primitive Data​

Tokens containing primitive data values, such as a single string or numeric value, can be easily detokenized within a request just by substituting the token data in place of the detokenization expression. For example, say you have the token:

{
  "id": "1d08babf-456a-4bef-993d-aece3c1a2f66",
  "type": "social_security_number",
  "data": "111-22-3333"
}

Then a request containing the detokenization expression:

{
  "ssn": "{{ token: 1d08babf-456a-4bef-993d-aece3c1a2f66 | json: '$.data' }}"
}

will be detokenized into the request:

{
  "ssn": "111-22-3333"
}

Detokenizing Complex Data​

Card, Bank, or generic tokens containing complex JSON objects can all be detokenized as well. Detokenization of complex token data is performed by embedding the token's JSON data within the request. For example, say you have the card token:

{
  "id": "5c20545b-52dc-4a60-b9b5-5b7c84f22369",
  "type": "card",
  "data": {
    "number": "4242424242424242",
    "expiration_month": 12,
    "expiration_year": 2025,
    "cvc": "123"
  }
}

Then you can embed the entire Card object within a request with the detokenization expression:

{
  "card": "{{ token: 5c20545b-52dc-4a60-b9b5-5b7c84f22369 | json: '$.data' }}"
}

which will be detokenized into the request:

{
  "card": {
    "number": "4242424242424242",
    "expiration_month": 12,
    "expiration_year": 2025,
    "cvc": "123"
  }
}

Using Token Properties​

In addition to the token's data, all properties of a token can be used within Detokenization Expressions. For example, say you have the card token:

{
  "id": "5c20545b-52dc-4a60-b9b5-5b7c84f22369",
  "type": "card",
  "data": {
    "number": "4242424242424242",
    "expiration_month": 12,
    "expiration_year": 2025,
    "cvc": "123"
  },
  "metadata": {
    "name": "John Doe"
  },
  "enrichments": {
    "bin_details": {
      // ...
      "card_brand": "Visa"
      // ...
    }
  }
}

The following example shows how to use the card_brand from the BIN Details Enrichment and the name from metadata:

{
  "card_brand": "{{ token: 5c20545b-52dc-4a60-b9b5-5b7c84f22369 | json: '$.enrichments.bin_details.card_brand' }}",
  "card_holder_name": "{{ token: 5c20545b-52dc-4a60-b9b5-5b7c84f22369 | json: '$.metadata.name' }}"
}

which will be detokenized into the request:

{
  "card_brand": "Visa",
  "card_holder_name": "John Doe"
}

Data-only Expressions​

These expressions are an ease of use addition to our Expression language, making it simpler to get access to just the token data instead of the full token results. Here are a few examples:

The following expressions are equivalent:

  {{ 815029c2-29ec-4fc2-8cd4-99feb3ee582c }}
  {{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data' }}
  {{ 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.number' }}
  {{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data.number' }}

Examples​

Each of the use cases shown below include an example for both Reactors and the Proxy - select the appropriate tab at the top of the right-hand column to choose which examples to show.

Prerequisites​

For the examples below, we will proxy requests to Spreedly using their Payment Methods API. This is the same request that is being made from within the Spreedly - Card Reactor.

This API endpoint accepts POST requests with a body of the form:

{
  "payment_method": {
    "credit_card": {
      "number": "4242424242424242",
      "month": "12",
      "year": "2025",
      "verification_value": "123",
      "full_name": "Card Owner"
    },
    "retained": true
  }
}

and it authenticates using basic auth - we will be passing a simulated value in the Authorization header on the example proxy requests - replace this with your own authentication credentials if you want to follow along.

Use Complex Tokens​

In this example, we show how you can use a Card token to create a Spreedly payment method.

Say you have created the card token:

{
  "id": "815029c2-29ec-4fc2-8cd4-99feb3ee582c",
  "type": "card",
  "data": {
    "number": "4242424242424242",
    "expiration_month": 11,
    "expiration_year": 2025,
    "cvc": "123"
  }
}

We have not tokenized the card owner's name, John Doe. Assume that we have this plaintext value directly available to our application to pass into the Reactor or Proxy request.

Original Request​

curl "https://api.basistheory.com/proxy" \
    -H "BT-API-KEY: <PRIVATE_API_KEY>" \
    -H "BT-PROXY-URL: https://core.spreedly.com/v1/payment_methods.json" \
    -H "Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" \
    -H "Content-Type: application/json" \
    -X "POST" \
    -d '{
        "payment_method": {
            "credit_card": {
                "number": "{{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data.number'}}",
                "month": "{{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data.expiration_month'}}",
                "year": "{{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data.expiration_year'}}",
                "verification_value": "{{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data.cvc'}}",
                "full_name": "John Doe"
            },
            "retained": true
        }
    }'

Detokenized Request​

{
  "payment_method": {
    "credit_card": {
      "number": "4242424242424242",
      "month": "11",
      "year": "2025",
      "verification_value": "123",
      "full_name": "John Doe"
    },
    "retained": true
  }
}

Override a Field on a Token​

In this example, we show how you can use a Card token to create a Spreedly payment method, but provide an updated CVC (987) that is different from the cvc value stored within the token. This could be desired if the updated CVC was collected directly from a user interface, possibly as a challenge to the user to prove they own the card.

Here, we will be using the same card token and card owner name from the previous Use Complex Tokens example.

Original Request​

curl "https://api.basistheory.com/proxy" \
      -H "BT-API-KEY: <PRIVATE_API_KEY>" \
      -H "BT-PROXY-URL: https://core.spreedly.com/v1/payment_methods.json" \
      -H "Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" \
      -H "Content-Type: application/json" \
      -X "POST" \
      -d '{
            "payment_method": {
              "credit_card": {
                "number": "{{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data.number'}}"
                "month": "{{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data.expiration_month'}}",
                "year": "{{ token: 815029c2-29ec-4fc2-8cd4-99feb3ee582c | json: '$.data.expiration_year'}}",
                "verification_value": "987",
                "full_name": "John Doe"
              },
              "retained": true
            }
          }'

Detokenized Request​

{
  "payment_method": {
    "credit_card": {
      "number": "4242424242424242",
      "month": "11",
      "year": "2025",
      "verification_value": "987",
      "full_name": "John Doe"
    },
    "retained": true
  }
}

Use Multiple Tokens​

In this example, we will show how you can use multiples tokens - a Card Number token and a generic PII token to hold the card owner's name:

{
  "id": "d9939ddc-d7be-423b-a0f5-69f65fec57df",
  "type": "card_number",
  "data": "5555555555554444",
  "containers": ["/pci/high/"]
}
{
  "id": "f4d86311-1254-4155-b532-b651279a8cc0",
  "type": "token",
  "data": "Jane Doe",
  "containers": ["/pii/moderate/"]
}

In this example, we have not tokenized the card's expiration date or CVC - say our application accepts these values in plaintext and forwards them directly into the request.

Original Request​

curl "https://api.basistheory.com/proxy" \
      -H "BT-API-KEY: <PRIVATE_API_KEY>" \
      -H "BT-PROXY-URL: https://core.spreedly.com/v1/payment_methods.json" \
      -H "Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" \
      -H "Content-Type: application/json" \
      -X "POST" \
      -d '{
            "payment_method": {
              "credit_card": {
                "number": "{{ token: d9939ddc-d7be-423b-a0f5-69f65fec57df | json: '$.data' }}",
                "month": "10",
                "year": "2024",
                "verification_value": "789",
                "full_name": "{{ token: f4d86311-1254-4155-b532-b651279a8cc0 | json: '$.data' }}"
              },
              "retained": true
            }
          }'

Detokenized Request​

{
  "payment_method": {
    "credit_card": {
      "number": "5555555555554444",
      "month": "10",
      "year": "2024",
      "verification_value": "789",
      "full_name": "Jane Doe"
    },
    "retained": true
  }
}

Combine Multiple Tokens within a Single Argument​

In this example, we will show how you can combine the data from multiple tokens within a single field on a request. Say we have chosen to store the card holder's first and last names as separate tokens:

{
  "id": "523949a9-e32f-4b5b-a0ad-7a435c79deb4",
  "type": "token",
  "data": "John"
}
{
  "id": "42af9170-e6ca-4ea7-a43b-730a0b47b6d0",
  "type": "token",
  "data": "Brown"
}

Also, we have the card token:

{
  "id": "b78b4bee-5499-42dd-8671-f1d23d32355b",
  "type": "card",
  "data": {
    "number": "5105105105105100",
    "expiration_month": 5,
    "expiration_year": 2025,
    "cvc": "123"
  }
}

Then we can concatenate the first and last name tokens within the request as shown in the sample requests.

Original Request​

curl "https://api.basistheory.com/proxy" \
      -H "BT-API-KEY: <PRIVATE_API_KEY>" \
      -H "BT-PROXY-URL: https://core.spreedly.com/v1/payment_methods.json" \
      -H "Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" \
      -H "Content-Type: application/json" \
      -X "POST" \
      -d '{
            "payment_method": {
              "credit_card": {
                "number": "{{token: b78b4bee-5499-42dd-8671-f1d23d32355b | json: '$.data.number'}}",
                "month": "{{token: b78b4bee-5499-42dd-8671-f1d23d32355b | json: '$.data.expiration_month'}}",
                "year": "{{token: b78b4bee-5499-42dd-8671-f1d23d32355b | json: '$.data.expiration_year'}}",
                "verification_value": "{{token: b78b4bee-5499-42dd-8671-f1d23d32355b | json: '$.data.cvc'}}",
                "full_name": "{{523949a9-e32f-4b5b-a0ad-7a435c79deb4}} {{42af9170-e6ca-4ea7-a43b-730a0b47b6d0}}" // Data only expression
              },
              "retained": true
            }
          }'

Detokenized Request​

{
  "payment_method": {
    "credit_card": {
      "number": "5105105105105100",
      "month": "5",
      "year": "2025",
      "verification_value": "123",
      "full_name": "John Brown"
    },
    "retained": true
  }
}

Using Custom Token Schemas​

In this example, we will store our card data within a custom generic token that contains additional fields relevant to our application:

{
  "id": "9a48a051-972b-4569-8fd5-cbe17a604f96",
  "type": "token",
  "data": {
    "card": {
      "card_number": "4000056655665556",
      "exp_month": 4,
      "exp_year": 2026,
      "owner": {
        "first_name": "John",
        "middle_name": "Andrew",
        "last_name": "Smith"
      }
    },
    "billing_address": {
      "street_address": "175 5th Ave",
      "city": "New York",
      "state": "NY",
      "zip": "10010"
    }
  },
  "containers": ["/pci/high/"]
}

Notice that the card owner's full name is constructed by concatenating the card.owner.first_name and card.owner.last_name properties into a single string value.

Original Request​

curl "https://api.basistheory.com/proxy" \
      -H "BT-API-KEY: <PRIVATE_API_KEY>" \
      -H "BT-PROXY-URL: https://core.spreedly.com/v1/payment_methods.json" \
      -H "Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" \
      -H "Content-Type: application/json" \
      -X "POST" \
      -d '{
            "payment_method": {
              "credit_card": {
                "number": "{{token: 9a48a051-972b-4569-8fd5-cbe17a604f96 | json: '$.data.card.card_number'}}",
                "month": "{{token: 9a48a051-972b-4569-8fd5-cbe17a604f96 | json: '$.data.card.exp_month'}}",
                "year": "{{token: 9a48a051-972b-4569-8fd5-cbe17a604f96 | json: '$.data.card.exp_year'}}",
                "full_name": "{{token: 9a48a051-972b-4569-8fd5-cbe17a604f96 | json: '$.data.card.owner.first_name'}} {{token: 9a48a051-972b-4569-8fd5-cbe17a604f96 | json: '$.data.card.owner.last_name'}}"
              },
              "retained": true
            }
          }'

Detokenized Request​

{
  "payment_method": {
    "credit_card": {
      "number": "4000056655665556",
      "month": "4",
      "year": "2026",
      "full_name": "John Smith"
    },
    "retained": true
  }
}

Transforming Token Data​

In this example, we will be using the card token:

{
  "id": "67d76cd0-bc2a-4a73-be8c-6539c81db465",
  "type": "card",
  "data": {
    "number": "4242424242424242",
    "expiration_month": 3,
    "expiration_year": 2025,
    "cvc": "123"
  }
}

We will transform the expiration_month property within this token to be padded on the left with a 0 to two characters, and to only provide the last two digits of the expiration year.

Original Request​

curl "https://api.basistheory.com/proxy" \
      -H "BT-API-KEY: <PRIVATE_API_KEY>" \
      -H "BT-PROXY-URL: https://core.spreedly.com/v1/payment_methods.json" \
      -H "Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" \
      -H "Content-Type: application/json" \
      -X "POST" \
      -d '{
            "payment_method": {
              "credit_card": {
                "number": "{{token: 67d76cd0-bc2a-4a73-be8c-6539c81db465 | json: '$.data.card_number'}}",
                "month": "{{token: 67d76cd0-bc2a-4a73-be8c-6539c81db465 | json: '$.data.expiration_month' | pad_left: 2, '0'}}",
                "year": "{{token: 67d76cd0-bc2a-4a73-be8c-6539c81db465 | json: '$.data.expiration_year' | to_string | slice: -2, 2}}",
                "full_name": "John Doe"
              },
              "retained": true
            }
          }'

Detokenized Request​

{
  "payment_method": {
    "credit_card": {
      "number": "4000056655665556",
      "month": "03",
      "year": "25",
      "full_name": "John Doe"
    },
    "retained": true
  }
}