Better data validation in TypeScript

Data validation tends to get out of hand and become difficult to maintain and understand. In this article, we explore a better way by using our open-source library
Shlomi Borovitz

May 18 2020 · 5 min read

SHARE ON

Better data validation in TypeScript

The Problem at Hand

It’s a fact that user input cannot be trusted and must always be validated. Yet, code like in the example below is too common. Although seemingly it returns the type we expect, in reality, it’s just lying to the type system and to ourselves!

const body = req.body as T;

We do not know what the body contains, and while wishing it to be of a specific type — we do not know this to be true.

Moreover, unlike static languages, JS does not have type-aware deserializers that validate correctness and validity (not even at the type level).

And, while JSON schema does allow us to validate inputs, it is difficult to understand and does not translate intuitively to TypeScript types. In complex scenarios, it is difficult to tell what is considered legitimate input and what is not. It is also difficult to figure out why a certain value did not pass one of the schema validations.

Usually, APIs that parse JSON and input would return the any type, on the false assumption that the developer knows what the input is — but when dealing with user-input, the developer does not (though we usually assume we do).

Example: The lunch delivery store

Suppose we are having an online service that allows its customers to place orders for their lunch meal: a pizza or a hamburger.

Clients may POST an Order:

interface Order {
    address: string
    servings: Serving[]
}

interface Serving {
    quantity: number
    dish: FoodOrder
}

type FoodOrder =
    | Pizza
    | Hamburger

interface Pizza {
    type: 'pizza'
    toppings: Topping[]
    size: PizzaSize
}

interface Hamburger {
    type: 'hamburger'
    doneness: Doneness
    tomatos?: boolean
    lettuce?: boolean
    pickles?: boolean
}

interface Topping {
    type: ToppingType
    cover: ToppingCover
}
type ToppingType =
    | 'olives'
    | 'onions'
    | 'bell-peppers'
    | 'chili'
    | 'pepperoni'

type ToppingCover =
    | '1st-half'
    | '2nd-half'
    | 'all'

type PizzaSize =
    | 'S'
    | 'M'
    | 'L'
    | 'XL'
    | 'XXL'

type Doneness =
    | 'R'
    | 'MR'
    | 'M'
    | 'MW'
    | 'WD'

And our handler function should parse an order and process it:

async function handler(req: Request) {
    const order = req.body as Order
    await processOrder(order)
}

As mentioned before, assuming the request body is an Order object is a mistake. A mistake that a hacker may exploit and might cause unforeseen consequences.

Using Type-Guards

TypeScript’s solution to runtime checking for the shape of a value is the user-defined type-guard: A boolean function which tells the type-checker what types a value can have in its true branch, and in its false branch.

While type-guards can bridge the gap between runtime type-checking and compile-time type-checking — it is on the developer to implement the runtime check, and thus, may require repeated code, that may or may not validate the input rigorously:

// Is this a rigorous input validation?
function isOrder(value: any): value is Order {
    return value !== null &&
        typeof value === 'object' &&
        typeof value.address === 'string' &&
        Array.isArray(value.servings)
}

async function handler(req: Request) {
    const order: unknown = req.body

    if (!isOrder(order)) {
        throw new Error('Invalid order')
    }

    await processOrder(order)
}

While this approach is much better than the “I trust all my users blindfolded” approach — it demands long and repetitive code, that in the end is not necessarily trustworthy.

Better Type-Guards with @altostra/type-validations

Neither JavaScript nor TypeScript makes it easy to validate user input (beyond the simplest examples), TypeScript makes it possible to have compile-time type-checking that is bound to runtime checks. We published an open-source library on NPM@altostra/type-validations that helps you easily compose complex data validators.

Let’s rewrite our type-guard from the previous example, but this time using the type-validations library:

import { arrayOf, objectOf, primitives } from '@altostra/type-validations'

const isOrder = objectOf({
    address: primitives.string,
    servings: arrayOf(primitives.any),
})

async function handler(req: Request) {
    const order: unknown = req.body;

    if (!isOrder(order)) {
        throw new Error('Invalid order')
    }

    await processOrder(order)
}

Now, the unknown request body is inferred correctly to be an Order type:

Inferred type for `order`
Inferred type for order

The inferred type of order variable is not exactly the Order type, but the type we actually tested for.

Coding an Order

Having the tools to describe types efficiently, we can now add the Order properties types, starting with the simplest and the best: Topping.

The Topping type is just an object with two string properties:

const isTopping = objectOf({
    // Specifying the type for enumOf would protect against specifying invalid values.
    // but it won't force you to specify all values (nor consider unspecified values as valid)
    type: enumOf<ToppingType>(
        'olives',
        'onions',
        'bell-peppers',
        'chili',
        'pepperoni'
    ),
    cover: enumOf<ToppingCover>('1st-half', '2nd-half', 'all')
})

While we can use the generic primitives.string type-validation for the type and cover properties, there is only a subset of valid string values these two properties can accept.

Having the Toppings, we can add the Pizza:

// Specifying a type for objectOf would protect you from forgetting or misspelling properties.
// It would also help TS to better infer `enumOf` and `is` types.
const isPizza = objectOf<Pizza>({
    // The `is` function creates a type-guard which validates a literal type, that is, a specific value.
    type: is('pizza'),
    toppings: arrayOf(isTopping),
    size: enumOf<PizzaSize>('S', 'M', 'L', 'XL', 'XXL')
})

Then the Hamburger in a similar way:

const isHamburger = objectOf<Hamburger>({
    type: is('hamburger'),
    doneness: enumOf<Doneness>(
        'R',
        'MR',
        'M',
        'MW',
        'WD'
    ),
    lettuce: primitives.maybeBoolean,
    tomatos: primitives.maybeBoolean,
    pickles: primitives.maybeBoolean,
})

And having all the food on the menu, we can finally code the rest of our order:

// `anyOf` creates a union of the provided types,
// which is the exact definition of `FoodOrder`
const isFoodOrder = anyOf(isPizza, isHamburger)

const isServing = objectOf<Serving>({
    quantity: primitives.number,
    dish: isFoodOrder,
})

// Is this a rigorous input validation?
export const isOrder = objectOf<Order>({
    address: primitives.string,
    servings: arrayOf(isServing)
})

Is the input-validation rigorous? Well, it is much more rigorous than our first try— but we can do better.

For example, quantity must be a positive integer. After all, NaN is a number but not quantity.

const isFoodOrder = anyOf(isPizza, isHamburger)

const isServing = objectOf<Serving>({
    // isQuantity is just a function that return boolean.
    // IRL, all type-guards are (but TS doesn't know).
    quantity: isQuantity,
    dish: isFoodOrder,
})

export const isOrder = objectOf<Order>({
    address: string,
    servings: arrayOf(isServing)
})

function isQuantity(value: unknown) {
    return typeof value === 'number' &&
        value > 0 &&
        Number.isInteger(value)
}

And this time, order is rightfully inferred to be an Order:

Inferred Order type
Inferred Order type

Handling Rejections

Now that we have our web-server, we can validate all user-input without assuming anything about it.

Later, our enthusiast mobile developer implemented a frontend application, and for a while, everything was well. But only for a while. After an update, more and more users are reporting that they’re failing to place an order. The logs are showing an increased number of 400 status code being returned to clients, saying "Invalid order". But why?

If the update included a small change, it could be reverted, or for the very least, inspected. But what if the update was a big refactor or a new version? Fortunately, all type-guards created by @altostra/type-validations can take a second argument — a function that would be called (possibly multiple times) if a value fails validation — with a reason for the rejection.

Can you spot the reason for the error?

async function handler(req: Request) {
    // Is this an order?
    const order: unknown = {
        address: '127.0.0.1',
        servings: [{
            quantity: 1,
            dish: {
                type: 'pizza',
                size: 'XL',
                toppings: [{
                    type: 'onions',
                    cover: 'all',
                }, {
                    type: 'chilli',
                    cover: '1st-half',
                }, {
                    type: 'bell-peppers',
                    cover: '2nd-half'
                },]
            }
        }]
    }

    if (!isOrder(order, console.log)) {
        throw new Error('Invalid order')
    }

    await processOrder(order)
}

From the validation output below, we can see that the value 'chilli' does not equal to any of the toppings. Careful inspection reveals that 'chilli' does not equal to 'chili', and that of course is the cause of the problem.

Using the path property, we can find where the offending property is, and trace its source.

{
  path: [ 'type', 1, 'toppings', 'dish', 0, 'servings' ],
  reason: "Value <'chilli'> is not equal to <'olives'>",
  propertyType: "'olives'"
}
{
  path: [ 'type', 1, 'toppings', 'dish', 0, 'servings' ],
  reason: "Value <'chilli'> is not equal to <'onions'>",
  propertyType: "'onions'"
}
{
  path: [ 'type', 1, 'toppings', 'dish', 0, 'servings' ],
  reason: "Value <'chilli'> is not equal to <'bell-peppers'>",
  propertyType: "'bell-peppers'"
}
{
  path: [ 'type', 1, 'toppings', 'dish', 0, 'servings' ],
  reason: "Value <'chilli'> is not equal to <'chili'>",
  propertyType: "'chili'"
}
{
  path: [ 'type', 1, 'toppings', 'dish', 0, 'servings' ],
  reason: "Value <'chilli'> is not equal to <'pepperoni'>",
  propertyType: "'pepperoni'"
}
{
  path: [ 'type', 'dish', 0, 'servings' ],
  reason: "Value <'pizza'> is not equal to <'hamburger'>",
  propertyType: "'hamburger'"
}

And selling back (buying a negative quantity of) hamburgers will not work either:

async function handler(req: Request) {
    // Is this an order?
    const order: Order = {
        address: '127.0.0.1',
        servings: [{
            quantity: -10,
            dish: {
                type: 'hamburger',
                doneness: 'MW',
            }
        }]
    }

    /* Errors with
     * {
     * path: [ 'quantity', 0, 'servings' ],
     * reason: 'Value [-10] failed validation',
     * propertyType: '* (isQuantity)'
     * }
     */
    if (!isOrder(order, console.log)) {
        throw new Error('Invalid order')
    }

    await processOrder(order)
}

Combining Everything with Assertions

Having backends with many endpoints can lead to repetition in both collection and logging rejections, and handling the invalid value itself (usually by throwing an error).

On the other hand, since TypeScript 3.7, control-flow analysis can use the so-called “assertion functions”, which assumed to return (and not throw) only if the type of the parameter is known.

Given that there is a general way to handle validation errors (e.g. log the rejections using your preferred logger, and throw BadRequest error), it could be encapsulated into a function.

We can then use that function to create per-type assertion, and use it to clean even more our code.

// An assertion must be explicitly typed in order to be used.
// The `Assertion<T>` type can be used for that.
const validateOrder: Assertion<Order> = assertBy(
    isOrder,
    invalidInputErrorFactory
)

async function handler(req: Request) {
    const order: unknown = req.body

    validateOrder(order)

    await processOrder(order)
}

function invalidInputErrorFactory(value: unknown, rejections: ValidationRejection[]): any {
    logger.warn('Invalid input', { value, rejections })
    // We return the error to be thrown
    return new FancyBadRequestError('Invalid request', { value, rejections })
}

In Conclusion

We’ve seen how by using @altostra/type-validations we create clean and clear code and yet it validates its input rigorously. If you feel that something is missing or can be improved, feel free to open issues and pull requests.

You can find the source code here https://github.com/altostra/type-validations, and on NPM https://www.npmjs.com/package/@altostra/type-validations

All code snippets from the post can be found here https://github.com/altostra/lunch-delivery-store

By submitting this form, you are accepting our Terms of Service and our Privacy Policy

Thanks for subscribing!

Ready to Get Started?

Request a Demo

Copyright © 2020 Altostra. All rights reserved.