Typeorm-where

Typeorm-where is an NPM library I published to solve a problem I had while working with Typeorm.

Type

NPM package

Stack

• TypeScript
• Jest
• Node.js
• Typeorm

The Problem

Typeorm is one of the most mature TypeScript ORMs in the JavaScript ecosystem. I've been using it for over 4 years. However, its find methods are limited to only querying the equal operation when passing the FindOptions object. I wanted to be able to pass the kind of object structure used with the likes of Prisma to filter data.

Project Assumptions and Goals

My aim was to take an object structure like the one below as input and transform it into an object Typeorm understands.

{
     quantity: {
         $between: [5, 12],
    },
 }

Challenges and Thought Process

I was glad to find that typeorm already exports utility filter operator functions like Not, LessThan, MoreThan, etc.

My thought process was to create a mini DSL that was a simple map of keywords to these utility functions as shown below:

const OperatorMap = {
  $not: Not,
  $lessThan: LessThan,
  $lessThanOrEqual: LessThanOrEqual,
  $moreThan: MoreThan,
  $moreThanOrEqual: MoreThanOrEqual,
  $equal: Equal,
  $like: Like,
  $ilike: ILike,
  $between: Between,
  $in: In,
  $any: Any,
  $isNull: IsNull,
  $arrayContains: ArrayContains,
  $arrayContainedBy: ArrayContainedBy,
  $arrayOverlap: ArrayOverlap,
  $raw: Raw,
};

The next step was to take a filter input object and traverse the object tree, converting each keyword to the corresponding typeorm utility function call. It took me three days to write two functions that did this namely: compileToOperatorAst and operatorAstToFilter.

compileToOperatorAst

This function traverses the input filter tree and its sub-nodes recursively and transforms it into an intermediate abstract syntax tree that matches each operator keyword to the corresponding typeorm filter operator function as shown below:

function compileToOperatorAst<Entity>(filterMap: Filter<Entity>) {
  const ast = filterMap as Record<string, any>;
  const keys = Object.keys(ast);
  for (let i = 0; i < keys.length; i++) {
    const key = keys[i];
    if (key.startsWith('$')) {
      const operatorFunc = getFindOperator(key, ast[key]);
      ast[`___$operator_${i}`] = operatorFunc;

      delete ast[key];
    } else if (isObjectLiteral(ast[key])) {
      compileToOperatorAst(ast[key]);
    }
  }

  return ast;
}

operatorAstToFilter

This function takes the output from compileToOperatorAst and recursively traverses the abstract syntax tree to join all operators and their values. The final output is an object made up of entity properties as keys and typeorm utility operator functions as values.

function operatorAstToFilter(ast: Record<string, any>) {
  const keys = Object.keys(ast);
  for (let i = 0; i < keys.length; i++) {
    const key = keys[i];
    if (!key.startsWith('___$operator_')) {
      const subVal = ast[key];
      const isOperatorSubKeys = Object.keys(subVal)[0].startsWith('___$operator_');

      if (isOperatorSubKeys) {
        ast[key] = joinOperatorValues(Object.values(subVal));
        if (!subVal) {
          delete ast[key];
        }
      } else {
        operatorAstToFilter(subVal);
      }
    }
  }

  return ast;
}

The two functions above constitute the core of the library. As I mentioned earlier, it took me three days to figure out how to recursively traverse every node of an object tree with an arbitrary depth, and perform operations on its entries.

The feeling of making it work and writing end-to-end tests to bug-proof it was sublime.

Lessons

JavaScript assigns objects by reference, not by value

This presents a huge problem which I'm yet to figure out how to solve because it occurs by JavaScript's language design. What it means is that any object passed to typeorm-where no matter how many "copies" are made all get mutated into the final output since they all point to the same location in memory. My workaround for now is using lodash's cloneDeep function to make true copies of the input filter object when I need to use the same object for multiple filters. An example below:

const { filter } = dto;
    const regularFilter = cloneDeep(filter);
    const premiumFilter = cloneDeep(filter);
    const expeditedFilter = cloneDeep(filter);

    const totalApplications = await ApplicationEntity.count({
      where: where<ApplicationEntity>(filter),
    });

    const totalRegularApplications = await ApplicationEntity.count({
      where: where<ApplicationEntity>({
        ...(regularFilter ?? {}),
        serviceType: {
          $equal: ServiceType.REGULAR,
        },
      }),
    });

    const totalPremiumApplications = await ApplicationEntity.count({
      where: where<ApplicationEntity>({
        ...(premiumFilter ?? {}),
        serviceType: {
          $equal: ServiceType.PREMIUM,
        },
      }),
    });
    const totalExpeditedApplications = await ApplicationEntity.count({
      where: where<ApplicationEntity>({
        ...(expeditedFilter ?? {}),
        serviceType: {
          $equal: ServiceType.EXPEDITED,
        },
      }),
    });