Node VM

6-minute read
nodejs • javascript • programming • CLI

I recently used Node’s VM module with a CLI tool to allow passing custom JavaScript via command line flags. The tool interacts with a REST API that returns JSON responses.

So for instance, we have a users end point that returns users. It accepts some filters but sometimes you may want to filter or transform the output in some way.

    myclitool get users limit 10

    [
      { name: 'jon', email: 'jon@doe.cp.com', city: 'paris' ....  },
      { name: 'jane', email: 'jane.doe@yahoo.com', city: 'paris'....  },
      { name: 'tom', email: 'tom@yahoo.com', city: 'reykjavik'....  },
      { name: 'chad', email: 'chad@gmail.com', city: 'tokyo' ....  },
      ...
      ...
    ]

What if you wanted to filter out users belonging to paris? You could use jq for it but what if you could do the following:

    myclitool get users --filter "(user) => user.city === 'paris'"

The vm module

The vm module can create isolated execution contexts. This means that user code cannot interfere with the main application, ensuring stability and security. This is quite useful when you want to run some un-trusted code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

const vm = require('vm');

// Create a context
const context = new vm.createContext();
const localVar = 34

// Define a variable within the context
context.myVariable = 42;

// Execute code within the context
const result = vm.runInContext('myVariable * 2', context);

console.log(result); // Output: 84

vm.createContext() is kind of like a sandbox where we run the actual code. It can’t access the variables defined in our code unless we allow it by passing in via the createContext constructor. Replacing myVariable * 2 with localVar * 2 on line number 11 won’t work as vm can’t access localVar.

Understanding createContext()

The vm.createContext() function is used to create a new JavaScript context or execution environment. A context is like a sandbox where you can run JavaScript code without affecting the global scope. When you create a context, it initially contains no variables, functions, or objects, making it an isolated environment.

While vm.createContext() establishes a clean slate, you’ll often want to execute code within that context. The vm.runInContext() function allows you to do precisely that. It runs JavaScript code within a specified context, providing isolation from the global scope.

To understand what I mean by that, let’s look at following piece of code:

1
2
3
4
5
6
7
8
9

const vm = require('vm');

const myVar = 'hello';
const script = new vm.Script('myVar');
const context = new vm.createContext();

const val = script.runInContext(context);

console.log(val);

This is going to error out with ‘myvar not defined’. It’s because myvar doesn’t exist within the context we created.

In order to make it work, we need to pass it to the context.

1
2
3
4
5
6
7
8
9

const vm = require('vm');

const myVar = 'hello';
const script = new vm.Script('myVar');
const context = new vm.createContext({ myvar });

const val = script.runInContext(context);

console.log(val);

This will work since the value of myVar is picked up from the context as we are passing it in the vm.createContext method.

So now that we have a basic understanding of how it works. Here’s how the transformation methods for our CLI tool are defined:

Transformation class 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import vm from 'node:vm';

export default class Transform {
  static createTransformFunction(data, operation, code, initialValue) {
    const script = new vm.Script(
      `(data) => data.${operation}(${code}${initialValue ? `, ${initialValue}` : ''})`,
    );
    const context = new vm.createContext({ data });
    return script.runInContext(context);
  }

  static filter(data, code) {
    const filterFunction = this.createTransformFunction(data, 'filter', code);
    return filterFunction(data);
  }

  static map(data, code) {
    const filterFunction = this.createTransformFunction(data, 'map', code);
    return filterFunction(data);
  }
}

This code defines a JavaScript class called Transform that is designed to make working with arrays and data transformation more flexible. It leverages the Node.js vm (Virtual Machine) module to execute custom JavaScript code within a controlled context. Let’s break down the key components and functionalities:

The code begins by importing the vm module, which provides a way to run JavaScript code in a separate execution context.

  • createTransformFunction Method:
1
2
3

static createTransformFunction(data, operation, code, initialValue) {
// ...
}

This method is a versatile utility for creating transformation functions. It accepts four parameters:

  1. data: The input data or array to be transformed.

  2. operation: The type of array operation to apply (e.g., “filter,” “map”).

  3. code: The custom JavaScript code that defines how the operation should be performed.

  4. initialValue (optional): An initial value to be used with some array operations (e.g., “reduce”).

Inside the method, a new JavaScript vm.Script is created. This script defines an anonymous function that encapsulates the specified array operation (operation) and custom JavaScript code (code) provided as parameters. A context is established using vm.createContext, with the data parameter included. This context isolates the execution environment for security and stability. The vm.Script is executed within the established context using script.runInContext(context). This allows the provided JavaScript code to operate on the data within a controlled environment.

Finally, the transformed result is returned.

Using the Transformer Class

Let’s take a look at how the Transformer class is used within my CLI tool:

1
2
3
4
5
6
7
8

// Fetching data from the API
const rawData = fetchSomeData();

// Applying a filter operation using the Transformer class
const filteredData = Transformer.filter(rawData, 'item => item.status === "completed"');

// Displaying the filtered data
console.log(filteredData);

In this example, I’ve fetched some data from the API and then applied a filter operation using the Transformer class. The provided JavaScript code snippet within the filter operation defines the filtering criteria, allowing me to extract only the data that matches the specified condition.

Passing Filters via the CLI

This can can further be integrated with something like Commander. Now, users can pass filter functions directly through the command line, customizing data retrieval on the fly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

import { Command } from 'commander';
import Transformer from './Transformer.js';

const program = new Command();

program
  .command('filter')
  .option('-f, --filter <filter>', 'JavaScript filter function')
  .action((cmd) => {
    const { filter } = cmd;

    const rawData = fetchDataFromAPI();

    // Apply the user-provided filter using the Transformer class
    const filteredData = Transformer.filter(rawData, filter);

    console.log(filteredData);
  });

program.parse(process.argv);

You can add find, map, reduce and other transformation functions along the similar lines and then you can invoke it like this.

./your-cli.js filter -f "item => item.status === 'completed'"

This command will apply the provided filter function to the data fetched from your API using the Transformer class and display the filtered results.

Conclusion

While it may seem like overkill or not the primary purpose of the vm module, it offers several advantages. It’s ideal for those who may not be familiar with jq (although I highly recommend learning it) or prefer to use JavaScript for all tasks. Moreover, since these transformation functions are part of your codebase, the possibilities are limitless.