Node VM

6-minute read
nodejs • javascript • development • CLI

I recently harnessed the power of Node.js’ vm module within a CLI tool to elevate its configurability. This upgrade empowers both me and fellow users to effortlessly inject custom JavaScript code via command line flags, expanding our horizons of flexibility.

This CLI tool, designed for internal use, communicates with a REST API that yields JSON responses. The API effectively functions as a database, offering its objects through REST endpoints while accepting an array of query operators as query parameters to filter specific objects.

I was content with using jq to manipulate JSON data, but then I discovered the vm module and decided to experiment with it by integrating it into my CLI tool. This decision was further encouraged by the fact that most people on my team are unfamiliar with jq, whereas almost everyone is well-versed in JavaScript.

The vm module

One of the remarkable features of the vm module is its ability to create isolated execution contexts. This means that user code cannot interfere with the main application, ensuring stability and security. It’s a game-changer when dealing with untrusted code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

const vm = require('vm');

// Create a context
const context = new vm.createContext();
const localVar = 34

// Define a variable within the context
context.myVariable = 42;

// Execute code within the context
const result = vm.runInContext('myVariable * 2', context);

console.log(result); // Output: 84

vm.createContext() is kind of like a sandbox where we run the actual code. It can’t access the variables defined in our code unless we allow it by passing in via the createContext constructor. Replacing myVariable * 2 with localVar * 2 on line number 11 won’t work as vm can’t access localVar.

Understanding createContext()

The vm.createContext() function is used to create a new JavaScript context or execution environment. A context is like a sandbox where you can run JavaScript code without affecting the global scope. When you create a context, it initially contains no variables, functions, or objects, making it an isolated environment.

While vm.createContext() establishes a clean slate, you’ll often want to execute code within that context. The vm.runInContext() function allows you to do precisely that. It runs JavaScript code within a specified context, providing isolation from the global scope.

To understand what I mean by that, let’s look at following piece of code:

1
2
3
4
5
6
7
8
9

const vm = require('vm');

const myVar = 'hello';
const script = new vm.Script('myVar');
const context = new vm.createContext();

const val = script.runInContext(context);

console.log(val);

This is going to error out with ‘myvar not defined’. It’s because myvar doesn’t exist within the context we created.

In order to make it work, we need to pass it to the context.

1
2
3
4
5
6
7
8
9

const vm = require('vm');

const myVar = 'hello';
const script = new vm.Script('myVar');
const context = new vm.createContext({ myvar });

const val = script.runInContext(context);

console.log(val);

This will work since the value of myVar is picked up from the context as we are passing it in the vm.createContext method.

So now that we have a basic understanding of how it works. Here’s how the transformation methods for our CLI tool are defined:

Transformation class 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import vm from 'node:vm';

export default class Transform {
  static createTransformFunction(data, operation, code, initialValue) {
    const script = new vm.Script(
      `(data) => data.${operation}(${code}${initialValue ? `, ${initialValue}` : ''})`,
    );
    const context = new vm.createContext({ data });
    return script.runInContext(context);
  }

  static filter(data, code) {
    const filterFunction = this.createTransformFunction(data, 'filter', code);
    return filterFunction(data);
  }

  static map(data, code) {
    const filterFunction = this.createTransformFunction(data, 'map', code);
    return filterFunction(data);
  }
}

This code defines a JavaScript class called Transform that is designed to make working with arrays and data transformation more flexible. It leverages the Node.js vm (Virtual Machine) module to execute custom JavaScript code within a controlled context. Let’s break down the key components and functionalities:

The code begins by importing the vm module, which provides a way to run JavaScript code in a separate execution context.

  • createTransformFunction Method:
1
2
3

static createTransformFunction(data, operation, code, initialValue) {
// ...
}

This method is a versatile utility for creating transformation functions. It accepts four parameters:

  1. data: The input data or array to be transformed.

  2. operation: The type of array operation to apply (e.g., “filter,” “map”).

  3. code: The custom JavaScript code that defines how the operation should be performed.

  4. initialValue (optional): An initial value to be used with some array operations (e.g., “reduce”).

Inside the method, a new JavaScript vm.Script is created. This script defines an anonymous function that encapsulates the specified array operation (operation) and custom JavaScript code (code) provided as parameters. A context is established using vm.createContext, with the data parameter included. This context isolates the execution environment for security and stability. The vm.Script is executed within the established context using script.runInContext(context). This allows the provided JavaScript code to operate on the data within a controlled environment.

Finally, the transformed result is returned.

Using the Transformer Class

Let’s take a look at how the Transformer class is used within my CLI tool:

1
2
3
4
5
6
7
8

// Fetching data from the API
const rawData = fetchSomeData();

// Applying a filter operation using the Transformer class
const filteredData = Transformer.filter(rawData, 'item => item.status === "completed"');

// Displaying the filtered data
console.log(filteredData);

In this example, I’ve fetched some data from the API and then applied a filter operation using the Transformer class. The provided JavaScript code snippet within the filter operation defines the filtering criteria, allowing me to extract only the data that matches the specified condition.

Passing Filters via the CLI

This can can further be integrated with something like Commander. Now, users can pass filter functions directly through the command line, customizing data retrieval on the fly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

import { Command } from 'commander';
import Transformer from './Transformer.js';

const program = new Command();

program
  .command('filter')
  .option('-f, --filter <filter>', 'JavaScript filter function')
  .action((cmd) => {
    const { filter } = cmd;

    const rawData = fetchDataFromAPI();

    // Apply the user-provided filter using the Transformer class
    const filteredData = Transformer.filter(rawData, filter);

    console.log(filteredData);
  });

program.parse(process.argv);

You can add find, map, reduce and other transformation functions along the similar lines and then you can invoke it like this.

./your-cli.js filter -f "item => item.status === 'completed'"

This command will apply the provided filter function to the data fetched from your API using the Transformer class and display the filtered results.

Conclusion

Of course, there are more refined approaches available. My preferred method is using jq, a powerful tool for manipulating JSON data. I rely on jq extensively to slice and dice JSON data effortlessly.

While it may seem like overkill or not the primary purpose of the vm module, it offers several advantages.It’s ideal for those who may not be familiar with jq (although I highly recommend learning it) or prefer to use JavaScript for all tasks. Moreover, since these transformation functions are part of your codebase, the possibilities are limitless.