Ever wanted to pass the path to a TypeScript file to some other library or module maybe to run it in a separate process or in a worker thread. You think you can just write the file in JavaScript, but you found out you will need to import another one or two of your TypeScript files beacuse it needs a function from it. Then I guess you have to refactor your whole codebase to use JavaScript instead, or not, if you read through this.

The problem is, using require with a TypeScript file won’t work, because Node.js does not and cannot handle .ts files. The extensions handled by the require function by default are .js, .mjs, .node, .json. The libaray or module you are passing the file path to would eventually require it at runtime and even if you add .ts to require.extensions, it would only resolve properly, but you get a syntax error on execution. This means sending it a TypeScript .ts file won’t work, require will choke on this.

import { Worker } from 'worker_threads'

const worker = new Worker('./path/to/typescript/worker.ts')

At runtime in the worker_threads module it would probably look somewhat like this

class Worker {
  constructor(filename) {
    const mod = require(filename)
  }
}

Well, the implementation is quite different, but my point is; filename will eventually go through require maybe not in this same process.

The magic

The only option is to precompile your TypeScript files, know where the compiled file would be output to before it would be compiled, then pass it the path. But what if you use a runtime like ts-node, which compiles on the fly and run the compiled files in-memory without emitting? There’s no way to do this, except:

The JavaScript worker file

A base JavaScript file to portal every worker file written in TypeScript to. In this file we register a ts-node runtime then go on to require the file passed to it through the workerData.aliasModule. Since we’ve registered the ts-node runtime on the file, it’s safe to import a TypeScript file using the require function. The file that workerData.aliasModule points to is a TypeScript file.

const { workerData } = require('worker_threads')

require('ts-node').register()
require(workerData.aliasModule)

The TypeScript worker file

The module containing the code to be run on a worker thread, which is actually written in TypeScript. All the code to be run on the worker thread is written in this TypeScript file which would later be required by worker.js. This file would be compiled on the fly when required and executed from worker.js.

const { parentPort, workerData } = require('worker_threads')

parentPort.postMessage(`Post back: ${workerData.whatever}`)

The worker entry point

This is the main file that needs to run a job on a worker thread. It begins the whole worker thread thingy.

import path from 'path'
import { Worker } from 'worker_threads'

const worker = new Worker('./worker.js', {
  workerData: {
    aliasModule: path.resolve(__dirname, 'worker.ts'), // worker.js uses this
    whatever: 'Hello, worker bee! The Queen greets you.', // worker.ts uses this
  },
})

worker.on('message', (message: string) => {
  console.log(message) // Post back: Hello, worker bee! The Queen greets you.
})

Most of the magic is done by ts-node using the require('ts-node').register() which registers the loader for future requires. The most beautiful thing about this magic is that you can dynamically set the module to load, because of the way the modules are structured. Therefore uisng worker.js for future workers but running different code in it is possible.

Re-creating the magic with a job queue like Bull

If you ever used a job queue in a Node.js application or more specifically, Bull, you will know you sometimes have to run a job in a different process (child process) from the main one (parent process). Bull lets you specify the path to the file or filename that conatins the code to process the job. Whenever you pass a file to queue.process, Bull knows to process that job in a different process.

In the case where a job processor is CPU intensive, it could stall the Node.js event loop and this could lead to double processing a job. Processing jobs on a separate process could prevent double processing it. Processing the job on a separate process would also make sure the main process doesn’t terminate even when the job process terminates maybe due to a runtime error.

The same problem as with worker threads happens here again if we are using TypeScript. We can’t do:

queue.process('./path/to/typescript/process-job.ts')

As we did with the worker thread example, although may not be as dynamic as that, we could do the same here also.

We create the queue and add a job to be processed to it. We then specify the code file that processes the job off of the queue. Bull will run this code file in a separate process, but it can’t handle TypeScript files.

import Bull from 'bull'

const queue = new Bull<IData>('job-queue', options)

queue.add('job-name', data)

queue.process('job-name', './path/to/processor.js')

Using the ts-node register method as before, we register a loader to be used for future require, then load the TypeScript code file, compile it and run it. Bull picks the top level export (default export or unnamed export) from module.exports and invokes it with the job object containing info sepcific to the job and the data, sent from queue.add, to be processed.

require('ts-node').register()
require('./processor.ts')

The processor.ts file is the file containing the original code to process the job.

export default async function (job: Bull.Job<IData>) {
  // do something with job.data
}