Richard Clayton

Thoughts on using Kafka with Node.js (node-rdkafka)

Let me start by saying, node-rdkafka is a godsend. When we first started using it, the library was the only one fully compatible with the latest version of Kafka and the SSL and SASL features. I owe webmakersteve and other contributors all a six-pack of beer for making this possible (thank you!!!!).

With that said, I guarantee you . . .

Read More

April 01, 2020

Avoid Synchronous Functions in Node.js

Sync Functions can dramatically slow down your app and they are almost never necessary.


Most synchronous methods in Node.js (statSync, readFileSync, etc.) have asynchronous alternatives (via callback). Using util.promisify can easily make these methods async/await compatible. Using the Promise-based equivalents can greatly speed up your application. In fact, I'd argue there's never a need for the . . .

Read More

Posted in: nodejstypescript

February 29, 2020

Distributed Locking with Postgres Advisory Locks


Postgres Advisory Locks are a great solution for arbitrary application locks, particularly in scenarios where you are already using Postgres and have a need to block concurrent mutations to a resource (that resource DOES NOT have to be data in Postgres).

Concurrency is hard

We ran into a problem . . .

Read More

February 16, 2020 TypeScript Client

TL;DR - TypeScript Client Gist:

If you are not familiar with WebSequenceDiagrams, it's an excellent tool for creating sequence diagrams for planning software flows. If you are not familiar with sequence diagrams, a quick Google search will . . .

Read More

June 22, 2019

Node.js and Kafka in 2018

Yes, Node.js has support for all of the Kafka features you need.

If you want to use Node and Kafka together and are struggling to find straightforward answers to some basic questions, I hope this helps you out. We've been building out an event system at Peachjar using Node and Kafka and some of this knowledge is the fruit of our research.

    . . .

Read More

Posted in: kafkanodejs

October 31, 2018

Use State Machines!

FSMs are not as complex as you think and they make your code better.

What is a State Machine?

Finite State Machines (FSM, or in the context of this post, simply "State Machines") are a methodology for modeling the behavior of an entity with an established lifecycle. The lifecycle is defined by an enumerated set of states known at the time of implementation (this is where the term . . .

Read More

March 17, 2018

Scheduling Execution in Node.js

Differences between setTimeout, setImmediate, and process.nextTick.

A commonly misunderstood concept in Node.js is the difference between setTimeout, setImmediate, and process.nextTick. While all three functions are used to schedule the execution of a function, it is important to understand when Node.js will execute them.

Consider the following example:

setImmediate(() => console.log('Set . . .

Read More

Posted in: nodejs

March 14, 2018