Richard Clayton
Stumbling my way through the great wastelands of enterprise software development.
Thoughts on using Kafka with Node.js (node-rdkafka)
Let me start by saying, node-rdkafka is a godsend. When we first started using it, the library was the only one fully compatible with the latest version of Kafka and the SSL and SASL features. I owe webmakersteve and other contributors all a six-pack of beer for making this possible (thank you!!!!).
With that said, I guarantee you . . .
Posted in: kafkamicroservicesnodejs
Avoid Synchronous Functions in Node.js
Sync Functions can dramatically slow down your app and they are almost never necessary.
tl;dr
Most synchronous methods in Node.js (statSync, readFileSync, etc.) have asynchronous alternatives (via callback). Using util.promisify can easily make these methods async/await compatible. Using the Promise-based equivalents can greatly speed up your application. In fact, I'd argue there's never a need for the . . .
Posted in: nodejstypescript
Distributed Locking with Postgres Advisory Locks
TL;DR
Postgres Advisory Locks are a great solution for arbitrary application locks, particularly in scenarios where you are already using Postgres and have a need to block concurrent mutations to a resource (that resource DOES NOT have to be data in Postgres).
Concurrency is hard
We ran into a problem . . .
Posted in: microservicesnodejspostgres
WebSequenceDiagrams.com TypeScript Client
https://gist.github.com/rclayton-the-terrible/493cd0811542ff9693ac02746517ba71
TL;DR - TypeScript Client Gist: https://gist.github.com/rclayton-the-terrible/493cd0811542ff9693ac02746517ba71
If you are not familiar with WebSequenceDiagrams, it's an excellent tool for creating sequence diagrams for planning software flows. If you are not familiar with sequence diagrams, a quick Google search will . . .
Posted in: architecturenodejstypescript
Node.js and Kafka in 2018
Yes, Node.js has support for all of the Kafka features you need.
If you want to use Node and Kafka together and are struggling to find straightforward answers to some basic questions, I hope this helps you out. We've been building out an event system at Peachjar using Node and Kafka and some of this knowledge is the fruit of our research.
What are the popular Kafka clients in Node?
- . . .
Use State Machines!
FSMs are not as complex as you think and they make your code better.
What is a State Machine?
Finite State Machines (FSM, or in the context of this post, simply "State Machines") are a methodology for modeling the behavior of an entity with an established lifecycle. The lifecycle is defined by an enumerated set of states known at the time of implementation (this is where the term . . .
Posted in: architecturenodejs
Scheduling Execution in Node.js
Differences between setTimeout, setImmediate, and process.nextTick.
A commonly misunderstood concept in Node.js is the difference between setTimeout, setImmediate, and process.nextTick. While all three functions are used to schedule the execution of a function, it is important to understand when Node.js will execute them.
Consider the following example:
setImmediate(() => console.log('Set . . .Posted in: nodejs
