If you want to use Node and Kafka together and are struggling to find straightforward answers to some basic questions, I hope this helps you out. We've been building out an event system at Peachjar using Node and Kafka and some of this knowledge is the fruit of our research.

What are the popular Kafka clients in Node?

• node-rdkafka wraps the librdkafka library, which is widely used by the Kafka community, particularly as the basis of clients in other languages.
• kafka-node is a pure JavaScript implementation of the Kafka protocol.

Is Kafka 2.0 Supported?

Yes, for both clients. I've been testing with node-rdkafka using the Confluent Enterprise Kafka 5.0 Docker image: https://hub.docker.com/r/confluentinc/cp-enterprise-kafka/

I am 99.9% certain this is also true for kafka-node whose documentation says it supports v0.8 and above.

Can I use SSL and SASL with Kafka?

Absolutely; SSL is supported by both brokers. In terms of authentication, SASL_PLAIN is supported by both, and I believe node-rdkafka theoretically supports "GSSAPI/Kerberos/SSPI, PLAIN, SCRAM" by virtue of passing configuration directly to librdkafka (but I haven't personally verified this).

In node-rdkafka, it's as simple as:

import { KafkaConsumer } from 'node-rdkafka'
import { hostname } from 'os'

const consumer = new KafkaConsumer({
  // My syntax highlighter doesn't support template strings...🙁
   'client.id':  ['myserver', hostname()].join('-'),
   'group.id': 'myconsumegroup',
   'rebalance_cb': true,
   'metadata.broker.list': 'localhost:9091',
   'security.protocol': 'SASL_SSL',
   'sasl.mechanisms': 'PLAIN',
   'sasl.username': 'myclient',
   'sasl.password': 'clientpassword',
   // Note: librdkafka does not access a string with the certificate/key
   'ssl.key.location': './etc/security/client.key',
   // This is only necessary if your key file is protected with a password
   'ssl.key.password': 'mysupersecretpassword',
   'ssl.certificate.location': './etc/security/client.certificate.pem',
   'ssl.ca.location': './etc/security/ca.pem',
}, {
   'auto.offset.reset': 'beginning',
})

Managing Kafka is scary, are there good hosted options?

Yaaaas.

• Confluent Cloud: https://www.confluent.io/confluent-cloud/
• Aiven Kafka: https://aiven.io/kafka
• Cloudkarafka: https://www.cloudkarafka.com
• Heroku Kafka: https://www.heroku.com/kafka)

Confluent Cloud I'm going to assume is the gold standard. This is a service offering by the founders and core contributors of Kafka. Unfortunately, I can't tell you much about it because I've never had a chance to use it. I'm assuming they are targeting big enterprises because getting a simple price from the company requires you to go through an army of sales engineers who want to "talk about your Kafka implementation."

Aiven Kafka and Cloudkarafka are straightforward, multi-cloud Kafka providers. Aiven specifically is easy to set up and use in AWS, GCP, Azure, and Digital Ocean. From the documentation, it appears Cloudkarafka is hosted in AWS, but it may also support other public clouds (IDK). One distinguishing feature of Aiven was the ability to store messages indefinitely (other offerings forced users into 30-day event storage limits unless they upgraded to "enterprise plans"). Either way, I do enjoy the pricing models both vendors CLEARLY DISPLAY on their marketing pages.

To be fair to Cloudkarafka, we didn't research the offering as thoroughly as Aiven. Aiven has other database offerings (specifically Postgres with the TimescaleDB extension) that really convinced us.

Finally, if you are on the Heroku platform, it probably makes sense to use Heroku Kafka. Heroku has a great reputation for providing stable products, so I imagine the same goes for this offering.

Conclusion

I hope this answers some of your questions. Kafka can be daunting for the uninitiated but is certainly worth the effort. More importantly, Node.js is a completely viable language for using the Kafka broker. If you do plan on choosing Kafka, consider using one of the hosted options. Confluent Cloud is probably the safest bet, but it's considerably more expensive. Aiven is a great alternative, especially if you are just testing out the platform.