Hopefully, at this juncture, you are very well aware of Kafka Producer APIs, their internal working, and common patterns of publishing messages to different Kafka topics. This section covers some of the best practices associated with Kafka producers. These best practices will help you in making some of the design decisions for the producer component.
Let's go through some of the most common best practices to design a good producer application:
- Data validation: One of the aspects that is usually forgotten while writing a producer system is to perform basic data validation tests on data that is to be written on the Kafka cluster. Some such examples could be conformity to schema, not null values for Key fields, and so on. By not doing data validation, you are risking breaking downstream consumer applications and affecting the load balancing of brokers as data...