Writing Custom Kafka Converter to Convert Debezium Source to Protobuf

postgres, debezium, kafka, zookeeper, consumer, producer

what is this

Problem

Debezium sends DB event to kafka in json schema but if you have kafka-consumer which expect protobuf schema then it will not work.But If you have kafka-consumer which expect json schema then it is not a problem.

debezium convert DB update event json and serialize into string before sending to kafka. So producer who expecting in protobuf schema will not work.

Debezium has out of box support for json schema.

architecture

postgres setup

kafka & zookeeper setup

kafka consumer setup json protobuf

debezium setup

CI/CD

docker setup