-
Task
-
Resolution: Done
-
Medium
-
None
Due to expected high data volume that is going to be sent to DMaaP, it has been decided to extend DMaaPClient to use Apache Kafka API.
It means that DMaaPClient is going to connect directly to Apache Kafka for publishing/reading messages to/from topics.
DMaapClient using/wrapping REST API will be left for backward compatibility.
Tasks (draft):
1 Check version of Apache Kafka/Zookeper used in DMaaP currently, upgrade if needed
2 Add dependency in DMaaP Client for Apache Kafka API Client correcpoding to Kafka version
3 Add Apache Kafka Producer using native API
4 Add Apache Kafka Consumer using native API
5 Check security possibility for TLS in Apache Kafka
6 Check security possibility for authorization in Apache Kafka
7 Check security possibility for authentication in Apache Kafka
8 Check how to use AAF in DMaaP
9 Data partitioning design and usage
10 Usage of several brokers - how to
11