Details
-
Bug
-
Status: Closed
-
High
-
Resolution: Done
-
Honolulu Release
-
Any environment with Kafka using snappy for compression.
-
Policy 2022-07-27->2022-08-10
Description
The Kafka plugin available in apex-pdp fails to load the snappy compression library, as it uses the default directory /tmp for extracting and installing the lib. And java is not able to load the classes from /tmp directory.
Log snippet
Exception in thread "Apex-org.onap.policy.apex.plugins.event.carrier.kafka.ApexKafkaConsumer:XYZ_NodeUpdateConsumer-58:0" java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy at org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:435) at org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:466) at java.base/java.io.DataInputStream.readByte(DataInputStream.java:270) at org.apache.kafka.common.utils.ByteUtils.readVarint(ByteUtils.java:168) at org.apache.kafka.common.record.DefaultRecord.readFrom(DefaultRecord.java:293) at org.apache.kafka.common.record.DefaultRecordBatch$1.readNext(DefaultRecordBatch.java:264) at org.apache.kafka.common.record.DefaultRecordBatch$RecordIterator.next(DefaultRecordBatch.java:569) at org.apache.kafka.common.record.DefaultRecordBatch$RecordIterator.next(DefaultRecordBatch.java:538) at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.nextFetchedRecord(Fetcher.java:1448) at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.fetchRecords(Fetcher.java:1483) at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.access$1600(Fetcher.java:1328) at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:641)
Possible solution
Update the Kafka version to 3.0.0 which transitively brings 1.1.8 snappy which has the fix and provides an argument flag to load the lib from a given directory structure.
Istanbul onwards the Kafka version was already upgraded to 3.0.0, we only need to get it upgraded in Honolulu version.