Hi Frontrunners,

 

I was just carrying out some testing on the PM Bulk Usecase and noticed the following behaviour. When multiple events are sent to Ves Collector in quick succession, the datafile collector can sometimes pull all these events down in the one message from message router but only one of these events are handled by DFC.

 

The snippet below shows FileReady events for sampleFile5,6,7,8 are consumed in the one message by DFC but only sampleFile5 is published. I have also attached the logs. 

 

 
2018-11-27 10:25:09.524 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.d.s.DmaapConsumerJsonParser      : original message from message router: ["{\"event\":{\"commonEventHeader\":{\"startEpochMicrosec\":8745745764578,\"eventId\":\"FileReady_1797490e-10ae-4d48-9ea7-3d7d790b25e1\",\"timeZoneOffset\":\"UTC+05.30\",\"internalHeaderFields\":

{\"collectorTimeStamp\":\"Tue, 11 27 2018 10:24:30 UTC\"}

,\"priority\":\"Normal\",\"version\":\"4.0.1\",\"reportingEntityName\":\"otenb5309\",\"sequence\":0,\"domain\":\"notification\",\"lastEpochMicrosec\":8745745764578,\"eventName\":\"Noti_RnNode-Ericsson_FileReady\",\"vesEventListenerVersion\":\"7.0.1\",\"sourceName\":\"oteNB5309\"},\"notificationFields\":{\"notificationFieldsVersion\":\"2.0\",\"changeType\":\"FileReady\",\"changeIdentifier\":\"PM_MEAS_FILES\",\"arrayOfNamedHashMap\":{\"name\":\"sampleFile5.gz\",\"hashMap\":{\"location\":\"s[ftp://admin:admin@10.42.236.38:22/sampleFile5.gz\",\"fileFormatType\":\"org.3GPP.32.435#measCollec\",\"fileFormatVersion\":\"V10\",\"compression\":\"gzip\"}}}}}%22,%22{%22event%22:{%22commonEventHeader%22:{%22startEpochMicrosec%22:8745745764578,%22eventId%22:%22FileReady_1797490e-10ae-4d48-9ea7-3d7d790b25e1%22,%22timeZoneOffset%22:%22UTC+05.30%22,%22internalHeaderFields%22:

{\%22collectorTimeStamp\%22:\%22Tue], 11 27 2018 10:24:30 UTC\"}

,\"priority\":\"Normal\",\"version\":\"4.0.1\",\"reportingEntityName\":\"otenb5309\",\"sequence\":0,\"domain\":\"notification\",\"lastEpochMicrosec\":8745745764578,\"eventName\":\"Noti_RnNode-Ericsson_FileReady\",\"vesEventListenerVersion\":\"7.0.1\",\"sourceName\":\"oteNB5309\"},\"notificationFields\":{\"notificationFieldsVersion\":\"2.0\",\"changeType\":\"FileReady\",\"changeIdentifier\":\"PM_MEAS_FILES\",\"arrayOfNamedHashMap\":{\"name\":\"sampleFile6.gz\",\"hashMap\":{\"location\":\"s[ftp://admin:admin@10.42.236.38:22/sampleFile6.gz\",\"fileFormatType\":\"org.3GPP.32.435#measCollec\",\"fileFormatVersion\":\"V10\",\"compression\":\"gzip\"}}}}}%22,%22{%22event%22:{%22commonEventHeader%22:{%22startEpochMicrosec%22:8745745764578,%22eventId%22:%22FileReady_1797490e-10ae-4d48-9ea7-3d7d790b25e1%22,%22timeZoneOffset%22:%22UTC+05.30%22,%22internalHeaderFields%22:

{\%22collectorTimeStamp\%22:\%22Tue], 11 27 2018 10:24:30 UTC\"}

,\"priority\":\"Normal\",\"version\":\"4.0.1\",\"reportingEntityName\":\"otenb5309\",\"sequence\":0,\"domain\":\"notification\",\"lastEpochMicrosec\":8745745764578,\"eventName\":\"Noti_RnNode-Ericsson_FileReady\",\"vesEventListenerVersion\":\"7.0.1\",\"sourceName\":\"oteNB5309\"},\"notificationFields\":{\"notificationFieldsVersion\":\"2.0\",\"changeType\":\"FileReady\",\"changeIdentifier\":\"PM_MEAS_FILES\",\"arrayOfNamedHashMap\":{\"name\":\"sampleFile8.gz\",\"hashMap\":{\"location\":\"s[ftp://admin:admin@10.42.236.38:22/sampleFile8.gz\",\"fileFormatType\":\"org.3GPP.32.435#measCollec\",\"fileFormatVersion\":\"V10\",\"compression\":\"gzip\"}}}}}%22,%22{%22event%22:{%22commonEventHeader%22:{%22startEpochMicrosec%22:8745745764578,%22eventId%22:%22FileReady_1797490e-10ae-4d48-9ea7-3d7d790b25e1%22,%22timeZoneOffset%22:%22UTC+05.30%22,%22internalHeaderFields%22:

{\%22collectorTimeStamp\%22:\%22Tue], 11 27 2018 10:24:30 UTC\"}

,\"priority\":\"Normal\",\"version\":\"4.0.1\",\"reportingEntityName\":\"otenb5309\",\"sequence\":0,\"domain\":\"notification\",\"lastEpochMicrosec\":8745745764578,\"eventName\":\"Noti_RnNode-Ericsson_FileReady\",\"vesEventListenerVersion\":\"7.0.1\",\"sourceName\":\"oteNB5309\"},\"notificationFields\":{\"notificationFieldsVersion\":\"2.0\",\"changeType\":\"FileReady\",\"changeIdentifier\":\"PM_MEAS_FILES\",\"arrayOfNamedHashMap\":{\"name\":\"sampleFile7.gz\",\"hashMap\":{\"location\":\"s[ftp://admin:admin@10.42.236.38:22/sampleFile7.gz\",\"fileFormatType\":\"org.3GPP.32.435#measCollec\",\"fileFormatVersion\":\"V10\",\"compression\":\"gzip\"}}}}}%22]]
2018-11-27 10:25:09.525 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.d.s.DmaapConsumerJsonParser      : starting to getJsonObjectFromAnArray!
2018-11-27 10:25:09.525 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.d.s.DmaapConsumerJsonParser      : starting to getFileDataFromJson!
2018-11-27 10:25:09.525 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.d.tasks.XnfCollectorTaskImpl     : Entering execute with FileData

{changeIdentifier=PM_MEAS_FILES, changeType=FileReady, name=sampleFile5.gz, location=s[ftp://admin:admin@10.42.236.38:22/sampleFile5.gz], compression=gzip, fileFormatType=org.3GPP.32.435#measCollec, fileFormatVersion=V10}

2018-11-27 10:25:09.525 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.d.tasks.XnfCollectorTaskImpl     : starting to collectFile
2018-11-27 10:25:09.525 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.datafile.ftp.FtpsClient          : collectFile called with fileServerData: FileServerData{serverAddress=10.42.236.38, userId=admin, password=admin, port=22}, remoteFile: /sampleFile5.gz, localFile: target/sampleFile5.gz
2018-11-27 10:25:09.525 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.datafile.ftp.FtpsClient          : retryCollectFile called
2018-11-27 10:25:11.140 DEBUG 1 — [reactor-http-client-epoll-7] o.o.d.c.datafile.ftp.FtpsClient          : File sampleFile5.gz Download Successfull from xNF
2018-11-27 10:25:11.141 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.datafile.ftp.FtpsClient          : retryCollectFile left with result: Download successful: true Error data: 
2018-11-27 10:25:11.141 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.d.tasks.XnfCollectorTaskImpl     : Exiting execute with ConsumerDmaapModel{name=sampleFile5.gz, location=target/sampleFile5.gz, compression=gzip, fileFormatType=org.3GPP.32.435#measCollec, fileFormatVersion=V10}
2018-11-27 10:25:11.141 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.d.tasks.DmaapPublisherTaskImpl   : Method called with arg ConsumerDmaapModel{name=sampleFile5.gz, location=target/sampleFile5.gz, compression=gzip, fileFormatType=org.3GPP.32.435#measCollec, fileFormatVersion=V10}
2018-11-27 10:25:11.141 TRACE 1 — [reactor-http-client-epoll-7] .c.d.s.p.DmaapProducerReactiveHttpClient : Entering getDmaapProducerResponse with ConsumerDmaapModel{name=sampleFile5.gz, location=target/sampleFile5.gz, compression=gzip, fileFormatType=org.3GPP.32.435#measCollec, fileFormatVersion=V10}
2018-11-27 10:25:11.142 TRACE 1 — [reactor-http-client-epoll-7] .c.d.s.p.DmaapProducerReactiveHttpClient : base64Creds...: ZHJhZG1pbjpkcmFkbWlu
2018-11-27 10:25:11.144 TRACE 1 — [reactor-http-client-epoll-7] .c.d.s.p.DmaapProducerReactiveHttpClient : Starting to publish to DR
2018-11-27 10:25:11.152 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : ===========================request begin================================================
2018-11-27 10:25:11.152 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : URI         : https://dmaap-dr-prov:8443/publish/1/sampleFile5.gz
2018-11-27 10:25:11.152 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : Method      : PUT
2018-11-27 10:25:11.152 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : Headers     : {Accept=[text/plain, application/json, application/*+json, */*], Content-Type=[application/octet-stream], X-ATT-DR-META=[

{"compression":"gzip","fileFormatType":"org.3GPP.32.435#measCollec","fileFormatVersion":"V10"}

], Authorization=[Basic ZHJhZG1pbjpkcmFkbWlu], Content-Length=[524288]}
2018-11-27 10:25:11.169 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : ==========================request end================================================
2018-11-27 10:25:11.206 DEBUG 1 — [reactor-http-client-epoll-7] o.o.d.c.d.web.PublishRedirectStrategy    : Redirect requested to location 'https://dmaap-dr-node:8443/publish/1/sampleFile5.gz'
2018-11-27 10:25:11.206 TRACE 1 — [reactor-http-client-epoll-7] o.o.d.c.d.web.PublishRedirectStrategy    : getRedirect...: PUT https://dmaap-dr-prov:8443/publish/1/sampleFile5.gz HTTP/1.1
2018-11-27 10:25:11.339 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : ============================response begin==========================================
2018-11-27 10:25:11.339 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : Status code  : 204 NO_CONTENT
2018-11-27 10:25:11.339 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : Status text  : No Content
2018-11-27 10:25:11.339 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : Headers      :

{Date=[Tue, 27 Nov 2018 10:25:11 GMT], Strict-Transport-Security=[max-age=2000; includeSubDomains], X-ATT-DR-PUBLISH-ID=[1543314311263.dmaap-dr-node], Server=[Jetty(9.3.8.RC0)]}

2018-11-27 10:25:11.339 DEBUG 1 — [reactor-http-client-epoll-7] .c.d.w.RequestResponseLoggingInterceptor : =======================response end=================================================
2018-11-27 10:25:11.339  INFO 1 — [reactor-http-client-epoll-7] o.o.d.c.datafile.tasks.ScheduledTasks    : Datafile consumed tasks. HTTP Response code 204 NO_CONTENT
2018-11-27 10:25:11.339  INFO 1 — [reactor-http-client-epoll-7] o.o.d.c.datafile.tasks.ScheduledTasks    : Datafile tasks have been completed
 

 

We are not sure if this is expected behaviour or not and would appreciate your opinion on this.

 

Kind Regards,

Conor