Hi

 

We are currently testing DFC with a real node. When we leave DFC running over a period of 12 hours we keep seeing this error in ONAP.

 

018-10-31 21:38:19.422 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.tasks.DmaapConsumerTaskImpl    : execute called with arg

2018-10-31 21:38:19.422 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : Request: GET http://message-router:3904/events/unauthenticated.VES_NOTIFICATION_OUTPUT/OpenDCAE-c12/C12

2018-10-31 21:38:19.422  INFO 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : Content-Type=application/json

2018-10-31 21:38:19.422 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : HTTP request headers: {Content-Type=[application/json]}

2018-10-31 21:38:19.422 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.tasks.DmaapConsumerTaskImpl    : consume called with arg MonoFlatMap

2018-10-31 21:38:25.063 TRACE 1 — [http-nio-8100-exec-4] o.o.d.c.d.c.HeartbeatController          : Receiving heartbeat request

2018-10-31 21:38:34.425 TRACE 1 — [pool-2-thread-1] o.o.d.c.datafile.tasks.ScheduledTasks    : Execution of tasks was registered

2018-10-31 21:38:34.426 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.tasks.DmaapConsumerTaskImpl    : execute called with arg

2018-10-31 21:38:34.426 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : Request: GET http://message-router:3904/events/unauthenticated.VES_NOTIFICATION_OUTPUT/OpenDCAE-c12/C12

2018-10-31 21:38:34.426  INFO 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : Content-Type=application/json

2018-10-31 21:38:34.426 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : HTTP request headers: {Content-Type=[application/json]}

2018-10-31 21:38:34.426 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.tasks.DmaapConsumerTaskImpl    : consume called with arg MonoFlatMap

2018-10-31 21:38:34.465 ERROR 1 — [reactor-http-client-epoll-32] r.n.resources.PooledConnectionProvider   : [id: 0xf8ddacf7, L:/10.42.37.201:58550 ! R:message-router/10.43.153.192:3904] Pooled connection observed an error

 

java.io.IOException: Connection closed prematurely

        at reactor.netty.http.client.HttpClientOperations.onInboundClose(HttpClientOperations.java:248)

        at reactor.netty.channel.ChannelOperationsHandler.channelInactive(ChannelOperationsHandler.java:121)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)

        at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)

        at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)

        at io.netty.handler.codec.http.HttpContentDecoder.channelInactive(HttpContentDecoder.java:205)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)

        at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)

        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelInactive(CombinedChannelDuplexHandler.java:420)

        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)

        at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)

        at io.netty.handler.codec.http.HttpClientCodec$Decoder.channelInactive(HttpClientCodec.java:282)

        at io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:223)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)

        at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)

        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1429)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)

        at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:947)

        at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)

        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)

        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)

        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:315)

        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)

        at java.lang.Thread.run(Thread.java:748)

 

2018-10-31 21:38:40.072 TRACE 1 — [http-nio-8100-exec-5] o.o.d.c.d.c.HeartbeatController          : Receiving heartbeat request

2018-10-31 21:38:49.427 TRACE 1 — [pool-2-thread-1] o.o.d.c.datafile.tasks.ScheduledTasks    : Execution of tasks was registered

2018-10-31 21:38:49.429 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.tasks.DmaapConsumerTaskImpl    : execute called with arg

2018-10-31 21:38:49.429 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : Request: GET http://message-router:3904/events/unauthenticated.VES_NOTIFICATION_OUTPUT/OpenDCAE-c12/C12

2018-10-31 21:38:49.429  INFO 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : Content-Type=application/json

2018-10-31 21:38:49.429 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient       : HTTP request headers: {Content-Type=[application/json]}

2018-10-31 21:38:49.430 TRACE 1 — [pool-2-thread-1] o.o.d.c.d.tasks.DmaapConsumerTaskImpl    : consume called with arg MonoFlatMap

2018-10-31 21:38:49.440 ERROR 1 — [reactor-http-client-epoll-36] r.n.resources.PooledConnectionProvider   : [id: 0x5b4267a1, L:/10.42.37.201:59248 ! R:message-router/10.43.153.192:3904] Pooled connection observed an error

 

java.io.IOException: Connection closed prematurely

        at reactor.netty.http.client.HttpClientOperations.onInboundClose(HttpClientOperations.java:248)

        at reactor.netty.channel.ChannelOperationsHandler.channelInactive(ChannelOperationsHandler.java:121)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)

        at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)

 

 

 

What we notice is that for roughly the first 4 hours after node starts sending events everything is fine. Then it keeps throwing the above error over and over again without getting any new messages off the topic.

So any help into what his happening here would be great thanks.

 

 

Also is the files coming off the node supposed to be stored under the “/target/” location ?

 

Datafile collector image: nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:1.0.3

 

Thanks

Paul Dennehy P