When Testing vLB ScaleOut, it seems that a change in timestamp format is making the interface with APPC fail (for HealthchecBB)

SO sends the following request to APPC:

020-12-01T10:45:09.553Z|69e67743-c9dc-462a-bce1-b3fef7758c19|o.o.s.b.i.flowspecific.tasks.GenericVnfHealthCheck - Running APP-C action: HealthCheck
2020-12-01T10:45:09.553Z|69e67743-c9dc-462a-bce1-b3fef7758c19|o.o.s.b.i.flowspecific.tasks.GenericVnfHealthCheck - VNFID: 5bc2493b-5047-4c5d-bc60-e578c5b1f927
2020-12-01T10:45:09.553Z|69e67743-c9dc-462a-bce1-b3fef7758c19|o.onap.so.client.appc.ApplicationControllerSupport - LCM Kit input message follows: {
{{ "common-header" : {}}
"timestamp" : "2020-12-01T10:45:09.553353Z",
{{ "api-ver" : "2.00",}}
{{ "originator-id" : "MSO",}}
{{ "request-id" : "69e67743-c9dc-462a-bce1-b3fef7758c19",}}
{{ "sub-request-id" : "02f42f96-66c8-440a-a163-d57184b75598",}}
{{ "flags" : {}}
{{ "mode" : "NORMAL",}}
{{ "force" : "FALSE",}}
{{ "ttl" : 65000}}
{{ }}}
{{ },}}
{{ "action" : "HealthCheck",}}
{{ "action-identifiers" : {}}
{{ "vnf-id" : "5bc2493b-5047-4c5d-bc60-e578c5b1f927"}}
{{ },}}
{{ "payload" : "{\"request-parameters\":{}}"}}
}

 

Which is then responded with :

{{2020-12-01T10:45:10.836Z||o.o.a.client.impl.protocol.APPCMessageReaderWriter - Received body : <{"output":{"common-header":{"timestamp":"2020-12-01T10:45:09.553353Z","originator-id":"MSO","request-id":"69e67743-c9dc-462a-bce1-b3fef7758c19","sub-request-id":"02f42f96-66c8-440a-a163-d57184b75598","flags":

{"mode":"NORMAL","ttl":65000,"force":"FALSE"}

,"api-ver":"2.00"},"status":color:#FF0000}"message":"REQUEST PARSING FAILED - Unparseable date: \"2020-12-01T10:45:09.553353Z\"","code":303}}}>}}

 

Looking at old request, I think that before, timestamps were set to ms :
DMAAP-source-APPC-LCM-WRITE][IN|DMAAP|APPC-LCM-WRITE] {"body":{"output":{"common-header":{"api-ver":"2.00","flags":{},"originator-id":"f0713429-4326-481c-a0a8-f0e59b4f0518","request-id":"f0713429-4326-481c-a0a8-f0e59b4f0518","sub-request-id":"2",*"timestamp":"2019-05-22T23:59:59.494Z"},*"status":{"code":401,"message":"Failure in retrieving VNF Hierachy"},"cambria.partition":"MSO","correlation-id":"f0713429-4326-481c-a0a8-f0e59b4f0518-2","rpc-name":"restart","type":"response"}
 

Moreover, this condition seems to lock the infra request somehow, with the following errors being pushed into SO logs :

 

020-12-01T10:45:11.578Z|69e67743-c9dc-462a-bce1-b3fef7758c19|org.camunda.bpm.engine.context - ENGINE-16004 Exception while closing command context: ENGINE-03083 Exception while executing Batch Database Operations with message '
### Error flushing statements. Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn=338920) Cannot delete or update a parent row: a foreign key constraint fails (`camundabpmn`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `act_ge_bytearray` (`ID_`))
### Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn=338920) Cannot delete or update a parent row: a foreign key constraint fails (`camundabpmn`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `act_ge_bytearray` (`ID_`))
java.sql.BatchUpdateException: (conn=338920) Cannot delete or update a parent row: a foreign key constraint fails (`camundabpmn`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `act_ge_bytearray` (`ID_`))
java.sql.SQLIntegrityConstraintViolationException: (conn=338920) Cannot delete or update a parent row: a foreign key constraint fails (`camundabpmn`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `act_ge_bytearray` (`ID_`))
java.sql.SQLException: Cannot delete or update a parent row: a foreign key constraint fails (`camundabpmn`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `act_ge_bytearray` (`ID_`))
Query is: delete from ACT_GE_BYTEARRAY where ID_ = '3ed80abd-33c2-11eb-a651-aa03d12a6f6e'
java thread: MariaDb-bulk-4
{{'. Flush summary: }}
{{ [...}}

 

Attaching full log for investigation