After completing Service instantiation, DB cannot be updated with COMPLETED

Hi, I notice that several times after sevice instantiated is completed by SO, the infra active request remains in IN_PROGRESS, and the so-bpmn logs shows that it received a failure when trying to update the requets in the DB with COMPLETE status.

I list below 2 examples:

one example showing error DB update and request status is stuck at "IN PROGRESS", preventing any further action on the instance,

and 2nd example showing good result and DB updated OK and request status is "COMPLETE"

Is this failure coming from SO DB or SDNC DB ?

Is there a way to overcome this failure in order to ensure consistent request status ?

 

SAMPLE SHOWING ERROR:

 

2019-04-28T06:31:53.663Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.client.RestClient - RestClientSSL using default SSL context!
2019-04-28T06:31:53.670Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| o.o.so.logging.jaxrs.filter.PayloadLoggingFilter - Making POST (overridden to PATCH) request to: https://aai.onap:8443/aai/v14/business/customers/customer/Demonstration/service-subscriptions/service-subscription/vFW/service-instances/service-instance/345d6bd7-cdf4-4e86-844e-8d4ca7c578ae
Request Headers:

{Authorization=[Basic YWFpQGFhaS5vbmFwLm9yZzpkZW1vMTIzNDU2IQ==], X-FromAppId=[MSO], X-TransactionId=[], X-HTTP-Method-Override=[PATCH], Accept=[application/json], Content-Type=[application/merge-patch+json], X-ONAP-RequestID=[c3b4a3ef-ace2-4d99-9447-43167ecb52bf], X-ONAP-InvocationID=[e57f22c6-64e1-48d3-98d5-55b95274825b], X-ONAP-PartnerName=[SO]}

2019-04-28T06:31:53.671Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| o.o.so.logging.jaxrs.filter.PayloadLoggingFilter - {"service-instance-id":"345d6bd7-cdf4-4e86-844e-8d4ca7c578ae","service-instance-name":"vFWNG_CDS_15","orchestration-status":"Active"}

2019-04-28T06:31:54.287Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.core.BPMNLogger - Entered SetSuccessIndicator Method
2019-04-28T06:31:54.288Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.core.BPMNLogger - Outgoing SuccessIndicator is: false
2019-04-28T06:31:54.297Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - CompleteMsoProcess Request: <aetgt:MsoCompletionRequest xmlns:aetgt="http://org.onap/so/workflow/schema/v1" xmlns:ns="http://org.onap/so/request/types/v1"><request-info xmlns="http://org.onap/so/infra/vnf-request/v1"><request-id>c3b4a3ef-ace2-4d99-9447-43167ecb52bf</request-id><action>createInstance</action><source>null</source></request-info><status-message>Macro-Service-createInstance request was executed correctly.</status-message><serviceInstanceId>345d6bd7-cdf4-4e86-844e-8d4ca7c578ae</serviceInstanceId><mso-bpel-name>WorkflowActionBB</mso-bpel-name></aetgt:MsoCompletionRequest>
2019-04-28T06:31:54.298Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - Incoming Request is: <aetgt:MsoCompletionRequest xmlns:aetgt="http://org.onap/so/workflow/schema/v1" xmlns:ns="http://org.onap/so/request/types/v1"><request-info xmlns="http://org.onap/so/infra/vnf-request/v1"><request-id>c3b4a3ef-ace2-4d99-9447-43167ecb52bf</request-id><action>createInstance</action><source>null</source></request-info><status-message>Macro-Service-createInstance request was executed correctly.</status-message><serviceInstanceId>345d6bd7-cdf4-4e86-844e-8d4ca7c578ae</serviceInstanceId><mso-bpel-name>WorkflowActionBB</mso-bpel-name></aetgt:MsoCompletionRequest>
2019-04-28T06:31:54.721Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - Incoming Instance Id Xml: <serviceInstanceId>345d6bd7-cdf4-4e86-844e-8d4ca7c578ae</serviceInstanceId>
2019-04-28T06:31:54.723Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - Outgoing Update Mso Request Payload is:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:req="http://org.onap.so/requestsdb">
<soapenv:Header/>
<soapenv:Body>
<req:updateInfraRequest>
<requestId>c3b4a3ef-ace2-4d99-9447-43167ecb52bf</requestId>
<lastModifiedBy>WorkflowActionBB</lastModifiedBy>
<statusMessage>Macro-Service-createInstance request was executed correctly.</statusMessage>
<requestStatus>COMPLETE</requestStatus>
<progress>100</progress>
<serviceInstanceId>345d6bd7-cdf4-4e86-844e-8d4ca7c578ae</serviceInstanceId>
</req:updateInfraRequest>
</soapenv:Body>
</soapenv:Envelope>
2019-04-28T06:31:54.725Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - setUpdateDBstatustoSuccessPayload:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:req="http://org.onap.so/requestsdb">
<soapenv:Header/>
<soapenv:Body>
<req:updateInfraRequest>
<requestId>c3b4a3ef-ace2-4d99-9447-43167ecb52bf</requestId>
<lastModifiedBy>WorkflowActionBB</lastModifiedBy>
<statusMessage>Macro-Service-createInstance request was executed correctly.</statusMessage>
<requestStatus>COMPLETE</requestStatus>
<progress>100</progress>
<serviceInstanceId>345d6bd7-cdf4-4e86-844e-8d4ca7c578ae</serviceInstanceId>
</req:updateInfraRequest>
</soapenv:Body>
</soapenv:Envelope>
2019-04-28T06:31:54.832Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - CompleteMsoProcess Response: onse xmlns:sdncadapterworkflow="http://org.onap/so/workflow/schema/v1">
<sdncadapterworkflow:out>BPEL WorkflowActionBB FAILED</sdncadapterworkflow:out>
</sdncadapterworkflow:MsoCompletionResponse>
2019-04-28T06:31:54.833Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - @@ CompleteMsoProcess Response @@

2019-04-28T06:31:54.836Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.ExceptionUtil - Building a WorkflowException for Subflow CompleteMsoProcess
2019-04-28T06:31:54.840Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.ExceptionUtil - Outgoing WorkflowException is WorkflowException[processKey=CompleteMsoProcess,errorCode=500,errorMessage=Complete MSO -- Update DB status to SUCCESS -- Failed,workStep=*]
2019-04-28T06:31:54.840Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.ExceptionUtil - Throwing MSOWorkflowException
2019-04-28T06:31:54.843Z|c3b4a3ef-ace2-4d99-9447-43167ecb52bf| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - Rethrowing MSOWorkflowException

 

 

ANOTHER SAMPLE SHOWING GOOD RESULT OK:

2019-04-28T09:03:52.200Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.client.RestClient - RestClientSSL using default SSL context!
2019-04-28T09:03:52.219Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| o.o.so.logging.jaxrs.filter.PayloadLoggingFilter - Making POST (overridden to PATCH) request to: https://aai.onap:8443/aai/v14/business/customers/customer/Demonstration/service-subscriptions/service-subscription/vFW/service-instances/service-instance/9c385d97-e817-4438-829a-1570218b3476
Request Headers: {Authorization=[Basic YWFpQGFhaS5vbmFwLm9yZzpkZW1vMTIzNDU2IQ==], X-FromAppId=[MSO], X-TransactionId=[], X-HTTP-Method-Override=[PATCH], Accept=[application/json], Content-Type=[application/merge-patch+json], X-ONAP-RequestID=[a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa], X-ONAP-InvocationID=[b07f1941-b2ae-4863-a9f0-f32c15dd3197], X-ONAP-PartnerName=[SO]}
2019-04-28T09:03:52.220Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| o.o.so.logging.jaxrs.filter.PayloadLoggingFilter - {"service-instance-id":"9c385d97-e817-4438-829a-1570218b3476","service-instance-name":"vFWNG_CDS_18","orchestration-status":"Active"}

2019-04-28T09:03:52.686Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.core.BPMNLogger - Entered SetSuccessIndicator Method
2019-04-28T09:03:52.687Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.core.BPMNLogger - Outgoing SuccessIndicator is: false
2019-04-28T09:03:52.687Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - CompleteMsoProcess Request: <aetgt:MsoCompletionRequest xmlns:aetgt="http://org.onap/so/workflow/schema/v1" xmlns:ns="http://org.onap/so/request/types/v1"><request-info xmlns="http://org.onap/so/infra/vnf-request/v1"><request-id>a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa</request-id><action>createInstance</action><source>null</source></request-info><status-message>Macro-Service-createInstance request was executed correctly.</status-message><serviceInstanceId>9c385d97-e817-4438-829a-1570218b3476</serviceInstanceId><mso-bpel-name>WorkflowActionBB</mso-bpel-name></aetgt:MsoCompletionRequest>
2019-04-28T09:03:52.688Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - Incoming Request is: <aetgt:MsoCompletionRequest xmlns:aetgt="http://org.onap/so/workflow/schema/v1" xmlns:ns="http://org.onap/so/request/types/v1"><request-info xmlns="http://org.onap/so/infra/vnf-request/v1"><request-id>a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa</request-id><action>createInstance</action><source>null</source></request-info><status-message>Macro-Service-createInstance request was executed correctly.</status-message><serviceInstanceId>9c385d97-e817-4438-829a-1570218b3476</serviceInstanceId><mso-bpel-name>WorkflowActionBB</mso-bpel-name></aetgt:MsoCompletionRequest>
2019-04-28T09:03:53.291Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - Incoming Instance Id Xml: <serviceInstanceId>9c385d97-e817-4438-829a-1570218b3476</serviceInstanceId>
2019-04-28T09:03:53.292Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - Outgoing Update Mso Request Payload is:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:req="http://org.onap.so/requestsdb">
<soapenv:Header/>
<soapenv:Body>
<req:updateInfraRequest>
<requestId>a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa</requestId>
<lastModifiedBy>WorkflowActionBB</lastModifiedBy>
<statusMessage>Macro-Service-createInstance request was executed correctly.</statusMessage>
<requestStatus>COMPLETE</requestStatus>
<progress>100</progress>
<serviceInstanceId>9c385d97-e817-4438-829a-1570218b3476</serviceInstanceId>
</req:updateInfraRequest>
</soapenv:Body>
</soapenv:Envelope>
2019-04-28T09:03:53.292Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - setUpdateDBstatustoSuccessPayload:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:req="http://org.onap.so/requestsdb">
<soapenv:Header/>
<soapenv:Body>
<req:updateInfraRequest>
<requestId>a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa</requestId>
<lastModifiedBy>WorkflowActionBB</lastModifiedBy>
<statusMessage>Macro-Service-createInstance request was executed correctly.</statusMessage>
<requestStatus>COMPLETE</requestStatus>
<progress>100</progress>
<serviceInstanceId>9c385d97-e817-4438-829a-1570218b3476</serviceInstanceId>
</req:updateInfraRequest>
</soapenv:Body>
</soapenv:Envelope>
2019-04-28T09:03:53.590Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - CompleteMsoProcess Response: <sdncadapterworkflow:MsoCompletionResponse xmlns:sdncadapterworkflow="http://ecomp.com/mso/workflow/schema/v1">
<sdncadapterworkflow:out>BPEL WorkflowActionBB completed</sdncadapterworkflow:out>
</sdncadapterworkflow:MsoCompletionResponse>
2019-04-28T09:03:53.591Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.core.BPMNLogger - Entered SetSuccessIndicator Method
2019-04-28T09:03:53.592Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.core.BPMNLogger - Outgoing SuccessIndicator is: true
2019-04-28T09:03:53.592Z|a1b96a1b-a78e-4f68-99fa-b4e3b63bb0fa| org.onap.so.bpmn.common.scripts.CompleteMsoProcess - @@ CompleteMsoProcess Response @@
<sdncadapterworkflow:MsoCompletionResponse xmlns:sdncadapterworkflow="http://ecomp.com/mso/workflow/schema/v1">
<sdncadapterworkflow:out>BPEL WorkflowActionBB completed</sdncadapterworkflow:out>
</sdncadapterworkflow:MsoCompletionResponse>