NAME                                               READY   STATUS                            RESTARTS   AGE
dev-cds-sdc-listener-845f9766c9-8xpkf              0/1     CrashLoopBackOff                  485        42h

dev-cds-sdc-listener-845f9766c9-8xpkf              0/1     CrashLoopBackOff                  485        42h

16:50:11,193 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@71d44a3 - Registering current configuration as safe fallback point
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::       (v2.3.12.RELEASE)2021-10-28 16:50:11,705|||main||||INFO||||||| Starting SdcListenerApplication v1.2.0 on dev-cds-sdc-listener-845f9766c9-8xpkf with PID 7 (/opt/app/onap/lib/org.onap.ccsdk.cds.sdclistener-sdclistener-application-1.2.0.jar started by onap in /app)
2021-10-28 16:50:11,707|||main||||INFO||||||| No active profile set, falling back to default profiles: default
2021-10-28 16:50:14,441|||main||||INFO||||||| Exposing 14 endpoint(s) beneath base path '/actuator'
2021-10-28 16:50:14,709|||main||||INFO||||||| HV000001: Hibernate Validator 6.1.7.Final
2021-10-28 16:50:15,129|||main||||INFO||||||| Using generated security password: 548bc3b1-47e8-4ec1-9c75-f1a2c37f7caf2021-10-28 16:50:15,326|||main||||INFO||||||| Netty started on port(s): 8080
2021-10-28 16:50:16,132|||main||||INFO||||||| Started SdcListenerApplication in 4.85 seconds (JVM running for 5.504)
2021-10-28 16:50:16,134|||main||||INFO||||||| Initialize the SDC distribution client
2021-10-28 16:50:16,436|||main||||INFO||||||| DistributionClient - init
2021-10-28 16:50:16,817|||main||||INFO||||||| POST http://message-router.onap:3904/apiKeys/create (anonymous) ...
2021-10-28 16:50:16,825|||main||||INFO||||||| 	--> HTTP/1.1 200 OK
2021-10-28 16:50:16,838|||main||||ERROR||||||| status from ASDC is org.onap.sdc.http.HttpAsdcResponse@44b99467
2021-10-28 16:50:16,839|||main||||ERROR||||||| DistributionClientResultImpl [responseStatus=ASDC_SERVER_PROBLEM, responseMessage=ASDC server problem]
2021-10-28 16:50:16,850|||main||||WARN||||||| DELETE http://message-router.onap:3904/apiKeys/QYWaBJ2yCjingA8x will send credentials over a clear channel.
2021-10-28 16:50:16,850|||main||||INFO||||||| DELETE http://message-router.onap:3904/apiKeys/QYWaBJ2yCjingA8x (as QYWaBJ2yCjingA8x) ...
2021-10-28 16:50:16,855|||main||||INFO||||||| 	--> HTTP/1.1 200 OK
2021-10-28 16:50:16,872|||main||||ERROR||||||| Application run failed
java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method
HandlerMethod details: 
Bean [org.onap.ccsdk.cds.sdclistener.client.SdcListenerClient]
Method [public void org.onap.ccsdk.cds.sdclistener.client.SdcListenerClient.initSdcClient() throws org.onap.ccsdk.cds.sdclistener.exceptions.SdcListenerException]
Resolved arguments: 	at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:322)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:190)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:153)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:404)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:361)
	at org.springframework.boot.context.event.EventPublishingRunListener.running(EventPublishingRunListener.java:108)
	at org.springframework.boot.SpringApplicationRunListeners.running(SpringApplicationRunListeners.java:77)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:327)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1247)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1236)
	at org.onap.ccsdk.cds.sdclistener.SdcListenerApplication.main(SdcListenerApplication.java:30)
Caused by: org.onap.ccsdk.cds.sdclistener.exceptions.SdcListenerException: SDC distribution client init failed with reason:ASDC server problem
	at org.onap.ccsdk.cds.sdclistener.client.SdcListenerClient.startSdcClientBasedOnTheResult(SdcListenerClient.java:75)
	at org.onap.ccsdk.cds.sdclistener.client.SdcListenerClient.initSdcClient(SdcListenerClient.java:69)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:305)
	... 13 common frames omitted
2021-10-28 16:50:16,885|||main||||INFO||||||| Stopping GRPC connection to CDS backend

NAME                                               READY   STATUS                            RESTARTS   AGE
dev-dcae-servicechange-handler-655c89dc-llzpq      0/1     CrashLoopBackOff                  285        25h

root@policy-k8s-nfs:~/oom/kubernetes# kubectl logs dev-dcae-servicechange-handler-655c89dc-llzpq -n onap
Certificate was added to keystore
16:52:03.438 [main] DEBUG sch.core - Entering run-distribution-client
16:52:03.444 [main] INFO org.onap.sdc.impl.DistributionClientImpl - DistributionClient - init
16:52:03.449 [main] DEBUG org.onap.sdc.impl.DistributionClientImpl - get ueb cluster server list from component(configuration file)
16:52:03.499 [main] DEBUG org.onap.sdc.http.SdcConnectorClient - about to perform getServerList. requestId= a35c3e82-5865-4986-94cf-fdfba30f8e7a url= /sdc/v1/artifactTypes
16:52:03.500 [main] DEBUG org.onap.sdc.http.HttpAsdcClient - url to send https://sdc-be.onap:8443/sdc/v1/artifactTypes
16:52:03.845 [main] DEBUG org.onap.sdc.http.HttpAsdcClient - GET Response Status 200
16:52:03.874 [main] DEBUG org.onap.sdc.impl.DistributionClientImpl - Artifact types: [DCAE_TOSCA, DCAE_JSON, DCAE_POLICY, DCAE_DOC, DCAE_EVENT, DCAE_INVENTORY_TOSCA, DCAE_INVENTORY_JSON, DCAE_INVENTORY_POLICY, DCAE_INVENTORY_DOC, DCAE_INVENTORY_BLUEPRINT, DCAE_INVENTORY_EVENT] were validated with ASDC server
16:52:03.882 [main] DEBUG org.onap.sdc.impl.DistributionClientImpl - create keys
16:52:03.886 [main] INFO com.att.nsa.apiClient.http.HttpClient - POST http://message-router.onap:3904/apiKeys/create (anonymous) ...
16:52:03.899 [main] INFO com.att.nsa.apiClient.http.HttpClient - 	--> HTTP/1.1 200 OK
16:52:03.900 [main] DEBUG com.att.nsa.apiClient.http.HttpClient - Consuming the entity.
16:52:03.909 [main] DEBUG org.onap.sdc.http.SdcConnectorClient - about to perform registerAsdcTopics. requestId= 9562155f-9cfe-429a-9d83-85c7bb3766d3 url= /sdc/v1/registerForDistribution
16:52:03.914 [main] ERROR org.onap.sdc.http.SdcConnectorClient - status from ASDC is org.onap.sdc.http.HttpAsdcResponse@4d0ec988
16:52:03.914 [main] ERROR org.onap.sdc.http.SdcConnectorClient - DistributionClientResultImpl [responseStatus=ASDC_SERVER_PROBLEM, responseMessage=ASDC server problem]
16:52:03.914 [main] DEBUG org.onap.sdc.http.SdcConnectorClient - error from ASDC is: {
  "requestError": {
    "policyException": {
      "messageId": "POL5000",
      "text": "Error: Internal Server Error. Please try again later.",
      "variables": []
    }
  }
}
16:52:03.926 [main] WARN com.att.nsa.apiClient.http.HttpClient - DELETE http://message-router.onap:3904/apiKeys/1MdA5L5Q2s1rcZ6U will send credentials over a clear channel.
16:52:03.926 [main] INFO com.att.nsa.apiClient.http.HttpClient - DELETE http://message-router.onap:3904/apiKeys/1MdA5L5Q2s1rcZ6U (as 1MdA5L5Q2s1rcZ6U) ...
16:52:03.932 [main] INFO com.att.nsa.apiClient.http.HttpClient - 	--> HTTP/1.1 200 OK
16:52:03.935 [main] DEBUG com.att.nsa.apiClient.http.HttpClient - Consuming the entity.
16:52:03.940 [main] ERROR sch.core - #object[org.onap.sdc.impl.DistributionClientResultImpl 0x879ce67 DistributionClientResultImpl [responseStatus=ASDC_SERVER_PROBLEM, responseMessage=ASDC server problem]]
16:52:03.940 [main] INFO sch.core - Done