Details
-
Bug
-
Status: Closed
-
Medium
-
Resolution: Done
-
Istanbul Release
-
None
-
None
-
istanbul deployment on a hardened RKE2 kubernetes solution
Description
We are trying to improve the backup and restore procedures.
We perform the following procedure
- deploy ONAP (deployment source A)
- perform tests on A (upload model, instantiate VM)
- backup cassandra deployment A
- deploy ONAP (B)
- check SDC B, redistribute a model (OK)
- restore A on B
- check SDC A (with B restored cassandra DB) => bad gateway => restart sdc-be needed
- check SDC => all B models available OK
- redistribute a B model on deployment A => error java exception (500 internal error on sdc-fe)
the backup and restore are performed through velero/restic solutions
the backup is stored in a S3 backend
the error in the SDC-FE is
org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET http://null:null/healthCheck HTTP/1.1 failed with exception: URI does not specify a valid host name: http://*null*:*null*/healthCheck
at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124)
at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66)
at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78)
at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51)
at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.addHostedComponentsFeHealthCheck(HealthCheckScheduledTask.java:119)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.buildHealthCheckListForComponent(HealthCheckScheduledTask.java:254)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.getFeHealthCheckInfos(HealthCheckScheduledTask.java:247)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.checkHealth(HealthCheckScheduledTask.java:203)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.run(HealthCheckScheduledTask.java:96)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: http://null:null/healthCheck
at org.apache.http.impl.client.CloseableHttpClient.determineTarget(CloseableHttpClient.java:95)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118)
... 15 common frames omitted
2021-12-13T15:55:00.775Z|||qtp817348612-53||SDC-FE||INFO|||sdc-fe.onap||o.o.l.f.base.AbstractAuditLogFilter||ActivityType=<?>, Desc=<Entering>
2021-12-13T15:55:00.776Z|||qtp817348612-53||SDC-FE|cs0008|INFO||10.42.1.40|sdc-fe.onap|10.253.0.93|o.o.sdc.fe.servlets.FeProxyServlet||ActivityType=<?>, Desc=<GET /sdc1/feProxy/rest/v1/catalog/services/a1a2ad70-dd42-4b00-b4f9-a39df9291676/conformanceLevelValidation HTTP/1.1>
2021-12-13T15:55:00.776Z|||qtp817348612-53||SDC-FE|cs0008|INFO||10.42.1.40|sdc-fe.onap|10.253.0.93|o.o.s.f.config.ConfigurationManager||ActivityType=<?>, Desc=<requested plugins configuration and got this:PluginsConfiguration[pluginsList=[org.openecomp.sdc.fe.config.PluginsConfiguration$Plugin@64763e49], connectionTimeout=1000]>
2021-12-13T15:55:00.777Z|||qtp817348612-53||SDC-FE|cs0008|INFO||10.42.1.40|sdc-fe.onap|10.253.0.93|o.o.l.f.base.AbstractAuditLogFilter||ActivityType=<?>, Desc=<Exiting.>
2021-12-13T15:55:00.939Z|||qtp817348612-48||SDC-FE||INFO|||sdc-fe.onap||o.o.l.f.base.AbstractAuditLogFilter||ActivityType=<?>, Desc=<Entering>
2021-12-13T15:55:00.940Z|||qtp817348612-48||SDC-FE|cs0008|INFO||10.42.1.40|sdc-fe.onap|10.253.0.93|o.o.sdc.fe.servlets.FeProxyServlet||ActivityType=<?>, Desc=<POST /sdc1/feProxy/rest/v1/catalog/services/2f7db495-a546-493a-bb8b-e01d511143d8/distribution/PROD/activate HTTP/1.1>
2021-12-13T15:55:00.940Z|||qtp817348612-48||SDC-FE|cs0008|INFO||10.42.1.40|sdc-fe.onap|10.253.0.93|o.o.s.f.config.ConfigurationManager||ActivityType=<?>, Desc=<requested plugins configuration and got this:PluginsConfiguration[pluginsList=[org.openecomp.sdc.fe.config.PluginsConfiguration$Plugin@64763e49], connectionTimeout=1000]>
2021-12-13T15:55:00.942Z|||qtp817348612-48||SDC-FE|cs0008|INFO||10.42.1.40|sdc-fe.onap|10.253.0.93|o.o.l.f.base.AbstractAuditLogFilter||ActivityType=<?>, Desc=<Exiting.>
2021-12-13T15:55:03.184Z|||FE-Health-Check-Thread||SDC-FE||ERROR|MAJOR||sdc-fe.onap||o.o.s.c.config.EcompErrorLogUtil||ActivityType=<?>, Desc=<Internal flow error. Operation: Execute request GET http://null:null/healthCheck HTTP/1.1 failed with exception: URI does not specify a valid host name: http://null:null/healthCheck>
2021-12-13T15:55:03.184Z|||FE-Health-Check-Thread||SDC-FE||ERROR|||sdc-fe.onap||o.o.s.f.i.HealthCheckScheduledTask||ActivityType=<?>, Desc=<CATALOG_FACADE_MS unexpected response >
org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET http://null:null/healthCheck HTTP/1.1 failed with exception: URI does not specify a valid host name: http://null:null/healthCheck
at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124)
at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66)
at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78)
at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51)
at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.addHostedComponentsFeHealthCheck(HealthCheckScheduledTask.java:119)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.buildHealthCheckListForComponent(HealthCheckScheduledTask.java:254)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.getFeHealthCheckInfos(HealthCheckScheduledTask.java:247)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.checkHealth(HealthCheckScheduledTask.java:203)
at org.openecomp.sdc.fe.impl.HealthCheckScheduledTask.run(HealthCheckScheduledTask.java:96)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: http://null:null/healthCheck
at org.apache.http.impl.client.CloseableHttpClient.determineTarget(CloseableHttpClient.java:95)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118)
... 15 common frames omitted
A manual restart is needed for the BE/cassandra connection => no reconenction mechanism?
I assume that some operations after the restore are needed to postconfigure some SDC elements (and avoid getting null values)
In a first step I assume that a procedure saying what shall be restarted could be enough
the goal is to be able to redistribute saved models and reuse them for instantiation.