On the last Master daily installation,

onap-vfc-vfc-ems-driver-845b6dbc96-cb6xd

is in crashLoopBackup because it is waiting for msb-ias:80

msb-ias is no more exposed on the port 80.

it is now reachable through https 443

msb-iag NodePort 10.233.59.231 <none> 443:30283/TCP

that is why the pod is crashing

pod logs

Name: onap-vfc-vfc-ems-driver-845b6dbc96-cb6xd
Namespace: onap
Priority: 0
PriorityClassName: <none>
Node: compute10-onap-master/10.253.0.28
Start Time: Mon, 25 Nov 2019 11:21:37 +0000
Labels: app=vfc-ems-driver
pod-template-hash=845b6dbc96
release=onap-vfc
Annotations: sidecar.istio.io/inject: true
Status: Running
IP: 10.233.75.254
Controlled By: ReplicaSet/onap-vfc-vfc-ems-driver-845b6dbc96
Containers:
vfc-ems-driver:
Container ID: docker://07006f52916400f993108b55fafc5ee6bc26b9aa87611ad94fd403351759b2f4
Image: nexus3.onap.org:10001/onap/vfc/emsdriver:1.3.0
Image ID: docker-pullable://nexus3.onap.org:10001/onap/vfc/emsdriver@sha256:2f6676ec622480e13994240778e87b8e35664116fd04159a0a53d330d36bcdd2
Port: 8206/TCP
Host Port: 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Mon, 25 Nov 2019 11:45:15 +0000
Finished: Mon, 25 Nov 2019 11:48:13 +0000
Ready: False
Restart Count: 7
Liveness: tcp-socket :8206 delay=120s timeout=1s period=10s #success=1 #failure=3
Readiness: tcp-socket :8206 delay=10s timeout=1s period=10s #success=1 #failure=3
Environment:
MSB_ADDR: msb-iag:80
VES_ADDR: dcae-ves-collector:8080
VES_AUTHINFO: :
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-hfhxr (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-hfhxr:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-hfhxr
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 31m default-scheduler Successfully assigned onap/onap-vfc-vfc-ems-driver-845b6dbc96-cb6xd to compute10-onap-master
Normal Created 28m kubelet, compute10-onap-master Created container vfc-ems-driver
Normal Started 28m kubelet, compute10-onap-master Started container vfc-ems-driver
Normal Killing 26m kubelet, compute10-onap-master Container vfc-ems-driver failed liveness probe, will be restarted
Warning Unhealthy 26m (x3 over 26m) kubelet, compute10-onap-master Liveness probe failed: dial tcp 10.233.75.254:8206: connect: connection refused
Normal Pulling 25m (x2 over 31m) kubelet, compute10-onap-master Pulling image "nexus3.onap.org:10001/onap/vfc/emsdriver:1.3.0"
Normal Pulled 25m (x2 over 28m) kubelet, compute10-onap-master Successfully pulled image "nexus3.onap.org:10001/onap/vfc/emsdriver:1.3.0"
Warning Unhealthy 6m14s (x127 over 28m) kubelet, compute10-onap-master Readiness probe failed: dial tcp 10.233.75.254:8206: connect: connection refused
Warning BackOff 81s (x20 over 4m53s) kubelet, compute10-onap-master Back-off restarting failed container
SERVICE_IP=10.233.75.254
MSB_ADDR=msb-iag:80
Wait for MSB initialization
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused
curl: (7) Failed connect to msb-iag:80; Connection refused

helm chart must be adapted to new msb configuration

this change has other impacts

• errors on the Multicloud healthchecks biny993
• errors on the MSB healthcheck Huabing_Zhao
• errors on the VF healthchecks yangyan

see https://gitlab.com/Orange-OpenSource/lfn/onap/xtesting-onap/-/jobs/360234314 for details