-
Bug
-
Resolution: Not a Bug
-
Medium
-
None
-
Frankfurt Release
-
None
-
None
-
- infrastructure: Openstack windriver lab VMs - 2 K8s kubernetes controllers, 5 K8s worker nodes. (4 vCPU, 8GB RAM)
- source base : oom/frankfurt branch
helm repo : https://nexus.onap.org/content/sites/oom-helm-staging/master and frankfurt
Reproduce step:
- helm deploy demo staging/onap --namespace onap -f onap/resources/environments/dev.yaml -f onap/resources/overrides/environment.yaml
Command line shows init containers of SDC's are failed after 3 attempts.
Here's `kubectl describe pod sdc-be-xxx -n onap` print
Name: demo-sdc-be-56954dbf89-bnqcc Namespace: onap Priority: 0 Node: yopp-k8s-worker-5/192.168.64.5 Start Time: Fri, 15 May 2020 11:24:33 -0400 Labels: app.kubernetes.io/instance=demo app.kubernetes.io/managed-by=Tiller app.kubernetes.io/name=sdc-be helm.sh/chart=sdc-be-6.0.0 pod-template-hash=56954dbf89 Annotations: cni.projectcalico.org/podIP: 10.42.6.79/32 cni.projectcalico.org/podIPs: 10.42.6.79/32 Status: Pending IP: 10.42.6.79 Controlled By: ReplicaSet/demo-sdc-be-56954dbf89 Init Containers: sdc-be-readiness: Container ID: docker://92b740d8471eff4ab2f46a28381c4d2ed3a6394979c54c91cfc9bc366fc54e3b Image: oomk8s/readiness-check:2.0.2 Image ID: docker-pullable://oomk8s/readiness-check@sha256:875814cc853d7d9b3227165740584c347884bc8c0d831b4148cb5f7952e4cbff Port: <none> Host Port: <none> Command: /root/ready.py Args: --container-name sdc-onboarding-be State: Running Started: Fri, 15 May 2020 11:24:35 -0400 Ready: False Restart Count: 0 Environment: NAMESPACE: onap (v1:metadata.namespace) Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-278tr (ro) sdc-be-job-completion: Container ID: Image: oomk8s/readiness-check:2.0.2 Image ID: Port: <none> Host Port: <none> Command: /root/job_complete.py Args: --job-name demo-sdc-onboarding-be-cassandra-init State: Waiting Reason: PodInitializing Ready: False Restart Count: 0 Environment: NAMESPACE: onap (v1:metadata.namespace) Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-278tr (ro) sdc-be-update-config: Container ID: Image: dibi/envsubst Image ID: Port: <none> Host Port: <none> Command: sh Args: -c cd /config-input && for PFILE in `find . -not -type d | grep -v -F ..`; do envsubst <${PFILE} >/config-output/${PFILE}; chmod 0755 /config-output/${PFILE}; done State: Waiting Reason: PodInitializing Ready: False Restart Count: 0 Environment: KEYSTORE_PASS: <set to the key 'keystore_password' in secret 'demo-sdc-cs-secrets'> Optional: false TRUSTSTORE_PASS: <set to the key 'truststore_password' in secret 'demo-sdc-cs-secrets'> Optional: false Mounts: /config-input/ from demo-sdc-be-environments (rw) /config-output/ from sdc-environments-output (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-278tr (ro) Containers: sdc-be: Container ID: Image: nexus3.onap.org:10001/onap/sdc-backend:1.6.4 Image ID: Ports: 8443/TCP, 8080/TCP Host Ports: 0/TCP, 0/TCP State: Waiting Reason: PodInitializing Ready: False Restart Count: 0 Liveness: http-get https://:api/sdc2/rest/healthCheck delay=120s timeout=15s period=60s #success=1 #failure=3 Readiness: exec [/var/lib/jetty/ready-probe.sh] delay=120s timeout=15s period=60s #success=1 #failure=3 Environment: ENVNAME: AUTO JAVA_OPTIONS: -Xmx1536m -Xms1536m cassandra_ssl_enabled: false HOST_IP: (v1:status.podIP) Mounts: /etc/localtime from demo-sdc-be-localtime (ro) /tmp/logback.xml from demo-sdc-be-logback (rw,path="logback.xml") /var/lib/jetty/chef-solo/cookbooks/sdc-catalog-be/files/default/org.onap.sdc.p12 from sdc-cert (rw,path="org.onap.sdc.p12") /var/lib/jetty/chef-solo/cookbooks/sdc-catalog-be/files/default/org.onap.sdc.trust.jks from sdc-cert (rw,path="org.onap.sdc.trust.jks") /var/lib/jetty/chef-solo/environments/ from sdc-environments-output (rw) /var/log/onap from demo-sdc-be-logs (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-278tr (ro) sdc-be-filebeat-onap: Container ID: Image: docker.elastic.co/beats/filebeat:5.5.0 Image ID: Port: <none> Host Port: <none> State: Waiting Reason: PodInitializing Ready: False Restart Count: 0 Environment: <none> Mounts: /usr/share/filebeat/data from demo-sdc-be-data-filebeat (rw) /usr/share/filebeat/filebeat.yml from demo-sdc-be-filebeat-conf (rw,path="filebeat.yml") /var/log/onap from demo-sdc-be-logs (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-278tr (ro) Conditions: Type Status Initialized False Ready False ContainersReady False PodScheduled True Volumes: demo-sdc-be-localtime: Type: HostPath (bare host directory volume) Path: /etc/localtime HostPathType: sdc-cert: Type: Secret (a volume populated by a Secret) SecretName: sdc-cert Optional: false demo-sdc-be-filebeat-conf: Type: ConfigMap (a volume populated by a ConfigMap) Name: demo-sdc-filebeat-configmap Optional: false demo-sdc-be-data-filebeat: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: <unset> demo-sdc-be-logback: Type: ConfigMap (a volume populated by a ConfigMap) Name: demo-sdc-be-logging-configmap Optional: false demo-sdc-be-environments: Type: ConfigMap (a volume populated by a ConfigMap) Name: demo-sdc-environments-configmap Optional: false sdc-environments-output: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: Memory SizeLimit: <unset> demo-sdc-be-logs: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: <unset> default-token-278tr: Type: Secret (a volume populated by a Secret) SecretName: default-token-278tr Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 5m48s default-scheduler Successfully assigned onap/demo-sdc-be-56954dbf89-bnqcc to yopp-k8s-worker-5 Normal Pulled 5m46s kubelet, yopp-k8s-worker-5 Container image "oomk8s/readiness-check:2.0.2" already present on machine Normal Created 5m46s kubelet, yopp-k8s-worker-5 Created container sdc-be-readiness Normal Started 5m46s kubelet, yopp-k8s-worker-5 Started container sdc-be-readiness
Screen shots are attached.