Uploaded image for project: 'ONAP Operations Manager'
  1. ONAP Operations Manager
  2. OOM-290

config_init pod fails when /dockerdata-nfs is nfs-mounted

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Medium
    • Resolution: Unresolved
    • Affects Version/s: Amsterdam Release
    • Fix Version/s: Beijing Release
    • Labels:
      None

      Description

      On some kubernetes deployments there is a conflict when a volume in docker image is mounted in kubernetes over NFS.

      The observed behavior is that files / directories created inside the container are not visible inside / outside of the container.

      Removing VOLUME declaration from the Dockerfile solve this problem.

      Related defect with similar symptoms: OOM-258

      Log:

      [root@kubernetes-master-borislavki-1867-09-04-17 config]# kubectl describe pods -n onap config-init
      Name: config-init
      Namespace: onap
      Node: kubernetes-node-borislavki-1867-09-04-17-2/10.247.58.31
      Start Time: Thu, 07 Sep 2017 13:17:00 +0300
      Labels: <none>
      Annotations: <none>
      Status: Succeeded
      IP: 10.244.3.117
      Containers:
      config-init:
      Container ID: docker://b6415e21bcb21b08ab1b97ebb9fca0517917f3ea4a07f948e0890c87a3352e58
      Image: oomk8s/config-init:1.1.0
      Image ID: docker-pullable://docker.io/oomk8s/config-init@sha256:8c9989e35f424110a82cae773bda785dedd9634ee7da019bb14bc1fdfe32a2fd
      Port: <none>
      State: Terminated
      Reason: Completed
      Exit Code: 0
      Started: Thu, 07 Sep 2017 13:17:05 +0300
      Finished: Thu, 07 Sep 2017 13:17:05 +0300
      Ready: False
      Restart Count: 0
      Environment:
      NAMESPACE: onap (v1:metadata.namespace)
      Mounts:
      /config-init/ from config-init-root (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-t8bfw (ro)
      Conditions:
      Type Status
      Initialized True
      Ready False
      PodScheduled True
      Volumes:
      config-init-root:
      Type: HostPath (bare host directory volume)
      Path: /dockerdata-nfs/
      default-token-t8bfw:
      Type: Secret (a volume populated by a Secret)
      SecretName: default-token-t8bfw
      Optional: false
      QoS Class: BestEffort
      Node-Selectors: <none>
      Tolerations: node.alpha.kubernetes.io/notReady:NoExecute for 300s
      node.alpha.kubernetes.io/unreachable:NoExecute for 300s
      Events:
      FirstSeen LastSeen Count From SubObjectPath Type Reason Message
      --------- -------- ----- ---- ------------- -------- ------ -------
      43s 43s 1 default-scheduler Normal Scheduled Successfully assigned config-init to kubernetes-node-borislavki-1867-09-04-17-2
      6s 6s 1 kubelet, kubernetes-node-borislavki-1867-09-04-17-2 Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "config-init-root"
      6s 6s 1 kubelet, kubernetes-node-borislavki-1867-09-04-17-2 Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "default-token-t8bfw"
      5s 5s 1 kubelet, kubernetes-node-borislavki-1867-09-04-17-2 spec.containers{config-init} Normal Pulling pulling image "oomk8s/config-init:1.1.0"
      3s 3s 1 kubelet, kubernetes-node-borislavki-1867-09-04-17-2 spec.containers{config-init} Normal Pulled Successfully pulled image "oomk8s/config-init:1.1.0"
      2s 2s 1 kubelet, kubernetes-node-borislavki-1867-09-04-17-2 spec.containers{config-init} Normal Created Created container
      1s 1s 1 kubelet, kubernetes-node-borislavki-1867-09-04-17-2 spec.containers{config-init} Normal Started Started container
      [root@kubernetes-master-borislavki-1867-09-04-17 config]# kubectl logs -n onap config-init
      cp: cannot create directory '/config-init/onap/aai/aai-config': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/aai-data': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/etc': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/model-loader': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/aai-resources': No such file or directory
      '/opt/config/src/aai/aai-traversal/docker-entrypoint.sh' -> '/config-init/onap/aai/aai-traversal/docker-entrypoint.sh'
      cp: cannot create regular file '/config-init/onap/aai/aai-traversal/docker-entrypoint.sh': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/elasticsearch': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/sparky-be': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/data-router': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/haproxy': No such file or directory
      cp: cannot create directory '/config-init/onap/aai/search-data-service': No such file or directory
      cp: cannot create directory '/config-init/onap/appc': No such file or directory
      cp: cannot create directory '/config-init/onap/dcae': No such file or directory
      cp: cannot create directory '/config-init/onap/message-router': No such file or directory
      cp: cannot create directory '/config-init/onap/mso': No such file or directory
      cp: cannot create directory '/config-init/onap/policy': No such file or directory
      cp: cannot create directory '/config-init/onap/portal': No such file or directory
      cp: cannot create directory '/config-init/onap/robot': No such file or directory
      cp: cannot create directory '/config-init/onap/sdc': No such file or directory
      cp: cannot create directory '/config-init/onap/sdnc': No such file or directory
      cp: cannot create directory '/config-init/onap/vid': No such file or directory
      mkdir: cannot create directory '/config-init/onap/appc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/dcae': No such file or directory
      mkdir: cannot create directory '/config-init/onap/portal': No such file or directory
      mkdir: cannot create directory '/config-init/onap/portal': No such file or directory
      mkdir: cannot create directory '/config-init/onap/sdnc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/vid': No such file or directory
      mkdir: cannot create directory '/config-init/onap/sdc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/sdc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/sdc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/sdc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/sdc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/sdc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/sdc': No such file or directory
      mkdir: cannot create directory '/config-init/onap/aai/opt': No such file or directory
      mkdir: cannot create directory '/config-init/onap/aai/model-loader': No such file or directory
      mkdir: cannot create directory '/config-init/onap/aai/haproxy': No such file or directory
      mkdir: cannot create directory '/config-init/onap/aai/aai-traversal/logs/': File exists
      mkdir: cannot create directory '/config-init/onap/aai/aai-resources': No such file or directory
      mkdir: cannot create directory '/config-init/onap/aai/sparky-be': No such file or directory
      mkdir: cannot create directory '/config-init/onap/aai/elasticsearch': No such file or directory
      mkdir: cannot create directory '/config-init/onap/aai/search-data-service': No such file or directory
      mkdir: cannot create directory '/config-init/onap/aai/data-router': No such file or directory
      mkdir: cannot create directory '/config-init/onap/mso': No such file or directory
      chmod: cannot access '/config-init/onap/sdc/logs/': No such file or directory
      chmod: cannot access '/config-init/onap/portal/logs/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/aai-config/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/aai-data/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/opt/aai/logroot/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/model-loader/logs/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/haproxy/log/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/aai-traversal/logs/': Stale file handle
      chmod: cannot access '/config-init/onap/aai/aai-resources/logs/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/sparky-be/logs/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/elasticsearch/es-data/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/search-data-service/logs/': No such file or directory
      chmod: cannot access '/config-init/onap/aai/data-router/logs/': No such file or directory
      chmod: cannot access '/config-init/onap/policy/mariadb/': No such file or directory

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            • Assignee:
              BorislavG Borislav Glozman
              Reporter:
              BorislavG Borislav Glozman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: