Uploaded image for project: 'CI-Management'
  1. CI-Management
  2. CIMAN-157

nexus3 config change breaks ONAP deployment for 50 images until PTLs request release - advise that the LF fix and explain first

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Medium Medium
    • Beijing Release
    • Beijing Release

      All unproxied deployments of master will fail
      based on the config change done friday
      https://lists.onap.org/pipermail/onap-discuss/2018-April/009317.html

      Any deployment that directly pull from nexus3 without a proxy will fail on the 30+ images below because they need to have a new release request to unblock snapshot images

      proposal - dont wait for PTL approval - we already were running with these 50 images - just reenable them

      All CD deployments fail except those with a proxy
      http://jenkins.onap.info/job/oom-cd-master/
      http://jenkins.onap.info/job/oom-cd-master2-aws/
      passing (with proxy?) https://jenkins.onap.org/view/External%20Labs/job/lab-tlab-beijing-oom-deploy/

      before
      http://jenkins.onap.info/job/oom-cd-master2-aws/24/console
      6

      13:25:59 onap          dev-so-db-684dffd45-cmsth                      0/1       ErrImagePull        0          1h
      13:25:59 onap          dev-smsdb-0                                    0/2       ErrImagePull        0          1h
      13:25:59 onap          dev-portal-zookeeper-db466fc-lfgf7             0/1       ErrImagePull        0          1h
      13:25:59 onap          dev-policydb-5dbb9b54df-ngxwd                  0/1       ErrImagePull        0          1h
      13:25:59 onap          dev-consul-b5d9d5564-85vn2                     0/1       ErrImagePull        0          1h
      13:25:59 onap          dev-consul-server-8f8f8767-6x8cm               0/1       ErrImagePull        0          1h
      unknown state
      13:25:59 onap          dev-vid-mariadb-9bbff4865-2cfkc                0/1       Pending             0          1h
      13:25:59 onap          dev-vnfsdk-647cc6bc8-bblpz                     0/1       Pending             0          1h
      

      after
      http://jenkins.onap.info/job/oom-cd-master2-aws/33/console
      1 fixed
      57 - 5 - (2 clustered) = 50

      after 7 hours a better list
      ubuntu@ip-10-0-0-206:~$ kubectl get pods --all-namespaces | grep Image 
      onap          dev-aaf-84dbb784f-wgqp2                        0/1       ErrImagePull   0          6h
      onap          dev-aai-babel-79d9cc58fb-2vv4j                 0/1       ErrImagePull   0          6h
      onap          dev-aai-data-router-cb4cf6b79-4mshj            0/1       ErrImagePull   0          6h
      onap          dev-aai-gizmo-657cb8556c-dr2qf                 1/2       ErrImagePull   0          6h
      onap          dev-aai-modelloader-7bf7598484-j4hsd           1/2       ErrImagePull   0          6h
      onap          dev-aai-resources-75c558cfd7-pqcwm             1/2       ErrImagePull   0          6h
      onap          dev-aai-search-data-8686bbd58c-rlbmn           1/2       ErrImagePull   0          6h
      onap          dev-appc-0                                     1/2       ErrImagePull   0          6h
      onap          dev-appc-1                                     1/2       ErrImagePull   0          6h
      onap          dev-appc-2                                     1/2       ErrImagePull   0          6h
      onap          dev-appc-cdt-747fb47876-kd647                  0/1       ErrImagePull   0          6h
      onap          dev-appc-dgbuilder-67579b8578-mmqfk            0/1       ErrImagePull   0          6h
      onap          dev-clamp-9dbb4686d-42zw5                      0/1       ErrImagePull   0          6h
      onap          dev-consul-b5d9d5564-hcgs8                     0/1       ErrImagePull   0          6h
      onap          dev-consul-server-8f8f8767-tvrr5               0/1       ErrImagePull   0          6h
      onap          dev-dcae-cloudify-manager-d8748c658-ncr98      0/1       ErrImagePull   0          6h
      onap          dev-dcae-healthcheck-67f5dfc95-fvl52           0/1       ErrImagePull   0          6h
      onap          dev-dcae-redis-0                               0/1       ErrImagePull   0          6h
      onap          dev-dmaap-5c876685bf-cqvgs                     0/1       ErrImagePull   0          6h
      onap          dev-drools-0                                   0/1       ErrImagePull   0          6h
      onap          dev-msb-discovery-6c6dbcd8f-27djv              1/2       ErrImagePull   0          6h
      onap          dev-multicloud-5d4648c6c5-5g7w5                1/2       ErrImagePull   0          6h
      onap          dev-multicloud-ocata-6465ddf889-l68n8          1/2       ErrImagePull   0          6h
      onap          dev-multicloud-vio-5fbf66464-hzqvl             1/2       ErrImagePull   0          6h
      onap          dev-multicloud-windriver-8579976b7c-6bddk      1/2       ErrImagePull   0          6h
      onap          dev-pap-bd6f978c5-mkql2                        1/2       ErrImagePull   0          6h
      onap          dev-portal-cassandra-5f477975d4-wj6bk          0/1       ErrImagePull   0          6h
      onap          dev-portal-db-745747866f-md9v9                 0/1       ErrImagePull   0          6h
      onap          dev-postgres-config-njx74                      0/1       ErrImagePull   0          6h
      onap          dev-robot-5fc4c7846b-t64tt                     0/1       ErrImagePull   0          6h
      onap          dev-sdc-cs-64f45d77dc-xxvp5                    0/1       ErrImagePull   0          6h
      onap          dev-sdc-es-57777d7789-xsl4t                    0/1       ErrImagePull   0          6h
      onap          dev-sdc-wfd-5b5d4f58f6-jmj7n                   0/1       ErrImagePull   0          6h
      onap          dev-sdnc-0                                     1/2       ErrImagePull   0          6h
      onap          dev-sdnc-dgbuilder-c85bdcd-zxx99               0/1       ErrImagePull   0          6h
      onap          dev-sms-857f6dbd87-5854h                       0/1       ErrImagePull   0          6h
      onap          dev-smsdb-0                                    0/2       ErrImagePull   0          6h
      onap          dev-so-6ddb6775b9-wfnhg                        1/2       ErrImagePull   0          6h
      onap          dev-vfc-catalog-7d89bc8b9d-h9276               1/2       ErrImagePull   0          6h
      onap          dev-vfc-ems-driver-864685477c-dsqbj            0/1       ErrImagePull   0          6h
      onap          dev-vfc-generic-vnfm-driver-544c977695-hp9sq   1/2       ErrImagePull   0          6h
      onap          dev-vfc-huawei-vnfm-driver-64665b4b4d-lfj9q    1/2       ErrImagePull   0          6h
      onap          dev-vfc-juju-vnfm-driver-856b9c78ff-ctbkj      1/2       ErrImagePull   0          6h
      onap          dev-vfc-multivim-proxy-767757dfd8-z8zcc        0/1       ErrImagePull   0          6h
      onap          dev-vfc-nokia-v2vnfm-driver-7fbc7dd6d6-vb49p   0/1       ErrImagePull   0          6h
      onap          dev-vfc-nokia-vnfm-driver-5f9c777fd8-fd67p     1/2       ErrImagePull   0          6h
      onap          dev-vfc-nslcm-76fd6648cc-4bsvk                 1/2       ErrImagePull   0          6h
      onap          dev-vfc-resmgr-7dfb9c7554-vcvk7                1/2       ErrImagePull   0          6h
      onap          dev-vfc-vnflcm-7467d9c9b7-tc6ch                1/2       ErrImagePull   0          6h
      onap          dev-vfc-vnfmgr-77855c55f-qmq9p                 1/2       ErrImagePull   0          6h
      onap          dev-vfc-vnfres-5658fc5d74-4sqt4                1/2       ErrImagePull   0          6h
      onap          dev-vfc-workflow-78f6466f9d-6qcvl              0/1       ErrImagePull   0          6h
      onap          dev-vfc-workflow-engine-79769874c7-gbtmb       0/1       ErrImagePull   0          6h
      onap          dev-vfc-zte-sdnc-driver-5b6c7cbd6b-mzfxp       0/1       ErrImagePull   0          6h
      onap          dev-vfc-zte-vnfm-driver-585cc96b6d-wbddb       1/2       ErrImagePull   0          6h
      onap          dev-vid-6686d58688-cfh88                       1/2       ErrImagePull   0          6h
      onap          sniro-emulator-7fc8658bcb-xszm4                0/1       ErrImagePull   0          6h
      ubuntu@ip-10-0-0-206:~$ kubectl get pods --all-namespaces | grep Image | wc -l
      57
      
      checking busted ones
      ubuntu@ip-10-0-0-144:~$ sudo docker pull nexus3.onap.org:10001/onap/babel:1.2-STAGING-latest
      Pulling repository nexus3.onap.org:10001/onap/babel
      Error: image onap/babel:1.2-STAGING-latest not found
      
      ones not in the list are ok
      ubuntu@ip-10-0-0-144:~$ sudo docker pull nexus3.onap.org:10001/openecomp/vid:v1.1.1
      v1.1.1: Pulling from openecomp/vid
      3e17c6eae66c: Pull complete 
      Digest: sha256:c09e2c2911531c3d528b1333ed8399b3f38ddc5c28ca06d01d953f58d08acbff
      Status: Downloaded newer image for nexus3.onap.org:10001/openecomp/vid:v1.1.1
      

            agrimberg agrimberg
            michaelobrien michaelobrien
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: