Uploaded image for project: 'Service Orchestrator'
  1. Service Orchestrator
  2. SO-1109

SO pods not starting 20181005

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Medium Medium
    • Casablanca Release
    • Casablanca Release
    • None

      Need to do pairwise testing - will also attempt just log, so and robot

      ubuntu@ip-172-31-20-99:~/oom/kubernetes$ kubectl get pods --all-namespaces | grep -E '0/|1/2'
      onap          onap-aai-champ-5dbbfb865-tghnz                          1/2       Running            0          2h
      onap          onap-aai-data-router-79d5c56486-vqht8                   1/2       CrashLoopBackOff   26         2h
      onap          onap-controller-blueprints-8b8d654db-rgmqt              0/1       Init:0/1           10         2h
      onap          onap-controller-blueprints-db-0                         0/1       Running            1          2h
      onap          onap-pomba-validation-service-5cf47bcbf-bzcrh           0/1       CrashLoopBackOff   16         2h
      onap          onap-so-77cf664485-bg9jn                                0/1       CrashLoopBackOff   18         2h
      onap          onap-so-bpmn-infra-69c7f9bd76-r92sr                     0/1       CrashLoopBackOff   17         2h
      onap          onap-so-catalog-db-adapter-7b4f44f75c-sfg5q             0/1       Error              19         2h
      onap          onap-so-request-db-adapter-96d778d8c-qcssc              0/1       CrashLoopBackOff   18         2h
      onap          onap-so-vfc-adapter-854d6665c5-bcn6m                    0/1       CrashLoopBackOff   19         2h
      
      ubuntu@ip-172-31-20-99:~/oom/kubernetes$ kubectl get pods --all-namespaces | wc -l
      194
      
      ubuntu@ip-172-31-20-99:~/oom/kubernetes$ kubectl get pods --all-namespaces
      NAMESPACE     NAME                                                    READY     STATUS             RESTARTS   AGE
      kube-system   heapster-6cfb49f776-6xg2c                               1/1       Running            1          11d
      kube-system   kube-dns-75c8cb4ccb-kd8lz                               3/3       Running            3          11d
      kube-system   kubernetes-dashboard-6f4c8b9cd5-x67tn                   1/1       Running            1          11d
      kube-system   monitoring-grafana-76f5b489d5-rm2qz                     1/1       Running            1          11d
      kube-system   monitoring-influxdb-6fc88bd58d-74lf4                    1/1       Running            1          11d
      kube-system   tiller-deploy-8b6c5d4fb-7x7xl                           1/1       Running            1          11d
      onap          dep-config-binding-service-5546cfd778-2vfc8             2/2       Running            0          1h
      onap          dep-dcae-datafile-collector-65d8f89f4d-knrtj            2/2       Running            0          40m
      onap          dep-dcae-hv-ves-collector-6848dbbd5c-dxgvh              2/2       Running            0          40m
      onap          dep-dcae-prh-68f58b46f9-kj2nh                           2/2       Running            0          40m
      onap          dep-dcae-snmptrap-collector-64bb78d7bc-qlsfs            1/1       Running            0          40m
      onap          dep-dcae-tca-analytics-5b55bbb658-88shk                 2/2       Running            0          40m
      onap          dep-dcae-ves-collector-86cc8fb9d4-h9dm7                 2/2       Running            0          40m
      onap          dep-deployment-handler-5bfbbcd9b6-s54sx                 2/2       Running            0          55m
      onap          dep-inventory-bcfcd5987-7mdg8                           1/1       Running            1          55m
      onap          dep-policy-handler-55ddfd69bd-hcb8z                     2/2       Running            0          55m
      onap          dep-pstg-write-77c89cb8c4-pw5px                         1/1       Running            0          55m
      onap          dep-service-change-handler-76b984b76-2grlz              1/1       Running            0          46m
      onap          onap-aaf-cm-65b685449-gnkt4                             1/1       Running            0          2h
      onap          onap-aaf-cs-6ddbb9d674-tm5v8                            1/1       Running            0          2h
      onap          onap-aaf-fs-7b96687d64-22wl6                            1/1       Running            0          2h
      onap          onap-aaf-gui-77d9f8f54-pvx7w                            1/1       Running            0          2h
      onap          onap-aaf-hello-76cb4d748c-mtz52                         1/1       Running            0          2h
      onap          onap-aaf-locate-7866799fd9-z6jm5                        1/1       Running            0          2h
      onap          onap-aaf-oauth-564bbbf568-l4pwz                         1/1       Running            0          2h
      onap          onap-aaf-service-657df4578d-v99d5                       1/1       Running            0          2h
      onap          onap-aaf-sms-844c974c6-k7cvg                            1/1       Running            0          2h
      onap          onap-aaf-sms-quorumclient-0                             1/1       Running            0          2h
      onap          onap-aaf-sms-quorumclient-1                             1/1       Running            2          2h
      onap          onap-aaf-sms-quorumclient-2                             1/1       Running            0          1h
      onap          onap-aaf-sms-vault-0                                    2/2       Running            1          2h
      onap          onap-aai-597c6f496c-8rczn                               1/1       Running            0          2h
      onap          onap-aai-babel-767f46c69d-ng8bx                         2/2       Running            0          2h
      onap          onap-aai-cassandra-0                                    1/1       Running            0          2h
      onap          onap-aai-cassandra-1                                    1/1       Running            0          2h
      onap          onap-aai-cassandra-2                                    1/1       Running            0          1h
      onap          onap-aai-champ-5dbbfb865-tghnz                          1/2       Running            0          2h
      onap          onap-aai-data-router-79d5c56486-vqht8                   1/2       CrashLoopBackOff   24         2h
      onap          onap-aai-elasticsearch-6d5889c967-dds8b                 1/1       Running            0          2h
      onap          onap-aai-gizmo-89c7bfdd-lm949                           2/2       Running            0          2h
      onap          onap-aai-graphadmin-55d5998fc8-rclz7                    2/2       Running            0          2h
      onap          onap-aai-modelloader-5748989fc-8zd2c                    2/2       Running            0          2h
      onap          onap-aai-resources-646c9c99c-vvrvb                      2/2       Running            0          2h
      onap          onap-aai-search-data-788f6ccdd7-szfth                   2/2       Running            0          2h
      onap          onap-aai-sparky-be-7c97d89984-qbltc                     2/2       Running            0          2h
      onap          onap-aai-spike-8488cd8b7f-z6tqb                         2/2       Running            0          2h
      onap          onap-aai-traversal-c55d9d478-6l7g5                      2/2       Running            0          2h
      onap          onap-appc-0                                             2/2       Running            0          2h
      onap          onap-appc-ansible-server-695d7bdf58-5jmmb               1/1       Running            0          2h
      onap          onap-appc-cdt-6b747775d5-f6wqk                          1/1       Running            0          2h
      onap          onap-appc-db-0                                          1/1       Running            0          2h
      onap          onap-appc-db-1                                          1/1       Running            2          1h
      onap          onap-appc-db-2                                          1/1       Running            2          1h
      onap          onap-appc-dgbuilder-789658d75d-grm58                    1/1       Running            0          2h
      onap          onap-brmsgw-74d69dcbd-2n4mz                             1/1       Running            0          2h
      onap          onap-clamp-dash-es-55d979c4b6-ffsb7                     1/1       Running            2          2h
      onap          onap-clamp-dash-kibana-554c57b8b9-dpdx5                 1/1       Running            0          2h
      onap          onap-clamp-dash-logstash-f69cc67f6-5ngn9                1/1       Running            0          2h
      onap          onap-clamp-ff79cd4fb-298kq                              2/2       Running            0          2h
      onap          onap-clampdb-6db5647d79-9pmz7                           1/1       Running            1          2h
      onap          onap-cli-65558c66cf-kb5qn                               1/1       Running            0          2h
      onap          onap-consul-68dfcd6d85-4bkzh                            1/1       Running            0          2h
      onap          onap-consul-server-0                                    1/1       Running            0          2h
      onap          onap-consul-server-1                                    1/1       Running            0          2h
      onap          onap-consul-server-2                                    1/1       Running            0          2h
      onap          onap-controller-blueprints-8b8d654db-rgmqt              0/1       Init:0/1           9          2h
      onap          onap-controller-blueprints-db-0                         0/1       Running            1          2h
      onap          onap-dbc-pg-0                                           1/1       Running            0          2h
      onap          onap-dbc-pg-1                                           1/1       Running            0          1h
      onap          onap-dcae-bootstrap-5b6bcdf77b-7krst                    1/1       Running            0          2h
      onap          onap-dcae-cloudify-manager-7484777bc6-mphwr             1/1       Running            7          2h
      onap          onap-dcae-db-0                                          1/1       Running            0          2h
      onap          onap-dcae-db-1                                          1/1       Running            0          1h
      onap          onap-dcae-healthcheck-7854bbbff4-8ms7p                  1/1       Running            0          2h
      onap          onap-dcae-redis-0                                       1/1       Running            0          2h
      onap          onap-dcae-redis-1                                       1/1       Running            1          2h
      onap          onap-dcae-redis-2                                       1/1       Running            0          1h
      onap          onap-dcae-redis-3                                       1/1       Running            0          1h
      onap          onap-dcae-redis-4                                       1/1       Running            0          56m
      onap          onap-dcae-redis-5                                       1/1       Running            0          49m
      onap          onap-dmaap-bus-controller-d5bc86bf9-qjlbb               1/1       Running            0          2h
      onap          onap-dmaap-dr-db-5b9898d7d9-4vn8b                       1/1       Running            0          2h
      onap          onap-dmaap-dr-node-6c9765bb64-fxd67                     1/1       Running            0          2h
      onap          onap-dmaap-dr-prov-559845765f-jnzbv                     1/1       Running            0          2h
      onap          onap-drools-0                                           1/1       Running            0          2h
      onap          onap-esr-gui-859fd85568-r4zqp                           1/1       Running            0          2h
      onap          onap-esr-server-7b6745d866-9kt24                        2/2       Running            0          2h
      onap          onap-kube2msb-fbbdf4499-5r2t7                           1/1       Running            0          2h
      onap          onap-log-elasticsearch-7557486bc4-nck8q                 1/1       Running            0          2h
      onap          onap-log-kibana-fc88b6b79-w2rnc                         1/1       Running            0          2h
      onap          onap-log-logstash-gjf9s                                 1/1       Running            0          2h
      onap          onap-message-router-546c549f8d-5mq7w                    1/1       Running            0          2h
      onap          onap-message-router-kafka-649c5d79cd-c2qbj              1/1       Running            0          2h
      onap          onap-message-router-zookeeper-6d8b98d584-262nq          1/1       Running            0          2h
      onap          onap-msb-consul-df5cdcdbb-rl6vh                         1/1       Running            0          2h
      onap          onap-msb-discovery-748d88fb76-xvtks                     2/2       Running            0          2h
      onap          onap-msb-eag-c89d486c4-7vghp                            2/2       Running            0          2h
      onap          onap-msb-iag-77f69996d9-p977s                           2/2       Running            0          2h
      onap          onap-multicloud-7955f9fb84-vw74n                        2/2       Running            0          2h
      onap          onap-multicloud-ocata-5ffc7654d8-lcv4d                  2/2       Running            0          2h
      onap          onap-multicloud-vio-85b5bfc64d-zmgn6                    2/2       Running            1          2h
      onap          onap-multicloud-windriver-74f647b456-w8qgh              2/2       Running            0          2h
      onap          onap-nbi-cd96754d7-fs2c2                                1/1       Running            3          2h
      onap          onap-nbi-mariadb-67db6865f4-ctsj4                       1/1       Running            1          2h
      onap          onap-nbi-mongo-0                                        1/1       Running            0          2h
      onap          onap-nengdb-0                                           1/1       Running            0          2h
      onap          onap-netbox-app-64bf7dcdff-9cc85                        1/1       Running            0          2h
      onap          onap-netbox-nginx-5bbfb86594-4tjgs                      1/1       Running            0          2h
      onap          onap-netbox-postgres-65967bccdd-fbx9g                   1/1       Running            0          2h
      onap          onap-network-name-gen-578bd9cf58-mdqw4                  1/1       Running            0          2h
      onap          onap-nexus-d8dd55b95-qvfgc                              1/1       Running            0          2h
      onap          onap-oof-7f5bfb86bc-427z5                               1/1       Running            0          2h
      onap          onap-oof-has-api-6bf66d5459-fc7jx                       1/1       Running            0          2h
      onap          onap-oof-has-cassandra-56dd9c466c-22qc2                 1/1       Running            0          2h
      onap          onap-oof-has-controller-64d658fcf8-8pdjj                1/1       Running            0          2h
      onap          onap-oof-has-data-864d4747c4-s7swl                      1/1       Running            0          2h
      onap          onap-oof-has-music-6f78f5565c-dd4db                     1/1       Running            0          2h
      onap          onap-oof-has-reservation-7744fd5698-7tn5x               1/1       Running            0          2h
      onap          onap-oof-has-solver-5457f775b4-vcjtl                    1/1       Running            0          2h
      onap          onap-oof-has-zookeeper-5bd9cdc875-c6h4v                 1/1       Running            0          2h
      onap          onap-pap-76486b4b54-rcrnv                               2/2       Running            0          2h
      onap          onap-pdp-0                                              2/2       Running            0          2h
      onap          onap-policy-apex-pdp-0                                  1/1       Running            0          2h
      onap          onap-policy-distribution-d7db45d9b-4xvvb                1/1       Running            0          2h
      onap          onap-policydb-5f876cf8bf-hqxqn                          1/1       Running            0          2h
      onap          onap-pomba-aaictxbuilder-56d98d7649-qfvnp               2/2       Running            0          2h
      onap          onap-pomba-contextaggregator-765844bbc9-4vt4p           1/1       Running            0          2h
      onap          onap-pomba-data-router-97fcf9597-jh4jk                  1/1       Running            4          2h
      onap          onap-pomba-elasticsearch-6c8d7bb4f9-tb8lh               1/1       Running            0          2h
      onap          onap-pomba-kibana-d76b6dd4c-tv7x7                       1/1       Running            0          2h
      onap          onap-pomba-networkdiscovery-85d76975b7-l7xmp            2/2       Running            0          2h
      onap          onap-pomba-networkdiscoveryctxbuilder-c89786dfc-hb98z   2/2       Running            0          2h
      onap          onap-pomba-sdcctxbuilder-5bc9cdcd77-62lgx               1/1       Running            0          2h
      onap          onap-pomba-search-data-65c86fbcf9-2cpg7                 2/2       Running            0          2h
      onap          onap-pomba-servicedecomposition-557dd9d669-rv5bv        2/2       Running            0          2h
      onap          onap-pomba-validation-service-5cf47bcbf-bzcrh           0/1       CrashLoopBackOff   15         2h
      onap          onap-portal-app-796bbfdd5b-zvklf                        2/2       Running            0          2h
      onap          onap-portal-cassandra-8588fbd698-xldxp                  1/1       Running            0          2h
      onap          onap-portal-db-55f464cb4f-dkc86                         1/1       Running            0          2h
      onap          onap-portal-sdk-74b9544457-2rdzx                        2/2       Running            0          2h
      onap          onap-portal-widget-654ff7859-hx76m                      1/1       Running            0          2h
      onap          onap-portal-zookeeper-5d8c598c4c-nzkrp                  1/1       Running            0          2h
      onap          onap-robot-7d6ccbf489-7bwdj                             1/1       Running            0          2h
      onap          onap-sdc-be-57ccbcdb5-6ltrq                             2/2       Running            0          2h
      onap          onap-sdc-cs-c76c4bf55-v5fpg                             1/1       Running            0          2h
      onap          onap-sdc-es-5778dc6bcd-p88x7                            1/1       Running            0          2h
      onap          onap-sdc-fe-6866dd9b6-45vv4                             2/2       Running            0          2h
      onap          onap-sdc-kb-5df7dc56dc-ljpkd                            1/1       Running            0          2h
      onap          onap-sdc-onboarding-be-7b95c87656-62kkc                 2/2       Running            0          2h
      onap          onap-sdc-wfd-be-7bfdb8bc69-hkgls                        1/1       Running            0          2h
      onap          onap-sdc-wfd-fe-6ff9ff5666-znbmb                        2/2       Running            0          2h
      onap          onap-sdnc-0                                             2/2       Running            0          2h
      onap          onap-sdnc-ansible-server-dc55dbb6-vjk8d                 1/1       Running            0          2h
      onap          onap-sdnc-db-0                                          2/2       Running            0          2h
      onap          onap-sdnc-dgbuilder-78688789fd-kcm5s                    1/1       Running            0          2h
      onap          onap-sdnc-dmaap-listener-b79df77c9-mgrpf                1/1       Running            0          2h
      onap          onap-sdnc-portal-7b779f87f5-6mnnq                       1/1       Running            0          2h
      onap          onap-sdnc-ueb-listener-5bcdbb8677-tww2f                 1/1       Running            0          2h
      onap          onap-sniro-emulator-f668fdb9-cwqtg                      1/1       Running            0          2h
      onap          onap-so-77cf664485-bg9jn                                0/1       CrashLoopBackOff   17         2h
      onap          onap-so-bpmn-infra-69c7f9bd76-r92sr                     0/1       CrashLoopBackOff   16         2h
      onap          onap-so-catalog-db-adapter-7b4f44f75c-sfg5q             0/1       CrashLoopBackOff   17         2h
      onap          onap-so-mariadb-84d684494d-sb5rs                        1/1       Running            1          2h
      onap          onap-so-openstack-adapter-6c4fd8d7fc-b5ctq              0/1       CrashLoopBackOff   15         2h
      onap          onap-so-request-db-adapter-96d778d8c-qcssc              0/1       CrashLoopBackOff   16         2h
      onap          onap-so-sdc-controller-55c479fb9b-ktls4                 0/1       CrashLoopBackOff   18         2h
      onap          onap-so-sdnc-adapter-76799b95d7-kkchj                   1/1       Running            1          2h
      onap          onap-so-vfc-adapter-854d6665c5-bcn6m                    1/1       Running            18         2h
      onap          onap-uui-cf7f9c4c4-thcrf                                1/1       Running            1          2h
      onap          onap-uui-server-6c8ff6544-xpzh7                         1/1       Running            0          2h
      onap          onap-vfc-catalog-5bd7d6bddf-pcrmz                       2/2       Running            3          2h
      onap          onap-vfc-db-678f484cdd-k45g6                            1/1       Running            0          2h
      onap          onap-vfc-ems-driver-b58764c48-pg474                     1/1       Running            6          2h
      onap          onap-vfc-generic-vnfm-driver-84bf45c5df-sfr8s           2/2       Running            0          2h
      onap          onap-vfc-huawei-vnfm-driver-64c9ddcffd-gpjlm            2/2       Running            0          2h
      onap          onap-vfc-juju-vnfm-driver-557758b87f-cjrwt              2/2       Running            0          2h
      onap          onap-vfc-multivim-proxy-5986c8695-gwcg2                 1/1       Running            0          2h
      onap          onap-vfc-nokia-v2vnfm-driver-56955fcd5d-9nlbb           1/1       Running            0          2h
      onap          onap-vfc-nokia-vnfm-driver-65fdfd478-xhzvd              2/2       Running            0          2h
      onap          onap-vfc-nslcm-74cfb48956-vkqcg                         2/2       Running            0          2h
      onap          onap-vfc-resmgr-5b55888bb4-tkcqv                        2/2       Running            0          2h
      onap          onap-vfc-vnflcm-646584d587-cqt92                        2/2       Running            3          2h
      onap          onap-vfc-vnfmgr-bcbc9c877-d9thq                         2/2       Running            0          2h
      onap          onap-vfc-vnfres-6848d65656-bjs9q                        2/2       Running            0          2h
      onap          onap-vfc-workflow-5c788dd6d5-7nkhh                      1/1       Running            0          2h
      onap          onap-vfc-workflow-engine-59d9f97858-9gxhd               1/1       Running            0          2h
      onap          onap-vfc-zte-sdnc-driver-856fc764f5-478rt               1/1       Running            0          2h
      onap          onap-vfc-zte-vnfm-driver-75d9b75b4b-l4gbw               2/2       Running            0          2h
      onap          onap-vid-67d4f5c468-wk5x6                               2/2       Running            0          2h
      onap          onap-vid-mariadb-galera-0                               1/1       Running            0          2h
      onap          onap-vnfsdk-94bb87c8b-fpzrt                             1/1       Running            0          2h
      onap          onap-vnfsdk-postgres-0                                  1/1       Running            0          2h
      onap          onap-vnfsdk-postgres-1                                  1/1       Running            0          1h
      

            mfour mfour
            michaelobrien michaelobrien
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: