-
Bug
-
Resolution: Done
-
Medium
-
Amsterdam Release
-
None
-
None
-
Openlab
network is the external on on openlab as of 20180203
public_net_id: 971040b2-7059-49dc-b220-4fab50cb2ad4 public_net_name: external ubuntu_1404_image: ubuntu-14-04-cloud-amd64 ubuntu_1604_image: ubuntu-16-04-cloud-amd64 flavor_small: m1.small flavor_medium: m1.medium flavor_large: m1.large flavor_xlarge: m1.xlarge flavor_xxlarge: m1.xxlarge vm_base_name: onap key_name: onap_key nexus_repo: https://nexus.onap.org/content/sites/raw nexus_docker_repo: nexus3.onap.org:10001 nexus_username: docker nexus_password: docker dmaap_topic: AUTO artifacts_version: 1.2.0-SNAPSHOT openstack_auth_method: password openstack_region: RegionOne horizon_url: http://10.12.25.2:5000/v3 keystone_url: http://10.12.25.2:5000 cloud_env: openstack dns_list: ["10.12.25.5", "8.8.8.8"] external_dns: 8.8.8.8 dns_forwarder: 110.12.25.5 oam_network_cidr: 10.0.0.0/16 dnsaas_config_enabled: true dnsaas_region: RegionOne dnsaas_keystone_url: http://10.12.25.5:5000/v3 dnsaas_tenant_name: Logging dnsaas_username: demo dnsaas_password: onapdemo dcae_keystone_url: http://10.12.25.5:5000/v2 dcae_centos_7_image: CentOS-7 dcae_domain: dcaeg2.onap.org dcae_public_key: ssh-rsa AAAAB3N....gUHlRcVWV7lNo+418Tt michaelobrien@obrienbiometrics dcae_private_key: -----BEGIN RSA PRIVATE KEY-----MIIEow...Zl3t5s-----END RSA PRIVATE KEY-----
obrienbiometrics:20180203 michaelobrien$ openstack stack create -t onap_openstack.yaml -e onap_openstack.env onap-20180203 +---------------------+------------------------------------------+ | Field | Value | +---------------------+------------------------------------------+ | id | da967f18-890c-497a-af10-4eb70ae880eb | | stack_name | onap-20180203 | | description | Heat template to install ONAP components | | creation_time | 2018-02-03T15:50:27Z | | updated_time | 2018-02-03T15:50:27Z | | stack_status | CREATE_IN_PROGRESS | | stack_status_reason | Stack CREATE started | +---------------------+------------------------------------------+ obrienbiometrics:20180203 michaelobrien$ openstack server list +--------------------------------------+---------------------+--------+----------------------------------------------------+--------------------------+ | ID | Name | Status | Networks | Image Name | +--------------------------------------+---------------------+--------+----------------------------------------------------+--------------------------+ | c2e6d564-f954-4245-83dc-92e6609e47f5 | onap-aai-inst1 | ACTIVE | oam_onap_0LrL=10.0.1.1, 10.12.6.118 | ubuntu-14-04-cloud-amd64 | | 3c79028d-d912-4c0c-800b-921cf0ec5b03 | onap-dcae-bootstrap | ACTIVE | oam_onap_0LrL=10.0.4.1, 10.12.6.113 | ubuntu-16-04-cloud-amd64 | | 217f70c8-d18f-431b-9765-284ccdbddffe | onap-policy | ACTIVE | oam_onap_0LrL=10.0.6.1, 10.12.6.132 | ubuntu-14-04-cloud-amd64 | | 8c60ad7b-bb31-4e15-868d-65e6e3f9823d | onap-portal | ACTIVE | oam_onap_0LrL=10.0.9.1, 10.12.6.117 | ubuntu-14-04-cloud-amd64 | | fd24feac-bb30-4b53-944b-fa153415dbd7 | onap-aai-inst2 | ACTIVE | oam_onap_0LrL=10.0.1.2, 10.12.6.130 | ubuntu-14-04-cloud-amd64 | | b5785e64-fcd7-479c-877d-1305dbda80fa | onap-robot | ACTIVE | oam_onap_0LrL=10.0.10.1, 10.12.6.119 | ubuntu-16-04-cloud-amd64 | | 444e62bf-7641-4968-9be1-b9c82faab23a | onap-appc | ACTIVE | oam_onap_0LrL=10.0.2.1, 10.12.6.124 | ubuntu-14-04-cloud-amd64 | | 94e4d421-ccfb-4d22-ae30-d7a2b084cd41 | onap-so | ACTIVE | oam_onap_0LrL=10.0.5.1, 10.12.6.126 | ubuntu-16-04-cloud-amd64 | | db4cc86f-5aac-421a-bdc7-7d9b5a39d484 | onap-vid | ACTIVE | oam_onap_0LrL=10.0.8.1, 10.12.6.121 | ubuntu-14-04-cloud-amd64 | | eb7cb65d-99c3-4fce-b1b2-e38c1395565c | onap-sdc | ACTIVE | oam_onap_0LrL=10.0.3.1, 10.12.6.131 | ubuntu-16-04-cloud-amd64 | | 31ca0ced-f768-4887-b4e7-bf186bc7f795 | onap-sdnc | ACTIVE | oam_onap_0LrL=10.0.7.1, 10.12.6.123 | ubuntu-14-04-cloud-amd64 | | 265c430b-4096-4e4f-8abd-d55ef6308e1e | onap-dns-server | ACTIVE | oam_onap_0LrL=10.0.100.1, 10.12.6.127 | ubuntu-14-04-cloud-amd64 | | eac383d1-d8e6-486b-827a-9347e16e1f9d | onap-multi-service | ACTIVE | oam_onap_0LrL=10.0.14.1, 10.12.5.141 | ubuntu-16-04-cloud-amd64 | | b0fd57d2-8a91-41c4-9951-5f245026c711 | onap-clamp | ACTIVE | oam_onap_0LrL=10.0.12.1, 10.12.6.120 | ubuntu-16-04-cloud-amd64 | | f39f47f8-8bda-4119-9765-18ec7513d748 | onap-message-router | ACTIVE | oam_onap_0LrL=10.0.11.1, 10.12.5.45 | ubuntu-14-04-cloud-amd64 | | 70b624b4-8bd9-4a1f-bbbf-234a17c8e2b2 | obrien-oom-1012 | ACTIVE | external=10.12.6.59 | | | d80f35ac-1257-47fc-828e-dddc3604d3c1 | oom-jenkins | ACTIVE | appc-multicloud-integration=10.10.5.14, 10.12.6.49 | | +--------------------------------------+---------------------+--------+----------------------------------------------------+--------------------------+
logged in to the DCAE controller and boostrapping the install - waiting on aai
brienbiometrics:20180203 michaelobrien$ ssh root@10.12.6.113 root@onap-dcae-bootstrap:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES root@onap-dcae-bootstrap:/opt# ls app config dcae2_install.sh dcae2_vm_init.sh docker root@onap-dcae-bootstrap:/opt# ./dcae2_vm_init.sh Login Succeeded Using proxyed DNSaaS service, performing additional registration and configuration ===> Waiting for A&AI to get ready for getting 200 from https://10.0.1.1:8443/aai/v11/examples/cloud-regions @ Sat Feb 3 16:01:43 UTC 2018 RESP CODE 000, not as expected RESP CODE 200 @ Sat Feb 3 16:01:43 UTC 2018. v1.1.1: Pulling from onap/org.onap.dcaegen2.deployments.bootstrap 50aff78429b1: Already exists f6d82e297bce: Already exists 275abb2c8a6f: Already exists 9f15a39356d6: Already exists fc0342a94c89: Already exists fd97135e26f3: Downloading [=========================================> ] 135.8MB/163.3MB fd97135e26f3: Downloading [===============================================> ] 153.9MB/163.3MB fd97135e26f3: Downloading [================================================> ] 158.7MB/163.3MB fd97135e26f3: Pull complete 65cb4d0362f2: Pull complete ca9b192fa64b: Pull complete cde868d37e2f: Pull complete e0c7601b44dd: Pull complete a7efac1be55b: Pull complete Digest: sha256:4df1298cb8876b0ff3caabdede096bc51dd916e52b59d82d28c2fd5ffff5fa82 Status: Downloaded newer image for nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.bootstrap:v1.1.1 Using default tag: latest RESP CODE 000, not as expected RESP CODE 200 @ Sat Feb 3 16:03:43 UTC 2018. latest: Pulling from library/nginx e7bb522d92ff: Downloading [===> ] 1.375MB/22.49MB e7bb522d92ff: Pull complete 6edc05228666: Downloading [=============================================> ] 20.17MB/21.95MB 6edc05228666: Pull complete cd866a17e81f: Pull complete Digest: sha256:285b49d42c703fdf257d1e2422765c4ba9d3e37768d6ea83d7fe2043dad6e63d Status: Downloaded newer image for nginx:latest RESP CODE 000, not as expected RESP CODE 200 @ Sat Feb 3 16:05:14 UTC 2018. RESP CODE 000, not as expected RESP CODE 200 @ Sat Feb 3 16:05:44 UTC 2018.
even after 24 min all of the docker containers on aai1 are not up and only 2 of 3 on aai2
checking logs
root@onap-aai-inst1:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES root@onap-aai-inst1:~# free obrienbiometrics:20180203 michaelobrien$ ssh root@10.12.6.130 root@onap-aai-inst2:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1118a72cd73d elasticsearch:2.4.1 "/docker-entrypoin..." 24 minutes ago Up 24 minutes 0.0.0.0:9200->9200/tcp, 9300/tcp elasticsearch 23e9bd0f5c77 aaionap/hbase:1.2.0 "/bin/sh -c \"/entr..." 25 minutes ago Up 25 minutes 0.0.0.0:2181->2181/tcp, 0.0.0.0:8080->8080/tcp, 0.0.0.0:8085->8085/tcp, 0.0.0.0:9090->9090/tcp, 0.0.0.0:16000->16000/tcp, 0.0.0.0:16010->16010/tcp, 9095/tcp, 0.0.0.0:16201->16201/tcp, 16301/tcp testconfig_aai.hbase.simpledemo.onap.org_1
applying fix from the latest on ---------- that I raised and several of us reopened since Nov in resolve.confAAI-460
https://jira.onap.org/browse/AAI-460however does look like a docker pull slowdown - after 36 min - 2 of 6 aai-vm1 containers are coming up
root@onap-aai-inst1:~# docker ps -a 2f94070c62ba nexus3.onap.org:10001/onap/aai-traversal "/bin/bash /opt/ap..." 6 minutes ago Up 6 minutes 0.0.0.0:8446->8446/tcp, 8447/tcp testconfig_aai-traversal.api.simpledemo.onap.org_1 e9b6bbf76f01 nexus3.onap.org:10001/onap/aai-resources "/bin/bash /opt/ap..." 10 minutes ago Up 10 minutes 0.0.0.0:8447->8447/tcp testconfig_aai-resources.api.simpledemo.onap.org_1 a reboot of aai1 - fixed everything root@onap-aai-inst1:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 343ec0122dcc nexus3.onap.org:10001/onap/model-loader "/opt/app/model-lo..." 45 minutes ago Up 45 minutes testconfig_model-loader_1 501a9a4e1caa nexus3.onap.org:10001/onap/data-router "/opt/app/data-rou..." 45 minutes ago Up 45 minutes 0.0.0.0:9502->9502/tcp testconfig_datarouter_1 d578ca45d7d7 nexus3.onap.org:10001/onap/search-data-service "/opt/app/search-d..." 45 minutes ago Up 45 minutes 0.0.0.0:9509->9509/tcp testconfig_aai.searchservice.simpledemo.openecomp.org_1 52fcd7cfd125 nexus3.onap.org:10001/onap/sparky-be "/bin/sh -c /opt/a..." 45 minutes ago Up 45 minutes 8000/tcp, 0.0.0.0:9517->9517/tcp testconfig_sparky-be_1 96b13d0d1067 aaionap/haproxy:1.2.0 "/docker-entrypoin..." 45 minutes ago Up 45 minutes 0.0.0.0:8443->8443/tcp testconfig_aai.api.simpledemo.onap.org_1 5dc72e5c76d1 nexus3.onap.org:10001/onap/aai-traversal "/bin/bash /opt/ap..." About an hour ago Up About an hour 0.0.0.0:8446->8446/tcp, 8447/tcp testconfig_aai-traversal.api.simpledemo.onap.org_1 a00450104506 nexus3.onap.org:10001/onap/aai-resources "/bin/bash /opt/ap..." About an hour ago Up About an hour 0.0.0.0:8447->8447/tcp testconfig_aai-resources.api.simpledemo.onap.org_1 rebooting the dcae_controller root@onap-dcae-bootstrap:~# ps -ef | grep docker root 1034 1 0 19:03 ? 00:00:00 /usr/bin/dockerd -H fd:// --mtu=1500 root 1164 1034 0 19:03 ? 00:00:00 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/containerd --shim docker-containerd-shim --runtime docker-runc root 1582 1564 0 19:05 pts/0 00:00:00 grep --color=auto docker running the init script * Connection #1 to host 10.0.14.1 left intact =====> After creation, zone 0LrL.dcaeg2.onap.org. ID is 50b76683-664b-4654-850e-55440dab4e5f registration and configuration for proxying DNSaaS completed. docker: Error response from daemon: Conflict. The container name "/boot" is already in use by container "c6e6d1170f634069511c330562c23f4b5708475178197a639f569d44ab5f1f2a". You have to remove (or rename) that container to be able to reuse that name. See 'docker run --help'. Waiting for Consul to become accessible . v1.1.1: Pulling from onap/org.onap.dcaegen2.deployments.bootstrap Digest: sha256:4df1298cb8876b0ff3caabdede096bc51dd916e52b59d82d28c2fd5ffff5fa82 Status: Image is up to date for nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.bootstrap:v1.1.1 Using default tag: latest deleted the stoped docker conatiner - reran init root@onap-dcae-bootstrap:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ca02d23a5e13 nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.bootstrap:v1.1.1 "/bin/sh -c 'exec ..." 30 seconds ago Up 30 seconds boot Checking out why DCAE is failing to orchestrate consul in HEAT. I previously rebooted AAI-vm1 to get the 7 containers up that were blocking the initial AAI rest calls from DCAE Goal is to be able to shut down all HEAT vms except the DCAE subset (cdap, consul...) getting a neutron network error - checking id's root@onap-dcae-bootstrap:/opt# docker logs -f boot Initiated ./blueprints/centos_vm.yaml If you make changes to the blueprint, run `cfy local init -p ./blueprints/centos_vm.yaml` again to apply them + cfy local execute -w install --task-retries=10 2018-02-03 19:16:19 CFY <local> Starting 'install' workflow execution 2018-02-03 19:16:19 CFY <local> [security_group_15499] Creating node 2018-02-03 19:16:19 CFY <local> [key_pair_a883e] Creating node 2018-02-03 19:16:19 CFY <local> [private_net_22910] Creating node 2018-02-03 19:16:19 CFY <local> [floatingip_vm00_3413c] Creating node 2018-02-03 19:16:19 CFY <local> [private_net_22910.create] Sending task 'neutron_plugin.network.create' 2018-02-03 19:16:19 CFY <local> [floatingip_vm00_3413c.create] Sending task 'neutron_plugin.floatingip.create' 2018-02-03 19:16:19 CFY <local> [key_pair_a883e.create] Sending task 'nova_plugin.keypair.create' 2018-02-03 19:16:19 CFY <local> [security_group_15499.create] Sending task 'neutron_plugin.security_group.create' 2018-02-03 19:16:19 CFY <local> [private_net_22910.create] Task started 'neutron_plugin.network.create' 2018-02-03 19:16:19 CFY <local> [floatingip_vm00_3413c.create] Task started 'neutron_plugin.floatingip.create' 2018-02-03 19:16:19 CFY <local> [key_pair_a883e.create] Task started 'nova_plugin.keypair.create' 2018-02-03 19:16:19 CFY <local> [security_group_15499.create] Task started 'neutron_plugin.security_group.create' 2018-02-03 19:16:19 CFY <local> [private_net_22910.create] Task failed 'neutron_plugin.network.create' -> {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} [status_code=401] 2018-02-03 19:16:19 CFY <local> 'install' workflow execution failed: Workflow failed: Task failed 'neutron_plugin.network.create' -> {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} [status_code=401] Workflow failed: Task failed 'neutron_plugin.network.create' -> {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} [status_code=401] root@onap-dcae-bootstrap:/opt# docker ps
Openlab network is the external on on openlab as of 20180203 public_net_id: 971040b2-7059-49dc-b220-4fab50cb2ad4 public_net_name: external ubuntu_1404_image: ubuntu-14-04-cloud-amd64 ubuntu_1604_image: ubuntu-16-04-cloud-amd64 flavor_small: m1.small flavor_medium: m1.medium flavor_large: m1.large flavor_xlarge: m1.xlarge flavor_xxlarge: m1.xxlarge vm_base_name: onap key_name: onap_key nexus_repo: https://nexus.onap.org/content/sites/raw nexus_docker_repo: nexus3.onap.org:10001 nexus_username: docker nexus_password: docker dmaap_topic: AUTO artifacts_version: 1.2.0-SNAPSHOT openstack_auth_method: password openstack_region: RegionOne horizon_url: http://10.12.25.2:5000/v3 keystone_url: http://10.12.25.2:5000 cloud_env: openstack dns_list: ["10.12.25.5", "8.8.8.8"] external_dns: 8.8.8.8 dns_forwarder: 110.12.25.5 oam_network_cidr: 10.0.0.0/16 dnsaas_config_enabled: true dnsaas_region: RegionOne dnsaas_keystone_url: http://10.12.25.5:5000/v3 dnsaas_tenant_name: Logging dnsaas_username: demo dnsaas_password: onapdemo dcae_keystone_url: http://10.12.25.5:5000/v2 dcae_centos_7_image: CentOS-7 dcae_domain: dcaeg2.onap.org dcae_public_key: ssh-rsa AAAAB3N....gUHlRcVWV7lNo+418Tt michaelobrien@obrienbiometrics dcae_private_key: -----BEGIN RSA PRIVATE KEY-----MIIEow...Zl3t5s-----END RSA PRIVATE KEY----- obrienbiometrics:20180203 michaelobrien$ openstack stack create -t onap_openstack.yaml -e onap_openstack.env onap-20180203 +---------------------+------------------------------------------+ | Field | Value | +---------------------+------------------------------------------+ | id | da967f18-890c-497a-af10-4eb70ae880eb | | stack_name | onap-20180203 | | description | Heat template to install ONAP components | | creation_time | 2018-02-03T15:50:27Z | | updated_time | 2018-02-03T15:50:27Z | | stack_status | CREATE_IN_PROGRESS | | stack_status_reason | Stack CREATE started | +---------------------+------------------------------------------+ obrienbiometrics:20180203 michaelobrien$ openstack server list +--------------------------------------+---------------------+--------+----------------------------------------------------+--------------------------+ | ID | Name | Status | Networks | Image Name | +--------------------------------------+---------------------+--------+----------------------------------------------------+--------------------------+ | c2e6d564-f954-4245-83dc-92e6609e47f5 | onap-aai-inst1 | ACTIVE | oam_onap_0LrL=10.0.1.1, 10.12.6.118 | ubuntu-14-04-cloud-amd64 | | 3c79028d-d912-4c0c-800b-921cf0ec5b03 | onap-dcae-bootstrap | ACTIVE | oam_onap_0LrL=10.0.4.1, 10.12.6.113 | ubuntu-16-04-cloud-amd64 | | 217f70c8-d18f-431b-9765-284ccdbddffe | onap-policy | ACTIVE | oam_onap_0LrL=10.0.6.1, 10.12.6.132 | ubuntu-14-04-cloud-amd64 | | 8c60ad7b-bb31-4e15-868d-65e6e3f9823d | onap-portal | ACTIVE | oam_onap_0LrL=10.0.9.1, 10.12.6.117 | ubuntu-14-04-cloud-amd64 | | fd24feac-bb30-4b53-944b-fa153415dbd7 | onap-aai-inst2 | ACTIVE | oam_onap_0LrL=10.0.1.2, 10.12.6.130 | ubuntu-14-04-cloud-amd64 | | b5785e64-fcd7-479c-877d-1305dbda80fa | onap-robot | ACTIVE | oam_onap_0LrL=10.0.10.1, 10.12.6.119 | ubuntu-16-04-cloud-amd64 | | 444e62bf-7641-4968-9be1-b9c82faab23a | onap-appc | ACTIVE | oam_onap_0LrL=10.0.2.1, 10.12.6.124 | ubuntu-14-04-cloud-amd64 | | 94e4d421-ccfb-4d22-ae30-d7a2b084cd41 | onap-so | ACTIVE | oam_onap_0LrL=10.0.5.1, 10.12.6.126 | ubuntu-16-04-cloud-amd64 | | db4cc86f-5aac-421a-bdc7-7d9b5a39d484 | onap-vid | ACTIVE | oam_onap_0LrL=10.0.8.1, 10.12.6.121 | ubuntu-14-04-cloud-amd64 | | eb7cb65d-99c3-4fce-b1b2-e38c1395565c | onap-sdc | ACTIVE | oam_onap_0LrL=10.0.3.1, 10.12.6.131 | ubuntu-16-04-cloud-amd64 | | 31ca0ced-f768-4887-b4e7-bf186bc7f795 | onap-sdnc | ACTIVE | oam_onap_0LrL=10.0.7.1, 10.12.6.123 | ubuntu-14-04-cloud-amd64 | | 265c430b-4096-4e4f-8abd-d55ef6308e1e | onap-dns-server | ACTIVE | oam_onap_0LrL=10.0.100.1, 10.12.6.127 | ubuntu-14-04-cloud-amd64 | | eac383d1-d8e6-486b-827a-9347e16e1f9d | onap-multi-service | ACTIVE | oam_onap_0LrL=10.0.14.1, 10.12.5.141 | ubuntu-16-04-cloud-amd64 | | b0fd57d2-8a91-41c4-9951-5f245026c711 | onap-clamp | ACTIVE | oam_onap_0LrL=10.0.12.1, 10.12.6.120 | ubuntu-16-04-cloud-amd64 | | f39f47f8-8bda-4119-9765-18ec7513d748 | onap-message-router | ACTIVE | oam_onap_0LrL=10.0.11.1, 10.12.5.45 | ubuntu-14-04-cloud-amd64 | | 70b624b4-8bd9-4a1f-bbbf-234a17c8e2b2 | obrien-oom-1012 | ACTIVE | external=10.12.6.59 | | | d80f35ac-1257-47fc-828e-dddc3604d3c1 | oom-jenkins | ACTIVE | appc-multicloud-integration=10.10.5.14, 10.12.6.49 | | +--------------------------------------+---------------------+--------+----------------------------------------------------+--------------------------+ logged in to the DCAE controller and boostrapping the install - waiting on aai brienbiometrics:20180203 michaelobrien$ ssh root@10.12.6.113 root@onap-dcae-bootstrap:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES root@onap-dcae-bootstrap:/opt# ls app config dcae2_install.sh dcae2_vm_init.sh docker root@onap-dcae-bootstrap:/opt# ./dcae2_vm_init.sh Login Succeeded Using proxyed DNSaaS service, performing additional registration and configuration ===> Waiting for A&AI to get ready for getting 200 from https://10.0.1.1:8443/aai/v11/examples/cloud-regions @ Sat Feb 3 16:01:43 UTC 2018 RESP CODE 000, not as expected RESP CODE 200 @ Sat Feb 3 16:01:43 UTC 2018. v1.1.1: Pulling from onap/org.onap.dcaegen2.deployments.bootstrap 50aff78429b1: Already exists f6d82e297bce: Already exists 275abb2c8a6f: Already exists 9f15a39356d6: Already exists fc0342a94c89: Already exists fd97135e26f3: Downloading [=========================================> ] 135.8MB/163.3MB fd97135e26f3: Downloading [===============================================> ] 153.9MB/163.3MB fd97135e26f3: Downloading [================================================> ] 158.7MB/163.3MB fd97135e26f3: Pull complete 65cb4d0362f2: Pull complete ca9b192fa64b: Pull complete cde868d37e2f: Pull complete e0c7601b44dd: Pull complete a7efac1be55b: Pull complete Digest: sha256:4df1298cb8876b0ff3caabdede096bc51dd916e52b59d82d28c2fd5ffff5fa82 Status: Downloaded newer image for nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.bootstrap:v1.1.1 Using default tag: latest RESP CODE 000, not as expected RESP CODE 200 @ Sat Feb 3 16:03:43 UTC 2018. latest: Pulling from library/nginx e7bb522d92ff: Downloading [===> ] 1.375MB/22.49MB e7bb522d92ff: Pull complete 6edc05228666: Downloading [=============================================> ] 20.17MB/21.95MB 6edc05228666: Pull complete cd866a17e81f: Pull complete Digest: sha256:285b49d42c703fdf257d1e2422765c4ba9d3e37768d6ea83d7fe2043dad6e63d Status: Downloaded newer image for nginx:latest RESP CODE 000, not as expected RESP CODE 200 @ Sat Feb 3 16:05:14 UTC 2018. RESP CODE 000, not as expected RESP CODE 200 @ Sat Feb 3 16:05:44 UTC 2018. even after 24 min all of the docker containers on aai1 are not up and only 2 of 3 on aai2 checking logs root@onap-aai-inst1:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES root@onap-aai-inst1:~# free obrienbiometrics:20180203 michaelobrien$ ssh root@10.12.6.130 root@onap-aai-inst2:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1118a72cd73d elasticsearch:2.4.1 "/docker-entrypoin..." 24 minutes ago Up 24 minutes 0.0.0.0:9200->9200/tcp, 9300/tcp elasticsearch 23e9bd0f5c77 aaionap/hbase:1.2.0 "/bin/sh -c \"/entr..." 25 minutes ago Up 25 minutes 0.0.0.0:2181->2181/tcp, 0.0.0.0:8080->8080/tcp, 0.0.0.0:8085->8085/tcp, 0.0.0.0:9090->9090/tcp, 0.0.0.0:16000->16000/tcp, 0.0.0.0:16010->16010/tcp, 9095/tcp, 0.0.0.0:16201->16201/tcp, 16301/tcp testconfig_aai.hbase.simpledemo.onap.org_1 applying fix from the latest on ----- AAI-460 ----- that I raised and several of us reopened since Nov in resolve.conf https://jira.onap.org/browse/AAI-460 however does look like a docker pull slowdown - after 36 min - 2 of 6 aai-vm1 containers are coming up root@onap-aai-inst1:~# docker ps -a 2f94070c62ba nexus3.onap.org:10001/onap/aai-traversal "/bin/bash /opt/ap..." 6 minutes ago Up 6 minutes 0.0.0.0:8446->8446/tcp, 8447/tcp testconfig_aai-traversal.api.simpledemo.onap.org_1 e9b6bbf76f01 nexus3.onap.org:10001/onap/aai-resources "/bin/bash /opt/ap..." 10 minutes ago Up 10 minutes 0.0.0.0:8447->8447/tcp testconfig_aai-resources.api.simpledemo.onap.org_1 a reboot of aai1 - fixed everything root@onap-aai-inst1:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 343ec0122dcc nexus3.onap.org:10001/onap/model-loader "/opt/app/model-lo..." 45 minutes ago Up 45 minutes testconfig_model-loader_1 501a9a4e1caa nexus3.onap.org:10001/onap/data-router "/opt/app/data-rou..." 45 minutes ago Up 45 minutes 0.0.0.0:9502->9502/tcp testconfig_datarouter_1 d578ca45d7d7 nexus3.onap.org:10001/onap/search-data-service "/opt/app/search-d..." 45 minutes ago Up 45 minutes 0.0.0.0:9509->9509/tcp testconfig_aai.searchservice.simpledemo.openecomp.org_1 52fcd7cfd125 nexus3.onap.org:10001/onap/sparky-be "/bin/sh -c /opt/a..." 45 minutes ago Up 45 minutes 8000/tcp, 0.0.0.0:9517->9517/tcp testconfig_sparky-be_1 96b13d0d1067 aaionap/haproxy:1.2.0 "/docker-entrypoin..." 45 minutes ago Up 45 minutes 0.0.0.0:8443->8443/tcp testconfig_aai.api.simpledemo.onap.org_1 5dc72e5c76d1 nexus3.onap.org:10001/onap/aai-traversal "/bin/bash /opt/ap..." About an hour ago Up About an hour 0.0.0.0:8446->8446/tcp, 8447/tcp testconfig_aai-traversal.api.simpledemo.onap.org_1 a00450104506 nexus3.onap.org:10001/onap/aai-resources "/bin/bash /opt/ap..." About an hour ago Up About an hour 0.0.0.0:8447->8447/tcp testconfig_aai-resources.api.simpledemo.onap.org_1 rebooting the dcae_controller root@onap-dcae-bootstrap:~# ps -ef | grep docker root 1034 1 0 19:03 ? 00:00:00 /usr/bin/dockerd -H fd:// --mtu=1500 root 1164 1034 0 19:03 ? 00:00:00 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/containerd --shim docker-containerd-shim --runtime docker-runc root 1582 1564 0 19:05 pts/0 00:00:00 grep --color=auto docker running the init script * Connection #1 to host 10.0.14.1 left intact =====> After creation, zone 0LrL.dcaeg2.onap.org. ID is 50b76683-664b-4654-850e-55440dab4e5f registration and configuration for proxying DNSaaS completed. docker: Error response from daemon: Conflict. The container name "/boot" is already in use by container "c6e6d1170f634069511c330562c23f4b5708475178197a639f569d44ab5f1f2a". You have to remove (or rename) that container to be able to reuse that name. See 'docker run --help'. Waiting for Consul to become accessible . v1.1.1: Pulling from onap/org.onap.dcaegen2.deployments.bootstrap Digest: sha256:4df1298cb8876b0ff3caabdede096bc51dd916e52b59d82d28c2fd5ffff5fa82 Status: Image is up to date for nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.bootstrap:v1.1.1 Using default tag: latest deleted the stoped docker conatiner - reran init root@onap-dcae-bootstrap:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ca02d23a5e13 nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.bootstrap:v1.1.1 "/bin/sh -c 'exec ..." 30 seconds ago Up 30 seconds boot Checking out why DCAE is failing to orchestrate consul in HEAT. I previously rebooted AAI-vm1 to get the 7 containers up that were blocking the initial AAI rest calls from DCAE Goal is to be able to shut down all HEAT vms except the DCAE subset (cdap, consul...) getting a neutron network error - checking id's root@onap-dcae-bootstrap:/opt# docker logs -f boot Initiated ./blueprints/centos_vm.yaml If you make changes to the blueprint, run `cfy local init -p ./blueprints/centos_vm.yaml` again to apply them + cfy local execute -w install --task-retries=10 2018-02-03 19:16:19 CFY <local> Starting 'install' workflow execution 2018-02-03 19:16:19 CFY <local> [security_group_15499] Creating node 2018-02-03 19:16:19 CFY <local> [key_pair_a883e] Creating node 2018-02-03 19:16:19 CFY <local> [private_net_22910] Creating node 2018-02-03 19:16:19 CFY <local> [floatingip_vm00_3413c] Creating node 2018-02-03 19:16:19 CFY <local> [private_net_22910.create] Sending task 'neutron_plugin.network.create' 2018-02-03 19:16:19 CFY <local> [floatingip_vm00_3413c.create] Sending task 'neutron_plugin.floatingip.create' 2018-02-03 19:16:19 CFY <local> [key_pair_a883e.create] Sending task 'nova_plugin.keypair.create' 2018-02-03 19:16:19 CFY <local> [security_group_15499.create] Sending task 'neutron_plugin.security_group.create' 2018-02-03 19:16:19 CFY <local> [private_net_22910.create] Task started 'neutron_plugin.network.create' 2018-02-03 19:16:19 CFY <local> [floatingip_vm00_3413c.create] Task started 'neutron_plugin.floatingip.create' 2018-02-03 19:16:19 CFY <local> [key_pair_a883e.create] Task started 'nova_plugin.keypair.create' 2018-02-03 19:16:19 CFY <local> [security_group_15499.create] Task started 'neutron_plugin.security_group.create' 2018-02-03 19:16:19 CFY <local> [private_net_22910.create] Task failed 'neutron_plugin.network.create' -> {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} [status_code=401] 2018-02-03 19:16:19 CFY <local> 'install' workflow execution failed: Workflow failed: Task failed 'neutron_plugin.network.create' -> {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} [status_code=401] Workflow failed: Task failed 'neutron_plugin.network.create' -> {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} [status_code=401] root@onap-dcae-bootstrap:/opt# docker ps
While running the latest master HEAT deployment on openlab - I am having problems with the dcae controller bootstrapping consul before cdap - specific to neutron
Running DCAE to be able to collect logs for LOG-167
Verifying my config
A search turned up
https://wiki.onap.org/questions/16002513/answers/16002516
status:20180204:
https://lists.onap.org/pipermail/onap-discuss/2018-February/007884.html
https://lists.onap.org/pipermail/onap-discuss/2018-February/007904.html
following this JIRA instead - from the OOM side instead of the HEAT side
https://jira.onap.org/browse/OOM-508
20180208: DCAE via OOM works OK
https://lists.onap.org/pipermail/onap-discuss/2018-February/008059.html
issue
Checking out why DCAE is failing to orchestrate consul in HEAT.
I previously rebooted AAI-vm1 to get the 7 containers up that were blocking the initial AAI rest calls from DCAE
Goal is to be able to shut down all HEAT vms except the DCAE subset (cdap, consul...)
getting a neutron network error - checking id's
root@onap-dcae-bootstrap:/opt# docker logs -f boot Initiated ./blueprints/centos_vm.yaml If you make changes to the blueprint, run `cfy local init -p ./blueprints/centos_vm.yaml` again to apply them + cfy local execute -w install --task-retries=10 2018 2018 2018 2018 2018 2018 2018 2018 2018 2018 2018 2018 2018 2018 Unknown macro: {401}
} |
Unknown macro: {
{]}}
20180203 19:16:19 CFY <local> 'install' workflow execution failed: Workflow failed: Task failed 'neutron_plugin.network.create' -> {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} [status_code=\\\\{\\\\{401}}
Unknown macro: {
{]}}
Workflow failed: Task failed 'neutron_plugin.network.create' -> {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} [status_code=\\\\{\\\\{401}}
Unknown macro: {
{]}}
root@onap-dcae-bootstrap:/opt# docker ps|
- blocks
-
DOC-245 Review of DCAEGEN2 docs - using DCEA cloudify-manager to orchestrate CDAP
- Closed
-
LOG-300 CD: OOM framework for continuous E2E deploy validation of tagged commit/merge trigger docker snapshots
- Closed
-
LOG-167 get DCAEGEN2/designate running on OOM-HEAT bridge - for closed-loop logging
- Closed
- is blocked by
-
INT-412 HEAT deployment docker containers timing out for 4h - requiring a VM reboot - breaking for example DCAE - AAI dependency
- Closed
-
AAI-460 vm1-aai-inst1 aai-resources fails to start
- Closed
-
INT-437 AAI-vm1 HEAT deploy failing to start all 6 containers
- Closed
- relates to
-
DCAEGEN2-128 fix integration issues for DCAEGEN2
- Closed
-
OOM-508 Add support to DCAEGEN2
- Closed
-
DCAEGEN2-218 Configure Openstack SSL Certificate for DCAE components
- Closed
-
INT-224 Workflow failed: Task failed 'dnsdesig.dns_plugin.aneeded' -> 'dns'
- Closed
-
MULTICLOUD-137 Problem in DCAE VMs in amsterdam release
- Closed
-
OPENLABS-161 TLAB - Create tenant for OOM-deployed ONAP instance
- Closed
-
OPENLABS-162 TLAB - Create tenant for HEAT-deployed ONAP instance
- Closed
- links to