Uploaded image for project: 'ONAP Operations Manager'
  1. ONAP Operations Manager
  2. OOM-4

deleteAll.bash fails to properly delete services and ports

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Low
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: Beijing Release
    • Labels:
      None

      Description

       Issue: after a create and delete we fail to run create again because of undeleted resources

       workaround is to remove the VM each time 

      obrienbiometrics:single-node michaelobrien$ rm -rf ~/.vagrant.d/boxes/coreos-alpha/

       reproduction all services (mostly works, only namespaces are left and ignored on recreate, some services fail to deploy because the ports are still held)

      root@oskub0:~/onap/oom/kubernetes/oneclick# ./deleteAll.bash -n onap
      
      ********** Cleaning up ONAP: sdc aai mso message-router robot vid sdnc portal policy appc
      SDC....
      deployment "sdc-es" deleted
      deployment "sdc-cs" deleted
      deployment "sdc-kb" deleted
      deployment "sdc-be" deleted
      deployment "sdc-fe" deleted
      AAI....
      deployment "hbase" deleted
      deployment "aai-service" deleted
      deployment "model-loader-service" deleted
      MSO....
      deployment "mariadb" deleted
      deployment "mso" deleted
      Message Router....
      deployment "zookeeper" deleted
      deployment "global-kafka" deleted
      deployment "dmaap" deleted
      Robot....
      deployment "robot" deleted
      VID....
      deployment "vid-mariadb" deleted
      deployment "vid-server" deleted
      SDNC....
      deployment "sdnc-dbhost" deleted
      deployment "sdnc" deleted
      deployment "sdnc-dgbuilder" deleted
      deployment "sdnc-portal" deleted
      Portal....
      deployment "portaldb" deleted
      deployment "portalapps" deleted
      deployment "vnc-portal" deleted
      Policy....
      deployment "mariadb" deleted
      deployment "nexus" deleted
      deployment "pap" deleted
      deployment "pdp" deleted
      deployment "brmsgw" deleted
      deployment "pypdp" deleted
      deployment "drools" deleted
      App-c....
      deployment "appc-dbhost" deleted
      deployment "appc" deleted
      deployment "appc-dgbuilder" deleted
      
      ********** Gone **********
      root@oskub0:~/onap/oom/kubernetes/oneclick# kubectl get pods --all-namespaces -o wide
      NAMESPACE     NAME                                    READY     STATUS             RESTARTS   AGE       IP              NODE
      kube-system   heapster-859001963-c6df8                1/1       Running            2          9h        10.42.25.134    oskub0
      kube-system   kube-dns-1759312207-pg37m               1/3       CrashLoopBackOff   20         9h        10.42.138.116   oskub0
      kube-system   kubernetes-dashboard-2463885659-vt5b2   1/1       Running            3          9h        10.42.70.46     oskub0
      kube-system   monitoring-grafana-1177217109-5006x     1/1       Running            1          9h        10.42.71.157    oskub0
      kube-system   monitoring-influxdb-1954867534-sbz23    1/1       Running            1          9h        10.42.204.19    oskub0
      kube-system   tiller-deploy-1933461550-m15lx          1/1       Running            2          9h        10.42.150.132   oskub0
      onap-appc     appc-4266112350-447gq                   0/1       Terminating        0          35m       10.42.126.107   oskub0
      onap-appc     appc-dgbuilder-939982213-9pmwb          0/1       Terminating        0          35m       10.42.104.173   oskub0
      onap-policy   brmsgw-554754608-x8xdz                  0/1       Terminating        0          35m       10.42.125.175   oskub0
      onap-policy   pypdp-3646772508-gcvxx                  0/1       Terminating        0          35m       10.42.168.0     oskub0
      root@oskub0:~/onap/oom/kubernetes/oneclick# kubectl get pods --all-namespaces -o wide
      NAMESPACE     NAME                                    READY     STATUS             RESTARTS   AGE       IP              NODE
      kube-system   heapster-859001963-c6df8                1/1       Running            2          9h        10.42.25.134    oskub0
      kube-system   kube-dns-1759312207-pg37m               2/3       CrashLoopBackOff   21         9h        10.42.138.116   oskub0
      kube-system   kubernetes-dashboard-2463885659-vt5b2   1/1       Running            3          9h        10.42.70.46     oskub0
      kube-system   monitoring-grafana-1177217109-5006x     1/1       Running            1          9h        10.42.71.157    oskub0
      kube-system   monitoring-influxdb-1954867534-sbz23    1/1       Running            1          9h        10.42.204.19    oskub0
      kube-system   tiller-deploy-1933461550-m15lx          1/1       Running            2          9h        10.42.150.132   oskub0
      
      
      root@oskub0:~/onap/oom/kubernetes/oneclick# ./createAll.bash -n onap
      
      ********** Creating up ONAP: sdc aai mso message-router robot vid sdnc portal policy appc
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-sdc" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../sdc/all-services.yaml": services "sdc-es" already exists
      Error from server (AlreadyExists): error when creating "../sdc/all-services.yaml": services "sdc-cs" already exists
      Error from server (AlreadyExists): error when creating "../sdc/all-services.yaml": services "sdc-kb" already exists
      Error from server (Invalid): error when creating "../sdc/all-services.yaml": Service "sdc-be" is invalid: spec.ports[0].nodePort: Invalid value: 30204: provided port is already allocated
      Error from server (Invalid): error when creating "../sdc/all-services.yaml": Service "sdc-fe" is invalid: spec.ports[0].nodePort: Invalid value: 30207: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-aai" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../aai/all-services.yaml": services "hbase" already exists
      Error from server (Invalid): error when creating "../aai/all-services.yaml": Service "aai-service" is invalid: spec.ports[0].nodePort: Invalid value: 30233: provided port is already allocated
      Error from server (Invalid): error when creating "../aai/all-services.yaml": Service "model-loader-service" is invalid: spec.ports[0].nodePort: Invalid value: 30229: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-mso" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (Invalid): error when creating "../mso/all-services.yaml": Service "mariadb" is invalid: spec.ports[0].nodePort: Invalid value: 30252: provided port is already allocated
      Error from server (Invalid): error when creating "../mso/all-services.yaml": Service "mso" is invalid: spec.ports[0].nodePort: Invalid value: 30223: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-message-router" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../message-router/all-services.yaml": services "zookeeper" already exists
      Error from server (AlreadyExists): error when creating "../message-router/all-services.yaml": services "global-kafka" already exists
      Error from server (Invalid): error when creating "../message-router/all-services.yaml": Service "dmaap" is invalid: spec.ports[0].nodePort: Invalid value: 30227: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-robot" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      The Service "robot" is invalid: spec.ports[0].nodePort: Invalid value: 30209: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-vid" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../vid/all-services.yaml": services "vid-mariadb" already exists
      Error from server (Invalid): error when creating "../vid/all-services.yaml": Service "vid-server" is invalid: spec.ports[0].nodePort: Invalid value: 30200: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-sdnc" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../sdnc/all-services.yaml": services "dbhost" already exists
      Error from server (AlreadyExists): error when creating "../sdnc/all-services.yaml": services "sdnctldb01" already exists
      Error from server (AlreadyExists): error when creating "../sdnc/all-services.yaml": services "sdnctldb02" already exists
      Error from server (Invalid): error when creating "../sdnc/all-services.yaml": Service "sdnc-dgbuilder" is invalid: spec.ports[0].nodePort: Invalid value: 30203: provided port is already allocated
      Error from server (Invalid): error when creating "../sdnc/all-services.yaml": Service "sdnhost" is invalid: spec.ports[0].nodePort: Invalid value: 30202: provided port is already allocated
      Error from server (Invalid): error when creating "../sdnc/all-services.yaml": Service "sdnc-portal" is invalid: spec.ports[0].nodePort: Invalid value: 30201: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-portal" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../portal/all-services.yaml": services "portaldb" already exists
      Error from server (Invalid): error when creating "../portal/all-services.yaml": Service "portalapps" is invalid: spec.ports[0].nodePort: Invalid value: 30213: provided port is already allocated
      Error from server (Invalid): error when creating "../portal/all-services.yaml": Service "vnc-portal" is invalid: spec.ports[0].nodePort: Invalid value: 30211: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-policy" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../policy/all-services.yaml": services "mariadb" already exists
      Error from server (AlreadyExists): error when creating "../policy/all-services.yaml": services "nexus" already exists
      Error from server (Invalid): error when creating "../policy/all-services.yaml": Service "drools" is invalid: spec.ports[0].nodePort: Invalid value: 30217: provided port is already allocated
      Error from server (Invalid): error when creating "../policy/all-services.yaml": Service "pap" is invalid: spec.ports[0].nodePort: Invalid value: 30219: provided port is already allocated
      Error from server (Invalid): error when creating "../policy/all-services.yaml": Service "pdp" is invalid: spec.ports[0].nodePort: Invalid value: 30220: provided port is already allocated
      Error from server (Invalid): error when creating "../policy/all-services.yaml": Service "pypdp" is invalid: spec.ports[0].nodePort: Invalid value: 30221: provided port is already allocated
      Error from server (Invalid): error when creating "../policy/all-services.yaml": Service "brmsgw" is invalid: spec.ports[0].nodePort: Invalid value: 30216: provided port is already allocated
      
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-appc" already exists
      Error from server (AlreadyExists): secrets "regsecret" already exists
      serviceaccount "default" not patched
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../appc/all-services.yaml": services "dbhost" already exists
      Error from server (AlreadyExists): error when creating "../appc/all-services.yaml": services "sdnctldb01" already exists
      Error from server (AlreadyExists): error when creating "../appc/all-services.yaml": services "sdnctldb02" already exists
      Error from server (Invalid): error when creating "../appc/all-services.yaml": Service "sdnhost" is invalid: spec.ports[0].nodePort: Invalid value: 30230: provided port is already allocated
      Error from server (Invalid): error when creating "../appc/all-services.yaml": Service "dgbuilder" is invalid: spec.ports[0].nodePort: Invalid value: 30228: provided port is already allocated
      
      
      
      ********** Creating deployments for  sdc aai mso message-router robot vid sdnc portal policy appc **********
      SDC....
      deployment "sdc-es" created
      deployment "sdc-cs" created
      deployment "sdc-kb" created
      deployment "sdc-be" created
      deployment "sdc-fe" created
      AAI....
      deployment "hbase" created
      deployment "aai-service" created
      deployment "model-loader-service" created
      MSO....
      deployment "mariadb" created
      deployment "mso" created
      Message Router....
      deployment "zookeeper" created
      deployment "global-kafka" created
      deployment "dmaap" created
      Robot....
      deployment "robot" created
      VID....
      deployment "vid-mariadb" created
      deployment "vid-server" created
      SDNC....
      deployment "sdnc-dbhost" created
      deployment "sdnc" created
      deployment "sdnc-dgbuilder" created
      deployment "sdnc-portal" created
      Portal....
      deployment "portaldb" created
      deployment "portalapps" created
      deployment "vnc-portal" created
      Policy....
      deployment "mariadb" created
      deployment "nexus" created
      deployment "pap" created
      deployment "pdp" created
      deployment "brmsgw" created
      deployment "pypdp" created
      deployment "drools" created
      App-c....
      deployment "appc-dbhost" created
      deployment "appc" created
      deployment "appc-dgbuilder" created
      **** Done ****root@oskub0:~/onap/oom/kubernetes/oneclick#

       

      reproduction (single service fails)

      [root@obrien-b2 oneclick]# ./createAll.bash -n onap -a vid
      
      ********** Creating up ONAP: vid
      
      Creating namespaces **********
      namespace "onap-vid" created
      
      Creating services **********
      service "vid-mariadb" created
      service "vid-server" created
      
      
      
      ********** Creating deployments for vid ********** 
      VID....
      deployment "vid-mariadb" created
      deployment "vid-server" created
      
      [root@obrien-b2 oneclick]# ./deleteAll.bash -n onap-vid -a vid
      
      ********** Cleaning up ONAP: vid
      VID....
      Error from server (NotFound): error when stopping "../vid/vid-mariadb-deployment.yaml": deployments.extensions "vid-mariadb" not found
      Error from server (NotFound): error when stopping "../vid/vid-server-deployment.yaml": deployments.extensions "vid-server" not found
      
      ********** Gone **********
      [root@obrien-b2 oneclick]# 
      [root@obrien-b2 oneclick]# ./deleteAll.bash -n onap -a vid
      
      ********** Cleaning up ONAP: vid
      VID....
      deployment "vid-mariadb" deleted
      deployment "vid-server" deleted
      
      ********** Gone **********
      [root@obrien-b2 oneclick]# ./createAll.bash -n onap -a vid
      
      ********** Creating up ONAP: vid
      
      Creating namespaces **********
      Error from server (AlreadyExists): namespaces "onap-vid" already exists
      
      Creating services **********
      Error from server (AlreadyExists): error when creating "../vid/all-services.yaml": services "vid-mariadb" already exists
      Error from server (Invalid): error when creating "../vid/all-services.yaml": Service "vid-server" is invalid: spec.ports[0].nodePort: Invalid value: 30200: provided port is already allocated
      
      
      
      
      

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

              People

              • Assignee:
                michaelobrien Michael O'Brien
                Reporter:
                michaelobrien Michael O'Brien
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: