Uploaded image for project: 'Logging analytics'
  1. Logging analytics
  2. LOG-378

cluster deletion strategy required to avoid --grace-period --force specific pod deletes

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Medium Medium
    • Dublin Release
    • None
    • None

      Cleaning the cluster and reinstalling onap via helm has issues - we need to formulate a strategy for full deletion - separate from a helm upgrade

      testing

      kubectl get pods --all-namespaces | grep Terminating | xargs -n 1 | grep onap- | kubectl delete pod -n onap --grace-period=0 --force
      
      

      The following deletion strategy is insufficient from OOM-716
      during
      http://jenkins.onap.info/job/oom-cd-master/2856/console

          if [ "$BRANCH" == "master" ]; then
            kubectl delete namespace $ENVIRON
            sudo helm delete --purge dev
          else
            oom/kubernetes/oneclick/deleteAll.bash -n $ENVIRON
          fi
      
          sleep 1
          # verify
          DELETED=$(kubectl get pods --all-namespaces -a | grep 0/ | wc -l)
          echo "verify deletion is finished."
          while [  $(kubectl get pods --all-namespaces | grep 0/ | wc -l) -gt 0 ]; do
            sleep 15
            echo "waiting for deletions to complete"
          done
          # wait for 0/1 before deleting
          echo "sleeping 30 sec"
          # delete potential hanging clustered pods
          kubectl delete pod $ENVIRON-aaf-sms-vault-0 -n $ENVIRON --grace-period=0 --force
          kubectl delete pod $ENVIRON-aai-cassandra-0 -n $ENVIRON --grace-period=0 --force
          kubectl delete pod $ENVIRON-aai-cassandra-1 -n $ENVIRON --grace-period=0 --force
          kubectl delete pod $ENVIRON-aai-cassandra-2 -n $ENVIRON --grace-period=0 --force
      
      ran
      ubuntu@ip-10-0-0-19:~$ kubectl delete namespace onap
      Error from server (NotFound): namespaces "onap" not found
      ubuntu@ip-10-0-0-19:~$  helm ls --all onap
      NAME	REVISION	UPDATED                 	STATUS  	CHART     	NAMESPACE
      onap	1       	Fri May  4 19:17:30 2018	DEPLOYED	onap-2.0.0	onap     
      ubuntu@ip-10-0-0-19:~$ helm del --purge onap
      release "onap" deleted
      
      

            michaelobrien michaelobrien
            michaelobrien michaelobrien
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: