Uploaded image for project: 'ONAP Operations Manager'
  1. ONAP Operations Manager
  2. OOM-428

Add log container healthcheck to mark failed creations - see OOM-427

    Details

      Description

      There was a change done to optimize the yamls - the log containers failed to create - but were not caught by the CD or the healtcheck because we are missing a call to one or all of the ELK containers

       

       The OOM CD job picked this up at 18:07 UTC 20171114 http://jenkins.onap.info/job/oom-cd/93/console

      18:07:15 ********** Creating deployments for log ********** 
      18:07:15 
      18:07:15 Creating namespace **********
      18:07:15 namespace "onap-log" created
      18:07:15 
      18:07:15 Creating service account **********
      18:07:15 clusterrolebinding "onap-log-admin-binding" created
      18:07:15 
      18:07:15 Creating registry secret **********
      18:07:15 secret "onap-docker-registry-key" created
      18:07:15 
      18:07:15 Creating deployments and services **********
      18:07:15 Error: parse error in "log/templates/logstash-deployment.yaml": template: log/templates/logstash-deployment.yaml:83: function "tpl" not defined
      18:07:15 The command helm returned with error code 1 
      18:07:15 wait for all pods up for 10 min
      
      A rollback of the 4 changes fixes the pods
      ubuntu@ip-172-31-25-96:~/oom/kubernetes/oneclick$ kubectl get pods --all-namespaces | grep log
      onap-log              elasticsearch-1942187295-kkg1n           1/1       Running            0          10m
      onap-log              kibana-3372627750-drmlf                  1/1       Running            0          10m
      onap-log              logstash-1708188010-1fsgl                1/1       Running            0          10m

      see kibana query

      http://kibana.onap.info:5601/app/kibana#/discover?_g=(refreshInterval:(display:'5%20seconds',pause:!f,section:1,value:5000),time:(from:now-7d,mode:quick,to:now))&_a=(columns:!(_source),index:AV-vxHzpUcDKD8zJ1zbn,interval:auto,query:(query_string:(query:'message:%20%22parse%20error%22')),sort:!('@timestamp',desc)) 

       

        Attachments

        1. loghealth1.PNG
          61 kB
          MIN Wen
        2. loghealth2.PNG
          47 kB
          MIN Wen
        3. loghealth3.PNG
          24 kB
          MIN Wen

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

              People

              • Assignee:
                mayankg2703 Mayank Gupta
                Reporter:
                michaelobrien Michael O'Brien
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: