Uploaded image for project: 'ONAP Operations Manager'
  1. ONAP Operations Manager
  2. OOM-443

Fix data loss due to no matching elasticsearch template

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Medium Medium
    • None
    • None
    • None
    • None

      Currently data is stored in the index onaplogs-%{+YYYY.MM.dd}

      In Elastisearch the only template is:

      ubuntu@onap:~$ curl localhost:30254/_template/
      {"logstash":{"order":0,"version":50001,"template":"logstash-*","settings":{"index":{"refresh_interval":"5s"}},"mappings":{"default":{"dynamic_templates":[\\\{"message_field":\\\{"path_match":"message","mapping":

      {"norms":false,"type":"text"}

      ,"match_mapping_type":"string"}},\\\{"string_fields":\\\{"mapping":\\\{"norms":false,"type":"text","fields":\\\{"keyword":

      {"type":"keyword"}

      }},"match_mapping_type":"string","match":"*"}}],"_all":{"norms":false,"enabled":true},"properties":{"@timestamp":{"include_in_all":false,"type":"date"},"geoip":{"dynamic":true,"properties":{"ip":{"type":"ip"},"latitude":{"type":"half_float"},"location":{"type":"geo_point"},"longitude":{"type":"half_float"}}},"@version":{"include_in_all":false,"type":"keyword"}}}},"aliases":{}}}ubuntu@onap:~$

       

      The fact that there is no matching template for the index onaplogs-%{+YYYY.MM.dd} seems to cause some data to be lost.

      This has been noted for logs that are written to rarely such as aai-ml metrics.log and audit.log.

       

      Possible solutions:

      1. Change index pattern in logstash conf to logstash-%{+YYYY.MM.dd} 
      2. Update Elasticsearch template by sending request to API after installation

            shanedaniel shanedaniel
            shanedaniel shanedaniel
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: