Uploaded image for project: 'Logging analytics'
  1. Logging analytics
  2. LOG-986

Some POMBA pods are not shipping logs to central logging ELK stack

    XMLWordPrintable

Details

    Description

      While working on adding missing filebeat sidecars, it appears that very few of them successfully are delivering the logs to the Logging ELK stack.  When running test calls, only search-data-service appeared to reliably be publishing logs.

       

      The filebeat.yml configuration for these pods appears correct to me, so I expect the logs are getting shipped to logstash successfully, but they're getting dropped from there. 

      NOTE: It's possible this is a broader issue, as other components in my deployment also didn't seem to be publishing their expected logs to the ELK stack either, although I lacked the ability to run test calls to verify this.

      Attachments

        Issue Links

          blocks

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. LOG-487 LOG Pipeline Integrity: Docker to Filebeat to Logstash to ElasticSearch to Kibana

          • Highest - Catastrophic defect that causes total failure of the software or unrecoverable data loss with no available workaround. Complete shut-down of the process, nothing can proceed further. Blocks testing of the product / feature. Must be fixed in the next build.
          • Closed

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. LOG-1041 Dublin pairwise testing for logging/pomba

          • Highest - Catastrophic defect that causes total failure of the software or unrecoverable data loss with no available workaround. Complete shut-down of the process, nothing can proceed further. Blocks testing of the product / feature. Must be fixed in the next build.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. LOG-1042 POMBA 2 of 7 healthchecks failing - see LOG-1018

          • Highest - Catastrophic defect that causes total failure of the software or unrecoverable data loss with no available workaround. Complete shut-down of the process, nothing can proceed further. Blocks testing of the product / feature. Must be fixed in the next build.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. LOG-852 POMBA: Some pods do not ship with filebeat

          • Medium - Failure of non-critical aspects of the system but results in an inconvenient situation. It causes some undesirable behavior, but the system is still functional. There is a reasonably satisfactory workaround. It is still a valid defect that should be corrected May be fixed after the release or in the next release.
          • Closed
          relates to

          Task - A task that needs to be done. LOG-230 Pairwise integration testing - verify single log makes it through ELK into Kibana

          • High - Severely impaired functionality. Erratic performance and reduced system availability. It may cause major components or features of the system to be unavailable although certain parts of the system remain functional. A workaround may exist but its use is unsatisfactory or at a high time cost (manual intervention required). Must be fixed in any of the upcoming builds and should be included in the current release.
          • Closed

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. LOG-707 Logging El-Alto (moved from) Dublin Scope

          • Medium - Failure of non-critical aspects of the system but results in an inconvenient situation. It causes some undesirable behavior, but the system is still functional. There is a reasonably satisfactory workaround. It is still a valid defect that should be corrected May be fixed after the release or in the next release.
          • Closed
          mentioned in

          Page Loading...

          Page Loading...

          Wiki Page

          Wiki Page Loading...

          Wiki Page Loading...

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              Unassigned Unassigned
              jmac James MacNider
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: