Uploaded image for project: 'Logging analytics'
  1. Logging analytics
  2. LOG-50

Reference ELK stack for testing

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Highest Highest
    • Beijing Release
    • None
    • None

      To aide unit testing/getting our use cases going – we may want to “in-parallel” get an elk stack going on the side – as we review the OOM elk stack.  Was going to write up a quick compose file but found a basic elk stack that we could use to at least get familiar with the stack (minus filebeat)

      Run the following docker compose to get an ELK stack up so we can experiment as the K8S pods are being merged.

      https://github.com/deviantony/docker-elk

       

      obrienbiometrics:wse_github michaelobrien$ git clone https://github.com/deviantony/docker-elk.git
      Cloning into 'docker-elk'...
      remote: Counting objects: 837, done.
      remote: Total 837 (delta 0), reused 0 (delta 0), pack-reused 837
      Receiving objects: 100% (837/837), 171.77 KiB | 0 bytes/s, done.
      Resolving deltas: 100% (308/308), done.
      obrienbiometrics:wse_github michaelobrien$ cd docker-elk/
      obrienbiometrics:docker-elk michaelobrien$ ls
      LICENSE	README.md	docker-compose.yml	elasticsearch	extensions	kibana	logstash
      obrienbiometrics:docker-elk michaelobrien$ docker-compose up -d
      Creating network "dockerelk_elk" with driver "bridge"
      Building elasticsearch
      Step 1/1 : FROM docker.elastic.co/elasticsearch/elasticsearch:5.5.2
      ---> ca27036dd5e7
      Successfully built ca27036dd5e7
      Successfully tagged dockerelk_elasticsearch:latest
      WARNING: Image for service elasticsearch was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
      Building logstash
      Step 1/1 : FROM docker.elastic.co/logstash/logstash:5.5.2
      5.5.2: Pulling from logstash/logstash
      364f9b7c969a: Already exists
      6a3de13dfd1b: Pull complete
      660fd2410629: Pull complete
      7bf7783fd403: Pull complete
      8e5b5fe9ef66: Pull complete
      c0260871811b: Pull complete
      83cc694df84a: Pull complete
      5318e6cd9947: Pull complete
      eba594e4857c: Pull complete
      231926073096: Pull complete
      ba95e0cbb687: Pull complete
      Digest: sha256:16b5ab50f2ed4b08e4bcf7ea99098a2ff2bd8947e81b2b46808d3fde36750f04
      Status: Downloaded newer image for docker.elastic.co/logstash/logstash:5.5.2
      ---> 499d074f7bd4
      Successfully built 499d074f7bd4
      Successfully tagged dockerelk_logstash:latest
      WARNING: Image for service logstash was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
      Building kibana
      Step 1/1 : FROM docker.elastic.co/kibana/kibana:5.5.2
      5.5.2: Pulling from kibana/kibana
      364f9b7c969a: Already exists
      e49bda868a31: Pull complete
      19392fbddf76: Pull complete
      7a6a24f9cca6: Pull complete
      945295f0f086: Pull complete
      ad5548b19b09: Pull complete
      52c4777d1729: Pull complete
      670929eeaf05: Pull complete
      568cf9e954fe: Pull complete
      Digest: sha256:9edc857d8c3dcd0f6833183c6b9fd7a14a96c39cf8e276dc57354f890672bf80
      Status: Downloaded newer image for docker.elastic.co/kibana/kibana:5.5.2
      ---> a5077718c01f
      Successfully built a5077718c01f
      Successfully tagged dockerelk_kibana:latest
      WARNING: Image for service kibana was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
      Creating dockerelk_elasticsearch_1 ...
      Creating dockerelk_elasticsearch_1 ... done
      Creating dockerelk_kibana_1 ...
      Creating dockerelk_logstash_1 ...
      Creating dockerelk_kibana_1
      Creating dockerelk_logstash_1 ... done
      obrienbiometrics:docker-elk michaelobrien$ docker ps
      CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                                            NAMES
      403a6005bbc5        dockerelk_logstash        "/usr/local/bin/do..."   10 seconds ago      Up 8 seconds        5044/tcp, 0.0.0.0:5000->5000/tcp, 9600/tcp       dockerelk_logstash_1
      3bec9fcde85a        dockerelk_kibana          "/bin/sh -c /usr/l..."   10 seconds ago      Up 8 seconds        0.0.0.0:5601->5601/tcp                           dockerelk_kibana_1
      eba7786fb4cd        dockerelk_elasticsearch   "/bin/bash bin/es-..."   11 seconds ago      Up 9 seconds        0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   dockerelk_elasticsearch_1
      obrienbiometrics:docker-elk michaelobrien$ curl http://127.0.0.1:9200
      {
        "name" : "pKyrWaY",
        "cluster_name" : "docker-cluster",
        "cluster_uuid" : "KIj80FDhShe2VdwHLbT9NQ",
        "version" : {
          "number" : "5.5.2",
          "build_hash" : "b2f0c09",
          "build_date" : "2017-08-14T12:33:14.154Z",
          "build_snapshot" : false,
          "lucene_version" : "6.6.0"
        },
        "tagline" : "You Know, for Search"
      }
      
      Issue is - installing docker-compose ends up upgrading us past docker 1.12
      apt install docker-compose
      root@ip-172-31-93-160:~# docker --version
      Docker version 17.05.0-ce, build 89658be
      root@ip-172-31-93-160:~# docker-compose -version
      docker-compose version 1.8.0, build unknown
      
      
      

            michaelobrien michaelobrien
            michaelobrien michaelobrien
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: