from aai team
"hector has discovered that the stress test jar (liveness probe?) in aai-cassandra is hammering the cpu/ram/hd on the vm that aai is on - this breaks the etcd cluster (not the latency/network issues we suspected that may cause pod rescheduling) "
20181017: update - reopen or re-raise AAI/Logstash specific JIRA for Dublin - in LOG-707 - as the issue is more of an AAI to logstash issue
12588 ubuntu 20 0 6397972 699192 22012 S 578.1 1.1 567:55.27 /usr/bin/java -Xmx500m -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbo+
find out what the reason is for the saturation - is it excessive logs from example the cluster heartbeat from all the db clusters
or a misconfiguration of the resource section
it looks like logs are still being processed up to 4 min after they come into logstash - getting an average of 200-400 logs per 30 sec on