In our labs (latest Casablanca release), after a couple of days, it is not possible to connect to pods via "kubectl exec" or "kubectl log", e.g.
Reason is a known bug in Kubernetes (see https://github.com/kubernetes/kubernetes/issues/67659), which might be fixed in version 1.12.0
So I logged in to the k8s-1, which hosts the dmaap-message-router and checked the open file descriptors of the kubelet (as described):
I tried the suggested workaround by restarting the kubelet (via Rancher UI) and the connection to the pod was working again.
I don't know yet, if this affects also the inter-pod communication or only the kubectl function, therefor I set the priority to "High"