Issue: during initial SDNC OOM creation we will time out using the new docker images introduced in SDNC-163
This will occur on a clean machine with no previous docker pullls
On a 2nd deployment we are ok on a pre-pulled vm

Underlying issue is that pod startup is sensitive to docker pull times - this is a separate jira

Marco,
Hi, there were changes done recently in SDNC to enable clustering (notice the new sdnc-0 pod name) https://jira.onap.org/browse/SDNC-163
– when this went in I had the same issue bringing up the pod tree – starting with dbhost. However it looks related to pulling the docker image where the readiness check times out before the retries are exhausted – itself a separate issue - OOM-544. If you delete/recreate SDNC you will be OK – also when running again - like we do on the CD server because by that point you have already loaded the docker image.
Part of this does not make sense to me yet because if the dynamic prepull has pulled the docker layers for a particular image - and later during pod startup the image is pulled again – then I should be seeing the same slowdown on the 2nd startup of SDNC with cached docker images – need to add logs from a clean startup on a clean VM – TODO.

So, restarting SDNC is not a proper workaround – I originally closed the following bug – but after talking with Rahul of the SDNC team – a proposal to not pull the image again (for even though the generated docker pre-pull works) – Kubernetes will still pull the image – slowing down the startup just enough to fail the first time we pull the new SDNC images in values.yaml – need to verify this theory with a test of the change tomorrow.

Previously closed
https://jira.onap.org/browse/OOM-537
created
https://jira.onap.org/browse/OOM-543

However, this also is not a proper workaround – I will raise another OOM jira that addresses the fact that our pod startup needs to handle “any” docker pull time – or be configurable. Because the underlying issue to all of this is the fact that we are exposed to startup failures on long docker pulls.

https://jira.onap.org/browse/OOM-544

/michael

From: PLATANIA, MARCO (MARCO) platania@research.att.com
Sent: Wednesday, January 3, 2018 14:43
To: Yury Novitsky <Yury.Novitsky@Amdocs.com>; alexis.de_talhouet@bell.ca; Michael O'Brien <Frank.Obrien@amdocs.com>; Roger Maitland <Roger.Maitland@amdocs.com>; Mike Elliott <Mike.Elliott@amdocs.com>
Subject: Re: VID/SDC error in ONAP K8S

Hi Yury and All,

I actually got past that error but issues between AAI and MSO happened, so I decided to rebuild ONAP from scratch. Since this morning, all my attempts failed due to incomplete SDNC installation. Please see below. I tried to reinstall ONAP a few times or bounce SDNC containers, but it didn’t work so far.

onap-sdnc nfs-provisioner-4051743034-vqvmm 1/1 Running 0 41m
onap-sdnc sdnc-0 0/2 Init:0/1 4 41m
onap-sdnc sdnc-dbhost-0 0/2 Pending 0 41m
onap-sdnc sdnc-dgbuilder-4267203648-6hqnk 0/1 Init:0/1 4 41m
onap-sdnc sdnc-portal-2558294154-bqsvb 0/1 Init:0/1 4 41m

Have you seen this issue before? If so, how can I get past it?