PAP pushPolicies.sh in startup fails due to race condition in some environments

pushPolicies.sh expects PDP to be up when run. This dependency is implemented currently by a single hard-coded 60s sleep in doStart.sh. If the startup happens to take longer,  pushPolicies.sh is just left out from startup. As a consequence, the state of policy after startup depends on random timing and cannot be trusted.

Relevant piece of pap container log in a case when this fails:

---------------------------------------------------------------

wget -O cl-amsterdam-template.drl https://git.onap.org/policy/drools-applications/plain/controlloop/templates/archetype-cl-amsterdam/src/main/resources/archetype-resources/src/main/resources/__closedLoopControlName__.drl?h=casablanca
+ wget -O cl-amsterdam-template.drl 'https://git.onap.org/policy/drools-applications/plain/controlloop/templates/archetype-cl-amsterdam/src/main/resources/archetype-resources/src/main/resources/_closedLoopControlName_.drl?h=casablanca'
-2019-03-11 06:17:45- https://git.onap.org/policy/drools-applications/plain/controlloop/templates/archetype-cl-amsterdam/src/main/resources/archetype-resources/src/main/resources/__closedLoopControlName__.drl?h=casablanca
Resolving git.onap.org (git.onap.org)... 198.145.29.92
Connecting to git.onap.org (git.onap.org)|198.145.29.92|:(443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 70247 (69K) [text/plain]
Saving to: 'cl-amsterdam-template.drl'

0K .......... .......... .......... .......... .......... 72% 139K 0s
50K .......... ........ 100% 14.0M=0.4s

2019-03-11 06:17:46 (191 KB/s) - 'cl-amsterdam-template.drl' saved [70247/70247]

sleep 2
+ sleep 2

curl -k -v --silent -X POST --header 'Content-Type: multipart/form-data' --header 'Accept: text/plain' --header 'ClientAuth: cHl0aG9uOnRlc3Q=' --header 'Authorization: Basic dGVzdHBkcDphbHBoYTEyMw==' --header 'Environment: TEST' -F "file=@cl-amsterdam-template.drl" -F "importParametersJson={\"serviceName\":\"ClosedLoopControlName\",\"serviceType\":\"BRMSPARAM\"}" 'https://pdp:8081/pdp/api/policyEngineImport'
+ curl -k -v --silent -X POST --header 'Content-Type: multipart/form-data' --header 'Accept: text/plain' --header 'ClientAuth: cHl0aG9uOnRlc3Q=' --header 'Authorization: Basic dGVzdHBkcDphbHBoYTEyMw==' --header 'Environment: TEST' -F file=@cl-amsterdam-template.drl -F 'importParametersJson={"serviceName":"ClosedLoopControlName","serviceType":"BRMSPARAM"}' https://pdp:8081/pdp/api/policyEngineImport

• Hostname was NOT found in DNS cache
• Could not resolve host: pdp
• Closing connection 0

 -----------------------------------------------------------------------------

In my opinion the best way to fix this would be to remove the arbitrary sleep from the start script and add the pdp dependency to pap chart so pap actually gets started only after pdp. Or if there are some hairier mutual dependencies buried here, at least make a retry loop in the start script that properly adapts to variable timings in different environments.