Uploaded image for project: 'Common Controller SDK'
  1. Common Controller SDK
  2. CCSDK-1874

Blueprint processor liveness probe fails during high load

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Highest Highest
    • Frankfurt Release
    • El Alto Release
    • cds
    • None

      The blueprint processor had its liveness probe fail occasionally when the rate of incoming requests is higher than the rate it can execute.
      This was causing disruption of service during that time where incoming requests rejected until the running requests finish running.

      This situation was causing the endpoint “health-check” to fail to answer.

      In order to replicate this behaviour with command executor, please take a look CBA I added and the following script which basically calls CDS multiple times.

      #!/bin/bash
      N=15
      echo "Will run N=${N} times the load test"
      echo "will sleep for 5 seconds before running,CTRL+C to cancel"
      #sleep 5for i in `seq 1 ${N}`; do
      (curl -X POST "http://<ip-to-reach-cds>:<port-to-reach-cds>/api/v1/execution-service/process" \
      -H 'Accept: */*' \
      -H 'Accept-Encoding: gzip, deflate' \
      -H 'Authorization: Basic Y2NzZGthcHBzOmNjc2RrYXBwcw==' \
      -H 'Content-Type: application/json' \
      -d "{ 
        \"commonHeader\": { 
          \"originatorId\": \"System\",
          \"requestId\": \"123456\",
          \"subRequestId\": \"1234-12234\"
        },
        \"actionIdentifiers\": {
          \"blueprintName\": \"remote_scripts\",
          \"blueprintVersion\": \"1.0.0\",
          \"actionName\": \"execute-remote-python\",
          \"mode\": \"sync\"
        },
        \"payload\": {
          \"execute-remote-python-request\": {
            \"input\": \"RUN_NUM_${i}\"
          }
        }
      }";

        1. Command-Executor-Logs
          73 kB
        2. Logs-From-Script
          25 kB
        3. Run-Time-CBA.zip
          15 kB
        4. Screen Shot 2019-10-29 at 6.32.42 PM.png
          Screen Shot 2019-10-29 at 6.32.42 PM.png
          129 kB

            prathameshmo prathameshmo
            prathameshmo prathameshmo
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: