Uploaded image for project: 'Configuration Persistence Service'
  1. Configuration Persistence Service
  2. CPS-1390

CPS-NCMP Processing stopped multiple times for minutes (intermittently)

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Won't Do
    • Icon: Medium Medium
    • Montreal Release
    • None
    • CPS-Core

      • This seems to happen after about every 3-4k cmHandles.
      • While PG container performs a clean up job that amy take a few minutes then, meanwhile CPS-NCMP does nothing (Processing seems to have stopped multiple times for minutes).
      2022-11-15 02:56:27.936  WARN [cps-application,,] 24128 — [ool-9-worker-27] o.h.engine.jdbc.spi.SqlExceptionHelper   : SQL Error: 0, SQLState: 53200
      2022-11-15 02:56:27.936 ERROR [cps-application,,] 24128 — [ool-9-worker-27] o.h.engine.jdbc.spi.SqlExceptionHelper   : ERROR: out of shared memory
        Hint: You might need to increase max_locks_per_transaction.
      • We tried to set up "max_transaction_per_lock" to a larger value (200 0r 1024) in the config on PG container and it made the error disappear, but overall performance was a lot slower.
      • CPS Commit ID : d2a1f36a78fdc69d43b39f0a7852d47d78924ed5 ( https://gerrit.onap.org/r/c/cps/+/132188)

                 Total Number of CM Handles : 10K

       ----------------------------------------------------------------------------------
        1st CM handle moved into READY state :   2022-11-15T03:18:38.128+0000
        Last CM handle moved into READY state : 2022-11-15T03:27:15.694+0000
        ---------------------------------------------------------------------------------
        517 seconds (8.61 minutes) > 19.34 cm handles / seconds
        ---------------------------------------------------------------------------------
      • CPS Commit ID : 8b37702647755949165dc822116ed10dca5c1c2f

                https://gerrit.onap.org/r/q/8b37702647755949165dc822116ed10dca5c1c2f

                Total Number of CM Handles : 10K (If CPS-NCMP processing does not stop ever)

                Approx time taken for all CmHandle READY state  3 min 37 sec 

            ToineSiebelink Toine Siebelink
            sourabh_sourabh Sourabh Sourabh
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: