-
Bug
-
Resolution: Done
-
High
-
Honolulu Maintenance Release 1
-
None
-
Tests executed on basic_weekly https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/28_22-58/
ONAP logs (AAI logs too big for Jira mpdellloader 40M, resources 61Mo)
Client logs can be found on LF backend: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/28_22-58/stability/results_instantiation_10_24h//
Tests executed on basic_weekly https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/28_22-58/ ONAP logs (AAI logs too big for Jira mpdellloader 40M, resources 61Mo) onap-so-.zip onap-sdc.zip status_pod.log Client logs can be found on LF backend: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/28_22-58/stability/results_instantiation_10_24h//
We automatically run stability tests consisting in 10 parallel basic_vm during 24h
The success rate is dropping and after a while all the tests are Fail.
The MariaDB is still up&running, the errors seem to be applicative now.
the different errors are listed as follows
first one that seems to be due to SDC
there is 1 error due to AAI the all the other ones seem to be due to SO or something behind SO (but from the tester perspective the error is on the a la carte so PBMN call).
Note that the same tests executed on honolulu-weekly (https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_honolulu/2021-05/27_10-47/)
are not fail after 24h
- first error (serie 27)
test 1,2,9 (other tests are OK) => occurs only once (+ MichaelMorris ChrisC )
_2021-05-29 04:30:52,469 ERROR 212:onap_service.py(1) - [SDC][Deep Load Service] request failed: Code: 500. Info: { "requestError": { "policyException": { "messageId": "POL5000", "text": "Error: Internal Server Error. Please try again later.", "variables": [] }}}._
- first full error (serie 34)
test1, 4, 5, 6, 7, 8, 9, 10
2021-05-29 05:32:11,555 ERROR 148:vf_module_ala_carte.py(1) - VfModule instantiation basic_vm0..BasicVm..base_ubuntu20..module-0 failed
test2 _2021-05-29 05:30:37,634 ERROR 212:onap_service.py(1) - [None][Create 9a0a4e09-37a9-48af-880e-5d15590bf338 VF moduledeletion request] request failed: Code: 409. Info: {"requestError":{"serviceException": {"messageId":"SVC2000","text":"*Error: Locked instance* - This vfModule (9a0a4e09-37a9-48af-880e-5d15590bf338) already has a request being worked with a status of IN_PROGRESS (RequestId - db4d6742-5328-4ec7-806f-7d5f6cb7214f). The existing request must finish or be cleaned up before proceeding."}}}._
test 3
_2021-05-29 05:29:33,835 ERROR 174:vf_module_ala_carte.py(1) - VfModule deletion AAI failed
_
wreehil
- last error (serie 86)
test1, 2, 3, 4, 7, 8, 9, 10
2021-05-30 02:06:05,151 ERROR 240:service_ala_carte.py(1) - Service instantiation basic_vm-99cc570a-0e84-4af0-970d-e03bd168c5e6 failed
test 5 _2021-05-30 01:45:47,153 ERROR 212:onap_service.py(1) - [None][*Instantiate basic_vm service* a'la carte] request failed: Code: 500. Info:
._
test 6 _2021-05-30 01:46:14,388 ERROR 212:onap_service.py(1) - [None][Instantiate basic_vm service a'la carte] request failed: Code: 502. Info: {"requestError":{"serviceException":
]</aetgt:ErrorMessage><aetgt:ErrorCode>7000</aetgt:ErrorCode></aetgt:WorkflowException>"}}}.
_
Honolulu weekly | master weekly | |
success rate after 100 tests | 100,00 % | 100,00 % |
success rate after 300 tests | 100,00 % | 100,00 % |
success rate after 500 tests | 93,00 % | 75,00 % |
success rate after 1000 tests | 82,00 % | 30,00 % |
success rate at t+24h | 80,00 % | 40,00 % |
First error in serie | 34 | 27 |
Last full successful serie | 110 | 33 |
Mean duration serie 1 (s) | 330 | 287 |
Mean duration serie 10 (s) | 358 | 299 |
Mean duration last successful serie (s) | 1104 | 394 |