The stability test campaign in Guilin allows to show memory leak

Such tests must be run systematically on Weekly CI chains including on weekly master in order to detect possible memory/regression/bottleneck as early as possible in the process (not 3 days before a release)

It shall be possible to run several existing tests (basic_*)

the KPI to be monitored

• success criteria PASS/FAIL
• test duration
• test component duration

For the test campaign, we used a bash script in charge of launching N xtesting dockers in //.

As an improvement a python project (that could be hosted in a first time on gitlab.com) could be initiated. It will use the python docker client.

it shall be possible to specify by config file and/or command line the test we want, a aggregated reporting shall be produced (html/log page).

As a prerequisite, it may suppose that the tests have been already run in CI/CD so the config files are already available in /tmp/xtesting as well as the dockers.

the python script will be run on the host (controller1)

python test_stability.py -h

-h: help

-t: test (basic_vm, basic_cnf, basic_onboarding, basic_network,..)

-l: get the list of available test cases

-n: nb of parallel occurrences

-d: test duration (in minutes)

-r: reporting path (/tmp/test_stability.html by default leveraging the same CSS than the one from pythonsdktests)

-------------------------

e.g.

python test_stability.py -t basic_vm -n 50 -d 10080 -r /home/debian/test01

it will trigger 50 basic_vm xtesting dockers during 1 week (7*24*60

 

Once developed, it shall be possible to integrate such test in CI (other Jira).