-
Bug
-
Resolution: Won't Do
-
High
-
Honolulu Release
ONAP pods use AAF to generate certificates during pod instantiation.
AAF Certificate manager stores the requests and the certificates in its CassandraDB.
But if a limit of 200 certificate request for a certain CN and user is reached, AAF refuses to accept the further requests and the Pod startups will fail.
As the DB is never cleaned up, the DB entries need to be removed manually.
Problem, e.g. in DCAE init-tls job:
...#### Place Certificates (by deployer)2021-03-31T07:10:25.472+0000 INIT [cadi] cadi_keyfile points to /opt/app/osaaf/local/org.onap.dcae.keyfile2021-03-31T07:10:25.485+0000 INIT [cadi] https.protocols set by cadi_protocols in CADI Properties2021-03-31T07:10:25.486+0000 INIT [cadi] jdk.tls.client.protocols set from Default Protocols2021-03-31T07:10:26.700+0000: Trans Info REMOTE Place Artifact 598.44006msFAILED to get CertificateInitialization complete+ /opt/app/aaf_config/bin/agent.sh aafcli showpass dcae@dcae.onap.org dcae
Logs of AAF CM :
2021-03-31T07:39:29.766+0000 INFO [service] user=deployer@people.osaaf.org[BAth],ip=10.233.68.159,port=42750,ms=211.37689,status=403,meth=PUT,path=/cert/local,msg="Request New Certificate/ErrResp [SVC1403] Forbidden: There are too many Certificates generated for CN=dcae for dcae@dcae.onap.org"
2021-03-31T07:39:31.223+0000 INFO [service] user=deployer@people.osaaf.org[BAth],ip=10.233.68.159,port=42792,ms=11.31259,status=200,meth=GET,path=/cert/artifacts/dcae@dcae.onap.org/dcae
2021-03-31T07:39:31.251+0000 INFO [service] user=deployer@people.osaaf.org[BAth],ip=10.233.68.159,port=42794,ms=1.228888,status=200,meth=GET,path=/cert/may/org.onap.dcae|certman|local|showpass
Intermediate solution executed on the AAF cassandra to delete the existing credentials (here for DCAE):
ubuntu@control01:/opt/oom/kubernetes$ kubectl -n onap exec -ti onap-aaf-cass-79f5d58f4b-rzdkk bash aaf@onap-aaf-cass-79f5d58f4b-rzdkk:/$ /opt/cassandra/bin/cqlsh cqlsh> use authz; cqlsh:authz> delete from cred where id = 'dcae@dcae.onap.org'; cqlsh:authz> select * from cred where id = 'dcae@dcae.onap.org'; id | type | expires | cred | notes | ns | other | prev | tag----+------+---------+------+-------+----+-------+------+-----(0 rows) cqlsh:authz> select count(*) from x509 where ca = 'local' and id = 'dcae@dcae.onap.org'; -> if there are more than 200, you need to delete them manually cqlsh:authz> select ca, serial from x509 where ca = 'local' and id = 'dcae@dcae.onap.org'; cqlsh:authz> delete from x509 where ca = 'local' and serial = 0x00e591f558ae34e9f0; ...
Proper solution:
Implementation of an automated cleanup of the x509 and cred tables
- mentioned in
-
Page Loading...