-
Bug
-
Resolution: Done
-
Medium
-
Dublin Release
-
Wind River lab Integration-Longevity tenant
-
SDNC El Alto Sprint 1
Trying to deploy a 3-node SDNC cluster in Integration-Longevity tenant.
It appears that all 3 nodes are trying to install the SDNC database, and deadlocked between the 3 mysql sessions. All 3 have been sitting in 1/2 Running for multiple hours with no restarts.
dev-sdnc-sdnc-0 1/2 Running 0 8h 10.42.9.18 long-k8s-08 <none> <none> dev-sdnc-sdnc-1 1/2 Running 0 8h 10.42.12.23 long-k8s-05 <none> <none> dev-sdnc-sdnc-2 1/2 Running 0 8h 10.42.11.16 long-k8s-10 <none> <none>
Logs from each of the 3 sdnc containers:
root@long-nfs:~# kubectl logs dev-sdnc-sdnc-0 sdnc Settings: ENABLE_ODL_CLUSTER=true SDNC_REPLICAS=3 SDNRWT=false Waiting for mysqlmysql ready Installing SDNC database ERROR 1005 (HY000) at line 52: Can't create table `sdnctl`.`CONFIG_RESOURCE_RELATION` (errno: 150 "Foreign key constraint is incorrectly formed") root@long-nfs:~# kubectl logs dev-sdnc-sdnc-1 sdnc Settings: ENABLE_ODL_CLUSTER=true SDNC_REPLICAS=3 SDNRWT=false Waiting for mysqlmysql ready Installing SDNC database ERROR 1007 (HY000) at line 1: Can't create database 'sdnctl'; database exists ERROR 1146 (42S02) at line 65: Table 'sdnctl.NETWORK_MODEL' doesn't exist ERROR 1136 (21S01) at line 54: Column count doesn't match value count at row 1 ERROR 1136 (21S01) at line 46: Column count doesn't match value count at row 1 root@long-nfs:~# kubectl logs dev-sdnc-sdnc-2 sdnc Settings: ENABLE_ODL_CLUSTER=true SDNC_REPLICAS=3 SDNRWT=false Waiting for mysqlmysql ready Installing SDNC database ERROR 1007 (HY000) at line 1: Can't create database 'sdnctl'; database exists ERROR 1050 (42S01) at line 25: Table 'IPAM_IP_POOL' already exists