Uploaded image for project: 'Microservices Bus'
  1. Microservices Bus
  2. MSB-299

MSB 1.2.0 pods enter crash loop

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Highest Highest
    • None
    • Casablanca Release

      dev-msb-msb-eag-7c4f98857c-8thb2 1/2 CrashLoopBackOff 86 7h
      dev-msb-msb-iag-775d79686d-rpmgw 1/2 CrashLoopBackOff 86 7h

       

      root@onap-oom-rancher:~# kubectl logs dev-msb-msb-eag-7c4f98857c-8thb2 -n onap msb-eag
      @RUNHOME@ /usr/local
      Convert SDCLIENT_IP to IP
      10.43.1.241

          1. Starting redis
            there is no /usr/local/redis/../redis-works
            =========== Redis config info =============================================
            Redis_HOME=/usr/local/redis
            config file=/usr/local/redis/conf/redis.conf
            Help:use /usr/local/redis/redis-server --help for help
            ===============================================================================
            /usr/local/redis
            @C_CMD@ /usr/local/redis/redis-server /usr/local/redis/conf/redis.conf
            \n\n### Starting openresty
            =========== prepare the symbolic links ========================================
            ================================================================================
            =========== create symbolic link for libluajit-5.1.so.2 ========================================
            ln: /lib64/libluajit-5.1.so.2: No such file or directory
            ===============================================================================
            =========== openresty config info =============================================
            HOME=/usr/local/openresty/nginx
            _NGINXCMD=/usr/local/openresty/nginx/sbin/nginx
            ===============================================================================
            /usr/local/openresty/nginx
            @WORK_DIR@ /usr/local/openresty/nginx
            @C_CMD@ /usr/local/openresty/nginx/sbin/nginx -p /usr/local/openresty/nginx/
            2018/10/19 13:56:43 [warn] 34#34: [lua] dns_util.lua:35: environment variable UPSTREAM_DNS_SERVERS:nil
            nginx: [warn] [lua] dns_util.lua:35: environment variable UPSTREAM_DNS_SERVERS:nil
            2018/10/19 13:56:43 [warn] 34#34: [lua] dns_util.lua:38: failed to parse the DNS Servers from the environment variable UPSTREAM_DNS_SERVERS Error:/usr/local/openresty/nginx//luaext/vendor/pl/stringx.lua:27: argument 1 expected a 'string', got a 'nil'
            nginx: [warn] [lua] dns_util.lua:38: failed to parse the DNS Servers from the environment variable UPSTREAM_DNS_SERVERS Error:/usr/local/openresty/nginx//luaext/vendor/pl/stringx.lua:27: argument 1 expected a 'string', got a 'nil'
            2018/10/19 13:56:43 [warn] 34#34: [lua] msb.lua:69: load_plugins(): Loading plugin: redirect-transformer
            nginx: [warn] [lua] msb.lua:69: load_plugins(): Loading plugin: redirect-transformer
            2018/10/19 13:56:43 [notice] 34#34: using the "epoll" event method
            2018/10/19 13:56:43 [notice] 34#34: openresty/1.13.6.1
            2018/10/19 13:56:43 [notice] 34#34: built by gcc 6.3.0 (Alpine 6.3.0)
            2018/10/19 13:56:43 [notice] 34#34: OS: Linux 4.4.0-64-generic
            2018/10/19 13:56:43 [notice] 34#34: getrlimit(RLIMIT_NOFILE): 1048576:1048576
            \n\n### Starting apiroute
            2018/10/19 13:56:43 [notice] 35#35: start worker processes
            2018/10/19 13:56:43 [notice] 35#35: start worker process 36
            2018/10/19 13:56:43 [notice] 35#35: start worker process 37
            2018/10/19 13:56:43 [notice] 35#35: start worker process 38
            2018/10/19 13:56:43 [notice] 35#35: start worker process 40
            2018/10/19 13:56:43 [notice] 35#35: start worker process 41
            2018/10/19 13:56:43 [notice] 35#35: start worker process 42
            2018/10/19 13:56:43 [notice] 35#35: start worker process 43
            2018/10/19 13:56:43 [notice] 35#35: start worker process 44
            2018/10/19 13:56:43 [notice] 35#35: start cache manager process 45
            2018/10/19 13:56:43 [notice] 35#35: start cache loader process 46
            @RUNHOME@ /usr/local/apiroute
            ================== ENV_INFO =============================================
            @RUNHOME@ /usr/local/apiroute
            @JAVA_HOME@ /usr/lib/jvm/java-1.8-openjdk
            @Main_Class@ org.onap.msb.apiroute.ApiRouteApp
            @APP_INFO@ msb-apiroute-service
            @Main_JAR@ apiroute-service.jar
            @Main_Conf@ conf/apiroute.yml
            ==========================================================================
            start msb-apiroute-service ...
            @JAVA_VERSION@ 1.8
            ================== RUN_INFO =============================================
            @JAVA_HOME@ /usr/lib/jvm/java-1.8-openjdk
            @JAVA@ /usr/lib/jvm/java-1.8-openjdk/bin/java
            @JAVA_OPTS@ -Xms16m -Xmx128m -XX:+UseSerialGC -XX:MaxMetaspaceSize=64m -XX:NewRatio=2 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/apiroute/../apiroute-works/logs/dump-msb-20181019135643.hprof
            @CLASS_PATH@ /usr/local/apiroute/:/usr/local/apiroute/apiroute-service.jar
            @EXT_DIRS@
            ==========================================================================
            @JAVA@ /usr/lib/jvm/java-1.8-openjdk/bin/java
            @JAVA_CMD@
            Exception in thread "main" java.lang.NoClassDefFoundError: com/codahale/metrics/jvm/JvmAttributeGaugeSet
            at io.dropwizard.setup.Bootstrap.registerMetrics(Bootstrap.java:84)
            at io.dropwizard.Application.run(Application.java:90)
            at org.onap.msb.apiroute.ApiRouteApp.main(ApiRouteApp.java:42)
            Caused by: java.lang.ClassNotFoundException: com.codahale.metrics.jvm.JvmAttributeGaugeSet
            at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
            at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
            ... 3 more

       

      root@onap-oom-rancher:~# kubectl logs dev-msb-msb-iag-775d79686d-rpmgw -n onap msb-iag
      @RUNHOME@ /usr/local
      Convert SDCLIENT_IP to IP
      10.43.1.241

          1. Starting redis
            there is no /usr/local/redis/../redis-works
            =========== Redis config info =============================================
            Redis_HOME=/usr/local/redis
            config file=/usr/local/redis/conf/redis.conf
            Help:use /usr/local/redis/redis-server --help for help
            ===============================================================================
            /usr/local/redis
            @C_CMD@ /usr/local/redis/redis-server /usr/local/redis/conf/redis.conf
            \n\n### Starting openresty
            =========== prepare the symbolic links ========================================
            ================================================================================
            =========== create symbolic link for libluajit-5.1.so.2 ========================================
            ln: /lib64/libluajit-5.1.so.2: No such file or directory
            ===============================================================================
            =========== openresty config info =============================================
            HOME=/usr/local/openresty/nginx
            _NGINXCMD=/usr/local/openresty/nginx/sbin/nginx
            ===============================================================================
            /usr/local/openresty/nginx
            @WORK_DIR@ /usr/local/openresty/nginx
            @C_CMD@ /usr/local/openresty/nginx/sbin/nginx -p /usr/local/openresty/nginx/
            2018/10/19 14:00:36 [warn] 34#34: [lua] dns_util.lua:35: environment variable UPSTREAM_DNS_SERVERS:nil
            nginx: [warn] [lua] dns_util.lua:35: environment variable UPSTREAM_DNS_SERVERS:nil
            2018/10/19 14:00:36 [warn] 34#34: [lua] dns_util.lua:38: failed to parse the DNS Servers from the environment variable UPSTREAM_DNS_SERVERS Error:/usr/local/openresty/nginx//luaext/vendor/pl/stringx.lua:27: argument 1 expected a 'string', got a 'nil'
            nginx: [warn] [lua] dns_util.lua:38: failed to parse the DNS Servers from the environment variable UPSTREAM_DNS_SERVERS Error:/usr/local/openresty/nginx//luaext/vendor/pl/stringx.lua:27: argument 1 expected a 'string', got a 'nil'
            2018/10/19 14:00:36 [warn] 34#34: [lua] msb.lua:69: load_plugins(): Loading plugin: redirect-transformer
            nginx: [warn] [lua] msb.lua:69: load_plugins(): Loading plugin: redirect-transformer
            2018/10/19 14:00:36 [notice] 34#34: using the "epoll" event method
            2018/10/19 14:00:36 [notice] 34#34: openresty/1.13.6.1
            2018/10/19 14:00:36 [notice] 34#34: built by gcc 6.3.0 (Alpine 6.3.0)
            2018/10/19 14:00:36 [notice] 34#34: OS: Linux 4.4.0-64-generic
            2018/10/19 14:00:36 [notice] 34#34: getrlimit(RLIMIT_NOFILE): 1048576:1048576
            2018/10/19 14:00:36 [notice] 35#35: start worker processes
            2018/10/19 14:00:36 [notice] 35#35: start worker process 36
            2018/10/19 14:00:36 [notice] 35#35: start worker process 38
            2018/10/19 14:00:36 [notice] 35#35: start worker process 39
            2018/10/19 14:00:36 [notice] 35#35: start worker process 40
            2018/10/19 14:00:36 [notice] 35#35: start worker process 41
            2018/10/19 14:00:36 [notice] 35#35: start worker process 42
            2018/10/19 14:00:36 [notice] 35#35: start worker process 43
            2018/10/19 14:00:36 [notice] 35#35: start worker process 44
            2018/10/19 14:00:36 [notice] 35#35: start cache manager process 45
            2018/10/19 14:00:36 [notice] 35#35: start cache loader process 46
            \n\n### Starting apiroute
            @RUNHOME@ /usr/local/apiroute
            ================== ENV_INFO =============================================
            @RUNHOME@ /usr/local/apiroute
            @JAVA_HOME@ /usr/lib/jvm/java-1.8-openjdk
            @Main_Class@ org.onap.msb.apiroute.ApiRouteApp
            @APP_INFO@ msb-apiroute-service
            @Main_JAR@ apiroute-service.jar
            @Main_Conf@ conf/apiroute.yml
            ==========================================================================
            start msb-apiroute-service ...
            @JAVA_VERSION@ 1.8
            ================== RUN_INFO =============================================
            @JAVA_HOME@ /usr/lib/jvm/java-1.8-openjdk
            @JAVA@ /usr/lib/jvm/java-1.8-openjdk/bin/java
            @JAVA_OPTS@ -Xms16m -Xmx128m -XX:+UseSerialGC -XX:MaxMetaspaceSize=64m -XX:NewRatio=2 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/apiroute/../apiroute-works/logs/dump-msb-20181019140036.hprof
            @CLASS_PATH@ /usr/local/apiroute/:/usr/local/apiroute/apiroute-service.jar
            @EXT_DIRS@
            ==========================================================================
            @JAVA@ /usr/lib/jvm/java-1.8-openjdk/bin/java
            @JAVA_CMD@
            Exception in thread "main" java.lang.NoClassDefFoundError: com/codahale/metrics/jvm/JvmAttributeGaugeSet
            at io.dropwizard.setup.Bootstrap.registerMetrics(Bootstrap.java:84)
            at io.dropwizard.Application.run(Application.java:90)
            at org.onap.msb.apiroute.ApiRouteApp.main(ApiRouteApp.java:42)
            Caused by: java.lang.ClassNotFoundException: com.codahale.metrics.jvm.JvmAttributeGaugeSet
            at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
            at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
            ... 3 more

       

            huabing_zhao huabing_zhao
            gwu gwu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: