Uploaded image for project: 'CI-Management'
  1. CI-Management
  2. CIMAN-174

Jenkins jobs failed by LF resource timeout - could use log parsing to auto retry the job - recheck

    Details

      Description

      From 20180709 PTL meeting
      Some jobs periodically fail due to nexus timeout for example - a retry mechanism (as suggested by Chris Donley - parsing the logs - would help to avoid manually verifying the failure and retrying the job
      will attach some feasibility analysis using my own jenkins.onap.org jobs to this jira

      example
      https://gerrit.onap.org/r/#/c/55339/

      14:56:58 FATAL: Failed to install https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.3.9/apache-maven-3.3.9-bin.zip to /w/tools/hudson.tasks.Maven_MavenInstallation/mvn33
      14:56:58 java.net.UnknownHostException: repo.maven.apache.org
      14:56:58 	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
      14:56:58 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
      14:56:58 	at java.net.Socket.connect(Socket.java:589)
      14:56:58 	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
      14:56:58 	at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
      14:56:58 	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
      14:56:58 	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
      14:56:58 	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
      14:56:58 	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
      14:56:58 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
      14:56:58 	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)
      14:56:58 	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)
      14:56:58 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
      14:56:58 	at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
      14:56:58 	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:776)
      14:56:58 Caused: java.io.IOException: Failed to install https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.3.9/apache-maven-3.3.9-bin.zip to /w/tools/hudson.tasks.Maven_MavenInstallation/mvn33
      
      

      Michael O'Brien
      Jul 3 9:22 AM

      Uploaded patch set 2.

      Michael O'Brien
      Jul 3 9:23 AM

      Patch Set 2:

      snapshot version from 1.2.0 to 1.2.2, added reference code to lukes library, retrofitting k8s/docker infrastructure

      ONAP Jobbuilder
      Jul 3 10:09 AM

      Patch Set 2:

      Build Started https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-python/5/ (1/3)

      ONAP Jobbuilder
      Jul 3 10:09 AM

      Patch Set 2:

      Build Started https://jenkins.onap.org/job/logging-analytics-master-verify-java/48/ (2/3)

      ONAP Jobbuilder
      Jul 3 10:09 AM

      Patch Set 2:

      Build Started https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-java/5/ (3/3)

      ONAP Jobbuilder
      Jul 3 10:18 AM

      Patch Set 2: Verified-1

      Build Failed

      https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-python/5/ : FAILURE

      No problems were identified. If you know why this problem occurred, please add a suitable Cause for it. ( https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-python/5/ )

      Logs: https://logs.onap.org/production/vex-yul-ecomp-jenkins-1/logging-analytics-pylog-master-pylog-verify-python/5

      https://jenkins.onap.org/job/logging-analytics-master-verify-java/48/ : SUCCESS

      Logs: https://logs.onap.org/production/vex-yul-ecomp-jenkins-1/logging-analytics-master-verify-java/48

      https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-java/5/ : SUCCESS

      Logs: https://logs.onap.org/production/vex-yul-ecomp-jenkins-1/logging-analytics-pylog-master-pylog-verify-java/5

      Michael O'Brien
      Jul 5 3:48 PM

      Patch Set 2:

      recheck

      ONAP Jobbuilder
      Jul 5 3:50 PM

      Patch Set 2: -Verified

      Build Started https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-python/6/ (1/3)

      ONAP Jobbuilder
      Jul 5 3:51 PM

      Patch Set 2:

      Build Started https://jenkins.onap.org/job/logging-analytics-master-verify-java/49/ (2/3)

      ONAP Jobbuilder
      Jul 5 3:51 PM

      Patch Set 2:

      Build Started https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-java/6/ (3/3)

      ONAP Jobbuilder
      Jul 5 3:53 PM

      Patch Set 2: Verified+1

      Build Successful

      https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-python/6/ : SUCCESS

      Logs: https://logs.onap.org/production/vex-yul-ecomp-jenkins-1/logging-analytics-pylog-master-pylog-verify-python/6

      https://jenkins.onap.org/job/logging-analytics-master-verify-java/49/ : SUCCESS

      Logs: https://logs.onap.org/production/vex-yul-ecomp-jenkins-1/logging-analytics-master-verify-java/49

      https://jenkins.onap.org/job/logging-analytics-pylog-master-pylog-verify-java/6/ : SUCCESS

      Logs: https://logs.onap.org/production/vex-yul-ecomp-jenkins-1/logging-analytics-pylog-master-pylog-verify-java/6

      Michael O'Brien
      Jul 5 4:40 PM

      Patch Set 2:

      Test results from
      http://localhost:8080/logging-demo/rest/health/health
      2018-07-05T20:21:34.794Z http-nio-8080-exec-2 INFO org.onap.demo.logging.ApplicationService InstanceUUID=ede7dd52-91e8-45ce-9406-fbafd17a7d4c, RequestID=f9d8bb0f-4b4b-4700-9853-d3b79d861c5b, ServiceName=/logging-demo/rest/health/health, InvocationID=8f4c1f1d-5b32-4981-b658-e5992f28e6c8, InvokeTimestamp=2018-07-05T20:21:26.617Z, PartnerName=, ClientIPAddress=0:0:0:0:0:0:0:1, ServerFQDN=localhost ENTRY
      2018-07-05T20:22:09.268Z http-nio-8080-exec-2 INFO org.onap.demo.logging.ApplicationService ResponseCode=, InstanceUUID=ede7dd52-91e8-45ce-9406-fbafd17a7d4c, RequestID=f9d8bb0f-4b4b-4700-9853-d3b79d861c5b, ServiceName=/logging-demo/rest/health/health, ResponseDescription=, InvocationID=8f4c1f1d-5b32-4981-b658-e5992f28e6c8, Severity=, InvokeTimestamp=2018-07-05T20:21:26.617Z, PartnerName=, ClientIPAddress=0:0:0:0:0:0:0:1, ServerFQDN=localhost, StatusCode= EXIT

      Michael O'Brien
      Patch Set 2: junit code coverage T E S T S ------------------------------------------------------- Running org.onap.logging.demo.ApplicationServiceTest 2018-07-05T21:13:55.295Z main INFO org.onap.demo.logging.ApplicationService Starting. 2018-07-05T21:13:55.383Z main INFO org.onap.demo.logging.ApplicationService InstanceUUID=c0284c80-7b6f-4cd3-b5d0-2f2e9bba69b4, RequestID=7ffc40bf-0c9b-4c05-8461-60ff228dd603, ServiceName=, InvocationID=2da4be8b-369c-4fdb-97e0-bbf4bfe7d42d, InvokeTimestamp=2018-07-05T21:13:55.308Z, PartnerName=, ClientIPAddress=127.0.0.1, ServerFQDN=localhost ENTRY 2018-07-05T21:13:55.383Z main INFO org.onap.demo.logging.ApplicationService ResponseCode=, InstanceUUID=c0284c80-7b6f-4cd3-b5d0-2f2e9bba69b4, RequestID=7ffc40bf-0c9b-4c05-8461-60ff228dd603, ServiceName=, ResponseDescription=, InvocationID=2da4be8b-369c-4fdb-97e0-bbf4bfe7d42d, Severity=, InvokeTimestamp=2018-07-05T21:13:55.308Z, PartnerName=, ClientIPAddress=127.0.0.1, ServerFQDN=localhost, StatusCode= EXIT health: true Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.638 sec - in org.onap.logging.demo.ApplicationServiceTestJul 5 5:15 PM
      Michael O'Brien

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

              People

              • Assignee:
                michaelobrien Michael O'Brien
                Reporter:
                michaelobrien Michael O'Brien
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: