Uploaded image for project: 'Data Collection, Analytics, and Events'
  1. Data Collection, Analytics, and Events
  2. DCAEGEN2-855

deployment handler uploadBlueprint which is not valid to cloudify (folder as file)

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Medium Medium
    • Dublin Release
    • Amsterdam Release, Beijing Release, Casablanca Release
    • None
    • DCAE R4 Sprint 3

      Hi,

      I have been investigating why I was getting the following error when trying to visualize Local Blueprints using Cloudify UI.
      Blueprint sources:
      An unexpected error occured
      EISDIR: illegal operation on a directory, open '/tmp/cloudifyBrowseSources/source1538574197244/extracted/work/'

      Here is the result of that investigation.

      CLOUDIFY
      =========
      Cloudify UI is using nodejs (/opt/nodejs/bin/node /opt/cloudify-stage/backend/server.js)
      and to display the local blueprint will download the blueprint (tar, or tar.gz or zip),
      and decompress it.

      The flow is
      /opt/cloudify-stage/backend/handler/SourceHandler.js browseArchiveTree()
      which will call
      /opt/cloudify-stage/backend/handler/ArchiveHelper.js .saveDataFromUrl()

      The decompress may fail as follow:
      [2018-10-03 13:43:17.295] [ERROR] Server - Error has occured

      { Error: EISDIR: illegal operation on a directory, open '/tmp/cloudifyBrowseSources/source1538574197244/extracted/work/' errno: -21, code: 'EISDIR', syscall: 'open', path: '/tmp/cloudifyBrowseSources/source1538574197244/extracted/work/' }

      }

      The reason is that cloudify UI is using the following library:
      https://registry.npmjs.org/decompress-unzip/-/decompress-unzip-4.0.1.tgz
      which has the following issue

      https://github.com/kevva/decompress/issues/46
      Error: EISDIR: illegal operation on a directory, open
      which happen if directories in that archive are marked as type: file
      Some people are advising to use an alternate library
      such as https://www.npmjs.com/package/extract-zip

      HOW TO SANITY TEST USING CLOUDIFY
      =================================
      Here is an example of good and bad zip that may be used to reproduce the issue

      unzip -l good.zip
      Archive: good.zip
      Length Date Time Name
      --------- ---------- ----- ----
      3027 10-02-2018 22:32 work/blueprint.yaml
      --------- -------
      3027 1 file

      unzip -l bad.zip
      Archive: bad.zip
      Length Date Time Name
      --------- ---------- ----- ----
      0 10-02-2018 22:32 work/
      3027 10-02-2018 22:32 work/blueprint.yaml
      --------- -------
      3027 2 files

      curl -X PUT http://dcae-cloudify-manager.onap-dgermain:80/api/v2.1/blueprints/good -H "Content-Type:application/octet-stream" -H "Accept:/" -H "Tenant:default_tenant" -H "Authorization: Basic YWRtaW46YWRtaW4=" --data-binary @good.zip

      curl -X PUT http://dcae-cloudify-manager.onap-dgermain:80/api/v2.1/blueprints/bad -H "Content-Type:application/octet-stream" -H "Accept:/" -H "Tenant:default_tenant" -H "Authorization: Basic YWRtaW46YWRtaW4=" --data-binary @bad.zip

      So if cloudify UI does not use a library that is more fault tolerant, then you need make sure that the zip was not created with folder as file.
      Note: you may also use tar.gz

      DCAE DEPLOYMENT_HANDLER
      =======================
      Why did I get a folder tag as a file, the reason is because of an issue in the deployment_handler.
      Deployment handler is also using nodejs (/usr/local/bin/node deployment-handler.js)
      It can be used to upload blueprint (as zip) to cloudify.

      PUT /dcae-deployments/deploymentId?cfy_tenant_name=cfy_tenant_name HTTP/1.1
      Host: localhost:7779
      Accept: application/json
      Content-Type: application/json
      Cache-Control: no-cache

      {
      "serviceTypeId": "0b90a7a7-4123-442e-800a-21e8c0f28f61",
      "inputs": {
      }
      }

      This will fetch the blueprint from the DCAE inventory for the specified serviceTypeId
      and create a zip to upload it to cloudify using Cloudify HTTP api showned above with curl

      dcaegen2/platform/deployment-handler/lib/dcae-deployments.js put()
      will call
      dcaegen2/platform/deployment-handler/lib/deploy.js deploy.launchBlueprint()
      will call
      dcaegen2/platform/deployment-handler/lib/cloudify.js cfy.uploadBlueprint()

      which is as follow

      288 // Uploads a blueprint via the Cloudify API
      289 exports.uploadBlueprint = function(req, bpid, blueprint) {
      290 logger.info(req.dcaeReqId, "uploadBlueprint " + bpid);
      291
      292 // Cloudify API wants a gzipped tar of a directory, not the blueprint text
      293 const zip = new admzip();
      294 zip.addFile('work/', new Buffer(0)); // ***********ERROR ADDING A FOLDER AS FILE *************
      295 zip.addFile('work/blueprint.yaml', new Buffer(blueprint, 'utf8'));
      296 const zip_buffer = zip.toBuffer();
      297
      298 // Set up the HTTP PUT request
      299 const reqOptions = {
      300 method : "PUT",
      301 uri : cfyAPI + "/blueprints/" + bpid,
      302 headers :

      { 303 "Content-Type" : "application/octet-stream", 304 "Accept" : "*/*" 305 }

      306 };
      307 addAuthToOptions(reqOptions, req);
      308
      309 // Initiate PUT request and return the promise for a result
      310 return doRequest(req, reqOptions, zip_buffer, CLOUDIFY);
      311 };

            alex_sh alex_sh
            dgermain dgermain
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: