aicsfiles-js

This is a JavaScript client to the AICS File Storage Service. It provides the API to initiate, process and complete an upload of files. The client requires access to the computer's filesystem in order to copy files to the destination directory. Thus the process using it must have access to NodeJS libraries such as fs, os, path, etc. If included inside of an electron application, this should be used in the "main" process.

Eventually it will also provide API to query for and access previously uploaded files.

Level of Support

We are not currently supporting this code for external use, but simply releasing it to the community AS IS. It is used for within our organization. We are not able to provide any guarantees of support. The community is welcome to submit issues, but you should not expect an active response.

Getting started

Install the library:

yarn add @aics/aicsfiles

Create FMS instance:

const uploadClient = new FileManagementSystem(LIMS_HOST, LIMS_PORT, "debug");

Example Usage

const startUpload = async (uploads, jobName) => {
    try {
        const startUploadResponse = await uploadClient.validateMetadataAndGetUploadDirectory(uploads);
        /* result will look something like this:
        {
            jobId: "2f55ea322a654e35b98d003a74f9170e",
            uploadDirectory: "/tmp/lk/fss/incoming/asdf
        }
        **/

        const result = await uploadClient.uploadFiles(startUploadResponse, uploads, jobName);
        /*  result will look something like:
        * {
        *   a-filename.txt: {
        *       fileId: "65ebf71c8f1046dcb52931e4aa9db0f4", 
        *       fileName: "Nunito-Regular.ttf",
        *       readPath: "/allen/aics/apps/stage/file-storage-service/data/f4/65ebf71c8f1046dcb52931e4aa9db0f4"
        *   }
        *   ...
        * } 
        * */
       console.log("Upload complete", result);
    } catch (e) {
        console.log("Upload failed", e)
    }
};

const retryUpload = async (uploadJob) => {
    Logger.debug("received start upload request from renderer");
    const uploadClient = new FileManagementSystem(LIMS_HOST, LIMS_PORT, "debug");
    try {
        const result = await uploadClient.retryUpload(uploadJob);
        /*  result will look something like:
        * {
        *   a-filename.txt: {
        *       fileId: "65ebf71c8f1046dcb52931e4aa9db0f4", 
        *       fileName: "Nunito-Regular.ttf",
        *       readPath: "/allen/aics/apps/stage/file-storage-service/data/f4/65ebf71c8f1046dcb52931e4aa9db0f4"
        *   }
        *   ...
        * } 
        * */
        console.log("Upload complete", result);
    } catch (e) {
        console.log("Upload failed", e)
    }
};

startUpload({
    "/home/foo/kitties.txt": {
        file: {
            originalPath: "/home/foo/kitties.txt", // required
            fileType: "other", // required
            numberCats: 3, // arbitrary fields are allowed
        },
        anotherProperty: "foo", // arbitrary fields here are also allowed
    }
});

retryUpload(uploadJobFromJSS);

Library development

NOTE: We're using yarn for dependency management. If you don't have yarn already, set it up by following these instructions:

https://yarnpkg.com/en/docs/install#debian-stable

Gradle Tasks

Description of Gradle tasks:

| script | comments | | ------ | -------- | | build | create CommonJS and ES module builds | | bundle | run Webpack to create a UMD bundle | | clean | remove generated artifacts | | format | run prettier on src directory | | generateTypes | generate type declarations | | lint | run eslint on src directory | | transpileCommonJs | run babel on src directory; transpile import/export statements for a CommonJS compatible build | | transpileES | run babel on src directory; do not transpile import/export statements for an ES module compatible build (used by bundlers for tree-shaking) | | test | run mocha; searches for any files matching the pattern "src/**/*.test.js" | | typeCheck | run tsc in type-check only mode |

Simultaneously working on application and library

You can test a development copy of the library in another application using yarn-link:

yarn link creates a symlink for the library that points to your development copy.

1. Run yarn link in the root of this repository
2. Run yarn link @aics/aicsfiles in the root of your application

Publishing

After merging to master, create a release build specifying the type of release ('patch', 'minor', 'major') and checking PUBLISH. https://jenkins.corp.alleninstitute.org/job/node-packages/job/aicsfiles/job/master/build?delay=0sec