FileBrowser Node

A widget to access SMB/CIFS Shares and FTP Repositories

Installation

The node can be installed via the Node-RED Palette manager. Search for "node-red-contrib-filebrowser" and install the node:

The Palette manager will install the new node and add a new storage-type "filebrowser":

The following flow, includes a configuration and a localozation node as well as a caching mechanism ( see Caching for more details ) to re-use retrieved data:

[{"id":"7543e93e.35a478","type":"http in","z":"d47cd009.b88c6","name":"","url":"/repositories/filters","method":"get","upload":false,"swaggerDoc":"","x":480,"y":380,"wires":[["a1ef4034.4952e"]]},{"id":"a1ef4034.4952e","type":"template","z":"d47cd009.b88c6","name":"params page","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"<form class=\"ui form\">\n\n  <div class=\"ui toggle checkbox\">\n      <input name=\"useFulltext\" type=\"checkbox\" checked=\"checked\">\n      <label>{{flow.LOCALIZED.useIndex}}</label>\n     \n    </div>\n\n <div></div>\n\n   <div class=\"field\">\n      <label>{{flow.LOCALIZED.itemType}}</label>\n      <select name=\"itemType\" class=\"ui dropdown\">\n         <option value=\"all\" selected=\"selected\">{{flow.LOCALIZED.itemAll}}</option>\n         <option value=\"folder\">{{flow.LOCALIZED.itemFolder}}</option>\n         <option value=\"file\">{{flow.LOCALIZED.itemFile}}</option>\n      </select>\n   </div>\n \n   <div class=\"field\">\n      <label>{{flow.LOCALIZED.maxResults}}</label>\n      <select name=\"maxResults\" class=\"ui dropdown\">\n         <option value=\"30\" selected=\"selected\">30</option>\n         <option value=\"50\">50</option>\n         <option value=\"100\">100</option>\n         <option value=\"*\">*</option>\n \n      </select>\n   </div>\n\n</form>","output":"str","x":870,"y":380,"wires":[["ff48f5df.fb2868"]]},{"id":"ff48f5df.fb2868","type":"http response","z":"d47cd009.b88c6","name":"","statusCode":"","headers":{},"x":1330,"y":380,"wires":[]},{"id":"b75c4b52.2b0078","type":"http in","z":"d47cd009.b88c6","name":"","url":"/repositories/sort","method":"get","upload":false,"swaggerDoc":"","x":480,"y":420,"wires":[["f4659889.6dd9e8"]]},{"id":"f4659889.6dd9e8","type":"template","z":"d47cd009.b88c6","name":"params page","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"<form class=\"ui form\">\n   <div class=\"field\">\n      <label>{{flow.LOCALIZED.sort}}</label>\n      <select name=\"order\" class=\"ui dropdown\">\n        <option value=\"lastmod\">{{flow.LOCALIZED.sort_default}}</option>\n         <option value=\"lastmod\">{{flow.LOCALIZED.sort_date}}</option>\n         <option value=\"name\">{{flow.LOCALIZED.sort_title}}</option>\n         <option value=\"size\">{{flow.LOCALIZED.sort_size}}</option>\n         <option value=\"type\">{{flow.LOCALIZED.sort_type}}</option>\n      </select>\n   </div>\n   \n   <div class=\"field\">\n      <label>{{flow.LOCALIZED.sortOrder}}</label>\n      <select name=\"sortorder\" class=\"ui dropdown\">\n         <option value=\"asc\">{{flow.LOCALIZED.sortOrderAsc}}</option>\n         <option value=\"desc\">{{flow.LOCALIZED.sortOrderDesc}}</option>\n      </select>\n   </div>\n</form>\n\n\n\n","output":"str","x":870,"y":420,"wires":[["55c2095.052a1f8"]]},{"id":"55c2095.052a1f8","type":"http response","z":"d47cd009.b88c6","name":"","statusCode":"","headers":{},"x":1330,"y":420,"wires":[]},{"id":"398813d3.21f60c","type":"inject","z":"d47cd009.b88c6","name":"Once","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":true,"onceDelay":"0.1","x":450,"y":140,"wires":[["e493f167.4baa5"]]},{"id":"e493f167.4baa5","type":"function","z":"d47cd009.b88c6","name":"Localization","func":"var localization = {\n    \"en_EN\": {\n        \"useIndex\" : \"Use fulltext index\",\n        \"maxResults\": \"Results per page\",\n        \"itemType\"  : \"Item type\",\n        \"itemAll\"   : \"All\",\n        \"itemFolder\" : \"Folder\",\n        \"itemFile\" : \"File\",\n        \"sort\":\"Sort by\",\n        \"sort_default\": \"Default\",\n        \"sort_date\":\"Date\",\n        \"sort_type\":\"Type\",\n        \"sort_size\":\"Size\",\n        \"sort_title\":\"Name\",\n        \"sortOrder\" : \"Sort order\",\n        \"sortOrderAsc\" : \"Ascending\",\n        \"sortOrderDesc\" : \"Descending\",\n       \n    },\n    \"de_DE\": {\n        \"useIndex\" : \"Volltextsuche verwenden\",\n        \"maxResults\": \"Ergebnisse pro Seite\",\n        \"itemType\"  : \"Typ\",\n        \"itemAll\"   : \"Alle\",\n        \"itemFolder\" : \"Verzeichnisse\",\n        \"itemFile\" : \"Dateien\",\n        \"sort\":\"Soriteren nach\",\n        \"sort_default\": \"Standart\",\n        \"sort_date\":\"Datum\",\n        \"sort_type\":\"Typ\",\n        \"sort_size\":\"Größe\",\n        \"sort_title\":\"Name\",\n        \"sortOrder\" : \"Sortiereihenfolge\",\n        \"sortOrderAsc\" : \"aufsteigend\",\n        \"sortOrderDesc\" : \"absteigend\",\n    },\n    \"fr_FR\": {\n        \"useIndex\" : \" Use fulltext index\",\n        \"maxResults\": \"Results per page\",\n        \"license\": \"License\",\n        \"ignore\": \"Ignore\",\n        \"lang\": \"Relevance language\",\n        \"lang_ar\": \"Arabic\",\n        \"lang_bg\": \"Bulgarian\",\n        \"lang_da\": \"Dänisch\",\n        \"lang_nl\": \"Niederländisch, Flämisch\",\n        \"lang_en\": \"Englisch\",\n        \"lang_fr\": \"Französisch\",\n        \"lang_de\": \"Deutsch\",\n        \"lang_es\": \"Spanisch\",\n        \"sort\":\"Sort by\",\n        \"sort_date\":\"Date\",\n        \"sort_type\":\"Rating\",\n        \"sort_size\":\"Video count\",\n        \"sort_title\":\"Title\",\n    }\n}\nvar language = flow.get(\"LANGUAGE\");\nvar localized = localization[language];\nflow.set(\"LOCALIZED\", localized);\n","outputs":1,"noerr":0,"x":590,"y":140,"wires":[[]]},{"id":"65b43013.6c2e5","type":"config","z":"d47cd009.b88c6","name":"CONFIG","properties":[{"p":"LANGUAGE","pt":"flow","to":"de_DE","tot":"str"},{"p":"OPENMEDIA_MAPPING","pt":"flow","to":"{\"templateId\":{\"default\":1388635,\"audio\":1388637,\"image\":1388639,\"video\":1388638},\"poolId\":101,\"folderId\":22219,\"systemId\":\"6290b86c-3cdd-4d52-8340-df3b9bf2aa15\",\"OMISCommandAfterCreation\":{\"pluginID\":123,\"commandID\":132},\"fields\":{\"title\":8,\"id\":3201,\"url\":401}}","tot":"json"},{"p":"FULLTEXTENGINE_URL","pt":"flow","to":"http://127.0.0.1:8090","tot":"str"}],"active":true,"x":460,"y":200,"wires":[]},{"id":"6a14dbab.4a3524","type":"http response","z":"d47cd009.b88c6","name":"","statusCode":"200","headers":{},"x":1560,"y":180,"wires":[]},{"id":"58b374dd.9f2b1c","type":"filebrowser in","z":"d47cd009.b88c6","filebrowser":"da48aead.84bd3","rules":[{"t":"da48aead.84bd3"},{"t":null}],"x":1110,"y":280,"wires":[["2862d76b.9b2a58"]]},{"id":"2862d76b.9b2a58","type":"function","z":"d47cd009.b88c6","name":"Pivot format","func":"var responseData = msg.payload;\nvar OPENMEDIA_MAPPING = flow.get(\"OPENMEDIA_MAPPING\");\n//we have to regenerate the fullpath for each item in the breadcrumb\n\n\nvar breadcrumb = [];\n    for (i=0;i<msg.fullPath.length;i++){\n        breadcrumb.push({\n                name:msg.fullPath[i].name,\n                payload:{\n                 remotePath: msg.fullPath[i].remotePath,\n                 fullPath: msg.fullPath.slice(0, i+1)\n             },\n             active:(i<msg.fullPath.length -1) ? false:true\n            })     \n    }\n\n\n\n\nmsg.payload = {\n  resultCount: responseData.items.length + \" results\",\n  navigation: {\n    breadcrumb: breadcrumb\n  },\n  dropdown: [],\n  items: []\n};\n\n\n\nif (responseData.items.length > 0) {\n    responseData.items =  sortResults(responseData, msg.sort, msg.sort_ascending);\n    node.warn(responseData);\n  for (var i = 0; i < responseData.items.length; i++) {\n      var type = responseData.items[i].type;\n      var mime = responseData.items[i].mime;\n      var iconName = \"file\"; // default icon \n      var destTempateType = \"default\";\n  if  (mime) {\n    if (mime.startsWith(\"audio\")) {\n        iconName = \"music\";\n        destTempateType =\"audio\"\n    }\n    if (mime.startsWith(\"video\")) {\n        iconName = \"video\"; \n        destTempateType =\"video\"\n    }\n    if (mime.startsWith(\"image\")) {\n        iconName = \"image\";   \n        destTempateType =\"image\"\n    }\n  }\n      \n    msg.payload.items.push({\n      key: responseData.items[i].id.videoId,\n      rawData: responseData.items[i],\n      presentation: {\n        title: responseData.items[i].name,\n        iconName: type===\"folder\" ? \"folder\" : iconName,\n        \n        iconColor:  type===\"folder\" ? \"yellow\" : \"grey\",\n        mediaType: null,\n    //    thumbnail: responseData.items[i].snippet.thumbnails.medium.url,\n    //    highres: responseData.items[i].snippet.thumbnails.high.url,\n        meta: [\n          {\n            name: \"Type\",\n            value: type===\"folder\" ? \"FOLDER\" : responseData.items[i].filename.split(\".\").pop().toUpperCase(),\n            icon: null\n          },\n          {\n            name: \"Last modified\",\n            value: timeAgo(responseData.items[i].lastmod),\n            icon: \"clock\"\n          },\n          {\n            name: \"Size\",\n           value: formattedSize(responseData.items[i].size),\n            icon: null\n          }\n          \n          \n        ]\n      },\n      navigation: {\n        type: type===\"folder\" ?\"query\" : \"external\", // possible value: external then value must be a string self, query must be a JSON object, openmedia\n         payload: type===\"folder\" ? {\n             remotePath: responseData.items[i].id,\n             fullPath: msg.fullPath.concat(\n                 {\n                     name:responseData.items[i].name,\n                     remotePath: responseData.items[i].id\n                 })\n         } : responseData.items[i].url\n        //type:\"query\",\n        //value:nextResultsPayload\n      },\n      openmedia: {\n        templateId: OPENMEDIA_MAPPING.templateId[destTempateType],\n        poolId: OPENMEDIA_MAPPING.poolId,\n        folderId: OPENMEDIA_MAPPING.folderId,\n        systemId: OPENMEDIA_MAPPING.systemId,\n        OMISCommandAfterCreation: {\n          pluginID: OPENMEDIA_MAPPING.OMISCommandAfterCreation.pluginID,\n          commandID: OPENMEDIA_MAPPING.OMISCommandAfterCreation.commandID\n        },\n        fields: [\n          {\n              //title\n            fieldId: OPENMEDIA_MAPPING.fields.title,\n            valueType:1,\n            value: responseData.items[i].name\n          },\n           {\n               //External Unique ID\n            fieldId: OPENMEDIA_MAPPING.fields.id,\n            valueType:1,\n           value: responseData.items[i].url\n          },\n           {\n               //url\n            fieldId: OPENMEDIA_MAPPING.fields.url,\n            valueType:1,\n           value: responseData.items[i].url\n          }\n        ]\n      },\n      dragAndDrop: []\n    });\n    \n   \n  }\n}\n\nreturn msg;\n\n\n\nfunction sortResults(responseData, prop, asc) {\n    \n    return responseData.items.sort(function(a, b) {\n        if (asc) {\n            return (a[prop] > b[prop]) ? 1 : ((a[prop] < b[prop]) ? -1 : 0);\n        } else {\n            return (b[prop] > a[prop]) ? 1 : ((b[prop] < a[prop]) ? -1 : 0);\n        }\n    });\n\n \n    \n}\n\n\n\n\nfunction getFormattedDate(date, prefomattedDate = false, hideYear = false) {\n        const MONTH_NAMES = [\n  'January', 'February', 'March', 'April', 'May', 'June',\n  'July', 'August', 'September', 'October', 'November', 'December'\n];\n    \n  const day = date.getDate();\n  const month = MONTH_NAMES[date.getMonth()];\n  const year = date.getFullYear();\n  const hours = date.getHours();\n  let minutes = date.getMinutes();\n\n  if (minutes < 10) {\n    // Adding leading zero to minutes\n    minutes = `0${ minutes }`;\n  }\n\n  if (prefomattedDate) {\n    // Today at 10:20\n    // Yesterday at 10:20\n    return `${ prefomattedDate } at ${ hours }:${ minutes }`;\n  }\n\n  if (hideYear) {\n    // 10. January at 10:20\n    return `${ day }. ${ month } at ${ hours }:${ minutes }`;\n  }\n\n  // 10. January 2017. at 10:20\n  return `${ day }. ${ month } ${ year }. at ${ hours }:${ minutes }`;\n}\n\n\n// --- Main function\nfunction timeAgo(dateParam) {\n\n\n  if (!dateParam) {\n    return null;\n  }\n\n  const date = typeof dateParam === 'object' ? dateParam : new Date(dateParam);\n  const DAY_IN_MS = 86400000; // 24 * 60 * 60 * 1000\n  const today = new Date();\n  const yesterday = new Date(today - DAY_IN_MS);\n  const seconds = Math.round((today - date) / 1000);\n  const minutes = Math.round(seconds / 60);\n  const isToday = today.toDateString() === date.toDateString();\n  const isYesterday = yesterday.toDateString() === date.toDateString();\n  const isThisYear = today.getFullYear() === date.getFullYear();\n\n\n  if (seconds < 5) {\n    return 'now';\n  } else if (seconds < 60) {\n    return `${ seconds } seconds ago`;\n  } else if (seconds < 90) {\n    return 'about a minute ago';\n  } else if (minutes < 60) {\n    return `${ minutes } minutes ago`;\n  } else if (isToday) {\n    return getFormattedDate(date, 'Today'); // Today at 10:20\n  } else if (isYesterday) {\n    return getFormattedDate(date, 'Yesterday'); // Yesterday at 10:20\n  } else if (isThisYear) {\n    return getFormattedDate(date, false, true); // 10. January at 10:20\n  }\n\n  return getFormattedDate(date); // 10. January 2017. at 10:20\n}\n\nfunction formattedSize(size) {\n    //dividing by 10 afterwards give the first decimal\n    if (size > 1000000000000) return Math.round(size/100000000000)/10 + \" TB\"\n    if (size > 1000000000) return Math.round(size/100000000)/10 + \" GB\"\n    if (size > 1000000) return Math.round(size/100000)/10 + \" MB\"\n    if (size > 1000) return Math.round(size/100)/10 + \" KB\"\n    else return size + \" KB\"\n}","outputs":1,"noerr":0,"x":1350,"y":280,"wires":[["6a14dbab.4a3524","feaba794.e6cbc8"]]},{"id":"1e4032e4.0d6d7d","type":"function","z":"d47cd009.b88c6","name":"Found in cache ?","func":"//first output uses cache data\nif (msg.hasOwnProperty(\"payload\") && !!msg.payload){\n    return [msg,null];\n}else{\n    //second output goes to actual repository search\n    return [null, msg];\n}\n\n","outputs":2,"noerr":0,"x":1090,"y":220,"wires":[["6a14dbab.4a3524"],["58b374dd.9f2b1c"]]},{"id":"cf4d65d2.181c38","type":"Cache in","z":"d47cd009.b88c6","name":"rep1","cache":"1708125e.da84de","keyType":"msg","keyProperty":"cached_query","valueType":"msg","valueProperty":"payload","useString":false,"x":890,"y":260,"wires":[["1e4032e4.0d6d7d"]]},{"id":"feaba794.e6cbc8","type":"Cache out","z":"d47cd009.b88c6","name":"rep1 ","cache":"1708125e.da84de","keyType":"msg","keyProperty":"cached_query","valueType":"msg","valueProperty":"payload","ttlType":"msg","ttlProperty":"","useString":false,"x":1530,"y":320,"wires":[]},{"id":"c426b8cc.830298","type":"function","z":"d47cd009.b88c6","name":"Prepare Query","func":"node.warn(\"Received payload:\" + JSON.stringify(msg.payload));\nmsg.payload.repIx =  msg.req.params[0];\nmsg.cached_query= JSON.stringify(msg.payload);\nmsg.fullPath = msg.payload.fullPath||[{name:\"Home\",remotePath:\"\"}]\nmsg.remotePath  = msg.payload.remotePath||\"\";\nmsg.fulltextEngineURL = flow.get(\"FULLTEXTENGINE_URL\");\nmsg.repIx  = msg.req.params[0] ||\"1\";\nnode.warn(\"Request for Repository ID :\" +msg.repIx )\nif (msg.payload.useFulltext===\"on\") {\nmsg.useFulltext =  \"true\";\n\n    \n}\nelse \n{\n    msg.useFulltext =  \"false\"\n}\n\n\nmsg.query       = msg.payload.query||\"\";\nif (msg.query  ===\"\") {\n    msg.operation   = \"list\";\n}\nelse {\n    msg.operation   = \"find\";\n    if (msg.payload.hasOwnProperty(\"breadcrumb\")){\n    msg.remotePath = msg.payload.breadcrumb[msg.payload.breadcrumb.length-1].payload.remotePath;\n    }\n}\nmsg.sort            = msg.payload.sort;\"lastmod\"; //type, name, size, mime, lastmod\nif (msg.payload.sortorder == \"asc\"){\n    msg.sort_ascending = true;\n}\nelse {\n    msg.sort_ascending = false;  \n}\n\nmsg.payload = {};\nmsg.payload.filedata= '{}'; // Needs to be a string\nreturn msg;","outputs":1,"noerr":0,"x":680,"y":260,"wires":[["cf4d65d2.181c38"]]},{"id":"73831b5d.cd08d4","type":"http in","z":"d47cd009.b88c6","name":"repository","url":"/repository/search/*","method":"get","upload":false,"swaggerDoc":"","x":440,"y":260,"wires":[["c426b8cc.830298"]]},{"id":"da48aead.84bd3","type":"filebrowser","z":"","repositoryType":"SMB","share":"\\\\SHARE1\\public","port":"445","domain":"","username":"MyUsername","password":"MyPassword","workdir":"","name":"SHARE1","sshKeyPath":"","additionalLftpCommands":"","addFulltextUrlPrefix":false},{"id":"1708125e.da84de","type":"Cache","z":"","name":"rep1","defaultTtl":"60","checkPeriod":"60"}]

Configuration

Add/Configure Repositories

Each Repository can be configured within the filebrowser node. The first drowdown "Repositories" allows to create or modify existing repository entries.

Once a repository has been created (and the filebrowser widget has been relaunched ) the repository can be added to the list of accessible repositories. Each accessible repository has an unique (ascending) number. This is required to identify the approperiate repository later on. The order of accessible repositores can be changed. Please take note that this will also require an to adjust the URL to the repository (Admin Tool)

SMB/CIFS Shares

• Type: Select Type to "SMB"
• Name: Select a Display name which will appear in the list of avaliable repositories
• Share: Set the SMB\CIFS\UNC Share
• Port: Default Port is 445
• Working Directory: Set entry path/working directory if needed
• User/Password/Domain: Enter the user credentials. ATTENTION: Please ensure, that the account has R/RW access to the Share on OS-Level (Windows Credentails Manager):

Indexing

"Everything" is search engine (tool) that locates files and folders by filename instantly for Windows. "Everything" provides a built-in HTTP web server that allows searching and accessing files from a web browser. Search parameters can be sent via http queries and the respone is a JSON. This mechanism is being used by the filebrowser node. It can query the content of a specific repsoitory instead of re-scanning all subfolders again, which safes time (especially with huge repositories)

In order to configure the search engine for SMB/CIFS repositories (FTP/SFTP/FTPS not supported) do the following:

• Install "Everything" (https://www.voidtools.com/downloads/). Select following options during installation

• Add the SMB/CIFS Repository under Tools>>>Options>Folders >> Add

• New Shares can be added by simply typing the full share name (e.g. \\SHARE1). Please ensure that the Share-name matches with the configured repository name in the filebrowser node

• Set an approperiate port number

• Start "Everything" from the TaskManager, using the spec. user account (where credentials are set in the windows credential manager):

• To test try to connect to the HTTP Endpoint:

FTP Shares

To create a new FTP repository set the Repsoitory Type to "FTP":

• Name: Select a Display name which will appear in the list of avaliable repositories
• FTP Protocol: Set a protocol type (ftp/ftps/sftp)
• Share: Set FTP Host name or IP-Adress Share
• Port: Default Port is 21 for FTP, 22 for SFTP and 990 for FTPS
• Working Directory: Set entry path/working directory if needed
• Additional FTP Commands: Additional commands to pass to lftp, splitted by ';'

Caching

In order to improve performance and to limit your API key usage, the cache node allows you to configure the duration during which a request and its answer remain in the cache. The default is 60 seconds.

Security

If your flow is available on the public internet, don't forget to secure your Node-Red instance !

Accessing the Tool

According to the configuration above the filebrowser/repository the repository can be accessed via the following URL:

http://[[NODE-RED-URL]]/repository/search/[[repositoryIndex]]?query=
e.g. http://127.0.0.1:1880/repository/search/2?query=

This will output the files (located on the repository) in JSON Format.

In order to display files and folder in a search widget view (see screenshot at top) a Generic Widget can be applied. The generic widget takes the JSON from the filebrowser response as source and allows navigating through folders and query files (search).

See (https://github.com/SCISYS-MS/GenericWidget) for more details.

E.g. the URL would be then:

https://127.0.0.1:1880/genericWidget?searchURL=./repository/search/2&filterFormURL=./repositories/filters&sortFormURL=./repositories/sort&refreshInterval=60