Batch Upload

A multi-threaded, event-based JavaScript batched upload library written in native JavaScript to be usable with any other framework or library.

Also includes the FileDropUpload class to connect HTML5 File Drag and Drop to a UI component

Preparing the Back End

Batch Upload uses the HTML5 FormData object to send chunks of file data along with information about the file data for the backend to gather, track and reassemble the chunks in the correct order.

An short example in PHP would be something like this (this example will be our reference to how the backend will expect the data):

// upload.php

// Specify where the chunks and files will be saved
$targetDir = FILE_UPLOAD_DIR;

// Get the file's name
$fileName = $_FILES["file"]["name"];

// Get which chunk is being recieved
$chunk = intval($_REQUEST["chunk"]) > 0 ? intval($_REQUEST["chunk"]) : 1;

// How many chunks are expected?
$chunks = intval($_REQUEST["chunks"]) > 0 ? intval($_REQUEST["chunks"]) : 1;

// Move the file data to the target directory, use the filename as an identifier along with the chunk info
$filePath = $targetDir . DIRECTORY_SEPARATOR . $fileName;
move_uploaded_file($_FILES["file"]["tmp_name"], $filePath.'-'.$chunk.'.part');
sleep(1); // Give it a moment

// Check if file has been uploaded
if (!$chunks || $chunk == $chunks) {
    $out = @fopen($filePath, 'wb');
    // Combine the chunks back into the correct order
    for ($i = 0; $i < $chunks; $i++) {
      $in = @fopen($filePath.'-'.$i.'.part', 'rb');
      stream_copy_to_stream($in, $out);
      @fclose($in);
      // Delete the chunk part, as it is no longer needed
      unlink($filePath.'-'.$i.'.part');
    }
    @fclose($out);
}

// Return Success Response, your front-end will handle progress logic
die(json_encode(array(
		'filename' => $fileName,
		'chunk' => $chunk,
		'chunks' => $chunks
	)));

FileUploadManager

The FileUploadManager class object is the easiest way to handle mutiple batch uploads. Batch uploading a file can be handled with only the FileUpload class object, but the FileUploadManager makes it easier to manage one or more file uploads overall.

FileUploadManager also handles some basic file validation before queuing the file for upload.

Settings

| Settings | Default | Description | | --- | --- | --- | | maxQueue | 6 | The maximum number of files to be simultaneously uploaded for this instantiation. | | url | null | The URL of the backend script that will handle the file upload, e.g. upload.php. | | method | POST | The HTTP method used to send the data. PUT is also acceptable, but symantically POST is most correct. | | autoStart | true | Automatically start the upload queue when files are provided. If false, the start() method will need to be called to start uploads. | | allowedTypes | [] | A list of permitted file MIME types. Defaulty, all file types are permitted to be uploaded. Any mime types not in the list will not be queued. e.g., image/png | | allowedExtensions | [] | A list of permitted file extensions. Defaultly, all extensions are permitted. | | maxFileSize | null | The maximum size of a file for upload (in bytes). Files larger than this size are rejected. Defaultly, there is no restriction on file size. | | maxChunkSize | 1048576 | The maximum size (in bytes) of the chunk sent to the server. Default is 1MB. | | formFileField | file | The name of the parameter the backend uses to capture the file data. | | chunkParameter | chunk | The name of the parameter the backend uses to determine which chunk is being uploaded. | | chunksParameter | chunks | The name of the parameter the backend uses to determine how many chunks to expect. |

• Note: File validation priority goes size, mime type, extension. If 'image/jpeg' is the allowedTypes list and the allowedExtentions list has values but 'jpg' is not in the list, the file will be rejected. If 'image/jpeg' is in the allowedTypes list and the allowedExtensions is empty, then the file will be accepted. And vice-versa. *

Methods

| Method | Description | | --- | --- | | new FileUploadManager(Object settings) | Constructor | | addFile(File file) | Adds a File object to the queue for uploading. If the queue isn't paused, queue autostarting is permitted, and the queue length is less than the configured maxQueue setting, the upload will automatically start. If the file is successfully loaded into the queue, given it passes validation checks, a queue event will be issued on the instantiated FileUploadManager object for which the File is queued. | | setFiles(FileList|Array[File] files) | Sets the list of files to upload to the queue with similar behavior to that of addFile() but is destructive. If any files are in the queue, they will be trashed. | | start() | Starts uploading the queue of files. This has no affect if the queue is already started. A start event is triggered. | | pause() | Pauses the upload queue. All FileUploads will finish uploading their current chunk and wait for to upload the next chunk until the queue is started. If the FileUpload has completed its last chunk, the complete event will trigger and the FileUploadManager will issue the file_complete event. | | clearQueue() | Clears all files that are queued for upload. Any current uploads will be trashed along with queued files | | clearErrors() | The FileUploadManager keeps errored and completed files in seperate queues to reference in the FileUploadEvent object. This method will clear the errors queue. | | clearCompleted() | Like clearErrors(), but clears the completed files. | | validateTypes(File file) | Validates a File's type with the FileUploadManagers settings if the allowedTypes setting is configured. If a File's type is not valid, the File will not be queue. | | validateExts(File file) | Validates a File's extensions if the allowedExtensions setting is configured. | | validateSize(File file) | Validates a File's file size if the maxFileSize setting is configured. | | validate(File file) | Runs the three previous validation functions on the provided File. | | setSettings(Object settings) | Sets the settings object for an instance of a FileUploadManager. | | on(event, callback) | Add a callback function to a FileUploadManager object to be trigger for a specific event. The callback function will receive a FileManagerEvent object as its only parameter. See Events below. | | off(event, callback) | Removes a callback function from a specific event. See Events below. | removeAllListeners() | Removes all event callbacks from the FileUploadManager object. | | addEventListener(event, callback) | See on() | | removeEventListener(event, callback) | See off() | | dispatchEvent(event) | Triggers all listener callbacks for the given event. See Events below. |

Events

| Event Name | Description | | --- | --- | | start | Dispatches when the start() method is successfully called. | | pause | Dispatched when the pause() method is successfully called. | | queue | Dispatched when a file is successfully add to the upload queue. | | invalid | Dispatched when a file has not met the validation requirements. | | complete | Dispatched when a FileUpload object has successfully uploaded all its chunks. | | progress | Dispatched when upload information on a FileUpload object is updated. | | error | Dispatched when a FileUpload object fails to upload a chunk. | | file_start | Dispatched when a FileUpload object in the queue begins its upload process. | | file_progress | Deprecated. See progress | | file_complete | Deprecated. See complete | | file_error | Deprecated. See error |

FileManagerEvent

An object with information about a FileUploadManagers state.

Properties

| Property Name | Description | | --- | --- | | bytesLoaded† | The bytes uploaded to the server for a the FileUpload object which triggered the event on the FileUploadManager. | | bytesTotal† | The total file size in bytes for the FileUpload object which triggered the event on the FileUploadManager. | | totalBytesLoaded | The total number of bytes that have been uploaded for the FileUploadManagers queue. | | totalBytesTotal | The sum of all the file sizes loaded into the FileUploadManagers queue. | | currentTarget† | The FileUpload which triggered the event on the FileUploadManager. | | file† | The File object the FileUpload is uploading. | | queue | The upload queue at the time the event was triggered. | | errors | Errors that have occurred by the time the event was triggered. | | fileList | Files that have not been queued for upload at the time the event was triggered. | | completed | FileUploads that have been completed by the time this event was triggered. | | data† | Only used for 'file_complete' events. The XHR response from the final upload request, JSON parsed. |

† This property is not used if the event was not triggered because of a FileUpload object.

FileUpload

Settings

| Settings | Default | Description | | --- | --- | --- | | filename | null | The filename the backend should use. If null, the filename from the File object is used. | | url | null | The URL of the backend script that will handle the file upload, e.g. upload.php | | method | POST | The HTTP method used to send the data. PUT is also acceptable, but symantically POST is most correct | | autoStart | true | Automatically start the upload queue when files are provided. If false, the start() method will need to be called to start uploads. | | maxChunkSize | 1048576 | The maximum size (in bytes) of the chunk sent to the server. Default is 1MB. | | formFileField | file | The name of the parameter the backend uses to capture the file data. | | chunkParameter | chunk | The name of the parameter the backend uses to determine which chunk is being uploaded. | | chunksParameter | chunks | The name of the parameter the backend uses to determine how many chunks to expect. |

Methods

| Method | Description | | --- | --- | | new FileUpload(Blob file, Object settings) | Constructor | | start() | Starts the upload process. Dispatches the 'start' event. | | pause() | Pauses the upload process. Dispatches the 'pause' event. | | reset() | Resets the upload process. Calls pause() and start() (if previously unpaused). | | setSettings(Object settings) | Sets the settings object. | | setUrl(String url) | Sets the URL to upload the file to. Do not change mid-upload. This will corrupt the upload. | | addEventListener(event, callback) | Add a callback function to a FileUpload object to be trigger for a specific event. The callback function will receive a FileUploadEvent object as its only parameter. See Events below. | | removeEventListener(event, callback) | Removes a callback function from a specific event. See Events below. | removeAllListeners() | Removes all event callbacks from the FileUpload object. | | on(event, callback) | See on() | | off(event, callback) | See off() | | dispatchEvent(event) | Triggers all listener callbacks for the given event. See Events below. |

Events

| Event Name | Description | | --- | --- | | start | Dispatches when the start() method is successfully called. | | pause | Dispatched when the pause() method is successfully called. | | progress | Dispatched when upload progress updated. | | complete | Dispatched when successfully uploaded all its chunks. | | error | Dispatched when fails to upload a chunk. |

FileUploadEvent

| Property Name | Description | | --- | --- | | bytesLoaded | Number of bytes sent to the server. | | bytesTotal | Total number of bytes to be uploaded. | | filename | The filename used by the server. | | file | The File object being uploaded. | | duration | The total duration (in seconds) the upload took. Doesn't include paused time. | | data | The JSON parsed response from the server. *Only used for 'error' and 'complete' events. | | header | The response headers from the server. *Only used for 'error' and 'complete' events. |

FileDropZone

Settings

| Settings | Default | Description | | --- | --- | --- | | manager | null | The FileUploadManager object to use. If null, a new FileUploadManager will be instatiated using settings from the FileDropZone. | | dragOverClass | drag-over | The class to be added to the DOM element when files are dragged over the element. | | maxQueue° | 6 | The maximum number of files to be simultaneously uploaded for this instantiation. | | url° | null | The URL of the backend script that will handle the file upload, e.g. upload.php. | | method° | POST | The HTTP method used to send the data. PUT is also acceptable, but symantically POST is most correct. | | autoStart° | true | Automatically start the upload queue when files are provided. If false, the start() method will need to be called to start uploads. | | allowedTypes° | [] | A list of permitted file MIME types. Defaulty, all file types are permitted to be uploaded. Any mime types not in the list will not be queued. e.g., image/png | | allowedExtensions° | [] | A list of permitted file extensions. Defaultly, all extensions are permitted. | | maxFileSize° | null | The maximum size of a file for upload (in bytes). Files larger than this size are rejected. Defaultly, there is no restriction on file size. | | maxChunkSize° | 1048576 | The maximum size (in bytes) of the chunk sent to the server. Default is 1MB. | | formFileField° | file | The name of the parameter the backend uses to capture the file data. | | chunkParameter° | chunk | The name of the parameter the backend uses to determine which chunk is being uploaded. | | chunksParameter° | chunks | The name of the parameter the backend uses to determine how many chunks to expect. |

° Not used when the manager setting is supplied.

Methods

| Method | Description | | --- | --- | | new FileDropZone(Element element, Object settings) | Constructor. Accepts a DOM Element and settings object as parameters. Constructor parameters are not required. | | setElement(Element element) | Binds a DOM Element with the FileDropZone object. | | setSettings(Object settings) | A settings object with properties as defined above. |

Utilities

List

Properties

| Property | Default | Description | | --- | --- | --- | | key | this.length | A string eval()'d to determine the item's key value when added to the list. Use item to reference the item when adding. | | length | 0 | The number of items in the list. | | list | {} | The object representing the list. Each item in the list is a value in this object with a key value as the item's property name. |

Methods

| Method | Description | | --- | --- | | new List() | Constructor for empty list with default settings. | | new List(String key, Array items) | Constructor specifying the key eval and a set of items to populate. | | new List(String key) | Constructor for empty list with custom key eval. | | new List(Array items) | Constructor with default key eval and pre-populated set of items. | | get(key) | Get an item from the list using a key value. | | add(item) | Add an item to the list. The item's associated key will be set dynamically by eval()'ing the List.key property. | | remove(item) | Removes an item from a list. Performs binary assertion. | | item(index) | Get an item at an indexed value (starting at 0) instead of a key. | | clear() | Clear and reset the list. | | toArray() | Get list items as an array. Key values are not passed to the array, only the items in the list. |