axloader
v1.0.6
Published
This is a library for loading assets files in batches. It works well for HTML5 games and for large web apps.
Downloads
5
Readme
AxLoader
On Node: npm install axloader
On Browsers: bower install axloader or download directly
This is a library for loading assets files in batches. It works well for HTML5 games and for large web apps.
Normally, the browser downloads your assets one at a time using one request for each, usually 4 - 8 requests at a time. This can be slow because of the added latency between the browser and the server. To work around this, you would merge your code files with browserify and maybe merge image files too or embed them into css. All of these approaches seriously hurt workflow & debugging. AxLoader solves this problem by allowing you use a normal workflow without the scalability problems. It does this by putting all your assets in an archive file and downloading them all in one request.
This approach has several advantages:
- Allows incremental updates
- Less round trips
- Easy zero downtime updates
- You control the cache
- You don't have to maintain a list of assets for the client to download
##How it works
AxLoader reads in your files and builds an archive and an index. The client will work out the most up to date version is, download it, and extract your assets in memory so you can use them as you wish.
It also has utility methods for turning data into strings or data urls, parsed JSON, and it will run your script files if you want, in a way that plays nicely with the debugger.
The best part: When it generates the archive, it also reads old versions of your app and generates patch files against them. If the client has an old version cached and a patch is available, it'll apply the patch file to the cached version automatically.
###Why is this better than If-Modified-Since? The browser still has to send one request for each file. Even with small latencies and, this can take some time, especially if nothing has changed. With AxLoader, it takes one round trip to know if you're up to date and one more to do the update.
Additionally, if you publish a new version of your app while a client is downloading it, the client might get files from both versions. Not only does AxLoader not do this, but you can manually tell your app to check for updates.
###Why is this better than AppCache? AppCache has the same problems as If-Modified-Since, and several other problems:
- You can't use query urls in AppCached apps.
- It has an awkward update mechanic.
- You have to build a list of all the files to be cached (manifest).
###Why is this better than Embedded Data URLs? Embedding data URLs is great if you have a few small image files, but doesn't scale up well. The workflow isn't great either.
##Setup Guide
To use AxLoader, run npm install axloader and do
require("axloader").devserver({/*config*/}).
And on the client, run bower install axloader or download this.
###Client To use AxLoader on the client, all you need to do is link the AxLoader JavaScript file as well as QuickArchive and define some callbacks:
<!DOCTYPE html5>
<html>
<head>
<script type='text/javascript' src='axloader.js'></script>
<script type='text/javascript' src='quickarchive.js'></script>
<script type='text/javascript'>
axLoader.startLoading("mydata", {
onComplete: function (assets) {
//Done loading. The assets variable wil contain all the data in
//the mydata folder.
},
onDecision: function (file) {
//At this point we know which version to download if any
},
onStatus: function (status) {
//Called with messages describing what's going on
},
onProgress: function (n) {
//Tells you how much has been downloaded. n is a number between 0 and 1.
},
onError: function (s) {
//Something died :(
},
}, [
//A list of transformations to do on the data.
//Convert all txt files to text (UTF8)
AxLoader.FILTER.WITH_EXT("txt", AxLoader.POST.STRINGIFY),
//Convert all png files to data urls.
AxLoader.FILTER.WITH_EXT("png", AxLoader.POST.BASE64IFY_AS_MIME("image/png")),
//Convert all jpg files to data urls.
AxLoader.FILTER.WITH_EXT("jpg", AxLoader.POST.BASE64IFY_AS_MIME("image/jpeg")),
//Remove file extensions from keys
AxLoader.POST.REMOVE_FILE_EXTS,
//Convert folders to objects. So instead of getting:
//{"path/to/data": "data"}
//you'll get:
//{path: {to: {data: "data"}}}
AxLoader.POST.EXPAND_FOLDERS
]);
</script>
</head>
<body>
</body>
</html>###Development Server
This library includes a development server that automatically generates archive files
every time one is requested. You can start it by running
require("axloader").devserver({/*config*/}).
The config object looks like this:
{
//Set this to the directory your assets are in. You can list multiple locations
//to generate multiple archives. The last part of each path should be unique.
"assetFolders": ["mydata"],
//Set this to the HTML file that should be served on /.
"index": "./static/myapp.html",
//Set this to the port you want to use.
"port": 8000,
//Set this to the folder you want to output generated files.
"output": "./output"
}###Production Server Whatever server platform you choose to run in production, all you have to do is statically serve the files output by AxLoader. By default, it looks for the files using relative paths. You can change this using syntax like this:
<script type='text/javascript' src='axloader.js?pathprefix=/assets/'></script>This example would cause it to look for assets in the /assets directory.
It's works like this because the script requests the index file immediatly.
The query string is read on the client and you don't need to do anything to support it.
####Generating Production Versions and Patches
Simply run require("axloader").generate({/*config*/}) where the config is the same as
specified above. This will generate a new production version of the project, as well
as incremental patches for all recent versions, and place them in the output
folder.
Once these new files are put on the production server, the client will know which files to download through the index file. It will download patch files if it has an old version and an appropriate patch exists.
##Misc
###Checking for Updates
AxLoader requests the index file on load, which is how it knows which version to download.
This means that if you publish a new version after the client loads the index, it won't
be able to see it. You can force reload using the checkForUpdates function.
The checkForUpdates function takes a callback argument, which will be passed a boolean,
whether there are any updates.
Be very careful with this as it updates the index for everything. If you have multiple archives you can end up with mismatches. It's up to you to unload an old version before getting a new one.
It is safe to start loading a new file while AxLoader is still checking for updates. You'll get the up to date version.
###Postprocessing Tasks
The third parameter of startLoading is an array of functions that process the data.
Since the data comes in the form of Uint8Arrays, you'll probably need to process it. A collection
of these functions are included.
The postprocessing functions are called for each item in the archive, so you need to filter them so only the ones you want get processed. Do this with the filter functions:
AxLoader.FILTER.WITH_EXT(ext, func)Only call func for files with a matching extention.AxLoader.FILTER.WITH_FILE(name, func)Only call func for files with a matching filename.AxLoader.FILTER.WITH_MATCHING(regex, func)Only call func for files whose name matches regex.AxLoader.FILTER.WITHOUT(regex, func)Call func with everything except those matching regex.
These are the postprocessing functions:
AxLoader.POST.EVAL_AS_JSRun each file as a UTF8 encoded javascript string.AxLoader.POST.STRINGIFYTurn each file into a UTF8 string.AxLoader.POST.PARSE_AS_JSONTurn each file into an object from JSON.AxLoader.POST.BASE64IFYTurn each file into a plain base 64 string.AxLoader.POST.BASE64IFY_AS_MIME(mime)Turn each file into a base 64 encoded data url with specified mime type.AxLoader.POST.REMOVE_FILE_EXTSRemove file extensions.AxLoader.POST.EXPAND_FOLDERSTurns folders into objects.
####Example This example turns all text files into strings, then runs all the javascript files in an archive, making sure the main file is run last, and then removes file extensions.
axLoader.startLoading("demo", {
//callbacks omitted for brevity
}, [
axLoader.FILTER.WITH_EXT("txt", axLoader.POST.STRINGIFY),
axLoader.FILTER.WITH_EXT("js", axLoader.FILTER.WITHOUT(/main\.js/, axLoader.POST.EVAL_AS_JS),
axLoader.FILTER.WITH_FILE("main.js", axLoader.POST.EVAL_AS_JS),
axLoader.POST.REMOVE_FILE_EXTS,
]);###Clearing the Cache
You can manually clear the cache using the clearCache method.
AxLoader.clearCache("myFolder", "myOtherFolder");##TODO list
- Audio features
- Allow filters and posts to support any input type
- Use HTML5 fs if availiable