userland storage

This is a simple standalone storage system that's designed to be integrated into any software that needs user-based storage.

We're using it here as part of the identity system we call userland. But it should be just as easy to integrate it with other identity systems.

I wanted once and for all to have a thoroughly thought-out design for this, based on many years of experience using and creating these kinds of storage systems.

It's influenced by many different file systems, from Unix, Mac, PC, S3, Frontier and most recently FeedLand.

It's influenced especially by the Mac OS -- each file has a metadata property, in JSON, that's shared between all the applications for each user. In the Mac it's like the refcon handle in objects and the resource fork of files.

Why userland?

It's called userland because like the company I started in the late 80s, it's the best name for describing what's going on here. This is functionality for developers to create products for users. It's taking the last step for turning the web into a virtual machine. There already exist good toolkits for user interface and serving content publicly. But there is no way for an app developer to create a utility for users without having to create their own OS. Obviously that has limited the creativity of developers. I've tried to explain many times why this is needed, but I guess the only way to make it happen is to give others a target to shoot at. I've done it for Node.js, but the apps that use it can run in any environment as long as they have the ability to make calls over the web, and there are good standards for that.

What is a file?

Each file is identified by a username and a relpath.

Each file is also assigned a unique integer id, and all the API functions use the id. There's a function that turns a username/relpath pair into an id, of course.

Files can be public or private. Public files can be served over the web, and a function is provided to interface to a web server. You have to provide the server function.

Files have contents, a type, dates for when the file was created and updated and how many times it has been updated.

SQL code

create table storage (
	id int auto_increment primary key, 
	username varchar (255) not null default '', 
	relpath varchar (255) not null default '', 
	flprivate boolean not null default true,
	type varchar (64) not null default 'application/json',
	filecontents text not null,
	whenCreated datetime not null default current_timestamp, 
	whenUpdated datetime not null default current_timestamp on update current_timestamp, 
	ctUpdates int unsigned not null default 0,
	metadata json not null default (json_object()),
	unique key (username, relpath)
	);