npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

valtio-persist

v1.0.2

Published

Flexible and performant saving of state to disk.

Downloads

2,738

Readme

valtio-persist

npm i valtio-persist allows flexible and performant saving of state to disk.

Quick Start - Basic Usage

import proxyWithPersist, { PersistStrategy } from 'valtio-persist';
import { subscribeKey } from 'valtio/utils';

const appStateProxy = proxyWithPersist({
  // must be unique, files/paths will be created with this prefix
  name: 'appState',

  initialState: {
    counter: 0,
  },
  persistStrategies: PersistStrategy.SingleFile,
  version: 0,
  migrations: {},

  // see "Storage Engine" section below
  getStorage: () => storage,
});

console.log('counter:', appStateProxy.counter);

subscribeKey(appStateProxy._persist, 'loaded', (loaded) => {
  if (loaded) {
    console.log('it is now safe to make changes to appStateProxy. the changes will now be persisted.');
  }
});

This will persist the entire object into one file, on every change.

You can read from appStateProxy immediately, however if you want changes persisted, wait until appStateProxy._persist.loaded goes to true.

This is obvious but to be safe, keep in mind the base value (initialState) must be an object. This applies to proxy as well from valtio, the argument to proxy is an object.

Every object returned by proxyWithPersist gets a special _persist key added to it. This key has the value of:

{
  status: 'loading' | 'loaded' | 'error';
  loading: boolean;
  loaded: boolean;
  error: null | Error;
}

You can use this section to figure out when loading has completed.

Storage Engine

You can use any storage engine as long as it respects the following interface:

export type ProxyPersistStorageEngine = {
  // returns null if file not exists
  getItem: (name: string) => string | null | Promise<string | null>;

  setItem: (name: string, value: string) => void | Promise<void>;
  removeItem: (name: string) => void | Promise<void>;
  getAllKeys: () => string[] | Promise<string[]>;
};

getItem should return null if file or path does not exist.

getAllKeys is used for the PersistStrategy.MultiFile. If you do not use this strategy, then you can make this function no-op.

To use this engine, set the getStorage option to a function that returns this. It can be async, it is only run once.


const stateProxy = proxyWithPersist({
  // ...
  getStorage: async () => {

    // do some async stuff, maybe create a directory you want to store this into

    // return storage interface
    return {
      getItem: () => { ... },
      setItem: () => { ... },
      removeItem: () => { ... },
      getAllKeys: () => { ... }
    }
  }
})

window.localStorage

Documentation on window.localStorage can be found here: https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage.

import proxyWithPersist from 'valtio-persist';
import type { ProxyPersistStorageEngine } from 'valtio-persist';

const storage: ProxyPersistStorageEngine = {
  getItem: name => window.localStorage.getItem(name),
  setItem: (name, value) => window.localStorage.setItem(name, value),
  removeItem: name => window.localStorage.removeItem(name),
  getAllKeys: () => Object.keys(window.localStorage);
};

const stateProxy = proxyWithPersist({
  getStorage: () => storage;
});

@react-native-async-storage/async-storage

Documentation on AsyncStorage can be found here: https://github.com/react-native-async-storage/async-storage.

npm i @react-native-async-storage/async-storage
import AsyncStorage from '@react-native-async-storage/async-storage';
import proxyWithPersist from 'valtio-persist';
import type { ProxyPersistStorageEngine } from 'valtio-persist';

const storage: ProxyPersistStorageEngine = {
  getItem: name => AsyncStorage.getItem(name),
  setItem: (name, value) => AsyncStorage.setItem(name, value),
  removeItem: name => AsyncStorage.removeItem(name),
  getAllKeys: () => AsyncStorage.getAllKeys();
};

const stateProxy = proxyWithPersist({
  getStorage: () => storage;
});

expo-file-system

Documentation on expo-file-system can be found here: https://docs.expo.dev/versions/latest/sdk/filesystem.

expo install expo-file-system
import * as FileSystem from 'expo-file-system';
import proxyWithPersist from 'valtio-persist';
import type { ProxyPersistStorageEngine } from 'valtio-persist';

const storage: ProxyPersistStorageEngine = {
  getItem: name => FileSystem.readAsStringAsync(FileSystem.documentDirectory + name),
  setItem: (name, value) => FileSystem.writeAsStringAsync(FileSystem.documentDirectory + name, value),
  removeItem: name => FileSystem.deleteAsync(FileSystem.documentDirectory + name),
  getAllKeys: () => FileSystem.readDirectoryAsync(FileSystem.documentDirectory);
};

const stateProxy = proxyWithPersist({
  getStorage: () => storage;
});

Persist Strategies

There are two techniques to persist, "single file" (PersistStrategy.SingleFile) or "multi-file" (PersistStrategy.MultiFile).

Single File

The single file strategy will stringify the value and store it into one file.

In the example here, any time a photo is added, or removed, or a key in the photo is updated, JSON.stringify runs on the entire photos object, and then this is written to file.

const stateProxy = proxyWithPersist({
  // ...

  initialState: {
    photos: {
      1: { id: 1, views: 0 },
      2: { id: 2, views: 0 },
      3: { id: 3, views: 0 },
      4: { id: 4, views: 0 }
    }
  }

  persistStrategies: {
    photos: PersistStrategy.SingleFile
  }
})

Multi-file

There is a second strategy called multi-file. This can only be used on keys that have an object-type value. Each key in the object will be turned into a file. This offers improved performance, because the entire value of of the object is not stringified, just individual values of the keys in the object are stringified, and then written to its own file.

In the example above, photos has an object-type value, so let's use multi-file strategy here.

const stateProxy = proxyWithPersist({
  // ...

  initialState: {
    photos: {
      1: { id: 1, views: 0 },
      2: { id: 2, views: 0 },
      3: { id: 3, views: 0 },
      4: { id: 4, views: 0 }
    }
  }

  persistStrategies: {
-    photos: PersistStrategy.SingleFile
+    photos: PersistStrategy.MultiFile
  }
})

Now adding a photo with key 5 and value of {id: 5, views: 0 } will only stringify this value and write it to disk. Updating the photos['2'].views to value of 99 will only stringify the photo at this position, and write it to it's individual file.

Whitelisting

To only persist certain keys, define an object for the persistStrategies option. The keys of this object are dot path notation for the paths you want to store. Here is an example:

const stateProxy = proxyWithPersist({
  // ...

  initialState: {
    entities: {
      tasks: {},
      schedules: {},
    },
  },

  persistStrategies: {
    'entities.tasks': PersistStrategy.SingleFile,
  },
});

In this example, only stateProxy.entities.tasks will get persisted. Any changes to stateProxy.entities.schedules or anywhere else, will not get persisted.

Migrations

The two keys in the config argument of proxyWithPersist related to migrations are version and migrations.

The version is required and must be a number. Any time persisted data is loaded, it compares the persisted version, to the current version passed into proxyWithPersist argument. If the persisted version is less than the one passed in to the argument, migrations will then be run in ascending order of numbered key.

The migrations option must be an object where each key is a version. The value is an async function, it receives no arguments, and returns nothing, it just mutates the proxy object. All the migrations will be run that have a number key that is greater than persisted version and less-than-or-equal-to the version passed into proxyWithPersist.

Example:

The last persisted version was 0.

const stateProxy = proxyWithPersist({
  // ...

  version: 2,
  migrations: {
    1: async () => {
      stateProxy.counter = {};
    },

    2: async () => {
      delete stateProxy.foo;
    },
  },
});

When the app runs, it finds the last persisted version was 0, but the current version is 2. It will first run migration with key of 1 and then it will run migration with key of 2 and then _persist.loaded will be set to true.

Recipes

Throttle Writes for Performance

Sometimes, writing to disk on every change immediately hurts performance. Here is a technique to changes get persisted at most once a second. It uses the throttle method from lodash. It will save to disk at most once a second.

Note: Debounce is not recommended as it could lead to starvation. For example, if writes are debounced to 1 second, but writes happen after 0.5s, then a write will never happen.

npm i lodash
import { throttle } from 'lodash';

const stateProxy = proxyWithPersist({
  // ...

  onBeforeBulkWrite: throttle(bulkWrite => bulkWrite(), 1000)
}