npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

federated-learning-server

v0.0.5

Published

Experimental server library for federated learning in Tensorflow.js

Downloads

9

Readme

Tensorflow.js Federated Learning Server

This library sets up a simple socket.io-based server for transmitting and receiving TensorflowJS model weights.

Usage

Basic

import * as http from 'http';
import * as federated from 'federated-learning-server';

const INIT_MODEL = 'file:///initial/model.json';
const httpServer = http.createServer();
const fedServer = new federated.Server(httpServer, INIT_MODEL);

fedServer.setup().then(() => {
  httpServer.listen(8080);
});

Setting the Initial Model

new federated.Server(httpServer, tfModel); // Initialize a federated server from an in-memory tf.Model
new federated.Server(httpServer, 'https://remote.server/tf-model.json'); // or from a URL pointing to one
new federated.Server(httpServer, 'file:///my/local/file/tf-model.json'); // (which can be a file URL in Node)
new federated.Server(httpServer, async () => { // or from an asynchrous function returning one
  const model = await tf.loadModel('file:///transfer/learning/model.json');
  model.layers[0].trainable = false;
  return model;
});
new federated.Server(httpServer, federatedServerModel); // if you need fully custom behavior; see below

The simplest way to set up a federated.Server is to pass a tf.Model. However, you can also pass a string that will be delegated to tf.loadModel (both https?:// and file:// URLs should work), or an asynchronous function that will return a tf.Model. The final option is to define your own FederatedServerModel, which has to implement various saving and loading methods. See its documentation for more details.

Note that by default, different tf.Model versions will be saved as files in subfolders of ${process.cwd()}/saved-models/. If you would like to change this directory, you can pass a modelDir configuration parameter, e.g. federated.Server(httpServer, model, { modelDir: '/mnt/my-vfs' }).

If you would like to skip the persistence layer, you can instead import FederatedServerInMemoryModel which will update a single model in memory. Furthermore, if you want a version of this library which omits socket.io in favor of a mocked-out version that works in the browser, check out the mock server library.

Setting Hyperparameters

new federated.Server(httpServer, model, {
  // These are true server parameters
  serverHyperparams: {
    aggregation: 'mean',      // how to merge weights (only mean supported now)
    minUpdatesPerVersion: 20, // server merges every 20 client weight updates
  },
  // These get broadcast to clients
  clientHyperparams: {
    learningRate: 0.01,    // client takes SGD steps of size 0.01
    epochs: 5,             // client takes 5 SGD steps per weight update
    examplesPerUpdate: 10, // client computes weight updates every 10 examples
    batchSize: 5,          // client subdivides `examplesPerUpdate` into batches
    noiseStddev: 0.001     // client adds N(0, 0.001) noise to their updates
  },
  verbose: false,           // whether to print debugging/timing information
  modelDir: '/mnt/my-vfs',  // server stores tf.Model-specific versions here
  modelCompileConfig: {     // tf.Model-specific compile config
    loss: 'categoricalCrossEntropy',
    metrics: ['accuracy']
  }
})

Many of these hyperparameters matter a great deal for the efficiency and privacy of learning, but the correct settings depend greatly on the nature of the data, the size of the model being trained, and how consistently the data is distributed across clients. In the future, we hope to support automated (and dynamic) tuning of these hyperparameters.

Listening to Events

You can add an event listener that fires each time a client uploads a new set of weights (and optionally, self-reported metrics of how well the model performed on the examples used in training):

fedServer.onUpload(message => {
  console.log(message.model.version); // version of the model
  console.log(message.model.vars); // serialized model variables
  console.log(message.clientId); // self-reported and usually random client ID
  console.log(message.metrics); // array of performance metrics for the update; only sent for clients configured to `sendMetrics`
});

You can also listen for whenever the server computes a new version of the model:

fedServer.onNewVersion((oldVersion, newVersion) => {
  console.log(`updated model from ${oldVersion} to ${newVersion}`);
});

TODO

Robustness:

  • median and trimmed-mean aggregations (for Byzantine-robustness)
  • client authentication (e.g. google oauth, captchas)
  • smoothing to limit individual clients' weight contributions (to prevent model from overfitting to most active clients and also create preconditions for Byzantine-robust learning if some clients are adversarial)
  • create virtual server-side clients who minimize train loss
  • discard client updates that increase server-side train loss (or subtract updates' projections onto the direction of increasing train loss)

Privacy:

  • determine how to set hyperparameters such that each version of the model is differentially private to individual clients' updates (i.e. prevent sensitive information from leaking from client->server->client)
  • consider implementing secure aggregation (i.e. prevent sensitive information from leaking from client->server)