🚂 chifley

🚨🚨                                                                    🚨🚨
🚨 Warning: This library is very new and should be considered pre-release. 🚨
🚨                         Do not use in production.                       🚨
🚨🚨                                                                    🚨🚨

A small, speedy reactive programming library with a simple API designed to be app ready.

• 👶 Simple reactivity model
• 🤖 Easy to debug
• 📉 Low memory use
• 🏃‍♂️‍➡️ Fast
• 🤏 Tiny (4kb gzipped) but with just enough features ⭐

Quick Start

import { Stream as c } from 'chifley'

const text = c('wow')

// perfectly valid email validation 🫠
const isEmail = 
    text.map( text => text.includes('@') )

isEmail.get()
//=> false

try {
    isEmail.set(true)
} catch (e) {
    // Cannot write to a readonly stream!
    console.error(e) 
}

text.set('[email protected]')

isEmail.get()
//=> true

Why Chifley?

What makes Chifley different from other stream libraries? Chifley is focused on one goal; to be the perfect stream library for managing state in UIs. In order to meet that goal, Chifley also has to be fast, easy to use (and debug!) and it has to be small.

You'll find Chifley familiar if you've used libraries like flyd or mithril-stream. If you've never used a stream library before then Chifley is a great place to start, it is designed to be easy to learn. If you're coming from libraries like Rx.js you need to unlearn what you have learned as Chifley has a very different philosophy and semantics.

API

Creating a stream

You can initialize a stream with an initial value or without. If you don't provide an initial value, the type of the stream will be T | undefined otherwise it will be T.

import { Stream as c } from 'chifley'

type Song = { 
    track: string
    duration: number 
}
const name = c('Ringo')
const songs = c<Song>()

type N = ReturnType<typeof name.get> 
// string

type S = ReturnType<typeof songs.get> 
// Song | undefined

Reading stream values

The last emitted value of a stream is always cached and can be retrieved via .get()

const name = c('Ringo')

c.get()
//=> 'Ringo'

If the stream had no initial value, the type will be T | undefined:

const name = c<string>();

c.get().toUpperCase() 
// 🚫 Not OK: Typescript Error 

c.get()?.toUpperCase() 
// 🟢 OK

Writing stream values

You can change a stream value via .set

name.set('George')

This will trigger dependent streams to also update.

If you need to access the previous value as part of the update you can use .update

name.update( 
    prev => 
        prev == 'John' 
            ? 'Paul' 
            : 'George' 
)

Subscribing to changes

You can subscribe to changes via .map. It will only emit when it has received a value. If you don't initialize a stream, don't worry, it won't emit until the source stream receives a value.

const name = c<string>();

name.map( x => 
    console.log('The name changed: ', x)
)

// At this point nothing has logged.

name.set('Paul')
// Logs: 'The name changed: Paul'

Ending a stream

You can end a stream and all of its dependent streams by calling .end(). You can also check if a stream has ended via stream.state === 'ended'

name.end()

If you want to subscribe to the end of a stream you use stream.ends.add(endCallback).

const callback () => {
    return () => console.log('name ended')
}
name.ends.add(callback)

Equally you can unsubscribe via name.ends.remove(callback).

🤓 Unlike mithril-stream and flyd we do not dynamically create end streams. This is one of those things that we really liked in principle but we rarely used. It also creates strange corner cases that need to be documented and explained and ultimately would only be worth it if was used frequently enough.

Opting out of an update

Within a visitor function, you can return c.SKIP which will be detected and skip propagation of that stream and its dependencies. The existing value of the stream will be retained and no dependencies will be updated.

const year = c(2000)

const olympicYear = 
    age.map( x => x % 4 === 0 ? x : c.SKIP )

// immediately logs: 'olympics! 2000'
olympicYear.map( x => {
    console.log('olympics!', x)
})

year(1999)
// no log occurs

year(2024)
// logs: 'olympics! 2024'

This allows .map (and most other operators) to act like .filter or .reject which can be incredibly powerful.

🤓 Note unlike mithril-stream SKIP only works as a return value within a stream transform. You can write SKIP to a writable stream but it will be treated like any other value. This removes two logical branches in the update path and speeds things up enough to warrant the sacrifice.

Static methods and properties

merge

merge is useful if you have two or more streams and you would like to combine their values.

You provide a list of streams and it will provide a stream of a list of values.

const organization_id = c<string>()
const project_id = c<string>()

const ids = 
    c.merge(
        [organization_id, project_id] 
        as const
    )

const path = 
    ids.map( ([o_id, p_id]) => 
        `/org/${o_id}/project/${p_id}`
    )

path.map( path => 
    console.log(
        'URL path changed: ', path
    )
)

merge will wait for all streams to have at least 1 value/emit before it emits its first update.

// So far nothing has logged
organization_id.set('1')

// still nothing has logged

project_id.set('2')
// now this logs:
//  'URL path changed: /org/1/project/2

isStream

Not very exciting but if you want to know if some arbitrary object is an instance from this library use this:

isStream(null) 
//=> false

isStream( c('hello') )
//=> true

Instance operators

map

When you have one input stream and you want to transform it and produce a new output stream use .map

const name = c('George')

const isCoolBeatle = 
    name.map( name => name === 'Ringo' )

dropRepeats / dropRepeatsWith

By default streams emit whenever they receive a new value even if the value is the same as the previous value.

If you'd like to skip emitting on values that have the same reference equality, then you can use dropRepeats.

const food = c<string>()

const foodOrder = food.dropRepeats()

foodOrder.map( x => console.log(
    'Hurry up!  I want my ', x + '!'
))

food.set('pretzel') 
// Hurry up!  I want my pretzel!
food.set('pretzel') 
food.set('pretzel')
food.set('pretzel')

food.set('doughnut') 
// Hurry up!  I want my doughnut!

If you'd like to change the equality function, use dropRepeatsWith( ... ) with a custom equality function e.g. for deep equality you could use Ramda's R.equal or Lodash's deepEqual.

const stateChanges = 
    state.dropRepeatsWith(R.equal)

filter / reject

For opting in and out of changes, we can use c.SKIP but sometimes defining control flow with simple predicates is clearer.

For these cases, filter and reject can help clean up things a bit.

E.g. only exit a terminal if the line feed exactly equals 'exit':

const linefeed = c<string>()

const exit = 
    linefeed.filter( x => x == 'exit' )

exit.map( () => killTerminalSubProcess() )

Or only execute statements that don't start with a comment:

const queryInput = c<string>()

const notAComment = 
    linefeed
    .reject( x => x.startsWith('--') )

const completedStatement = 
    notAComment
    .filter( x => x.includes(';') )

completedStatement
    .map( query => {
        execSqlQuery(query)
    })

default

If you want to guarantee a stream has a value you can use default.

🤓 default isn't used for coalescing undefined or null values, it is for handling streams that may not have emitted a value yet.

const projects = c<Project[]>()

{
    // 🚫 Type and runtime Error 
    // as `projects.get()`
    // may not have emitted yet.
    const length = 
        projects.get().length
}

{
    // 🟢 OK
    const length = 
        projects.default([]).get().length
}

skip

Sometimes we have a stream with an initial value but we don't want to be notified until we receive a new value. This is when we use skip:

const form = c<InitialState>(init())

const changed = form.skip(1)

changed.map( x => console.log(
    'Form changed', x
))

take / once

We can use .once() if we want only want to receive the first emit.

const firstClick = click.once()

We can use .take(n) if we want the first n emits.

const firstThreeClicks = click.take(3)

In both cases, the stream will end when it reaches the specified number of events.

🤓 If you want only 1 event but not the initial value you can use .skip(1).once()

scan

scan makes it convenient to produce a new value based on the combination of the latest and previous emit. scan is often used as a lightweight alternative to redux or for building reactive state machines.

Here is a simple counter example:

type Action = 'inc' | 'dec'
const action = c<Action>()

const count = 
    action.scan(0, (prev, action) => {
        if (action === 'inc' ) {
            return prev + 1
        } else if (action == 'dec') {
            return prev - 1
        }
        return prev
    })

awaitLatest

Unlike many other stream/observable libraries, Chifley doesn't implicitly coerce promises or arrays into stream events. A Promise is as much a value as a number or a string.

But we do provide two basic operators to make working with promises easier.

awaitLatest allows you to preference the promise resolution that was triggered by the most recent source emit - even if a prior promise resolves first. This is most useful for network requests:

const results = 
    searchInput
    .awaitLatest((search) =>
        fetch(`${endpoint}?q=${search}`)
            .then( x => x.json())
    )

search('hello')

results
    .map( x => console.log(
        ...x.status === 'fulfilled' 
            ? ['🟢', x.value] 
            : ['🚫', x.reason]
    ))

In the above example, the user may constantly be updating the value of searchInput but we are guaranteed to only get the network response corresponding to the latest searchInput.

Because Promise's can reject, we coerce the result to the allSettled API

type Settled = 
    | { status: 'fulfilled'
      , value: T 
      }
    | { status: 'rejected'
      , reason: unknown 
      }

This will help us with our control flow but it won't actually cancel the network request. awaitLatest will simply ignore the response of older requests.

If you'd like to also cancel the unused network requests you can use the abort signal which is conveniently passed in as a second argument.

const results = 
    searchInput
    .awaitLatest((search, { signal }) => 
        fetch(
            `${endpoint}?q=${search}`
            , { signal }
        )
        .then( x => x.json())
    )

awaitEvery

Sometimes we want a corresponding emit for every single promise resolution.

const results = 
    searchInput
    .awaitEvery((search, { signal }) => 
        fetch(
            `${endpoint}?q=${search}`
            , { signal }
        )
        .then( x => x.json())
    )

awaitEvery will emit every time a returned Promise settles. We still provide the ability to cancel on abort signal but this will only fire if the stream is ended before a request can resolve.

Like awaitLatest, the resulting emit is of the Settled<T> type.

throttle

Usually we don't want to fire a network request for every single corresponding user keystroke.

One strategy is to throttle the input, sending at most 1 request every n milliseconds.

const searchInput = c('')

searchInput
    .throttle(100)
    .awaitLatest((search, { signal }) => 
        fetch(
            `${endpoint}?q=${search}`
            , { signal }
        )
    )

afterSilence

Other times we only want to fire off a request when the user stops typing for a bit.

const searchInput = c('')

searchInput
    .afterSilence(100)
    .awaitLatest( ... )

Advanced

Stream States

Each stream is in one of three states: pending | active | ended. You can check the state of a stream via someStream.state

let name = c<string>()

name.state // 'pending'

name.set('Scotty')

name.state // 'active'

name.end()

name.state // 'ended'

Error / Promise Rejection propagation

This library deliberately has no opinions on synchronous error handling. If you throw an error within a stream update, it will simply bubble up the stack uninterrupted. However, for awaitLatest and awaitEvery we return the same type as the standard Promise.allSettled API.

Stream End Behaviour

When you end a stream, the last emitted value will still be cached so .get() will continue to work forever. Any upstream changes will not propagate to an ended stream. The value will stay the same and no operators will fire.

All children streams will also be ended (recursively).

If you end a writable stream and change the value via .set or .update no exception will be thrown and the value will change. If you subscribe to an ended stream via an operator, no exceptions will be thrown but as the upstream dependency will never again emit, neither will the new dependent stream.

When using .merge, if any dependency ends, the merged stream is ended. If another (unended) dependency emits, the merged stream won't emit.

.abortSignal

Every chifley stream has an associated .abortSignal. AbortSignals are a fairly recent Javascript standard for supporting cancellation. You can use it natively with fetch and with addEventListener.

let mousemove$ = Stream<{ x: number, y: number }>()
window.addEventListener('mousemove', e => {
    mousemove$.set({ x: e.clientX, y: e.clientY })
}, { signal: mousemove$.signal })

// later
mousemove$.end()

// now the event listener has automatically ended when the stream ended

Read more about AbortSignal on MDN

Tracking references and creations

This feature is designed to be used primarily by framework authors and not directly by Chifley users in normal application code. It will seem a little verbose at first but it is designed to give the framework author precise control over how tracking contexts cascade.

To track stream references, use the static trackReferenced function:

const f = 
    someFunctionThatMightReferenceStreams

const returnValue = 
    c.trackReferenced(() => {
        return f()
    }, new Set<c>)

Any synchronous calls to .get() while that function runs will result in that stream being added to the provided set Set<c>. You can then inspect the set after executing the function and use it to inject dependencies for some framework process, e.g. automatic rendering when a stream emits.

To track stream creation, you can do the same thing via trackCreated:

const f = 
    someFunctionThatMightCreateStreams

const returnValue = c.trackCreated(() => {
    return f()
}, new Set<c>)

This is useful if you'd like to automatically destroy streams created within a particular scope. The most likely use case would be during a render, or within a component initialization.

To track both creations and references use c.track

const returnValue = 
    c.track(() => {
        return someFunction()
    }, new Set<c>)

You can opt out of a tracking context using the sample function. There are sampleCreated and sampleReferenced variants (which do what you would expect).

const g = weWillNotTrackThisFunctionCall
const returnValue = 
    c.track(() => {
        f()

        return sample(() => {
            return g()
        })
    }, new Set<c>)

You can also untrack a stream that has already been tracked. This removes it from whatever tracking context is currently active.

const returnValue = c.track(() => {
    f()

    // even if `f` referenced 
    // `someStream` 
    // it won't appear in the set.
    untrack(someStream);
}, new Set<c>)

From this feature set, you can build S.js like computations and any kind of conceivable reactive framework feature you'd like to build. If you'd like tracking contexts to cascade, reuse the same set for multiple contexts. If you'd like each tracking context to have sole responsibility then use separate sets. If you'd like to use generator functions then you can start and end a tracking set for each invocation of iterator.next(...) - the sky is the limit.

Comparisons

👶 Simple reactivity model

Chifley is heavily inspired by stream libraries like flyd and mithril-stream. All Chifley streams are immediately active. What does "active" mean? It means there is no extra subscribe() call before your stream transforms start evaluating. If you put a console.log(...) in a call to filter or map or any other operator, you'll see they are immediately invoked.

Additionally, every stream caches the last emitted value, and you can always access it via .get(). Source streams can be written to via .set and .update while dependent streams are readonly and will prevent you from writing to them both at runtime and via Typescript.

When you end a parent stream, all dependent streams will also end.

If you are coming from libraries like Rx.js this may feel a little foreign as you don't typically manually end a source stream. In Rx.js, you would instead unsubscribe from a sink stream and the source will end automatically when there are no more subscribers.

For UI programming having to subscribe to a stream before it becomes active introduces unnecessary constraints. By removing the subscribe step we have to specifically and explicitly end source streams but we think, for UI programming, this is the correct trade off. Ending streams created in a component or view context is not difficult and can be automated via trackCreated and trackReferenced.

Unlike traditional FRP libraries, you are encouraged to write to streams (even while changes are already propagating). You are expected to guard against potential infinite loops yourself but Chifley is designed to make it easy to diagnose issues in your stream graph by using a transactional / atomic clock model: each new write is deferred until the immediate effects of the current write are complete.

🤖 Easy to debug

Most stream libraries have a complex recursive propagation algorithm. If you are trying to debug your stream graph, traditional debuggers aren't very helpful as you will find yourself stepping through generic internal functions recursively.

So, instead, we find ourselves using custom tools like Rx marbles (and more commonly logs 🫠).

Chifley was deliberately designed to be sustainable at scale with large complex UI stream graphs. When you update a stream, Chifley traverses all the possible dependencies in a single while loop.

Additionally, any writes that occur during a propagation are deferred until the immediate impact of the previous write is complete.

This doesn't prevent infinite loops but it makes it very clear when you are entering and exiting a propagation caused by a single write. It also helps identify how many dependent transactions are spawned by a single write.

📉 Low memory use

Chifley streams are instances of a simple class. Each instance has many useful methods available but thanks to prototypical inheritance, the memory for those methods are all shared. Each stream has a few instance specific properties but everything else is shared and stored once.

🏃‍♂️‍➡️ Fast

Chifley isn't the fastest stream library available but it is competitive and strikes a healthy balance between memory / CPU usage and features.

For more information on performance check out the comparisons section here.

🤏 Tiny but with just enough features ⭐

Chifley originally started as a fork of mithril-stream which, in turn, was a rewrite of flyd. Chifley is a formalization of patterns that emerged while using these libraries. We've both added and removed functionality based on actual usage.

flyd was already small to begin with because you don't need to predefine operators for all possible scenarios, as writing to source streams is encouraged.

So, here's a short list of things we have included in this tiny library:

• 💪 14 commonly use stream operations: (map, filter, reject, merge, scan, dropRepeats, throttle, default, skip, take, once, afterSilence, awaitLatest and awaitEvery)
• 🕵️ Automatic dependency tracking (like S.js / Signal libraries)
• 🔒 Read-only streams (both at runtime and via Typescript)
• 💾 toJSON serialization
• 🔥 Sin.js observable protocol
• 🕰️ Predictable atomic clock update (similar to S.js)

Traditional FRP vs Reactive Store

There are many stream libraries in JS, with varying philosophies, feature sets and performance profiles.

Chifley is laser focused on UI state management, nothing more. It is designed to act as a simple reactive store with a few core transforms/combinators. You are encouraged to write to a chifley stream the same way you write to setState in React, or a signal in Solid.js.

But in traditional FRP, this is considered problematic. You will often read advice like:

If you find yourself writing to a Subject you are probably doing things the wrong way

And in some contexts this is true. You really can create a mess very quickly if you don't take the time to learn how to use these tools in the way they were designed to be used.

But part of the benefit of Chifley is you don't need to explicitly think about hot vs cold, MemoryStreams, share/publish etc. It's effectively a state store with a few tricks up its sleeve.

Apples to Apples comparisons are complicated...

Often with traditional FRP libraries. you can really tweak performance if you know what you are doing but its also easy to get awful performance if you are not very careful. It is quite simple to find yourself with duplicated effects (e.g. network requests) because multiple components are referencing the same observable and each subscription duplicates all the transforms.

This will never happen with libraries where all streams are "hot" (xstream, flyd, mithril-stream) but can be avoided with specific operators in "cold" by default libraries (Rx.js, most.js).

xstream is a very interesting case. It has amazing performance and while being Rx inspired, it is deliberately designed to sidestep common confusing aspects. xstream is a great choice if you are writing mostly acyclic stream graphs and you are happy to avoid writing to subject/source streams. It is still very much in the spirit of what most would consider "real FRP". In my benchmarks (which are very tailored towards UI workloads) xstream was frequently the fastest stream library that I tested against. xstream is roughy 30-40% faster than Chifley. While benchmarking we observed that removing the transactional writes brought us in line with xstream's performance, we feel this is a healthy trade off. There may be other changes we can make to catch up.

flyd and mithril-stream both could be considered direct ancestors of Chifley. The programming model is less focused on linear dataflow and raw performance and instead is designed to assist with UI programming. Chifley is roughly 4x faster than flyd, and 18x faster than mithril-stream.

Chifley approaches (and sometimes exceeds) the performance of most.js but only in the specific context of how we benchmark (multiple subscribers, no fromArray, no special caching operators). In a different context most.js is often the leader in performance, especially in server workloads where there is 1 subscriber per request. For those specific use cases I would recommend checking out most.js. Additonally our most benchmarks do not use @most/core. It is likely we'd get very different results then.

Benchmarking considerations

In our benchmarks, we completely avoid fromArray and any kind of operator like share(), remember() or publish(). We avoid fromArray because this is easy to optimize with specialized code paths but it is not a good representation of performance when processing asynchronous events. We avoid operators like share(), remember(), publish() (etc) because having to opt in to this specialized code path requires a level of expertise that isn't representative of most users of a stream library. We want to know how Chifley compares to the sort of code you would actually see in the wild. Equally, Chifley benchmarks are not allowed to use any special operator to improve performance. If we're filtering, mapping and reducing, then that's the only operators we can use in the benchmark.

Additionally, it's rare in a UI for a stream to only have a single subscriber so we will often subscribe to the same pipeline n times to see how performance degrades (this heavily impacts benchmarking scores for cold observable libraries).

filter -> map -> reduce 1000000 integers (10 consumers)
-------------------------------------------------------
xstream              34.83 op/s ±  1.89%   (82 samples)
chifley              21.05 op/s ±  7.83%   (53 samples)
flyd                  4.70 op/s ±  1.26%   (27 samples)
mithril-stream        1.76 op/s ±  3.62%   (13 samples)
most.js               8.46 op/s ±  8.69%   (44 samples)
-------------------------------------------------------

From Github Actions

Note we won't be adding many more libraries to our benchmarks (unless if you are faster than xstream) as our benchmarks are there to gauge where Chifley sits in comparison to its ancestor libraries (flyd, mithril-stream) and the fastest libraries we've benchmarked (xstream, most.js).

Prior Art and Thanks

This library has many influences and is greatly shaped by years of working with FP and FRP libraries in large complex applications.

Many thanks, first of all, to Simon Friis Vindum, this library is greatly influenced by his library flyd.

Much appreciaton to Adam Haile for creating S.js and to Ryan Carniato for popularising it through his work with Solid.js. This library has been inspired by S.js in its atomic update algorithm and with the automatic dependency tracking that we also see in S.computation.

After benchmarking, we greatly revised the internals to mimic some techniques xstream uses to improve performance. Much credit goes to André Staltz, Tyler Steinberger and all the other contributors to xstream for making it so fast.

This library originally started as a port of mithril-stream, and mithril-stream was originally a rewrite of flyd. mithril-stream was first implemented by Leo Horie (Original author of Mithril.js) and later rewritten by Rasmus Porsager. Thanks again goes to Rasmus for the Sin.js observable protocol which this library has adopted.

Finally, special thanks to Barney Carroll and Scotty Simpson for consistently offering helpful feedback.