#+title: flow-degen README #+author: Logan Barnett #+email: [email protected] #+date: <2018-05-08 Tue> #+language: en #+file_tags: readme flow deserialization

• flow-degen

This is a deserialization generator for JavaScript objects that are under [[http://flow.org][Flow]]. Some deserializer/validator projects use =$ObjMap= and some clever casting to achieve a type safe means of deserialization/validation. =flow-degen= aims to leverage Flow itself in ensuring type safety by generating all of that ugly deserialization/validation code you would have written by hand.

Pros:

1. =flow-degen= introduces no runtime dependencies to consumers, other than itself.
2. =flow-degen= emits generators that use plain Flow type checking. There is no =any= casting internally, and there are no magic types.

Cons:

1. =flow-degen= does not currently provide a list of deserializer errors, but instead bails on the first error.
2. There is potentially a memory/storage footprint concern for non-trivial sizes and amounts of deserializers. A minifier may significantly mitigate this con. Using =degenRefiner= to reference other refiners can also reduce the footprint by calling other refiners instead of duplicating refinement logic in multiple places.

** installation

#+begin_src sh yarn add -E -D flow-degen #+end_src

** config structure

You'll need a config file to work as inputs to =flow-degen=. It has the structure below. Any paths or file names that start with =./= are intended to show a relative directory.

#+begin_src json { "baseDir": "", "generatedPreamble": "", "generators": [ { "exports": { "fooGenerator": "fooRefiner" }, "inputFile": "./dir/generator-input-file.js", "outputFile": "deserializer-output-file.js" } ], "importLocations": { "importName": "./dir/runtime.js" }, "typeLocations": { "TypeName": "./dir/type.js" } } #+end_src *** baseDir =baseDir= is the directory that the generated file will be assumed to be living out of relative to the imports. *** generatedPreamble =generatedPreamble= is the code or text you want to appear at the top of the file. You can use this to insert a copyright comment block, include linter rules (such as disabling ESLint's [[https://eslint.org/docs/rules/no-unused-expressions][no-used-expressions]] rule which can be an issue for disjoint unions). *** generators This is a list of generators and how they produce refiners. Generators here have the following structure:

#+begin_src json
  {
    "exports": {
      "fooGenerator": "fooRefiner",
      "barGenerator": "barRefiner",
      "bazGenerator": "bazRefiner"
    },
    "inputFile": "foo-generator.js",
    "outputFile": "foo-refiner.js"
  }
#+end_src

**** exports =exports= is a mapping of identifiers exported from the generator file that =flow-degen= can find, and it maps to identifiers it will generate as refiners for that associated generator. In the sample configuration, =fooGenerator= is found in =foo-generator.js=, and it will emit a =fooRefiner= to =foo-refiner.js= that you can then =import= or =require= to use.

 =exports= is also implicitly added to =importLocations= such that your
 refiners can refer to each other, and even achieve recursive calls if your
 structure requires recursion.

*** importLocations This is a mapping of import names (which must be valid JavaScript identifiers) to files. The identifiers must map to =export= entities inside of your module. These will be hoisted to the top of the generated deserializer files if they are used. Including entries in here does not mean it will be used in your files, it is simply a lookup for =flow-degen= to use. *** typeLocations Just like =importLocations=, =typeLocations= is a reference for =export type= identifiers so the generated deserializer can find them if any of the combined deserializers use it. ** usage *** internal deserializer generators

=import= from =flow-degen= to get deserialization generators.

**** degenField =degenField= is meant to be used in conjunction with =degenObject=. **** degenFilePath This is just an alias for =degenString= currently, but could one day encompass a Flow opaque type that, while represented by a string, is ensured to be a valid file path. **** degenList Requires a deserializer to be used for the element type, which is provided as its only argument. This will produce an =Array=.

 Suppose we have a =foos-generator.js=:

 #+begin_src js
   import { degenList, degenNumber} from 'flow-degen'

   const numberType = { name: 'number', typeParams: [] }
   export const foosGenerator = () => degenList(numberType, degenNumber())
 #+end_src

 Upon importing the emitted file, you can now refine into an =Array= of
 =number=:

 #+begin_src js
   import { deFoos } from './foos-refiner.js'

   deFoos([1, 2, 3]) // Produces [1, 2, 3].
   deFoos('farsnaggle') // Produces Error object.

   declare var someInput: mixed

   const eitherResult = deFoos(someInput)

   if(eitherResult instanceof Error) {
     // Here the result did not refine correctly.
     console.error('How did this happen?', eitherResult)
   } else {
     // Now you have an Array of number.
     console.log(eitherResult.map(x => x + 1))
   }
 #+end_src

**** degenMapping A "mapping" is of the type ={[A]: B}= although usually it will be ={[string]: mixed}=. It takes the key meta type, the value meta type, a key deserializer, and a value deserializer for =A= and =B= respectively. **** degenMaybe The =degenMaybe= generator is for creating refiners for maybe types (e.g. type Foo = ?string). The maybe type will still require additional refinement after passing through the refiner. For example, given the type:

 #+begin_src js
   export type Foo = {
     bar: ?string,
   }
 #+end_src

 And generator:
 #+begin_src js
   import { degenObject, degenField, degenMaybe, degenString } from 'flow-degen'

   const fooType = { name: 'Foo' }
   const stringType = { name: 'string' }
   export const fooGenerator = () => degenObject(fooType, [
     degenField('bar', degenMaybe(stringType, degenString())),
   ])
 #+end_src

 The refiner would be used like so:
 #+begin_src js
   import { deFoo } from './foo-refiner.js'

   declare var someInput: mixed

   const eitherResult = deFoo(someInput)

   if(eitherResult instanceof Error) {
     // Here the result did not refine correctly.
     console.error('How did this happen?', eitherResult)
   } else {
     // We have a foo, but bar may not have been present
     if (eitherResult.bar != null) {
       console.log(eitherResult.bar + ' was refined')
     } else {
       console.log('result had a null bar')
     }
   }
 #+end_src

**** degenNumber The =degenNumber= deserializer simply deserializes a value as a =number=. **** degenObject An "Object" can be thought of as a collection of "fields". See =degenField= as these go together except for empty objects. =degenObject= takes the type of the object and a list of required fields that =degenField= can emit, and a second list of =degenField= results that represent the optional fields.

 Assume the object =Cat=.

 #+begin_src js

   export type Cat = {
     // Cats always have demands.
     demands: number,
     // Cats can have no love sometimes.
     love?: number,
   }

   const catType = { name: 'Cat' }
   const catGenerator = () => degenObject(catType, [
     degenField('demands', degenNumber()),
   ], [
     degenField('love', degenNumber()),
   ])
 #+end_src

 It is well known that cats always have =demands= but only sometimes have
 =love=. It is fallacious to assume =love= will always be present.

 #+begin_src js
   import { catRefiner } from './cat-refiner.js'

   // It's pretty easy to get an unsanitized cat from anywhere, really.
   handleUnsanitizedCat((input) => {
     const catOrError: string | Error = catRefiner(input)
     if(catOrError instanceof Error) {
       goGetADog()
     } else {
       // We have a cat! But we can't expect love.
       // Flow will also settle for a null check for love.
       if(catOrError.hasOwnProperty('love')) {
         console.log(`My cat loves me ${catOrError.love} love units!`)
       } else {
         console.log('My cat does not have any love for me at all...')
       }
     }
   })
 #+end_src

**** degenString The =degenString= deserializer simply deserializes a value as a =string=.

 Say we have a =name-generator.js=:
 #+begin_src js
   import { degenString } from 'flow-degen'
   export const nameGenerator = () => degenString()
 #+end_src

 And this is configured to produce a =name-refiner.js=, this is how it would
 be used:
 #+begin_src js
   import { nameRefiner } from './name-refiner.js'

   // This could be an HTTP POST handler on a server, or a form handler on a UI
   handleUnsanitizedInput((input) => {
     const nameOrError: string | Error = nameRefiner(input)
     if(nameOrError instanceof Error) {
       console.error(nameOrError)
     } else {
       // Here can we use the name.
       storeName(nameOrError)
     }
   })
 #+end_src

**** degenSentinelValue This deserializer is to be used in conjunction with =degenSum= to produce deserializers for a sum type. This represents one member of the union. It needs a =key=, which is a string value for the sentinel value, and the object deserializer itself, which will likely be =degenObject=. **** degenSum The =degenSum= deserializer handles sum type objects. It takes the type of the union, the sentinel field name, the sentinel field type, and a list of sentinel object deserializers (which can just come from =degenObject=) from =degenSentinelValue=. **** degenValue The =degenValue= deserializer takes a =type= (as a string) and a =value= (which could be anything). It checks for the literal equivalence of that value. This can be helpful when using Flow's sentinel properties for sum types of objects. **** degenRefiner The =degenRefiner= refiner simply imports a symbol for use. This allows recursion to work when the refined data structure is recursive. Also it allows for reuse of other refiners of any kind. This reduces the size of generated refiners significantly. Otherwise the refiners are inlined.

 Suppose we have a =foo-generator.js= whose generator builds the =deFoo=
 refiner:
 #+begin_src js
   import { degenObject, degenField, degenString } from 'flow-degen'

   const fooType = { name: 'Foo' }
   export const fooGenerator = () => degenObject(fooType, [
     degenField('first', degenString()),
     degenField('last', degenString()),
   ])
 #+end_src

 And we have a =bar-generator.js=:
 #+begin_src js
   import {
     degenObject,
     degenField,
     degenString,
     degenRefiner,
   } from 'flow-degen'

   // This is the same fooType in foo-generator.js, and could be imported.
   const fooType = { name: 'Foo' }
   const barType = { name: 'Bar' }
   export const fooGenerator = () => degenObject(barType, [
     degenField('foo', degenRefiner(fooType, 'deFoo')),
   ])
 #+end_src

 Here the generator will simply invoke =deFoo= to refine the =foo= field.
 Any import, type, and hoist information will be made available in this
 refiner.

 Note that this symbol must be one that is managed by =flow-degen= in your
 configuration file, or your configuration file must specify in the
 =imports= how to find this symbol.

*** creating meta types

Objects of type =MetaType= are passed into many generator functions and
contain information =flow-degen= uses to build imports and type signatures
in the generated code. The =MetaType= type can be found in
=src/generator.js= but at a minimum contains the type name:

#+begin_src js
type Foo = {
  bar: string,
}

const fooType = { name: 'Foo' }
#+end_src

In the case of generic types, the optional =typeParams= field in =MetaType=
can be used to list the meta types to be specified in the type signature:

#+begin_src js
type Foo<K: string, V: string> = {
  [K]: V,
}

const stringType = { name: 'string' }
const fooType = { name: 'Foo', typeParams: [stringType, stringType]}
#+end_src

Some types (for example flow utility types like =$PropertyType=) take
literal strings or numbers instead of a type. The =MetaType= has an optional
=literal= boolean to indicate these usages:

#+begin_src js
type Foo = {
  bar: {
    baz: string,
  },
}

const fooType = { name: 'Foo' }
const bazPropertyType = { name: "'baz'", literal: true }
const barType = { name: '$PropertyType', typeParams: [fooType, bazPropertyType] }
#+end_src

Note that string literals (and other literals with delimiters) need to
include the delimiters in the name (e.g. "'baz'" instead of "baz" or 'baz').

*** building custom deserializer generators All deserializers must satisfy the following contract:

+ They must be a function.
+ The function returns a =DeserializerGenerator<CustomType: string,
  CustomImport: string>=, which is a tuple of a function that returns a
  =string= (the code) and a =CodeGenDep<CustomType: string, CustomImport:
  string>=. The exacts of these types can be found in =./src/generator.js=.
+ The code returned by the function must accept a =mixed= as a parameter.
  This is your input provided from your mystery variable. It is assumed to
  be "deserialized" already in the sense that it is not a string of JSON but
  perhaps the result of =JSON.parse=.
+ If any imports are used, they must be enumerated in the =imports= list of
  the =CodeGenDep=. Any imports used by the generated function will also
  need to be part of the =CustomImport= type parameter of the generator as
  well as included in =importLocations= in your =flow-degen=
  configuration file (adding an import to =importLocations= is not necessary
  if the import is an export from a refiner defined in your =flow-degen=
  config).
+ If any type imports are used, they must be enumerated in the =types= list
  of the =CodeGenDep=. Any types used by the generated function will also
  need to be part of the =CustomType= type parameter of the generator as
  well as included in =typeLocations= in your =flow-degen= configuration
  file.
+ Consider that your generated code could likely be embedded deep within a
  function chain. If you need some "root" access to the module to declare
  things such as throw-away types, use the =hoists= list to place code.
+ If your generator delegates to other generators (such as =degenList=
  delegating to a deserializer for the elements), you must honor the results
  of its =CodeGenDep= when you call the generator. This could mean merging
  the =CodeGenDep= with your own. The =mergeDeps= function in
  =./src/generator.js= does this for you. It is found by =flow-degen=
  consumers as a top-level export (=import { mergeDeps } from
  'flow-degen'=).
+ Try testing your refiner with an opaque type. This seems to be a good way
  to ensure Flow cannot run into issues with type inferencing. We suspect
  this is a good test because opaque types can never be inferred, and
  therefore will always need explicit types at the call site of a refiner.

Let's create an custom generator example where we have an uppercase string.

#+begin_src js
  import {
    degenString,
    mergeDeps,
    type DeserializerGenerator,
  } from 'flow-degen'
  import {
    type UppercaseString,
    uppercase,
  } from './my-string-utils.js'

  type UppercaseGeneratorType =
    | 'UppercaseString'

  type UppercaseGeneratorImport =
    | 'uppercase'

  export const degenUppercaseString = (
  ): DeserializerGenerator<UppercaseGeneratorType, UppercaseGeneratorImport> => {
    const [ stringGenerator, stringDeps ] = degenString()
    return [
      () => {
        return `(x: mixed): UppercaseString => {
           return uppercase(${stringGenerator()})
        }`
      },
      mergeDeps(
        stringDeps,
        {
          hoists: [],
          imports: [ 'uppercase' ],
          types: [ { name: 'UppercaseString' } ],
        },
      ),
    ]
  }
#+end_src

Custom generators are no different from the built-in generators.

#+begin_src js
  import {
    degenUppercaseString,
  } from './custom-degens.js'

  export const generateUppercaseStringRefiner = () => degenUppercaseString()
#+end_src

The built-in generators in =src/generators.js= can be used as more complex
examples for building your own generators.

*** command line Once installed, you can use the =flow-degen= script to generate your deserializers:

#+begin_src sh yarn flow-degen degen-config.json #+end_src

*** consuming generated deserializers

The output files you indicate will export refiner functions defined in the =exports= config for the generator. The refiner functions take the form of =(mixed) => T | Error=.

#+begin_src javascript import fs from 'fs' import { fooDeserializer } from './foo.deserializer.js'

const unvalidatedFoo = JSON.parse(fs.readFileSync('foo.json', 'utf8')) const fooOrError = fooDeserializer(unvalidatedFoo)

// Refine the result. if(fooOrError instanceof Error) { console.error('Error deserializing foo:', fooOrError) } else { doStuffWithFoo(fooOrError) } #+end_src

*** editing generated deserializers Do not edit these files directly except for debugging purposes. The files will be overwritten on subsequent runs of the generator. Also, the code written there is not designed with human maintainability as its chief concern.

*** source control Tooling could be built to make the generation process opaque to a consumer, but at the time that method is not known to =flow-degen= maintainers. It is fine and even recommended to check your generated deserializers into source control.

** known issues *** no-unused-expressions When using =degenSum=, ESLint has a [[https://eslint.org/docs/rules/no-unused-expressions][no-unused-expressions]] rule that fails during a cast in the =default= case. This expression doesn't do anything in the runtime, but Flow needs it to tie the "everything else" match to the =default= case. This makes Flow flag an error when a member of the union isn't enumerated in the =switch=. To work around this issue, you can add =// eslint-disable no-unused-expressions= to your configuration's =generatedPreamble=. ** bragging rights

The config object above is generated from =config-generator.js= which in turn must deserialize itself in order to build the generator. =mind-blown.gif=