npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

ipfs-car-utils

v0.0.3

Published

Convert files to content-addressed archives (.car) and back

Downloads

71

Readme

ipfs-car 🚘✨⬢

Convert files to content-addressable archives (.car) and back

Build dependencies Status JavaScript Style Guide Downloads Minzipped size

Description

ipfs-car is a library and CLI tool to pack & unpack files from Content Addressable aRchives (CAR) file. A thin wrapper over @ipld/car and unix-fs.

Content-addressable archives store data as blocks (a sequence of bytes) each prefixed with the Content ID (CID) derived from the hash of the data; typically in a file with a .car extension.

Use ipfs-car to pack your files into a .car; a portable, verifiable, IPFS compatible archive.

$ ipfs-car --pack path/to/files --output my-files.car

or unpack files from a .car, and verify that every block matches it's CID

$ ipfs-car --unpack my-files.car --output path/to/write/to

Fetch and locally verify files from a IPFS gateway over http

curl -X POST "https://ipfs.io/api/v0/dag/export?arg=bafybeidd2gyhagleh47qeg77xqndy2qy3yzn4vkxmk775bg2t5lpuy7pcu" | 🚘

Install

# install it as a dependency
$ npm i ipfs-car

# or use the cli without installing via `npx`
$ npx ipfs-car --help

Usage

--pack files into a .car

# write a content addressed archive to the current working dir.
$ ipfs-car --pack path/to/file/or/dir

# specify the car file name.
$ ipfs-car --pack path/to/files --output path/to/write/a.car

--unpack files from a .car

# unpack files to a specific path.
$ ipfs-car --unpack path/to/my.car --output /path/to/unpack/files/to

# unpack specific roots
$ ipfs-car --unpack path/to/my.car --root <cid1> [--root <cid2>]

# unpack files from a .car on stdin.
$ cat path/to/my.car | ipfs-car --unpack

List the contents of a .car

# list the files.
$ ipfs-car --list path/to/my.car

# list the cid roots.
$ ipfs-car --list-roots path/to/my.car

# list the cids for all the blocks.
$ ipfs-car --list-cids path/to/my.car

API

To pack files into content-addressable archives, you can use the functions provided in:

  • ipfs-car/pack for consuming a CAR writer async iterable
  • ipfs-car/pack/blob for getting a blob with the CAR file
  • ipfs-car/pack/fs for storing in the local file system (Node.js only)
  • ipfs-car/pack/stream for writing to a writable stream (Node.js only)

⚠️ While packing files into CAR files, a Blockstore is used for temporary storage. All pack functions provide a default, but you can use other options (if supported on your runtime).

To unpack content-addressable archives to files, you can use the functions provided in:

  • ipfs-car/unpack for getting an async iterable of the UnixFS entries stored in the CAR file
  • ipfs-car/unpack/fs for writing the unpacked files to disk (Node.js only)

ipfs-car/pack

Takes an ImportCandidateStream and returns a a CAR writer async iterable.

import { pack } from 'ipfs-car/pack'
import { MemoryBlockStore } from 'ipfs-car/blockstore/memory' // You can also use the `level-blockstore` module

const { root, out } = await pack({
  input: [new Uint8Array([21, 31, 41])],
  blockstore: new MemoryBlockStore(),
  wrapWithDirectory: true // Wraps input into a directory. Defaults to `true`
  maxChunkSize: 262144 // The maximum block size in bytes. Defaults to `262144`. Max safe value is < 1048576 (1MiB)
})

const carParts = []
for await (const part of out) {
  carParts.push(part)
}

ipfs-car/pack/blob

Takes an ImportCandidateStream and writes it to a Blob.

import { packToBlob } from 'ipfs-car/pack/blob'
import { MemoryBlockStore } from 'ipfs-car/blockstore/memory' // You can also use the `level-blockstore` module

const { root, car } = await packToBlob({
  input: [new Uint8Array([21, 31, 41])],
  blockstore: new MemoryBlockStore()
})

ipfs-car/pack/fs

Takes a path on disk and writes it to CAR file (Node.js only).

import { packToFs } from 'ipfs-car/pack/fs'
import { FsBlockStore } from 'ipfs-car/blockstore/fs'

await packToFs({
  input: `${process.cwd()}/path/to/files`,
  output: `${process.cwd()}/output.car`,
  blockstore: new FsBlockStore()
})
// output.car file now exists in process.cwd()

ipfs-car/pack/stream

Takes a writable stream and pipes the CAR Writer stream to it (Node.js only).

import fs from 'fs'
import { packToStream } from 'ipfs-car/pack/stream'
import { FsBlockStore } from 'ipfs-car/blockstore/fs'

const writable = fs.createWriteStream(`${process.cwd()}/output.car`)

await packToStream({
  input: `${process.cwd()}/path/to/files`,
  writable,
  blockstore: new FsBlockStore()
})
// output.car file now exists in process.cwd()

ipfs-car/unpack

Takes a CAR reader and yields files to be consumed.

import fs from 'fs'
import { unpack } from 'ipfs-car/unpack'

const inStream = fs.createReadStream(`${process.cwd()}/output.car`)
const carReader = await CarReader.fromIterable(inStream)

const files = []
for await (const file of unpack(carReader)) {
  // Iterate over files
}

Takes an AsyncIterable and yields files to be consumed.

import fs from 'fs'
import { unpackStream } from 'ipfs-car/unpack'

const inStream = fs.createReadStream(`${process.cwd()}/output.car`)

const files = []
for await (const file of unpackStream(inStream)) {
  // Iterate over files
}

unpackStream takes an options object, allowing you to pass in a BlockStore implementation. The blocks are unpacked from the stream in the order they appear, which may not be the order needed to reassemble them into the Files and Directories they represent. The blockstore is used to store the blocks as they are consumed from the stream. Once the stream is consumed, the blockstore provides the random access by CID to the blocks, needed to assemble the tree.

The default is a MemoryBlockStore, that will store all the blocks in memory. For larger CARs in the browser you can use IndexedDB by passing in an IdbBlocksStore, and in Node.js you can provide a [FsBlockStore] instance to write blocks to the tmp dir.

/* browser */
import { unpackStream } from 'ipfs-car/unpack'
import { IdbBlockStore } from 'ipfs-car/blockstore/idb'

const res = fetch(
  'https://ipfs.io/api/v0/dag/export?arg=bafkreigh2akiscaildcqabsyg3dfr6chu3fgpregiymsck7e7aqa4s52zy',
  { method: 'POST' }
)
const files = []
const blockstore = new IdbBlockStore()
for await (const file of unpackStream(res.body, { blockstore })) {
  // Iterate over files
}
blockstore.destroy()

When providing a custom Blockstore, it is your responsibiltiy to call blockstore.destroy() when you're finished. Failing to do so will fill up the users storage.

ipfs-car/unpack/fs

Takes a path to a CAR file on disk and unpacks it to a given path

import { unpackToFs } from 'ipfs-car/unpack/fs'

await unpackToFs({
  input: `${process.cwd()}/my.car`,
  output: `${process.cwd()}/foo`
})
// foo now exists in process.cwd()
// it is either a file or a directory depending on the contents of the .car

Takes a stream to a CAR file and unpacks it to a given path on disc

import fs from 'fs'
import { unpackStreamToFs } from 'ipfs-car/unpack/fs'

const input = fs.createReadStream(`${process.cwd()}/my.car`)

await unpackStreamToFs({
  input,
  output: `${process.cwd()}/foo`
})
// foo now exists in process.cwd()
// it is either a file or a directory depending on the contents of the .car