npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

ipfs-car

v1.2.0

Published

Convert files to content-addressed archives (.car) and back

Downloads

50,196

Readme

ipfs-car 🚘✨⬢

Convert files to content-addressable archives (.car) and back

Build dependencies Status JavaScript Style Guide Downloads Minzipped size

Description

ipfs-car is a library and CLI tool to pack & unpack files from Content Addressable aRchives (CAR) file. A thin wrapper over @ipld/car and unix-fs.

Content-addressable archives store data as blocks (a sequence of bytes) each prefixed with the Content ID (CID) derived from the hash of the data; typically in a file with a .car extension.

Use ipfs-car to pack your files into a .car; a portable, verifiable, IPFS compatible archive.

$ ipfs-car pack path/to/files --output my-files.car

or unpack files from a .car, and verify that every block matches it's CID

$ ipfs-car unpack my-files.car --output path/to/write/to

Fetch and locally verify files from a IPFS gateway over http

curl "https://ipfs.io/ipfs/bafybeidd2gyhagleh47qeg77xqndy2qy3yzn4vkxmk775bg2t5lpuy7pcu?format=car" | ipfs-car unpack -o images

Install

# install it as a dependency
$ npm i ipfs-car

# OR use the cli without installing via `npx`
$ npx ipfs-car --help

Usage

Pack files into a .car

# write a content addressed archive to stdout.
$ ipfs-car pack path/to/file/or/dir
# note: CAR data streamed to stdout will not have roots set in CAR header!

# specify the car file name.
$ ipfs-car pack path/to/files --output path/to/write/a.car

# by default, ipfs-car will wrap files in an IPFS directory.
# use --no-wrap to avoid this.
$ ipfs-car pack path/to/file --no-wrap --output path/to/write/a.car

Unpack files from a .car

# unpack files to a specific path.
$ ipfs-car unpack path/to/my.car --output /path/to/unpack/files/to

# unpack a specific root.
$ ipfs-car unpack path/to/my.car --root <cid1>

# unpack files from a .car on stdin.
$ cat path/to/my.car | ipfs-car unpack

Show the files and directories in a .car

# show the files and directories.
$ ipfs-car ls path/to/my.car

# show the files and directories, their CIDs and byte sizes.
$ ipfs-car ls path/to/my.car --verbose

Show the root CIDs in a .car

# show the CID roots found in the CAR header.
$ ipfs-car roots path/to/my.car

# show the CID roots found implicitly from the blocks in the file.
$ ipfs-car roots --implicit path/to/my.car

Show the block CIDs in a .car

# show the CIDs for all the blocks.
$ ipfs-car blocks path/to/my.car

Get other information about a CAR

# generate CID for a CAR.
$ ipfs-car hash path/to/my.car

API

To pack files into content-addressable archives, you can use the following:

  • createFileEncoderStream a factory function for creating a ReadableStream that encodes a single file into DAG Blocks.
  • createDirectoryEncoderStream a factory function for creating a ReadableStream for encoding a directory of files into DAG Blocks.
  • CAREncoderStream a TransformStream sub-class that you can write Blocks to and read Uint8Array CAR file data from.

To unpack content-addressable archives to files, you should use @ipld/car and ipfs-unixfs-exporter modules.

Examples

Basic single file pack

import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'

const file = new Blob(['Hello ipfs-car!'])
const carStream = createFileEncoderStream(file).pipeThrough(new CAREncoderStream())

// carStream.pipeTo(somewhereWritable)

Directory pack to file system in Node.js

import { Writable } from 'stream'
import { createDirectoryEncoderStream, CAREncoderStream } from 'ipfs-car'
import { filesFromPaths } from 'files-from-path'

const files = await filesFromPaths(process.argv.slice(2))

await createDirectoryEncoderStream(files)
  .pipeThrough(new CAREncoderStream())
  .pipeTo(Writable.toWeb(process.stdout))

Usage: node script.js file0 file1 dir0 > my.car.

Obtaining the root CID

The root CID is the final block generated by the file/directory encoder stream. Use a transform stream to record the CID of the last block generated:

import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'

const file = new Blob(['Hello ipfs-car!'])
let rootCID

await createFileEncoderStream(file)
  .pipeThrough(new TransformStream({
    transform (block, controller) {
      rootCID = block.cid
      controller.enqueue(block)
    }
  }))
  .pipeThrough(new CAREncoderStream())
  .pipeTo(new WritableStream())

console.log(rootCID.toString())

Adding root CIDs to the CAR header

If you need root CIDs in the CAR header, there are two approaches you can use:

  1. Buffer all the DAG blocks, then encode with known root:
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'

const file = new Blob(['Hello ipfs-car!'])
const blocks = []

// buffer the output
await createFileEncoderStream(file)
  .pipeTo(new WritableStream({ write: b => blocks.push(b) }))

const rootCID = blocks.at(-1).cid
const blockStream = new ReadableStream({
  pull (controller) {
    if (blocks.length) {
      controller.enqueue(blocks.shift())
    } else {
      controller.close()
    }
  }
})

await blockStream
  .pipeThrough(new CAREncoderStream([rootCID])) // pass root to CAR encoder
  .pipeTo(new WritableStream())
  1. Write to disk with placeholder CID, then update after DAG is completely generated (Note: Node.js only):
import fs from 'fs'
import { Writable } from 'stream'
import { CarWriter } from '@ipld/car/writer'
import { CID } from 'multiformats/cid'
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'

// Root CID written in CAR file header before it is updated with the real root CID.
const placeholderCID = CID.parse('bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi')

const file = new Blob(['Hello ipfs-car!'])
let rootCID

await createFileEncoderStream(file)
  .pipeThrough(new TransformStream({
    transform (block, controller) {
      rootCID = block.cid
      controller.enqueue(block)
    }
  }))
  .pipeThrough(new CAREncoderStream(placeholderCID))
  .pipeTo(Writable.toWeb(fs.createWriteStream('path/to/my.car')))

// update roots in CAR header
const fd = await fs.promises.open(opts.output, 'r+')
await CarWriter.updateRootsInFile(fd, [rootCID])
await fd.close()

Unpacking files from a CAR

This functionality is not provided by this library, but is easy to do with @ipld/car and ipfs-unixfs-exporter modules:

import { CarIndexedReader } from '@ipld/car/indexed-reader'
import { recursive as exporter } from 'ipfs-unixfs-exporter'

const reader = await CarIndexedReader.fromFile('path/to/my.car')
const roots = await reader.getRoots()

const entries = exporter(roots[0], {
  async get (cid) {
    const block = await reader.get(cid)
    return block.bytes
  }
})

for await (const entry of entries) {
  if (entry.type === 'file' || entry.type === 'raw') {
    console.log('file', entry.path, entry.content)
  } else if (entry.type === 'directory') {
    console.log('directory', entry.path)
  }
}

Contributing

Feel free to join in. All welcome. Open an issue!

License

Dual-licensed under MIT + Apache 2.0