dep-copy-file

Blazing-fast file and directory copy for Node.js — constant memory, maximum throughput.

Features

• ⚡ Constant memory — sequential DFS tree walk keeps heap bounded regardless of tree size (tested: 182k files, 34k dirs → 73 MB peak)
• 🔒 No OOM risk — works with millions of files without exploding memory
• ✅ Zero dependencies — pure Node.js, no native modules
• 🔗 Symlink aware — detects and re-creates symlinks correctly
• 📁 Empty directory safe — empty directories are preserved
• 🛡️ Error resilient — one bad file won't cancel the whole operation
• 🎯 Filter support — skip files/directories with a callback
• 🔄 Dual CJS/ESM — works with both require() and import
• TypeScript types included

Install

npm install dep-copy-file --save

Quick start

const copy = require('dep-copy-file');
// or: import copy from 'dep-copy-file';

// Copy a single file
await copy('./src/config.json', './dist/config.json');

// Copy an entire directory tree
await copy('./source', './backup');

// With options
const result = await copy('./source', './dest', {
  concurrency: 64,                  // max parallel copies (default: cpu × 2)
  overwrite: false,                 // skip existing files
  filter: (srcPath) => !srcPath.includes('node_modules'),
  onProgress: (stats) => console.log(`${stats.completed}/${stats.total}`),
});

console.log(result);
// {
//   files: 1500,
//   dirs: 42,
//   symlinks: 3,
//   succeeded: 1503,
//   failed: 0,
//   errors: []
// }

API

copy(source, dest [, options]) → Promise<CopyResult>

| Option | Type | Default | Description | |--------|------|---------|-------------| | concurrency | number \| 'auto' | cpu × 2 (min 4) | Max parallel file/symlink copies. Pass 'auto' for cpu × 4 (max 128). Lower values reduce memory; higher values may increase throughput on fast SSDs. | | overwrite | boolean | true | Allow overwriting existing files | | filter | (srcPath: string, entry: Dirent) => boolean | — | Skip entries when returning false | | excludeSymlinks | boolean | false | Skip symlinks entirely | | onProgress | (stats: CopyProgress) => void | — | Called after each file copy completes |

CopyResult

interface CopyResult {
  files: number;        // total files discovered
  dirs: number;         // total directories created (including root)
  symlinks: number;     // total symlinks discovered
  succeeded: number;    // items copied successfully (files + symlinks)
  failed: number;       // items that failed
  errors: CopyError[];  // detailed failure info
}

Architecture

1. Sequential DFS tree walk — directories are processed one at a time, depth-first. Only a single readdir result is in memory at any moment, keeping heap usage constant regardless of tree size.

2. Parallel file/symlink I/O — copies run through a concurrency pool (default cpu × 2 in-flight). The pool is shared by both file copies and symlink creation, so the system is never overwhelmed.

3. No deferred accumulation — file copies start immediately as entries are discovered. No arrays of [src, dst] tuples are ever accumulated (those are the #1 cause of OOM on large trees).

4. macOS APFS systems get COPYFILE_FICLONE — copy-on-write clones that are near-instant regardless of file size.

5. mkdir deduplication — dest directories are created with { recursive: true }, eliminating redundant syscalls and race conditions.

Why sequential directory walking?

Parallel directory walking sounds faster, but readdir is microsecond-fast on modern file systems. The overhead of coordinating thousands of concurrent readdir calls — and holding all their results in memory — far outweighs any perceived speed gain. A sequential DFS walk keeps memory flat while letting the real bottleneck (file copy I/O) run at full parallelism.

Migration from v1.x

The default export (copyDep) still works identically — just await the result:

- copy(source, dest).then(() => { ... }).catch(...)
+ const result = await copy(source, dest)

The return value is now a CopyResult object instead of void, but this is backward-compatible.