fetch-md

A CLI tool for fetching web pages and converting them to Markdown format while preserving images.

Features

• 🚀 Fast and efficient web page fetching
• 📝 Clean Markdown conversion with proper formatting
• 🖼️ Automatically downloads and saves images
• 🎨 Supports background images (optional)
• 📊 Progress bars and status indicators
• 📥 Pipe support for batch processing
• 🔄 Self-update capability
• 🔍 Debug mode for troubleshooting

Installation

# Install globally with npm
npm install -g fetch-md

# Or install with bun if you prefer
bun install -g fetch-md

# Verify installation
fetmd --version

# Check for updates
fetmd update

After installation, the fetmd command will be available globally in your terminal.

Usage

Basic Usage

# Output markdown to terminal
fetmd https://example.com

# Save to a directory (with images)
fetmd https://example.com output

# Save to a specific path
fetmd https://example.com ~/Documents/notes

# Only output the markdown file path
fetmd https://example.com -p

Advanced Options

# Include background images
fetmd https://example.com -b

# Quiet mode (only errors)
fetmd https://example.com -q

# Debug mode for troubleshooting
fetmd https://example.com -d

# Wait for a specific element
fetmd https://example.com -s "#content"

# Wait before processing
fetmd https://example.com -w 2000

Batch Processing

# Process multiple URLs from a file (output to terminal)
cat urls.txt | fetmd

# Process multiple URLs and save to directory
cat urls.txt | fetmd output

# Process multiple URLs directly
echo "https://example.com" | fetmd
echo "https://github.com" | fetmd output

Command Line Options

| Option | Description | |--------|-------------| | [url] | URL to fetch (optional if using pipe) | | [output] | Output directory (optional) | | -b, --background | Include background images | | -q, --quiet | Quiet mode - only show errors | | -d, --debug | Enable debug output | | -p, --path-only | Only output the markdown file path | | -w, --wait <ms> | Wait time in milliseconds before processing | | -s, --selector <selector> | Wait for a specific CSS selector | | -h, --help | Show help information | | -V, --version | Show version number | | update | Check for and install updates |

Output Structure

When saving to a directory, the tool creates the following structure:

output/
├── [title]/
│   ├── [title].md    # Markdown content
│   └── images/       # Downloaded images
│       ├── image1.jpg
│       ├── image2.png
│       └── ...

Examples

1. Simple webpage to markdown:

   fetmd https://example.com

2. Save blog post with images:

   fetmd https://blog.example.com/post-1 blog-posts

3. Save with background images:

   fetmd https://example.com output -b

4. Get only the output file path:

   fetmd https://example.com -p
   # Output: /path/to/output/example-domain/example-domain.md

5. Debug mode with path output:

   fetmd https://example.com -p -d

6. Wait for specific content:

   fetmd https://example.com -w 2000 -s "#main-content"

7. Batch process multiple URLs:

   # Create a file with URLs
   echo "https://example.com

https://github.com" > urls.txt

Process all URLs and save to directory

cat urls.txt | fetmd output

## Requirements

- Node.js 14.17.0 or higher
- Supported platforms:
- macOS
- Linux
- Windows (via WSL)

## License

MIT