npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@push.rocks/smarts3

v2.2.5

Published

A Node.js TypeScript package to create a local S3 endpoint for simulating AWS S3 operations using mapped local directories for development and testing purposes.

Downloads

372

Readme

# @push.rocks/smarts3

A Node.js TypeScript package to create a local S3 endpoint for development and testing using mapped local directories, simulating AWS S3.

## Install

To integrate `@push.rocks/smarts3` with your project, you need to install it via npm. Execute the following command within your project's root directory:

```sh
npm install @push.rocks/smarts3 --save
```

This command will add @push.rocks/smarts3 as a dependency in your project's package.json file and download the package into the node_modules directory.

Usage

Overview

The @push.rocks/smarts3 module allows users to create a mock S3 endpoint that maps to a local directory using s3rver. This simulation of AWS S3 operations facilitates development and testing by enabling file uploads, bucket creation, and other interactions locally. This local setup is ideal for developers looking to test cloud file storage operations without requiring access to a real AWS S3 instance.

In this comprehensive guide, we will explore setting up a local S3 server, performing operations like creating buckets and uploading files, and how to effectively integrate this into your development workflow.

Setting Up the Environment

To begin any operations, your environment must be configured correctly. Here’s a simple setup procedure:

import * as path from 'path';
import { promises as fs } from 'fs';

async function setupEnvironment() {
  const packageDir = path.resolve();
  const nogitDir = path.join(packageDir, './.nogit');
  const bucketsDir = path.join(nogitDir, 'bucketsDir');

  try {
    await fs.mkdir(bucketsDir, { recursive: true });
  } catch (error) {
    console.error('Failed to create buckets directory!', error);
    throw error;
  }

  console.log('Environment setup complete.');
}

setupEnvironment().catch(console.error);

This script sets up a directory structure required for the smarts3 server, ensuring that the directories needed for bucket storage exist before starting the server.

Starting the S3 Server

Once your environment is set up, start an instance of the smarts3 server. This acts as your local mock S3 endpoint:

import { Smarts3 } from '@push.rocks/smarts3';

async function startServer() {
  const smarts3Instance = await Smarts3.createAndStart({
    port: 3000,
    cleanSlate: true,
  });

  console.log('S3 server is up and running at http://localhost:3000');
  return smarts3Instance;
}

startServer().catch(console.error);

Parameters:

  • Port: Specify the port for the local S3 server. Defaults to 3000.
  • CleanSlate: If true, clears the storage directory each time the server starts, providing a fresh test state.

Creating and Managing Buckets

With your server running, create buckets for storing files. A bucket in S3 acts similarly to a root directory.

async function createBucket(smarts3Instance: Smarts3, bucketName: string) {
  const bucket = await smarts3Instance.createBucket(bucketName);
  console.log(`Bucket created: ${bucket.name}`);
}

startServer()
  .then((smarts3Instance) => createBucket(smarts3Instance, 'my-awesome-bucket'))
  .catch(console.error);

Uploading and Managing Files

Uploading files to a bucket uses the SmartBucket module, part of the @push.rocks/smartbucket ecosystem:

import { SmartBucket } from '@push.rocks/smartbucket';

async function uploadFile(
  smarts3Instance: Smarts3,
  bucketName: string,
  filePath: string,
  fileContent: string,
) {
  const s3Descriptor = await smarts3Instance.getS3Descriptor();
  const smartbucketInstance = new SmartBucket(s3Descriptor);
  const bucket = await smartbucketInstance.getBucket(bucketName);

  await bucket.getBaseDirectory().fastStore(filePath, fileContent);
  console.log(`File "${filePath}" uploaded successfully to bucket "${bucketName}".`);
}

startServer()
  .then(async (smarts3Instance) => {
    await createBucket(smarts3Instance, 'my-awesome-bucket');
    await uploadFile(smarts3Instance, 'my-awesome-bucket', 'hello.txt', 'Hello, world!');
  })
  .catch(console.error);

Listing Files in a Bucket

Listing files within a bucket allows you to manage its contents conveniently:

async function listFiles(smarts3Instance: Smarts3, bucketName: string) {
  const s3Descriptor = await smarts3Instance.getS3Descriptor();
  const smartbucketInstance = new SmartBucket(s3Descriptor);
  const bucket = await smartbucketInstance.getBucket(bucketName);

  const baseDirectory = await bucket.getBaseDirectory();
  const files = await baseDirectory.listFiles();

  console.log(`Files in bucket "${bucketName}":`, files);
}

startServer()
  .then(async (smarts3Instance) => {
    await createBucket(smarts3Instance, 'my-awesome-bucket');
    await listFiles(smarts3Instance, 'my-awesome-bucket');
  })
  .catch(console.error);

Deleting a File

Managing storage efficiently involves deleting files when necessary:

async function deleteFile(smarts3Instance: Smarts3, bucketName: string, filePath: string) {
  const s3Descriptor = await smarts3Instance.getS3Descriptor();
  const smartbucketInstance = new SmartBucket(s3Descriptor);
  const bucket = await smartbucketInstance.getBucket(bucketName);

  await bucket.getBaseDirectory().fastDelete(filePath);
  console.log(`File "${filePath}" deleted from bucket "${bucketName}".`);
}

startServer()
  .then(async (smarts3Instance) => {
    await createBucket(smarts3Instance, 'my-awesome-bucket');
    await deleteFile(smarts3Instance, 'my-awesome-bucket', 'hello.txt');
  })
  .catch(console.error);

Scenario Integrations

Development and Testing

  1. Feature Development: Use @push.rocks/smarts3 to simulate file upload endpoints, ensuring your application handles file operations correctly before going live.

  2. Continuous Integration/Continuous Deployment (CI/CD): Integrate with CI/CD pipelines to automatically test file interactions.

  3. Data Migration Testing: Simulate data migrations between buckets to perfect processes before implementation on actual S3.

  4. Onboarding New Developers: Offer new team members hands-on practice with mock setups to improve their understanding without real-world consequences.

Stopping the Server

Safely shutting down the server when tasks are complete ensures system resources are managed well:

async function stopServer(smarts3Instance: Smarts3) {
  await smarts3Instance.stop();
  console.log('S3 server has been stopped.');
}

startServer()
  .then(async (smarts3Instance) => {
    await createBucket(smarts3Instance, 'my-awesome-bucket');
    await stopServer(smarts3Instance);
  })
  .catch(console.error);

In this guide, we walked through setting up and fully utilizing the @push.rocks/smarts3 package for local development and testing. The package simulates AWS S3 operations, reducing dependency on remote services and allowing efficient development iteration cycles. By implementing the practices and scripts shared here, you can ensure a seamless and productive development experience using the local S3 simulation capabilities of @push.rocks/smarts3.


## License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.

**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

### Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

### Company Information

Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany

For any legal inquiries or if you require further information, please contact us via email at [email protected].

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.