google-drive-to-s3-sync
v1.0.0
Published
Google Drive to Amazon S3 Sync
Downloads
4
Readme
Overview
This is a Node.js package that simplifies the process of transferring files from Google Drive to Amazon S3. This package utilizes AWS S3 and Google Drive APIs to securely and efficiently sync files from a specified Google Drive folder to a designated bucket on Amazon S3.
Features
- Sync files from Google Drive to Amazon S3.
- Secure access with AWS IAM user credentials.
- Authenticate using Google Drive service account.
- Effortless file transfer between Google Drive and Amazon S3.
System Requirements
Before using the package, make sure you have the following:
AWS Account and IAM User:
- Create an IAM user in your AWS account with S3FULLACCESS permissions.
- Create an S3 bucket where the Google Drive files will be synced. Google Cloud Platform Account:
Create a project on Google Cloud Platform (GCP).
- Generate a service account for the project and download the credentials as a JSON file.
- Enable the Google Drive API for the project in the GCP dashboard.
Google Drive Setup:
- Identify the folder in your Google Drive that you want to sync with Amazon S3.
- Share the folder with the client_email present in the service account credentials JSON.
Node.js and NPM:
- Ensure you have Node.js >=18.17.0 and NPM >=9.6.7 (Node Package Manager) installed on your system.
Installation
Install the package via NPM:
npm install google-drive-to-s3-sync
Configuration
Before using the package, set up the configuration:
- Copy the downloaded service account credentials json to project folder and rename it as 'service_account_key.json'
- Set GOOGLE_DRIVE_FOLDER_ID to the folder ID of the Google Drive folder you want to sync.
- Set S3_BUCKET_NAME to the name of your AWS S3 bucket.
- Set S3_BUCKET_PREFIX if you want to specify.
- Set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION to access the s3 bucket and sync.
Usage
Use the package in your Node.js application:
const GoogleDriveToS3 = require('google-drive-to-s3-sync');
const googleServiceAccountCredentials = require('./service_account_key.json');
const googleDriveFolderId = 'GOOGLE_DRIVE_FOLDER_ID';
const s3BucketName = 'S3_BUCKET_NAME';
const s3BucketPrefix = 'S3_BUCKET_PREFIX';
const awsAccessKeyId = 'AWS_ACCESS_KEY_ID';
const awsSecretAccessKey = 'AWS_SECRET_ACCESS_KEY';
const awsRegion = 'AWS_REGION';
try {
const credentials = {
aws: {
accessKeyId: awsAccessKeyId,
region: awsRegion,
secretAccessKey: awsSecretAccessKey,
},
google: googleServiceAccountCredentials,
};
const migration = new GoogleDriveToS3(
credentials,
googleDriveFolderId,
s3BucketName,
s3BucketPrefix
);
migration.startMigration();
} catch (error) {
console.error(error);
}
The package will start syncing files from the specified Google Drive folder to the Amazon S3 bucket.
Note: Ensure that you have proper network connectivity, and both your AWS and GCP credentials are correctly configured before using the package.
License
This project is licensed under the MIT License