Backup One S3 Bucket’s To Another S3 Bucket

Create an IAM user in AWS

You have to create an IAM user for code Programming AccessAmazoneS3FullAccess. Aws provides an access key and one secrete key saves these keys.

IAM User

User’s Policy(AmazoneS3FullAccess)

Config IAM user in EC2 CLI

For config user in CLI, you should connect ec2 via ssh then install AWS CLI, run some commands. These commands are given below:-

$ sudo apt install awscli
$ aws configure7aRicbzJ16vHQDcmoB3xoK
$ AWS Access Key ID [None]: AKIAQTEMCLRYXXX5FGXQXP
$ AWS Secret Access Key [None]: 4xyXKelNSOpDZXJS32el4xyX32elNSOpDZXJS
$ Default region name [None]: ap-southeast-1
$ Default output format [None]: json

Make task.sh file

task.sh file is a combination of some code that will sync data from local to s3 bucket.

#!/bin/bash
# S3 bucket name
SOURCE_BUCKET=ankit-tesing
DESTINATION_BUCKET=backup-ankit-tesing

# Sync Data from SOURCE to BUCKET
/usr/bin/aws s3 sync s3://$SOURCE_BUCKET/ s3://$DESTINATION_BUCKET/

# All done
echo “Backup completed”
echo “Backup available at https://s3.amazonaws.com/$DESTINATION_BUCKET"

Benefits:

• Scalable — supports huge S3 buckets
• Multi-threaded — syncs the files faster by utilizing multiple threads
• Smart — only syncs new or updated files
• Fast — thanks to its multi-threaded nature and smart sync algorithm

Accidental Deletion:

Conveniently, the sync command won't delete files in the Destination S3 Bucket. If they have missed from the Source S3 Bucket, and vice-versa. This is perfect for backing up S3 Bucket in case files get deleted from Source Bucket, re-syncing it will not delete them Destination S3 Bucket. And in case you delete a Destination S3 Bucket, it won't be deleted from the Source S3 Bucket either.

Setup crontab

Setup schedules a job for sync data from Source S3 to Destination S3 Bucket every day at 1:00 AM, You can use it https://crontab.guru/ for reference for making crontab.

crontab -e
0 1 * * * /home/ubuntu/task.sh

Note:- It will take minimum time(first sync time) every sync.

Note:- For any query, you may comment in this section or message me on Facebook.