How to Mount S3 Bucket on Amazon EC2

In this tutorial we can check how to mount S3 bucket on your server.

S3FS is a FUSE (File System in User Space) will mount Amazon S3 as a local file system. S3FS has an ability to manipulate Amazon S3 bucket in many useful ways. If you wish to access your Amazon S3 bucket without mounting it on your server, you can use s3cmd command line utility to manage S3 bucket.

What is an Amazon S3 bucket?

Amazon S3 is a cloud based web service interface that you can used to store and retrieve any amount of data. To upload your data, first you need to create an S3 bucket in one of the Amazon regions.

Creating a Bucket

S3 provides an API for creating and managing buckets. You can create a maximum of 100 buckets from your AWS console. When you create a bucket, you need to provide a name and AWS region where you want to create the bucket. In each bucket, you can store any number of objects. You can use your AWS account root credentials to create a bucket, but it is not recommended. Instead just create an IAM user and add full permission to that user on S3 bucket. You can access your S3 bucket from your Amazon S3 console. 

Remove Existing Packages

Before installing any package, first you need to check if you have any existing fuse or S3FS on your server. If it is already existing, then remove it from your server to avoid further conflicts. Use the following command to check if you have any existing fuse or S3FS on your server

CentOS users:

# yum remove fuse fuse-s3fs

Ubuntu Users:

# apt-get remove fuse

Install Packages

Install all dependency packages for fuse and s3cmd using the below command.

CentOS users:

# yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap

Ubuntu Users:

# apt-get install build-essential libcurl4-openssl-dev libxml2-dev mime-support

Download and Compile Fuse

Move to /usr/src then download and compile fuse source code. After compiling, add fuse to kernel. In my case the latest version of fuse is fuse-3.0.0

# cd /usr/src/
# wget https://github.com/libfuse/libfuse/releases/download/fuse-3.0.0/fuse-3.0.0.tar.gz
# tar xzf fuse-3.0.0.tar.gz
# cd fuse-3.0.0
# ./configure –prefix=/usr/local
# make && make install
# export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
# ldconfig
# modprobe fuse

Download and compile S3FS

Navigate to /usr/src, Download and compile s3fs source code.

For CentOS
# yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel
For Ubuntu
# apt-get install automake autotools-dev fuse g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config
# git clone https://github.com/s3fs-fuse/s3fs-fuse.git
# cd s3fs-fuse
# ./autogen.sh
# ./configure
# make
# make install

Setup Access Key

In the next step we will create a file which stores the access key and secret key of our account (you can find the same under AWS Menu -> Your Account -> Security Credentials).

Both access key and secret key of your s3 AWS account is required for configuring S3FS. Replace the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with your actual key values.

# vi /etc/passwd-s3fs
AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY

Make sure that the file has proper permission.

# chmod 640 /etc/passwd-s3fs

Now we will set the allow_other mount option so even if we are mounting the bucket using non root account, they can have permission to use allow_other option.

For this (to uncomment user_allow_other in fuse configuration file), we will execute the following command,

#  sed -ie 's/#user_allow_other/user_allow_other/g' /etc/fuse.conf

Mount S3 Bucket

You can run the below command to mount s3fs.

# s3fs mybucket /path/to/mountpoint -o passwd_file=/etc/passwd-s3fs

For other user access:

# s3fs mybucket /path/to/mountpoint -o use_cache -o allow_other

You can also mount the s3 bucket on boot by following below commands.

# mkdir /tmp/cache
# mkdir /path/to/mountpoint
# chmod 777 /tmp/cache /path/to/mountpoint
# vi /etc/fstab

# s3fs <mybucket> /path/to/mountpoint fuse allow_other,use_cache=/tmp/cache,uid=userid,gid=groupid 0 0

# mount -a

Now you can access you S3 on your server..