In this tutorial we can check how to mount S3 bucket on your server.
S3FS is a FUSE (File System in User Space) will mount Amazon S3 as a local file system. S3FS has an ability to manipulate Amazon S3 bucket in many useful ways. If you wish to access your Amazon S3 bucket without mounting it on your server, you can use s3cmd command line utility to manage S3 bucket.
What is an Amazon S3 bucket?
Amazon S3 is a cloud based web service interface that you can used to store and retrieve any amount of data. To upload your data, first you need to create an S3 bucket in one of the Amazon regions.
Creating a Bucket
S3 provides an API for creating and managing buckets. You can create a maximum of 100 buckets from your AWS console. When you create a bucket, you need to provide a name and AWS region where you want to create the bucket. In each bucket, you can store any number of objects. You can use your AWS account root credentials to create a bucket, but it is not recommended. Instead just create an IAM user and add full permission to that user on S3 bucket. You can access your S3 bucket from your Amazon S3 console.
Remove Existing Packages
Before installing any package, first you need to check if you have any existing fuse or S3FS on your server. If it is already existing, then remove it from your server to avoid further conflicts. Use the following command to check if you have any existing fuse or S3FS on your server
# yum remove fuse fuse-s3fs
# apt-get remove fuse
Install all dependency packages for fuse and s3cmd using the below command.
# yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap
# apt-get install build-essential libcurl4-openssl-dev libxml2-dev mime-support
Download and Compile Fuse
Move to /usr/src then download and compile fuse source code. After compiling, add fuse to kernel. In my case the latest version of fuse is fuse-3.0.0
# cd /usr/src/ # wget https://github.com/libfuse/libfuse/releases/download/fuse-3.0.0/fuse-3.0.0.tar.gz # tar xzf fuse-3.0.0.tar.gz # cd fuse-3.0.0 # ./configure –prefix=/usr/local # make && make install # export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig # ldconfig # modprobe fuse
Download and compile S3FS
Navigate to /usr/src, Download and compile s3fs source code.
For CentOS # yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel For Ubuntu # apt-get install automake autotools-dev fuse g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config # git clone https://github.com/s3fs-fuse/s3fs-fuse.git # cd s3fs-fuse # ./autogen.sh # ./configure # make # make install
Setup Access Key
In the next step we will create a file which stores the access key and secret key of our account (you can find the same under AWS Menu -> Your Account -> Security Credentials).
Both access key and secret key of your s3 AWS account is required for configuring S3FS. Replace the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with your actual key values.
# vi /etc/passwd-s3fs AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY
Make sure that the file has proper permission.
# chmod 640 /etc/passwd-s3fs
Now we will set the allow_other mount option so even if we are mounting the bucket using non root account, they can have permission to use allow_other option.
For this (to uncomment user_allow_other in fuse configuration file), we will execute the following command,
# sed -ie 's/#user_allow_other/user_allow_other/g' /etc/fuse.conf
Mount S3 Bucket
You can run the below command to mount s3fs.
# s3fs mybucket /path/to/mountpoint -o passwd_file=/etc/passwd-s3fs
For other user access:
# s3fs mybucket /path/to/mountpoint -o use_cache -o allow_other
You can also mount the s3 bucket on boot by following below commands.
# mkdir /tmp/cache # mkdir /path/to/mountpoint # chmod 777 /tmp/cache /path/to/mountpoint # vi /etc/fstab
# s3fs <mybucket> /path/to/mountpoint fuse allow_other,use_cache=/tmp/cache,uid=userid,gid=groupid 0 0
# mount -a
Now you can access you S3 on your server..