Tutorial: How to upload images directly to Amazon S3 from local Web Server
A step-by-step guide explaining how to create a Web Front End Server on localhost that Uploads Images securely to Amazon S3 Storage.
In modern web servers or applications, there is a common need or feature to upload documents or media files to the storage. And usual approach is to have User upload the file to Application Server then App Server saves the upload to Temporary space for processing and then the Application Server transfers the file to a Object Storage (S3)
Inspired and
Here I am trying to create a web server that allows users to directly upload images to the s3 bucket and retrieve them from the S3 bucket later rather than having to go to "the server" to process, hold and transfer upload and download the images with S3.
The Benefit of having to do it this way is that the load on the server significantly decreases because it never has to process image data and upload happens directly to S3 Bucket.
Prerequisites: VS Code, Node.js, AWS Account
Two-Parts to this project
- Bucket Policy, User IAM Policy, and Permissions
- The Front End Web (HTML and CSS) Application server deployed on Localhost Server with Node.js and Express.js
Part 1. Create S3 Bucket and Configure Security. Public Read Access.
How to create public access to AWS S3 Bucket Step-by-step
Note : S3 Bucket Cross-origin resource sharing (CORS)
Since I am going to use this Demo Test Project on localhost:8080, I will set the CORS policy to "AllowedOrigins": ["*"] which means any web domain is allowed for PUT and Get requests.(Not a Production Best Practice, I know) Must Change your CORS policy appropriately as required. CORS configuration Examples
S3 bucket is now setup so that I can get images to the Public Get allowed bucket. BUT adding(Post) new images to the S3 Bucket is something ONLY my Application/Authorised Users should be able to do.
Part 1.1 IAM User Programmatic Access Setup
In order to allow put access on the bucket, We need IAM User with "Programmatic Access" and Secure IAM Policy.
Create an IAM user with "Access key - Programmatic access" and attach "PutObject Allowed" IAM Policy. Note : Copy/Download the access key ID and secret access key for later.
Part 2. Install VS Code (how-to) and Node.js with npm
Checking your version of npm and Node.js
node -v
npm -v
Part 2.1 FrontEnd comprise of simple index.html and index.js
Index.js First gets secure url ("/s3Url") from the server then Fetch image ("imageURL") directly to the server and PUT directly to the S3 bucket. The function calls the upload method of the Amazon S3 service object to upload the photo. After uploading the photo, the function redisplays the album so the uploaded photo appears.
Part 2.2 BackEnd comprise of s3.js and server.js
Also installed NPM Lirbary List
It imports all the npm library and Bucket configuration along with IAM User's programatic access defined in environment variables of "accessKeyId" and "secretAccessKey". Also adds RandomBytes added on generated "uploadURL" when someone uploads an Image.
AWS_ACCESS_KEY=<your_access_key>
AWS_SECRET_ACCESS_KEY=<your_secret_key>
And finally Deploying Localhost Server with Express.js.
Deploying A Localhost Server With Node.js and Express.js
$ npm install express
Running server.js which deploys webserver and listens to port 8080 (localhost:8080)
Clicked upload , it successfully uploaded to my S3 Bucket "s3-direct-upload-from-web"
Hakuna Matata!
Simple and Secure Image uploader from my Localhost Server to directly to the S3 Bucket which eliminates the Application server processing time and resources. It's Secure because Bucket is only open to POST / PUT Objects from IAM User with its Programmatic Access (Secret Access Key ID) and uploads through randomized Signed S3url.
Some Serverless and other methods of Uploading to S3 from Web
Amazon S3 Browser Examples
Serverless-S3-Uploader
I learned a lot through these Demo projects and others and will continue to improve my knowledge to enhance features or work on similar kinds of projects in the future.
In the next blog, I am thinking set an S3 Bucket Event Notification (Email) using Amazon SNS (Simple Notification Service).
If you like this tutorial, feel free to check my other Blogs and Follow for more. Happy Learning !!
Thanks,
Jineshkumar Patel