Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Amazon S3 Cookbook (n)
  • Table Of Contents Toc
Amazon S3 Cookbook (n)

Amazon S3 Cookbook (n)

By : Hashimoto
3 (2)
close
close
Amazon S3 Cookbook (n)

Amazon S3 Cookbook (n)

3 (2)
By: Hashimoto

Overview of this book

Amazon S3 is one of the most famous and trailblazing cloud object storage services, which is highly scalable, low-latency, and economical. Users only pay for what they use and can store and retrieve any amount of data at any time over the Internet, which attracts Hadoop users who run clusters on EC2. The book starts by showing you how to install several AWS SDKs such as iOS, Java, Node.js, PHP, Python, and Ruby and shows you how to manage objects. Then, you’ll be taught how to use the installed AWS SDKs to develop applications with Amazon S3. Furthermore, you will explore the Amazon S3 pricing model and will learn how to annotate S3 billing with cost allocation tagging. In addition to this, the book covers several practical recipes about how to distribute your content with CloudFront, secure your content with IAM, optimize Amazon S3 performance, and notify S3 events with Lambada. By the end of this book, you will be successfully implementing pro-level practices, techniques, and solutions in Amazon S3.
Table of Contents (14 chapters)
close
close
13
Index

Learning AWS SDK for Java and basic S3 operations with sample code

This section tells you about how to configure an IAM user to access S3 and install AWS SDK for Java. It also talks about how to create S3 buckets, put objects, and get objects using the sample code. It explains how the sample code runs as well.

Getting ready

AWS SDK for Java is a Java API for AWS and contains AWS the Java library, code samples, and Eclipse IDE support. You can easily build scalable applications by working with Amazon S3, Amazon Glacier, and more.

To get started with AWS SDK for Java, it is necessary to install the following on your development machine:

  • J2SE Development Kit 6.0 or later
  • Apache Maven

How to do it…

First, we set up an IAM user, create a user policy, and attach the policy to the IAM user in the IAM management console in order to securely allow the IAM user to access the S3 bucket. We can define the access control for the IAM user by configuring the IAM policy. Then, we install AWS SDK for Java by using Apache Maven and git.

Tip

For more information about AWS Identity and Access Management (IAM), refer to http://aws.amazon.com/iam/.

There are two ways to install AWS SDK for Java, one is to get the source code from GitHub, and the other is to use Apache Maven. We use the latter because the official site recommends this way and it is much easier.

  1. Sign in to the AWS management console and move to the IAM console at https://console.aws.amazon.com/iam/home.
  2. In the navigation panel, click on Users and then on Create New Users.
    How to do it…
  3. Enter the username and select Generate an access key for each user, then click on Create.
    How to do it…
  4. Click on Download Credentials and save the credentials. We will use the credentials composed of Access Key ID and Secret Access Key to access the S3 bucket.
    How to do it…
  5. Select the IAM user.
    How to do it…
  6. Click on Attach User Policy.
    How to do it…
  7. Click on Select Policy Template and then click on Amazon S3 Full Access.
    How to do it…
  8. Click on Apply Policy.
    How to do it…

Next, we clone a repository for the S3 Java sample application and run the application by using the Maven command (mvn). First, we set up the AWS credentials to operate S3, clone the sample application repository from GitHub, and then build and run the sample application using the mvn command:

  1. Create a credential file and put the access key ID and the secret access key in the credentials. You can see the access key ID and the secret access key in the credentials when we create an IAM user and retrieve the CSV file in the management console:
    $ vim ~/.aws/credentials
    [default]
    aws_access_key_id = <YOUR_ACCESS_KEY_ID>
    aws_secret_access_key = <YOUR_SECRET_ACCESS_KEY>

    Tip

    Downloading the example code

    You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

  2. Download the sample SDK application:
    $ git clone https://github.com/awslabs/aws-java-sample.git
    $ cd aws-java-sample/
    
  3. Run the sample application:
    $ mvn clean compile exec:java
    

How it works…

The sample code works as shown in the following diagram; initiating the credentials to allow access Amazon S3, creating and listing a bucket in a region, putting, getting, and listing objects into the bucket, and then finally deleting the objects, and then the bucket:

How it works…

Now, let's run the sample application and see the output of the preceding command, as shown in the following screenshot, and then follow the source code step by step:

How it works…

Then, let's examine the sample code at aws-java-sample/src/main/java/com/amazonaws/samples/S3Sample.java.

The AmazonS3Client method instantiates an AWS service client with the default credential provider chain (~/.aws/credentials). Then, the Region.getRegion method retrieves a region object, and chooses the US West (Oregon) region for the AWS client:

AmazonS3 s3 = new AmazonS3Client();
Region usWest2 = Region.getRegion(Regions.US_WEST_2);
s3.setRegion(usWest2);

Tip

Amazon S3 creates a bucket in a region you specify and is available in several regions. For more information about S3 regions, refer to http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region.

The createBucket method creates an S3 bucket with the specified name in the specified region:

s3.createBucket(bucketName);

The listBuckets method lists and gets the bucket name:

for (Bucket bucket : s3.listBuckets()) {
System.out.println(" - " + bucket.getName());

The putObject method uploads objects into the specified bucket. The objects consist of the following code:

s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile()));

The getObject method gets the object stored in the specified bucket:

S3Object object = s3.getObject(new GetObjectRequest(bucketName, key));

The ListObjects method returns a list of summary information of the object in the specified bucket:

ObjectListing objectListing = s3.listObjects(new ListObjectsRequest()

The deleteObject method deletes the specified object in the specified bucket.

The reason to clean up objects before deleting the S3 bucket is that, it is unable to remove an S3 bucket with objects. We need to remove all objects in an S3 bucket first and then delete the bucket:

s3.deleteObject(bucketName, key);

The deleteBucket method deletes the specified bucket in the region.

The AmazonServiceException class provides the error messages, for example, the request ID, HTTP status code, the AWS error code, for a failed request from the client in order to examine the request. On the other hand, the AmazonClientException class can be used for mainly providing error responses when the client is unable to get a response from AWS resources or successfully make the service call, for example, a client failed to make a service call because no network was present:

s3.deleteBucket(bucketName);
} catch (AmazonServiceException ase) {
  System.out.println("Caught an AmazonServiceException, which means your request made it " + "to Amazon S3, but was rejected with an error response for some reason.");
  System.out.println("Error Message:  " + ase.getMessage());
  System.out.println("HTTP Status Code:  " + ase.getStatusCode());
  System.out.println("AWS Error Code:  " + ase.getErrorCode());
  System.out.println("Error Type:  " + ase.getErrorType());
  System.out.println("Request ID:  " + ase.getRequestId());
  } catch (AmazonClientException ace) {
    System.out.println("Caught an AmazonClientException, which means the client encountered " + "a serious internal problem while trying to communicate with S3," + "such as not being able to access the network.");
    System.out.println("Error Message: " + ace.getMessage());

See also

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Amazon S3 Cookbook (n)
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon