While enjoying holidays, I read the book “Programming Amazon Web Services” by James Murty. As explained in my earlier post, I was most interested to learn how cloud computing could be leveraged for developing integration solutions.
The book discusses 5 Amazon Web Services (AWS):
- Simple Storage Service (S3)
- Elastic Cloud Computing (EC2), virtual Linux servers on demand
- Simple Queue Service (SQS), to deliver short messages
- Flexible Payment Service
- SimpleDB – simple database with no SQL support
The book goes into quite some technical detail and has code snippets showing in detail how to interact with the Amazon services. All the samples are written in Ruby. I don’t know Ruby, but the code is quite readable (should read Enterprise Integration with Ruby some day). The author prefers the REST and the Query API. Unfortunately, he does not show anywhere the use of the SOAP API to access Amazon WS.
The 1st chapter is introductory and e.g. explains how to use self-signed certificates to connect with AWS, explains how AWS were developed for internal use by Amazon and later turned into a products, come without an SLA (except for S3) and without real support.
In the 2nd chapter, the author builds up a library of Ruby code to access the Amazon Web Services. This is very well written and gives an immediate feeling for some aspects to take into account, e.g. clock differences.
S3 is covered in chapters 3 and 4. No standard file access but the use of buckets and objects through a non-standard API (REST or SOAP); no FTP, WebDAV or SFTP. And objects cannot be modified: only deleted and re-created (after the deletion has propagated). Ruby code is shown for all the options the API offers: bucket creation/lookup/deletion, object creation/listing/deletion, ACL update/retrieval and access logging file retrieval. Tricks with HTTP header fields (object metadata), posting data through forms, alternative hostnames and BitTorrent are discussed. The last part discusses signed URI’s: this is a neat trick to make S3 resources temporarily accessible to users without Amazon account.
Chapter 4 shows some applications of the S3 service: large file transfer, backup, turning S3 into a file system (with FTP or WebDAV). Interesting to note that the author has his doubts wrt. exposing S3 as a file system. The author also discusses his own Java open source application: JetS3t. This application is a “gatekeeper” for S3 resources and authorizes local agent applications after acquiring signed URL to upload files to S3 and download files from S3.
Chapter 5, 6 and 7 dive into EC2 and how virtual Linux systems (based on Xen) can be configured using Amazon Machine Images. Ruby code is shown for every available API: keypairs (for SSH access), network security (dynamically configure the firewall), images and instances. Chapter 6 explains instances in more detail and discusses how to create new images. This involves quite some commands and scripts at the Linux command prompt. Chapter 7 discusses some sample applications: VPN server, web photo album thereby backing up data on S3. Chapter 7 also discusses issues around dynamically assigned IP addresses and the use of dynamic DNS.
The Simple Queue Service (SQS) is discussed in chapters 8 and 9. Because of the small message size, SQS is clearly meant for events with actual data stored on S3 (or elsewhere). Again Ruby code to manipulate queues and messages. Chapter 9 describes a Messaging Simulator application, not that relevant in my opinion. The 2nd application – leveraging a video conversion tool – shows how to build generic service for implementing “batch” services (Command Message pattern). The 3rd application – LifeGuard – leverages SQS to manage EC2 instance pools and dynamically scale the number of EC2 instances.
The chapter on payment service I skipped and I only skimmed through the SimpleDB chapter. Enough to learn that SimpleDB is not an RDBMS but a basic storage mechanism (no data types) with proprietary query facilities (no SQL).
The author writes fluently and gives a non-biased view on the Amazon Web Services. Sometimes the code goes into too much detail, showing how to invoke every available method of the API. Although the book is very recent (March 2008), important new features such as elastic IP addresses, persistent storage for EC2 and availability zones weren’t yet available at the time of writing. The book definitely taught me that AWS is quite proprietary and not that trivial. And to use Amazon’s cloud computing and AWS, you’d better “think like Amazon”.
So now I have my own simple Windows 2003 server with a fixed IP address and DNS name. Accessing the server goes fine with Remote Desktop. The responsiveness is not always top, but similar to a local VMWare instance. By the way, this is a perfect alternative for VMWare and a serious! I have the smallest server instance (AMI) running, which is obviously virtualised at Amazon. But it looks like dual-core Opteron with 1.66 GB of memory. And bandwidth is phenomenal: downloading Acrobat at more than 8 MByte/s.