Mashed Articles

EDIINT AS2 is a very popular B2B protocol. Apart from file transfer, I think this is currently the most popular B2B protocol. Although EDIINT was initially meant to replace EDI over VAN connections by EDI over the INTernet (EDIINT), the AS2 protocol can transport any data in an asynchronous manner.

AS2 provides:

• Firewall friendly as it uses HTTP(S) underneath
• Straightforward: adds a number of HTTP header field and MIME based message structure
• Reliable: send message until you get a 200, duplicates are filtered based on message-id
• Signing: based on S/MIME message structure and PKCS7 signing (self signed certificates are often used in practice)
• Encyption: also leveraging the S/MIME message structure
• Non-repudiation: signed acknowledgement or message receipt, called the Message Disposition Notification
• Any payload: EDI, XML, binary, text
• Many implementations: free commercial versions available at low price and one or two open source implementations

As is typical in B2B scenario’s, AS2 servers are mostly located behind firewalls that only allows inbound connections from well know IP addresses.

AS2 and its family
AS2 has some family: AS1, which goes over SMTP. And a younger brother AS3 which uses FTP as its transport. And now a 4th child joins the family: AS4! AS4 uses uses ebXML messaging v3 as its transport. But what the heck is ebXML?

ebXML
ebXML was an initiative to define *the* new B2B standard for the 21st century: new transport layer, new message definitions, XML schema building blocks, process layer, protocol profiles and so on. ebXML didn’t really take off, mostly because it was considered some sort of threat to the Web Services story. A pitty that the WS-* world and ebXML world weren’t able to come together in 2001. ebXML gained a bit of popularity in some European countries like the Netherlands and Denmark, where it is used on a limited scale.

Work continued very quiet and a new version of the transport layer was released in 2007: ebXML Messaging v3. ebXML Messaging v2 and v3 are actually SOAP over HTTP(s) with some extra whistles.

Conclusion
The feature of AS4 is pulling or polling. Polling is one of the reasons why file transfer is so popular: it is “asymmetric” and allows one side to stay behind a closed firewall. ebMS 3.0 supports polling and AS4 das well. Well done! WS-Polling was a similar initiative in the WS-* world to introduce polling.

But AS4 will not become that popular in my opinion. The spec is rather “heavy” and ebMS 3.0 has very little traction. I’m not aware of any implementation. I would have preferred an extension of AS2 with support for polling, completely independent of the ebMS and WS-* specs. And so we’ll simply continue using file transfer (SFTP, FTPS) as our most popular polling mechanism.

Notes:
- Brik, thanks for pointing me to AS4
- Pictures are from a talk I gave on AS2
- Interesting link for those speaking Dutch: ebXML en ebMS: veilig en betrouwbaar berichten uitwisselen

While enjoying holidays, I read the book “Programming Amazon Web Services” by James Murty. As explained in my earlier post, I was most interested to learn how cloud computing could be leveraged for developing integration solutions.

The book discusses 5 Amazon Web Services (AWS):

• Simple Storage Service (S3)
• Elastic Cloud Computing (EC2), virtual Linux servers on demand
• Simple Queue Service (SQS), to deliver short messages
• Flexible Payment Service
• SimpleDB – simple database with no SQL support

The book goes into quite some technical detail and has code snippets showing in detail how to interact with the Amazon services. All the samples are written in Ruby. I don’t know Ruby, but the code is quite readable (should read Enterprise Integration with Ruby some day). The author prefers the REST and the Query API. Unfortunately, he does not show anywhere the use of the SOAP API to access Amazon WS.

The 1st chapter is introductory and e.g. explains how to use self-signed certificates to connect with AWS, explains how AWS were developed for internal use by Amazon and later turned into a products, come without an SLA (except for S3) and without real support.

In the 2nd chapter, the author builds up a library of Ruby code to access the Amazon Web Services. This is very well written and gives an immediate feeling for some aspects to take into account, e.g. clock differences.

S3 is covered in chapters 3 and 4. No standard file access but the use of buckets and objects through a non-standard API (REST or SOAP); no FTP, WebDAV or SFTP. And objects cannot be modified: only deleted and re-created (after the deletion has propagated). Ruby code is shown for all the options the API offers: bucket creation/lookup/deletion, object creation/listing/deletion, ACL update/retrieval and access logging file retrieval. Tricks with HTTP header fields (object metadata), posting data through forms, alternative hostnames and BitTorrent are discussed. The last part discusses signed URI’s: this is a neat trick to make S3 resources temporarily accessible to users without Amazon account.

Chapter 4 shows some applications of the S3 service: large file transfer, backup, turning S3 into a file system (with FTP or WebDAV). Interesting to note that the author has his doubts wrt. exposing S3 as a file system. The author also discusses his own Java open source application: JetS3t. This application is a “gatekeeper” for S3 resources and authorizes local agent applications after acquiring signed URL to upload files to S3 and download files from S3.

Chapter 5, 6 and 7 dive into EC2 and how virtual Linux systems (based on Xen) can be configured using Amazon Machine Images. Ruby code is shown for every available API: keypairs (for SSH access), network security (dynamically configure the firewall), images and instances. Chapter 6 explains instances in more detail and discusses how to create new images. This involves quite some commands and scripts at the Linux command prompt. Chapter 7 discusses some sample applications: VPN server, web photo album thereby backing up data on S3. Chapter 7 also discusses issues around dynamically assigned IP addresses and the use of dynamic DNS.

The Simple Queue Service (SQS) is discussed in chapters 8 and 9. Because of the small message size, SQS is clearly meant for events with actual data stored on S3 (or elsewhere). Again Ruby code to manipulate queues and messages. Chapter 9 describes a Messaging Simulator application, not that relevant in my opinion. The 2nd application – leveraging a video conversion tool – shows how to build generic service for implementing “batch” services (Command Message pattern). The 3rd application – LifeGuard – leverages SQS to manage EC2 instance pools and dynamically scale the number of EC2 instances.

The chapter on payment service I skipped and I only skimmed through the SimpleDB chapter. Enough to learn that SimpleDB is not an RDBMS but a basic storage mechanism (no data types) with proprietary query facilities (no SQL).

The author writes fluently and gives a non-biased view on the Amazon Web Services. Sometimes the code goes into too much detail, showing how to invoke every available method of the API. Although the book is very recent (March 2008), important new features such as elastic IP addresses, persistent storage for EC2 and availability zones weren’t yet available at the time of writing. The book definitely taught me that AWS is quite proprietary and not that trivial. And to use Amazon’s cloud computing and AWS, you’d better “think like Amazon”.