Mashed Articles

Messaging systems such as IBM’s WebSphereMQ and similar use proprietary messaging protocols. So some library is always needed at the client side to talk the proprietary language to the messaging server. If such library is not available for your programming language, you’re out of luck. Regarding standard API’s, JMS seems the only one ever defined.

If a messaging server exposes a simple protocol over HTTP, it becomes possible to talk to the messaging server from any programming language. ActiveMQ is a good example in that area with their STOMP protocol. IBM has the “MQ Bridge for HTTP“. And OpenMQ 4.3 now has the UMS protocol.

From quickly skimming over the REST versions of these protocols – UMS of OpenMQ in particular – they are not “pure REST” but rather “REST-RPC Hybrid” (cf. the great book “RESTful Web Services“): 1) HTTP POST is used instead of GET, PUT or DELETE, 2) the actual action is part of the URL parameters and 3) the interactions become stateful through Logon service request.

When I think about a “pure REST” approach for messaging, I expect to see URLs such as http://mq.my-org.be/…/domain/queue. Sending a message becomes a HTTP PUT action. Peeking a message is a HTTP GET action. And receiving a message should become HTTP GET followed by HTTP DELETE (receiving a message is normally “destructive” in the messaging world). How to avoid concurrency issues in this receive scenario with multiple clients receiving the same message is a REST concurrency question that I gladly pass on

Closing remarks:

• Another approach is taken by AMQP: this initiative standardizes a binary protocol between client and messaging server. Any AMQP client library in whatever programming language should be able to communicate with any AMQP compliant server. Adoption of AMQP is rather limited.
• Existing messaging products (WebSphereMQ, SonicMQ, …) can tunnel their protocol over HTTP(S), but that still requires use of their respective client libraries.
• Most JMS messaging solutions support .Net (and optionally COM).
• At Devoxx 2008, Linda Schneider will talk about “Connectivity with OpenMQ” and Bruce Snyder does university session about “ActiveMQ and ServiceMix“.

When the Amazon Simple Queue Service appeared about 2 years ago, I looked into it as a solution of message exchange between business partners. With its limited message size (only 4K) and message retrieval based on simple Amazon user accounts, I put the offering aside. The Amazon offering did however trigger me into wondering when the big Internet players (Google, Microsoft, Salesforce or Amazon) would enter the integration market.

But the ‘big’ players aren’t entering the integration world (yet?). Alternatively, software vendors could cloud-enable their software. Or anyone could leverage the cloud and develop an integration system on system on top of it.

Some scenario’s that I can envision, probably lacking some imagination here

• upload messages to the cloud from which they can be polled and retrieved (e.g. some central FTP or JMS server)
• service composition (something like Splice)
• business processes managed in the cloud through BPEL process engine (like RunMyProcess)
• service enabled integration solution (like Grand Central Communications once tried)
• XML gateway/firewall that filters traffic, enforces policies and forwards requests to different back-ends
• B2B hub (re-invention of the VAN)
• SOA governance as a service (sharing services within a community)
• Centralized SAML provider used by federation(s) of business partners
• WS-Trust Security Token Service (STS)

To learn about cloud computing and see how it could be used for cloud based integration solutions, I went through the book “Programming Amazon Web Services” by James Murty.

Initial conclusion/impression after reading the book was that Amazon services still lack some important features. Some limitations of the Amazon services:

• no fixed IP addresses, use of dynamic DNS required; with no URL pointing to the Amazon servers, own server in own data center required as main entry point
• EC2 instances loose all state when they stop or die (partially addressed with backups at short intervals)
• no SLA (except for S3), EC2 and SimpleDB are still “beta” (but so is gmail)
• payment via credit card (no formal ordering/invoicing)
• only forums to report and track problems, no formal communication channels
• propagation latency of newly or updated S3 objects (without even guarantee that you retrieve the latest document yourself because there is no guarantee that S3 requests will be directed to the same location)
• no transactions, e.g. when retrieving messages from SQS service
• no relational database
• no guarantees, everything on best effort

But since the publication of the book, a number of shortcomings were already addressed.

• Elastic IP addresses now offer fixed IP addresses and do away with dynamic DNS. A single EC2 instance can load balance request to other instances.
• Availability zones allow instances to be started in specific zones (read data center).
• Persistent storage for EC2 now provides a real file system for EC2 surviving restarts (although file system can only be mounted by 1 instance).
• And AWS Premium Support starts addressing the support issue.

So it seems that AWS can be used for some of my envisioned scenario’s. Open source project will definitely offer their integration solution As A Service on the Amazon cloud. I’m curious to see if and when the 1st closed source integration vendor takes the same step.

All existing messaging solutions (WebSphereMQ, JMS, …) use proprietary protocols. This is not a problem within a single organization. But between organizations, standard protocols are needed. Therefore, the B2B world uses protocols such as AS2, RNIF (RosettaNet) or good old (S)FTP(S).

AMQP is an initiative to bring a standard binary wire protocol to the messaging world. Just like POP3+SMTP allows you to retrieve and send emails using whatever email server, AMQP will allow any AMQP client to receive and send messages via any AMQP compliant server.

But when I read the spec, AMQP is focusing on the client-server protocol, contrary to SMTP that is (also) used for communication between mail servers. The AMQP spec states that a bridge should be used for server-2-server communication, but doesn’t provide any details. As such, AMQP is focusing on messaging within the corporate firewall.

AMQP can be used for unbalanced B2B scenario’s, where one side runs the AMQP broker. This is a setup similar to one big company or intermediary running an (S)FTP(S) server and smaller organizations putting and retrieving files from it. But for good decoupling, server-to-server communication is preferred. The server at the sending side will take care of delivering the message to the server at the opposite side. Like e.g. AS2 does: once an organization has an AS2 server in place, it becomes equal to all its AS2 counterparts.

With all this in mind, I was a bit puzzled by Paul Fremantle’s enthusiasm about AMQP. In particular because he is the WS-RM spec lead.

WS-ReliableMessaging should have brought reliable async messaging to the WS-world. But it didn’t. The WS-RM spec doesn’t mention message persistence and so (most) vendors have an in-memory implementation, which is not reliable.

I still remember going through the book “Programming Indigo” and learning about the ReliableSessionEnabled binding property. What a disappointment to learn that for real reliability, one had to use the MsmqIntegration Binding and thus the proprietary MSMQ transport layer.