Mashed Articles

Big news this week, the rumor finally became true: Google App Engine supports Java, next to Python. So Google AppEngine is now a big Servlet Engine in the cloud.

Along with the Java on Google App Engine announcement, I noticed another component: the Secure Data Connector. This SDC allows applications running in the Google cloud to inter operate with Intranet applications. Through the Secure Data Connector, Intranet applications can be accessed.

Scenario:

• The Secure Data Connector is installed on a Linux server within the Enterprise.
• An administrator configures the SDC to access certain resources within the Intranet.
• The SDC is started and runs continuously as a background process.
• The SDC connects to Google (https://apps-secure-data-connector.google.com) on port 443 (HTTPS). The connection is made from the enterprise to Google, so no need to configure the firewall at Entrprise side to allow inbound connections (from Google into the Enterprise).
• The SDC authenticates itself using username and password.
• Once the SSL connection is established, the connection remains open.
• An application running in the Google cloud (AppEninge, Google Spreadsheet, …) needs to access data from the Intranet or send data to the Intranet.
• In AppEngine, this is done using the URLFetchService. To specify that an Intranet resource, HTTP header use_intranet=true is set.
• From the Google AppEngine, a call is made to the SDC deployed in the Enterprise. Remember, TCP connections are bidirectional.
• The SDC verifies if the access the local resources, e.g. using the local DNS from within the Enterprise.
• The SDC accesses the local resource or web service and returns the data back to the applicaton running in te Google cloud. The size of request and response messages is limited to 1 MB.

The access to protected data within the enterprise is somewhat of a challenge. The only mechanism the SDC can provide credentials to Intranet application/service/resource is OpenSocial and OAuth signatures. And

One of the evolutions that I envision is that ESB’s or B2B services will embed the SDC logic as an adapter. The ESB is able to transform the requests coming from Google into other protocols or formats and add the necessary credentials.

Some more thoughts and remarks I made while going through the Secure Data Connector docs:

• How is the configuration file of the SDC protected? In particular the username and password contained therein.
• Support is limited to Linux. What prevents this open source code to be ported to other platforms?
• How about load balancing or failover?
• How about interoperability between clouds: anyone already tried to deploy the SDC on EC2?
• Where is the SOAP support? How to invoke SOAP web services using the URLFetchService?
• How about identity services and mapping the identity of a Google user to an internal Enterprise user account?
• The SDC is not comparable to the .Net Services Bus of Windows Azure.
• Can the SDC access the Internet through a proxy?
• To deploy the Secure Data Connector in a large enterprise, you might have a hard time convincing the security department.

A basic mechanism to make a service idempotent is duplicate detection. For one-way services, duplicate detection may be sufficient, Request/response services require also that exactly the same response message is returned.

Duplicate detection isn’t that hard:

1. Unique value in the request. Multiple options exist:
   • business identifier such as purchase order number
   • hash of the input message or
   • technical identifier such as WS-Addressing MessageID
2. Database table with unique constraint to manage the identifiers.
3. Cleanup job to remove old entries from the above database table.

The ideal location to do duplicate detection is in the back-end services themselves. The service is invoked in a transactional context and has already access to a database. But it can also make sense to do duplicate detection in an intermediate such as an ESB. E.g. when an ESB switches between an unreliable protocol such as HTTP and a reliable persistent queuing protocol.

Integration solutions often need to maintain information about messages anyway. For
message monitoring, message archiving or non-repeduation (keep copy of signed messages). Duplicate detection can also be combined with protection against replay attacks (even required by the core WS-Security spec).

Mule comes with an Idempotent Receiver that provides duplicate detection.
Still, why other ESB’s don’t come with duplicate detection mechanisms is a mystery to me!

Definition of idempotent: Refers to an operation that produces the same results no matter how many times it is performed.” See also the “Idempotent receiver” pattern.

An idempotent service makes life much easier in the unreliable web services world. A service can be invoked multiple times without any side effects. This is highly useful when a service invocation times out (was the service executed yes/no). But also to re-try execution of (part of) a business process.

Making a service idempotent consists of 2 parts: duplicate detection and caching of response messages. Duplicate detection I discussed in an earlier posting. Returning of earlier response message is somewhat harder: each response message must be stored for a reasonable amount of time.

Idempotency is not mentioned in the WS-* specs nor in the WS-I basic profile. Making web services idempotent is left as an exercise to the designer/developer.

With REST, a number of verbs such as GET and DELETE are idempotent. POST (create) is obviously not idempotent. PUT (update) should be idempotent. I can believe PUT to be idempotent in a document world. But PUT is an update and therefore a change in my opinion.

As already explained in my earlier post, the ideal location to do duplicate detection and make services idempotent in the back-end services themselves. Interesting to learn about support for idempotency in recent SAP products (Netweaver 2004S SPS09 / ECC SE 600 SP03). SAP stores the response messages in a “response store”.

Besides duplicate detection, let’s ask ESB vendors to implement support for idempotent services through a “response store”.