Nimble Code: Tag soap

The Silliness of SOA

harrisj@nimblecode.com (Jacob Harris) — Thu, 10 Nov 2005 09:06:00 -0800

There seems to be a lot of buzz around the concept of Service-Oriented Architecture (SOA), as long as you don’t ask any of its proponents what it actually means. For an example of the ways in which English can be mangled by technical jargon beyond recognition, check out some of the definitions from A Defining Moment for SOA and Revisiting the Definitive SOA Definition to see what I mean. My personal favorite from the bunch is

SOA is a form of technology architecture that adheres to the principles of service orientation. When realized through the Web services technology platform, SOA establishes the potential to support and promote these principles throughout the business process and automation domains of an enterprise.

which really tells me nothing at all about why this is cool and where you would start if your CEO suddenly decided you needed it. Cutting through the bombast and hype, it seems like SOA is basically

Wrapping a backend database or similar resource in a small component for business logic you can talk to with XML.

This is definitely cool for many reasons (loose coupling, security, building big things from small parts), but I don’t understand why there’s so much hype about it now. Not to boast, but we’ve been doing this at Alacra for 8 years now, and I’m really gobsmacked that this sort of architecture would be a revelation to anyone, especially since it builds on design patterns good sense and the older Unix and later Internet philosophy that Small Pieces Loosely Joined can make great things. And SOA is pretty much what the earliest XML-driven web services were.

I can only assume that the latest hype about SOA is less about what the technology is, but what it is not. Hint: add a P and you can see what businesses find so refreshing about SOA. SOAP is a heavyweight protocol that can be good for some external APIs, but is a lot more work than a simple quick and dirty internal application often needs. Imagine being able to drop the requirements of WSDL bindings, XML typing, even limited data structures to get the most lightweight, low-ceremony solutions you can feel comfortable with using. And since it’s internal, you don’t have to worry about the “shame” of not being fully SOAP+WS-whatever compliant for your API. Looking at it this way, no wonder they’re excited. I’d be excited too if I were able to stop documenting and start hacking for change.

Extracted Standards

harrisj — Mon, 02 May 2005 21:39:00 -0700

Still busy, but there is a great article at O'Reilly Radar on how great frameworks are sometimes extracted wholesale from successful applications rather than built from scratch up. That is, it's often better to extract the framework from the application, rather than build the application on top of a framework:

That is, at 37signals, they try to design the usability and function of the application first, and that drives the implementation. And if they can then extract a re-usable framework, all the better. For example, basecamp wasn't built on top of Ruby on Rails. Rather, Ruby on Rails was extracted from basecamp.

This echoes some of the complaints I was making earlier about the unwieldiness of SOAP for web standards. And it fits in with the philosophy of nimble technologies perfectly. Create a product, solve a need, and make it work great and the frameworks will follow, but complicated frameworks in themselves solve nothing.

The New Web Math, Or How SOAP Adds Up To Nothing

harrisj — Mon, 25 Apr 2005 00:06:00 -0700

Last week saw the intriguing web mashups of combining two popular web services in one place. So, first there was combining Craigslist Real Estate listings with Google Maps, which although slow is really quite amazing to behold. Then someone else hacked Gmail + Del.icio.us, using Google's mail service to capture contents on bookmarked pages at the bookmarking time. Not to mention Adrian's trailblazing mix of Google Maps and Chicago transit info.

All of these are amusing and technically well-executed hacks, but what is most amazing to me is not the front-end combinations, but the ease in which it is possible to mix web tools in the back end to create new powerful services. And I have been especially struck by how SOAP has been utterly unnecessary for this to happen. In fact, I would argue that the strict COM-style model of SOAP is exactly the problem here, in that SOAP requires such effort and planning, it resists such freeform discovery of capabilities beyond what the creators of products imagine. REST seems to have a natural advantage here in that it builds on the inherent procedural model of HTTP and thus can evolve as naturally as websites do. Combine that with dynamic HTML tricks and we can see how nimble code can outmaneuver the heavyweight mechanics of SOAP again and again.

For a more lengthy and detailed critique of SOAP, see my earlier post, Radical Simplification

Radical Simplification

harrisj — Sat, 16 Apr 2005 10:43:00 -0700

David Heinemeier Hansson has posted an interested find over at his blog Loud Thinking about IBM’s stated new goal to pursue Radical Simplification in their enterprise work. Essentially, Big Blue is starting to acknowledge that most enterprise web development is just too cumbersome and daunting for agile and powerful web development. Things like SOAP and WSDL are examples of this. What should be a simple task like retrieve a weather forecast from a remote site becomes a complicated mess of debugging SOAP calls, tweaking WSDL specifications and using wizards on the server to generate code that is impossible for the end user to debug, let alone understand.

Or as Sam Ruby puts in his excellent presentation to IBM on the topic Hello From The Open Source World:

For normal people, the perceived usefulness of a computer language is inversely proportional to the amount of theory the language forces you to learn.

Sam illustrates this in his slides by showing the classic “Hello World” example in a variety of programming languages on different slides. But when he reaches WSDL to specify it, it’s a blank. Because, quite frankly, WSDL is so unwieldy to use, it’s hard to just build it on the fly, forcing many people to plunk down bucks to Microsoft or IBM to get their compilers to build it for them. It’s good business for those companies, but bad news for the Internet as a whole I think.

Ruby says that essentially for a framework to succeed on the web, it has to enable a situation he terms Zero Training, a state where it is easy to get going in the language and to adapt examples to your own needs. Good programming languages have it, some web technologies have it, SOAP and WSDL don’t. Indeed, I feel like the growth of SOAP/WSDL in the enterprise has been in spite of the difficulties of developing for it (mainly because of Microsoft and IBM pushing it). Because, quite frankly, it’s a beast for anything but the most basic RPC-style calls. The reasons:

SOAP allows you to abstract away the underlying transport protocol. But for 99% of SOAP communication, this protocol is HTTP, and the abstraction limits the control you have over HTTP.
SOAP needs to validate the entire message before applications can operate on it. This usually means it has to load it up into a DOM as well, so calls that return lots of data or might take a while (the fun ones) are frowned upon
SOAP also assumes the COM/CORBA marshalling mechanism, so no streaming data or parsing data until the entire document has been received, parsed into an XML DOM tree, and then mapped into an in-memory object tree. For large amounts of data, forget it.
WSDL has a lot of syntax to map procedure names/arguments onto HTTP. Contrast this to REST, which embraces the inherent naming conventions of HTTP and applies them to a simple procedural call model
The assumption of these tools is that you will use their SOAP wizards instead of rolling your own code. The wizard is meant to be the only way, not an additional way.
In fact, the idea that SOAP = Simple Object Access Protocol has become a profound irony.

In his presentation, Mr. Ruby points to PHP as an alternate model. PHP is not really a pretty programming model and it tends to shy away from enforcing higher-level abstractions in favor of lower-level models. But, it is precisely for this reason that PHP has succeeded where many other dynamic web application programming models have failed. Indeed, as it says in the page Do You PHP?

Database abstraction is mostly a myth. There is nothing wrong with direct database calls’ making use of all the tricks and cheats your chosen database has to offer, to tweak as much performance as possible out of it.

And this is the kicker here. Complicated database models tend to abstract away the lower level details of individual databases, even when those details can be the difference between fast and sluggish performance. Remote invocation models like SOAP abstract away the details of the underlying calling mechanism, even when tweaking HTTP requests can be the difference between a fast web app and a slow one. The lower-level mechanism might require more work for the developer, it might be ugly as sin, and it might be wrong and inefficient on some levels, but it works, and more importantly, it works quickly. Because ultimately, the customer doesn’t care what technology is used on the backend, they just want their data fast.

Bringing it back to my own experience, I once had to work on an application where we searched a remote document repository via SOAP and got metadata back on matching documents in XML. The service would send back all matching results for a query and in some cases, this would result in returning 150000 documents over 5 minutes of streaming XML to us. Knowing that the user probably would want to see data as quickly as possible, I decided it would be good to render whatever data I got after I received 10000 documents and tell the user to perhaps narrow their search down. And I tried to do this in SOAP in Visual Studio, and I failed. I could error out completely, but the mechanism had abstracted away the underlying XML-over-HTTP communication that I had no hooks to get lower level.

And so, I chucked it out and went back to basics. The upcoming call to the remote service was faked by filling in an XML string I grabbed from the original SOAP call via a packet sniffer. On the return, my program screams through the data with stream-based XML parsing. Once I hit 10000 records, then I can stop reading the input stream, display what I have, and tell the user that they need to narrow their search criteria. All within 30 seconds.

Those people who use SOAP are probably horrified by this and would rightly point out that I had to do a lot of lower-level work to get some of the same functionality promised by the one-click higher-level SOAP abstraction. But I think that’s precisely the problem. I needed something to work, and I didn’t need as much for it to be pretty. The SOAP framework locked me into a particular model, and once I needed more than that, it failed to deliver. Abstraction is nice, but abstraction always reduces speed, abstraction always reduces flexbility. In most cases, it is possible to strike a balance where the abstraction helps more than it hurts, but until SOAP is able to handle the heavy lifting needed for serious web work, it isn’t worth my time.