Progress Software - getting past "Who"?

A couple of months back I had a brief Twitter exchange with David Bressler of Progress Software (@djbressler), following a comment I'd seen from Judith Hurwitz (@jhurwitz) at Progress' analyst day regarding the lack of brand awareness that the company has out there in industry. What I said was: "Progress is a bit like Unilever - top-level brand is vanilla, sub-brands have chops". What I meant is that these days, there's little knowledge of what Progress does (a typical response is either "Who?" or possibly "oh, they used to sell a 4GL and a database in the 1990s, didn't they") - whereas there's much more recognition of brands like Sonic (SOA infrastructure), Actional (SOA management / governance), IONA (middleware, SOA infrastructure), Apama (event processing), DataXtend (data integration) and DataDirect (data connectivity, legacy application integration).

David replied that Progress is a technology company's company - which is absolutely correct: Progress has a long and successful history of providing a platform for other software vendors to embed in their application offerings. And he followed up with this blog entry, saying "We'd love for the Progress brand to have some chops, and we're trying but it's not trivial."

Well, for a few weeks I'd been meaning to write a blog post of my own exploring this - but in the general headlong rush that we've been experiencing so far this year, I'd forgotten to write that post. When I saw today's news that there's been a change at the top at Progress, though, I was finally prompted to write some thoughts down. (Thanks for the pointer Miko).

The main thought in my head all those weeks ago was that it's all very well for Progress to be a bit like Unilever - with the sub-brands (Sonic, Actional, Apama, DataDirect, and so on) having much more visibility in industry than the parent brand - as long as the company doesn't want to start pulling together broader IT and business infrastructure propositions that tie together pieces from the different brands. Unilever is well-known for owning a vast portfolio of products, many of which actually compete with others in the portfolio (Dove v Lux; or Persil v Surf, for example. The invisibility of the parent brand is fine for Unilever, but it's bad news for Progress if it wants to really make the most of its potential within enterprises (by cross-selling or bundling its products to help customers with broader opportunities, for example).

So this is the point where the company has to undergo a pretty radical shift. As reported in PCWorld, the new Progress Software CEO (formerly the COO) has established a target of doubling the company's annual revenue to around $1bn, by "reorienting sales towards multi-product suites, as well as aiming marketing messages more at business executives than IT workers" - that is, precisely what it's not currently suited to doing.

This goal makes absolute sense, and in fact it has made sense for ages. The majority of the markets where Progress' brands play are growth markets where there's real opportunity, right now; and what's more, the combination of the offerings could have real power, too.

The required shift will be no picnic, but there are worse times for Progress to be trying to make it happen. There's a new man at the top with a new broom, no doubt; and what's more, there's still a small window of opportunity open for another medium-to-large-sized specialist infrastructure software vendor to pick up business, following BEA's acquisition by Oracle a few months back. TIBCO and Software AG have recently been making much of BEA's disappearance as an "independent" infrastructure software vendor, and it's surely no coincidence that both these companies also have aspirations to reach $1bn in annual revenues (Software AG has been particularly vocal about this of late). Progress has long had the potential to join Software AG and TIBCO as a serious contender for enterprises wanting to avoid getting into bed with the MISO pack (Microsoft, IBM, SAP or Oracle) for whatever reason, but until now it just never seemed to be able to be bothered to do what was necessary.

With a new CEO at the top, it'll be fascinating to see whether Progress can move up a gear. If it succeeds, then enterprises wanting to avoid giving too much technology supplier power to the MISO pack may well have a new choice - and in a market where consolidation has recently been rampant, more choice would be refreshing for everyone.

Labels: industry, integration, Progress, SOA, Software AG, TIBCO

posted by Neil Ward-Dutton @ 9:30 AM 2 comments permalink

Cloud computing, SaaS and SOA - the universal service network

Something that's been sloshing gently around in my head for a little while came into focus the other day on reading a post by Brenda Michelson: Unintentional Cloud Watching >> Cloud Computing for Enterprise Architects. Namely, that the link between cloud computing and SOA has multiple angles.

It's becoming clearer that, true to Tim O'Reilly's initial Web 2.0 noodling, by providing open infrastructure services and APIs, the poster children of the Web 2.0 era - Amazon, Salesforce, Zoho, and so on - are now treading the path that companies like StrikeIron started out on in 2002.

As a result, as other commentators have noted, it looks like the bulk of the service-oriented IT that many organisations will interact with will be "stuff from outside" (commercially provided services) rather than "stuff from inside" (internally developed services). And it's not just hosted SaaS providers who are playing here of course: there's the issue of newer versions of on-premise commercial packaged application software products and integrations between them - SOA is coming in by the back door there, too (see SAP's ESOA, Oracle's Fusion Architecture). In fact, perhaps unsurprisingly, there are strong parallels here with the component-based development (CBD) hype-wave of the 1990s - a lot of the initial hype was around tools and development for enterprise IT groups, but ultimately the vast bulk of development was actually carried out by commercial software vendors, for consumption by enterprise IT teams. What we're seeing here is a repeat of "componentware" market development, with a 21st Century twist.

Regardless of where services come from (and indeed because they will come from multiple places, creating cross-enterprise service networks), it's increasingly the case that in order to deliver effective IT capabilities in the 21st Century, you need to understand SOA principles and build technology and management structures that really support the principles of service orientation. Much has been written about the "consumerisation of IT" and how new generations of people entering the workplace are asking difficult questions about why enterprise IT applications are so unintuitive to use. But what happens when business teams that are using SaaS-based offerings learn about the infrastructure side of the story - how easy they can be to customise, extend, and integrate with - and ask why internally-developed systems don't exhibit the same qualities? Another slab of SOA pressure, that's what.

Back in 2006 I was doing some research for an event that we were considering running on "IT Sourcing in the 21st Century". As part of trying to work out what might be in scope and what might be out of scope, I drew this picture:



As the picture attempts to show, from a technology architecture point of view, the software-as-a-service (SaaS) model relates to the prior Application Service Provider (ASP) model in much the same way as SOA relates to monolithic on-premise applications - to deliver value, both SaaS and SOA need to "crack the box open" and enable IT capabilities to be customised, composed and remixed while also being shareable/reusable. There's much I disagreed with about Anne Thomas Manes' recent "SOA is dead" post (see Schrodinger's SOA), but she nailed this parallel between SaaS and SOA pretty well I think.

With the increasing visibility of cloud-based infrastructure and application services in mind, anyone seriously pursuing SOA should be looking to the world of SaaS for insight, or at least inspiration. In the IT industry's rush to SOA, many of the nuances and implications of SOA have often been condensed into simplistic advice targeted at software developers (something that's been written about a great deal, not least by us). One of the themes we've returned to again and again in our SOA advice is that the concept of a "service" isn't primarily about something you build - it's about something you experience. If you're going to deliver business value from your SOA efforts, you have to grasp the implications of this and make the necessary changes - not only in terms of tools and technologies, but also (crucially) in terms of your governance approach. One of the most popular posts from arch-architect Todd Biske, on ITIL and SOA, digs nicely into how it's crucial to consider the full service lifecycle when you do SOA, and drive governance to ensure that you can deliver "real service" - not just code wrapped in XML-based interfaces.

The providers of cloud-based services should have this idea etched on their brains - and there are plenty of examples of this principle in action for any eager student of SOA. Witness the grumbles that echo when Twitter barfs, for example, or see the grumpy users trying to get to grips with Zoho's CRM API in the face of almost non-existent documentation. These are great examples of situations where the provider's idea of service doesn't match the consumer's expectation.

The rise of cloud computing and SaaS should entice us to revisit our development-centred assumptions about SOA and search for a "bigger picture" that focuses on consumer expectation and value first. The pressure to deliver business value from IT capabilities, and the increasingly diverse mix of IT capability sources, demands that we do so.

Labels: architecture, cloud computing, ITSM, SaaS, service, SOA

posted by Neil Ward-Dutton @ 9:53 AM 0 comments permalink

Schrödinger's SOA

I've spent a couple of days wavering over whether to jump into an ongoing blogosphere debate over the "Death of SOA". For those who haven't yet read any of the debate online, here's the catalyst: SOA is Dead; Long Live Services - a fictional obituary of SOA by Anne Thomas Manes of the Burton Group. On one hand I wanted to avoid adding another voice to a growing crescendo of opinion and counter-opinion and the risk of not really shedding any extra light: but on the other hand I thought that some people might expect MWD to have something to say! Particularly given past blog posts like SOA 2.0? Stop the madness, Little SOA vs Big SOA, More big vs small thinking: SOA vs BPM, The pointless search for SOA ROI, and On SOA governance: for SOA, read CPOA?.

In the end, I couldn't help myself. Call it New Year exuberance. So what's our take? Is SOA dead or alive? Should anyone care?

I think there are a couple of key points to consider. First, what's wrong with SOA today? And second, what should we do about it? I agree with Anne on the first; but I think I really disagree on the second.

When I look at Anne's post in detail I believe she's making 4 basic points:

1. The majority of SOA projects have failed to deliver what they promised.
2. Business people are disillusioned with SOA.
3. "SOA projects" will be killed by today's difficult economic conditions.
4. Despite all this, the requirement for SOA is greater than ever.

I'll happily back two of those points up - the first (UPDATE: to a degree - I have anecdotal evidence that suggests many organisations struggle with SOA, but that's not quite the same)and the fourth. There's one section towards the end of Anne's post I'm particularly in agreement with:

SOA is not simply a matter of deploying new technology and building service interfaces to existing applications...it requires a massive shift in the way IT operates. ... The latest shiny new technology will not make things better. Incremental integration projects will not lead to significantly reduced costs and increased agility. If you want spectacular gains, then you need to make a spectacular commitment to change.

This is pretty much what we've been saying about SOA for some time (see Little SOA vs Big SOA from April 2007). In short: SOA won't succeed if you take an overly technical, product-based view of it (and indeed we're far from alone in this - ZapThink bangs this drum a lot, too, for example).

So we agree that not all is well in the State of SOA: but where I diverge with Anne's post is in the prognosis for SOA and the treatment for the illness.

Anne's prognosis/suggested treatment is that SOA (or at least the term) is (or will soon be) dead. Her view appears to be that because its death is inevitable, we should accept it and move on - stopping talking about SOA and starting talking about other things instead. This appears to be bound up with her observations that "business people are disillusioned with SOA" and "SOA projects will be killed by the economic downturn".

Before I move on - I don't know about you, but I think that if any IT group has been trying to sell SOA directly to business people, they deserve everything they get. Regardless of economic conditions. No, no, no! You don't sell methodologies and architectural patterns: you sell outcomes. Argh! And here's a hint: if you work for an IT group that used to try to sell SOA to business people, and is thinking about now trying to sell "mashups", "cloud computing", or similar things - don't bother, you'll get the same result. Sell the outcome and the benefits, not the mechanism or the technology. Double Argh!

There's another risk with changing the way we speak about what Miko Matsumura jokingly now calls "The Artist Formerly Known as SOA": by doing so, we continue to spread the perception that the IT industry is a fashion industry unable to kick its habit of (re)inventing terms to reinvigorate markets when earlier promises go unfulfilled. Whether she knew it or not when she wrote the post, by referring to a change in terminology (away from SOA and towards "services", "mashups", "cloud computing", etc) as an active and influential commentator, Anne is contributing to the fall from grace of the term.

Well, as I said here when I railed against SOA 2.0,

I sincerely believe that analysts should be good stewards of the influence they have - educating, clarifying, abstracting, comparing, acting independently, being measured, etc. It's about filtering out hype and trying to provide practical, independent advice and insight.

Just because SOA is difficult to do, we shouldn't start calling it something else in the hope that we can start over without anyone noticing. And it's no surprise that SOA is tough to sell to business people - I don't believe that was ever up for debate, and it shouldn't be seen as any kind of broader indicator.

Let's acknowledge that we all have more work and education to do - but let's not jump the shark on this. "SOA is Dead" is a headline that no-one needs.

Labels: architecture, hype, SOA

posted by Neil Ward-Dutton @ 7:50 PM 2 comments permalink

On SOA governance: for SOA, read CPOA?

A couple of weeks ago I was the happy recipient of a review copy of the excellent Todd Biske's SOA Governance book. Todd's "Outside the Box" blog is one of those rarities where every post is worth reading twice - so I was very interested to see whether his writing ability might stretch to something the length of a book! Todd's clearly established himself as someone who has a lot of insight on the topic of SOA Governance, so I was pretty sure I wouldn't be disappointed.

A number of other bloggers have posted detailed reviews of Todd's book, so I'm not going to do exactly the same here. Take a look at the Amazon comments if you'd like to see what they said.

For me, I'll be brief: SOA Governance is a very good book indeed, in that it does something that so many technology and business management books fail to do: it breaks a complex and hype-laden subject down into very manageable chunks, and walks through the topic clearly and at a steady pace - but it still manages to move quickly enough to prevent the reader getting bored. It's not a perfect(*) book, but then nothing is - and if it had been, I would have been too jealous to write this. We need more technology/business management books like this, and we needed just such a book on SOA Governance. Well done Todd!

I knew this was a good book because it made me revisit some conclusions I'd already had washing around in my own head for a couple of years.

One of the things that I still find as I travel around is that when I get into discussions about SOA, there's way too much focus on the "S" and not enough focus on the "A". It's almost as if we've been blinded by technologies and standards which have "service" somewhere in their names, and aren't able to look at the bigger picture.

What Todd's book reminded me is that if you want to get real value out of service orientation, then it's the "A"rchitecture that really makes things happen. Todd's narrative keeps coming back to his definition of Governance, which revolves around People, Policies and Processes. And it also talks a lot about the concept of "contracts" in the context of analysing how service providers and consumers should work together in order to interact. Without People, Policies and Processes in place to guide your organisation down the right path, and without the concept of "Contract" to focus on the responsibilities that need to be described and assigned when service consumers and providers interact, such an architecture effort will likely lead nowhere. You'll end up with "just a bunch of services".

So - and this was the thought that occurred to me after reading Todd's book - perhaps we shouldn't really be thinking about "service" oriented "architecture" at all. It seems to me that what architects might find more productive to focus on is policies and contracts, not "services". Maybe "service" is better thought of as a concept that describes the outcome of this kind of architecture approach. And so maybe it's the case that there are two things in play here, and we're getting them mixed up: contract- and policy-oriented architecture (CPOA ;-) and service-oriented IT delivery?

What do you think?

(*) one thing I found rather strange was that despite a word at the front to reassure readers that they didn't need to know any technology detail in order to read the book, at a number of points you're suddenly confronted, out of nowhere, by XML fragments which (as far as I could tell) didn't really add any value. That's a tiny niggle though.

Labels: contracts, governance, policy, SOA, toddbiske

posted by Neil Ward-Dutton @ 8:43 PM 1 comments permalink

Software Delivery InFocus podcast - ALM challenges and direction in the real world

Following the first Software Delivery InFocus podcast which we published in September, October sees Bola Rotibi's second podcast episode, in which she discusses a series of questions focused on the topic of Application Lifecycle Management (ALM). Her guests are John Leegte (ICT Architect at the Dutch Tax and Customs department, Belastingdienst) and Steve Jones (Head of SOA and SaaS for Capgemini?s global outsourcing business).

Application Lifecycle Management (ALM) is a topical subject that has garnered significant column inches in recent months, as many of the leading players in the market have launched new versions of their ALM solutions, and make strategic announcements concerning future directions and customer services and support. Over the last few months we have either heard about or seen previews of products from the likes of Borland, Compuware, HP, IBM, Microsoft, MKS, Polarion and Serena, to name but a few. Software is seen by many organisations as a key enabler of business value - whether that be through improving operational efficiency, competitive differentiation or business/ product innovation. With this in mind, an ad hoc approach to software application lifecycle management cannot provide the predictability, visibility and traceability that organisations require of a process that has such a significant impact on the balance sheet. So - how relevant and applicable is ALM now and in the future, particularly in light of today's technology and business environments, when issues such as agility are so much to the fore?

The episode is slightly longer than normal (clocking in at 41'55"). There was so much good material in the conversation, we didn't want to cut anything! Thanks very much to both John and Steve for such a great conversation.

You can download the audio here or alternatively you can subscribe to the podcast feed to make sure you catch this and all future podcasts!

As with all the episodes in this podcast series, we've also published a companion report which summarises the discussion and "key takeaways". You can find it here, and it's free to download for all MWD's Guest Pass research subscribers (joining is free).

Labels: agile, development, podcast, SOA

posted by Neil Ward-Dutton @ 11:58 AM 0 comments permalink

SOA governance and data governance - separate or one in the same?

Labels: governance, SOA

posted by Neil Macehiter @ 11:23 AM 0 comments permalink

Hmm indeed Mr McKendrick - that should be "an over-simplistic definition of 'SOA'"

Labels: SOA

posted by Neil Macehiter @ 10:26 AM 0 comments permalink

IBM, Business Event Processing, and CEP: behind the bag of spanners

Earlier this week I attended an IBM press and analyst summit on the topic of "Business Event Processing". To coincide with this, the company made some announcements on its "BEP leadership", with over 3700 "BEP customers". This is fairly early days in IBM's attempts to tell a coherent story about what it's doing in event processing, although it's been helping customers for many years prior to the January 2008 acquisition of AptSoft (which catalysed IBM's use of the BEP term). However the company isn't doing itself any favours with language like this: it's all too easy for it to come across as disingenuous, or at the very least the result of a significant amount of Vaseline being spread on a camera lens.

Why? Well, it's a question of boundaries and definitions. Specifically, IBM defines BEP in the following way: (1) an event is "something that happens"; (2) a business event is "an event that has relevance to the business"; and (3) Business Event Processing (BEP) is "the correlation of heterogeneous events in order to achieve smarter business outcomes".

BEP is a term which has come about primarily through IBM's own marketing since the AptSoft acquisition. But a similar term, Complex Event Processing (CEP), preceded this by some significant time (it was popularised by David Luckham's 2002 book The Power of Events). With a nod to the debate that still simmers about what the CEP term should really signify (is it the processing of compound/aggregated, or "complex" events - or is it application of "complex processing" to events?), IBM's definition of BEP is very close indeed to what seems to be the pre-eminent definition of CEP. This is no accident: on a call to discuss the AptSoft acquisition, IBM's Sandy Carter explicitly called out her interest in trying to rename the "CEP category".

All's fair in love and software marketing, I suppose, even when it risks leaving everyone confused (most people are still trying to get their heads around CEP). The real problem I have with this, though, that it's not only a simple renaming that's at work here. When IBM then goes on to talk about the event processing work it's doing, it tells a story that's much broader than the AptSoft technology area, and also broader than what most event-processing folks would think of when they think of CEP. Likewise when IBM refers to "3700 customers" these aren't AptSoft customers (apart from a tiny minority of them). As well as focusing on WebSphere Business Events (the former AptSoft technology) IBM's view of BEP encompasses event-processing capabilities that exist in Tivoli/NetCool systems and network management products, WebSphere MQ Event Broker and MQ Low Latency Messaging, a new stream-based event processing platform called InfoSphere Streams (formerly "System S"), the SolidDB in-memory database, and many other bits and pieces as well.

In short, it looks like IBM's taking a term that only in January 2008 it used to describe a "business friendly" event processing toolset (the AptSoft technology), and is now instead applying it to any software technology that IBM offers which involves the processing of events. A cynic would say it's got a big plumbers' canvas toolbag marked "BEP", and is shoving as much into it as possible.

Now the intention of this post isn't just to bash IBM for attempting to subvert the CEP concept. The reason I don't want to leave things here is that there's more to what IBM's doing than meets the eye.

As the press and analyst day unfolded, it became clear (not through the presentations from the IBM execs, tellingly, but through Q&A and one-on-one meetings) that behind the scenes, IBM is actually trying to do something pretty interesting. It's got a big kit-bag of event-processing technologies, yes - but behind this, it's working to put together a common event processing technology framework that will allow its individual technology components to be assembled in different combinations to support different types of business and IT requirements. So, rather than having separate event processing stovepipes in the WebSphere, InfoSphere and Tivoli software portfolios, together with assorted miscellaneous other components, it's enabling all of these distinct efforts to cross-fertilize. One of the first outcomes of this work is the snappily-named WebSphere Business Events eXtreme Scale, which marries the former AptSoft technology (never marketed as a platform for high-volume processing) to the WebSphere eXtreme Scale platform.

With this in mind, I think it's a bit of a shame if IBM sticks to referring to its overall event processing architecture effort as BEP. BEP, like CEP and one of my other favourites Composite Application Development, is a "how" name - it says more about a method than its outcome. Or, in other words, although it might appeal to geeks, it says nothing about why anyone would invest in it. What would work much better, in my opinion, would be for IBM to complement (or even replace) its discussions about BEP with discussions more centred around a "what" idea that is more about the business value of event processing - what all these technologies, and the framework being built, actually enable customers to achieve. If IBM hadn't already walked away from the term, I'd suggest something like "on-demand business"... but seeing as that's verboten these days, I'll keep my thinking cap on. Perhaps "event-driven business"?

Putting all the vendor marketing stuff to one side, what does all this mean for enterprises? First, try and look behind the cynical BEP and CEP marketing stuff. IBM might be mis-stepping in its marketing, but you can be sure that it's serious about building a set of technology offerings to help customers with a variety of event-processing related problems and opportunities. Second, although algorithmic trading and other capital markets applications are where most of the technology market activity in this space has been over the past year or so, try and take a broader perspective of event processing and how it might offer value. Event processing is a useful computing approach, even when the event volumes aren't colossal, or the processing requirement isn't near-real-time, or the processing requirement isn't highly complex.

Lastly: watch out for a free Guest Pass report we'll be publishing on event processing before Christmas, where we?ll attempt to unpick the different scenarios and technology requirements in play.

Labels: BEP, CEP, ibm, industry, SOA

posted by Neil Ward-Dutton @ 12:30 PM 0 comments permalink

Software AG goes in an interesting direction for SOA governance

As part of yesterday's release of the latest iteration of its webMethods Insight product Software AG announced an OEM partnership with Progress Software. This announcement adds the Actional runtime SOA management and monitoring technology (which Progress acquired back in January 2006) to Software AG's existing Centrasite design-time governance capabilities (which were bolstered by the acquisition of Infravio in September 2006) and the runtime policy enforcement provided by its webMethods X-Broker and partner Layer 7's XML Firewall.

The incorporation of runtime SOA management and monitoring functionality into Insight is a necessary evolution of Software AG-webMethods integration strategy that we commented on just over a year ago. It's long been our position that SOA is more than a standards-based approach to software development and integration. The business value of a service-oriented initiative depends on a recognition that software services are experienced, just like their real-world analogues. The quality of that experience depends on a governance approach that extends throughout the service lifecycle, where the contracts defined when services are designed are subsequently enforced through policies once they are deployed and running - and where runtime metrics are captured to provide insight into the service level quality that is actually exeprienced. Furthermore, those metrics can be used to inform and support change management processes, so closing the SOA lifecycle loop.

Whilst the announcement doesn't come as any great surprise, the source of the runtime management and monitoring functionality does. When Oracle confirmed it's intention to acquire BEA, I said:

It [the acquisition] leaves some of the other bigger specialist players - TIBCO, SoftwareAG (and to a lesser extent Progress and Red Hat) in an interesting position. On the one hand they will be more attractive, particularly for SOA and BPM, to customers looking for an application-independent infrastructure offering.

Software AG has gone to a potential competitor for the mantle of best-of-breed, specialist alternative to the likes of IBM, Microsoft and Oracle. If you had told me on Friday that Software AG was going to strike an OEM deal for SOA management and monitoring I'd have put my money on AmberPoint, which has historically been the OEM of choice for the likes of BEA and TIBCO.

I am not quite sure what to make of this decision. AmberPoint doesn't compete with Software AG directly and has established a healthy and growing customer base, as well as partnerships with some of the leading systems integrators - and a technology partnership with Software AG! Software AG's decision comes not long after Oracle's decision to drop AmberPoint. As we pointed out in our analysis of Oracle's roadmap for the BEA integration, we don't have any hard evidence for Oracle's claims that it had received negative feedback from BEA customers but it's something we will continue to explore. In light of the decision to go with Actional, it will be intriguing to see how the partnership evolves and how things pan out when Software AG and Progress are in a competitive situation.

This acquisition should be welcome news to Software AG customers that have invested in the company's SOA offerings as it will save them the time and effort of plugging the runtime governance gap that existed prior to the partnership. Those embarking on a significant SOA intiative should also give Software AG careful consideration, particularly if they are not wedded to one of the mega-platform providers.

Labels: AmberPoint, BEA, governance, ibm, Oracle, Progress, SOA, Software AG, TIBCO, webMethods

posted by Neil Macehiter @ 2:32 PM 6 comments permalink

BPM isn't yet "the killer app for SOA"

As part of our BPM Continuous Advisory Service, we're carrying out twice-yearly research studies which capture information about the state of maturity and practice of BPM in European businesses. We've finally finished analysing the results of our first study, and published the report.

One of the most interesting things for me in processing all the findings from the study was that for now at least, making too simplistic a connection between BPM and SOA looks like being a risky thing to do. Where BPM and SOA initiatives exist in organisations today, all the signs point to them being conducted in completely separate worlds, by completely separate teams with wildly differing sets of assumptions and expectations.

We've highlighted other key findings of the study over on our BPM blog, which is a key element of our BPM service but is freely accessible by anyone.

Labels: BPM, MWD, SOA

posted by Neil Ward-Dutton @ 11:00 AM 0 comments permalink

Businesses aren't machines, and enterprise architecture can't make them so

Via Service Oriented Enterprise, I recently picked up an Infoworld blog post by SOA journeyman David Linthicum, where he makes a couple of very strange points about SOA and ESBs. It may be, of course, that the post is pure link bait: certainly, David appears to have said some relatively sane things in the past, so that might be it. If it is link bait, I'm going to fall for it now.

In his piece David quotes a representative from iTKO, who repeats the now well-understood insight that in some large organisations, there's a challenge that arises because different groups with SOA initiatives find themselves having to integrate different ESBs in order to pull together implementations for cross-silo scenarios. He then goes on to comment (I?ve taken the liberty of extracting elements of his piece below):

First, if there is indeed 'enterprise architecture' and an 'enterprise architect' then the different divisions should not be using different ESBs, or even an ESB for that matter... Second, considering that my first point is correct (which it is), why the heck are you attempting to integrate these integration engines when they should perhaps be displaced altogether. Because, call me crazy again, that would be cheaper... just to be clear, you determine your requirements, and then the solutions patterns, and then the technology.

The most charitable explanation I can come up with for David's position is that he believes passionately in the transformational power of enterprise architecture. The problem with this perspective is that even where EA is highly effective (and in many places it isn't), it can at best only be one of many forces shaping the way that IT evolves to support changing business conditions and requirements, and each force has its own vector. Some forces, like a good EA team, try and combine business and technology focus, and promote the value of global optimisations, good practice and standardisation. But most of the most powerful forces are business forces, and in 99.99% of organisations, their power, when something really big and important happens, will trump any righteous splutterings emanating from IT departments.

As any experienced IT staffer who's been on the sharp end of a big business merger or acquisition, or even a radical change of leadership, will tell you, businesses don't act like machines that EAs can simply steer so that they tend towards technology optimisation. In fact, it's the opposite: business change forces (new competitors, new product launches, new market launches, new regulations, mergers and acquisitions, and so on) will always drive entropy, tending to push IT estates towards chaos. The best value an EA team can really provide in this environment is to help the IT organisation to absorb these changes with as little stress as possible, and drive consistent, planned responses.

Still, there's always been a vocal (and quite often technically brilliant) part of the IT industry that seems to misunderstand this, and persists in maintaining that once people see beautiful models, they'll modify their behaviour to enable the models to be made real. They see businesses as deterministic machines that can be algorithmically decomposed, analysed, predicted, and shaped. I can only think that David is one of these people.

Unfortunately, no matter how clever your EA team might be, no amount of architecture astronautics can make businesses change their nature.

Labels: EA, ESB, inside-out IT, SOA

posted by Neil Ward-Dutton @ 6:50 PM 2 comments permalink

Which comes first: process or service? Part 2

The question of how to combine BPM and SOA came up a lot here at TIBCO's TUCON user event - and, a little disappointingly, the standard response seems to typically revolve around reinventing the three-tier architecture of the 1990s, just with more scope and scale.

I pointed out in the previous part of this post a few ways in which that perspective is too short-sighted. It's OK to view BPM and SOA as both essentially technology approaches to building and integrating applications, but this is a perspective that misses a big part of the potential business value of the combination.

What we're starting to see, though, in a few advanced organisations, is how top-down, business-driven approaches to service orientation and business process thinking can combine with bottom-up, technology-driven approaches. The model that we see links an approach to business architecture that leans on the concepts of process and service to describe business fundamentals, with an approach to technology architecture that uses the same concept to describe the operation of automated systems. The same concepts are used at multiple levels of abstraction and composition/decomposition, so the link is seamless.

To make this link, the concepts of process and service are united through a third concept: outcome. The middle section of the diagram below, which outlines a process- and service-oriented view of business architecture, calls this out.

This is how it lays out:

• Outcomes are desired results. An outcome at the highest level is likely to be something concerned with the core value of the organisation, financial performance, etc. At this level, outcomes might link very straightforwardly to mission statements. At lower levels outcomes are going to be concerned with operational results - for example "product is delivered".


• Services are commitments to achieve outcomes.


• Processes are the methods through which outcomes are achieved.

One of the realisations that should come when you think about this approach is that service-oriented thinking can "drive" process thinking - but not only because technical process implementations can be wrapped with technical service interfaces, as I mentioned in the last post. More importantly, service-oriented thinking should drive process work because when you define business services (commitments to achieve outcomes) you're actually providing business context that shapes the KPIs and goals that you need to set for processes in BPM initiatives.

Another way to explain this aspect of service orientation is like this: when you model a process, and define a KPI and a target for that KPI, you're actually modelling aspects of a service "wrapper" for the process, as well as the process. You're defining what the commitment to achieve the outcome looks like, as well as the method you'll use to achieve the outcome. It's only when you start to think in terms of outcomes (and then services and processes) that this becomes clear, though.

There are other ways in which SOA and BPM can be intelligently combined to add value to both that aren't just about simplistic views of integration (and I'll try and get to some of those in future posts, watch this space) - but I think this is one of the most important. It's important because it helps people get their heads around a way of linking business architecture work with technical architecture work - with one consistent set of concepts. To date, there haven't been many ways to do this, and our research suggests that few organisations manage to make the link effectively today.

To come back to the post title: when you start to think about outcomes as a core concept in business and technology architecture, it becomes clear that it's not accurate to say either that services come first, or processes come first: the truth is that *outcomes* come first, and services and processes are two sides of the same coin in achieving the right results.

Labels: BPM, EA, SOA

posted by Neil Ward-Dutton @ 3:50 PM 1 comments permalink

Which comes first: process or service? Part 1

Over the years I've been helping to run MWD I've been to quite a few events, talked to many enterprise IT folks and talked to many tech vendors, too - and one of the topics that comes up most often is the relationship between BPM and SOA. There's been a bit of a run on the topic in the blogosphere lately. First, I was alerted to this post on CIO.com via the EDS fellows' blog in a post called SOA is a Business Process Architecture. At around the same time, I read The Problem with Process by Nick Malik (always good to read).

Too often, in presentations and papers, I see diagrams that replicate the old three-tier architecture of the '90s, but with a twist: instead of user interface, business logic and data access layers, now I keep seeing portal, BPM and SOA layers. Portals provide user interaction, and invoke processes; processes invoke services. Job done.

Looking at BPM and SOA purely in this way is short-sighted, disingenuous and dangerous. It looks at both initiatives through a very focused lens, and essentially says "BPM and SOA are about building and integrating applications". Not your father's applications I'll grant, but applications nonetheless - things with hard boundaries that have little awareness of other systems or resources that might exist, and little conceptual relationship to broader architectural or business considerations.

The first, and most obvious, point to make that blows this kind of picture apart is that it's mind-numbingly straightforward to wrap a process (or at least the initial invocation of a process) as a service. So *now* what's "on top" - process or service? This realisation has led to more enlightened commentators advocating that organisations consider processes and services more like a lasagne, with alternating layers of the two concepts. Services expose processes, processes call services, etc etc ad infinitum.

However even here we're only part way to the real answer, because this view also stems from seeing both BPM and SOA primarily from a "bottom up", software development- and integration-centric perspective. Of course, many people refute a view like this, and point out that "BPM is a business management approach, and SOA is a technology architecture approach - you can't lump them together so easily". In other (very simplistic) words BPM is a top-down, business-driven initiative, whereas SOA is a bottom-up, technology-driven initiative.

The two articles I've linked to above are great because they start to take a deeper look at the architectural and conceptual relationship between service and process. Both EDS Fellow Fred Cummins and Nick Malik start to pick away at the simplistic views that seem to hold sway in the minds of many at the moment.

Neither quite take the argument as far as they might, though. Through our industry research (for our book as well as our various free reports and consulting gigs), we've come to see that the value of both BPM and SOA comes from considering them neither purely as bottom-up initiatives focused on improving the development / integration of applications and resources, nor as top-down initiatives focused on exploring and elaborating business architecture. We already have a champion in the case of "top-down SOA" - the OASIS SOA Adoption Blueprints team. The real insights come, I think, when you see how top-down *and* bottom-up perspectives of service-orientation *and* process-orientation can all come together in a more holistic view of business and technology architecture.

I'll expand on what such a view entails in a Part 2 of this post, in the next few days.

Labels: BPM, EA, SOA

posted by Neil Ward-Dutton @ 9:55 AM 0 comments permalink

HP tightens up its SOA governance proposition

Labels: governance, HP, service, SOA, UDDI

posted by Neil Macehiter @ 9:38 AM 0 comments permalink

SOA's five benefits in one picture

In recent discussions with one customer, I ended up drawing a series of little pictures to try and summarise the five potential benefits that can come from pursuing SOA. It seemed to work for them, so I thought I'd reproduce it here and see what our readers think.

In order to test the old adage that a picture speaks 1,000 words, I'm not going to write a whole lot to explain what the diagram is showing: if it's a good diagram then you should be able to work that out pretty quickly. Of course, if I get an avalanche of comments asking for an explanation then (1) of course, I'll post one; and (2) it's not as cute a diagram as I thought it was!

UPDATE: I managed to cut off the bottom of the original image when I uploaded it, so here's another one.

Labels: SOA

posted by Neil Ward-Dutton @ 4:46 PM 1 comments permalink

Please don't hire a VP of SOA

This might sound like an odd title for a post, but I was prompted by this ZapThink ZapFlash, via the ever-watchful Todd Biske.

The ZapThink note starts off talking about the challenges in SOA adoption that come from organisational issues - specifically, challenges that arise from situations where tactical decisions continue to trump strategic decisions. All these are good points well made (FWIW, when trying to educate people about the importance of enterprise architecture, governance, BPM and SOA, we talk about the strategic importance of global vs local business optimisations).

However the note all goes wrong when it goes on to say:

Among the various approaches organizations take to overcome such obstacles, one technique is increasing dramatically in popularity: bringing on board a new executive responsible for the enterprise's SOA initiatives.

Really? I haven't seen that. If some organisations (maybe US based ones?) are doing this, I hope they're more focused on business transformation than on IT implementation - and that SOA isn't in their job title.

The note then goes on to suggest some ideal characteristics for such an executive:

The ideal candidate will first and foremost be a business process guru who also has broad experience in IT. Must have a background in architecture and ten-plus years in increasingly senior management roles. Must be able to communicate to both business and technical audiences. The successful candidate will be part team builder, part evangelist, and part bean counter.

Although it's slightly off the topic of this post, I wonder how many people fitting this description are out there? If SOA success relies on you hiring someone with all these capabilities, then I think we're going to see a hell of a lot of failures. This is where I get seriously worried though:

This position reports directly to the CIO with dotted line responsibility to the COO, and will be responsible for a seven-figure annual budget... job responsibilities include:

• Provide executive-level management leadership to all architecture efforts across the enterprise. The directors of Business Architecture, Enterprise Architecture, Technical Architecture, Data Architecture, and Network Architecture will all be your direct reports.


• Drive all Business Process Management (BPM) initiatives enterprisewide. Coordinate with process specialists across all lines of business, and drive architectural approaches to business process.

Ummmm... so this role reports to the CIO, but drives all BPM efforts across the company? Even though the note says "even though the VP of SOA reports to the CIO, the role is primarily a business role" this is pure fantasy. Unless you're in a small-to-medium business it's just not practical to make this happen.

Think of a concrete example of a process transformation - something related to CRM. There's a wealth of salutory tales out there about the folly of driving CRM initiatives (which are process improvement/transformation initiatives) from within IT: CRM initiatives need to be owned and driven by the business. Extrapolating out to process improvement/transformation more broadly, even if transformation of some process areas isn't in the same league as that involved in CRM (and you'd have to argue hard to convince me of that) then I'd still argue that business leaders have to share ownership of process improvement and transformation initiatives. Real BPM cannot be driven by someone reporting to your average CIO - not even by a $200k-a-year uber-architect-cum-process-guru who's equally happy wearing patent leather shoes or pizza-stained trainers.

Of course there is a need to bring people together to push through significant IT and business transformations, such as those required to make the most of the promise of SOA and BPM initiatives. I would wholeheartedly back ZapThink and others in that. But in the real world - particularly in large organisations - people who drive these change programs aren't able to directly push everything, as ZapThink seems to be advocating: they have to be influencers and coordinators first and foremost. Think of an enterprise architect you know. If they're successful, chances are one of their key skills is in how they influence others' behaviour and get different stakeholders working together.

Even if you believe that one role can drive all this - especially from within the IT organisation - then your VP of SOA will be a transitory role. If your SOA initiative succeeds in its mission, then SOA becomes part of the furniture, and when that happens, roles like this one melt into the responsibilities of other, "business-as-usual" roles. If your SOA initiative doesn't succeed, then SOA is seen as yet another over-hyped industry silver bullet - and your $200k-plus hire is now seen as an expensive mistake.

Labels: BPM, SOA, toddbiske

posted by Neil Ward-Dutton @ 4:50 PM 0 comments permalink

Who do you put in a Centre of Excellence?

I've lost count of the number of times I've seen throwaway comments exhorting companies to "create a centre of excellence (CoE" (mostly, for initiatives like SOA or BPM). Vendor / pundit / analyst / journalist: "Having trouble? Establish a centre of excellence!" Customer: "Oh, that's OK then, I'll do that."

But let's take a deeper look. Quite aside from the role that one of these beasts plays (something I'll attack in a future post), what does a best practice CoE look like?

From what I've seen out there in real-world implementations of SOA and BPM initiatives, I suspect that the best results come from having a good mix of responsibilities / personalities in the group. Something like an even distribution across this matrix of perspectives:

Although it's tempting to staff a CoE with good dependable technical people that you understand, you need a good mix of business-focused types and technology-focused types, because those business-focused types will help keep expectations practical, and help keep business people from outside the CoE engaged and willing to help. And it's vital to get a good mix of practical and visionary focus, because rolling out new concepts and approaches to delivering IT capabilities requires both "selling" and getting things done.

That's my view. What do you think?

Labels: BPM, governance, SOA

posted by Neil Ward-Dutton @ 8:20 PM 1 comments permalink

Oracle proposes to buy BEA

Oracle today confirmed

that it delivered a letter to the Board of Directors of BEA Systems, Inc. (NASDAQ: BEAS) on October 9 in which Oracle proposes to acquire BEA for $17.00 per share in cash. The $17.00 per share offer is a 25% premium over yesterday's closing price of $13.62.

This acquisition has been long-discussed so I can't say I find the news particularly surprising, particularly with Carl Icahn recently upping his stake in the company. I think this just makes it more likely that Oracle's proposal will be accepted.

This is primarily as a market share grab by Oracle. It does plug some gaps in the portfolio - particularly around business process management (based on BEA's Fuego acquisition), where Oracle only has basic BPEL web services orchestration; adds some telecoms vertical market capabilities to complement Oracle's vertical market push and the virtualisation work that BEA has done with the WebLogic Virtual Server Edition. Also, there's the opportunity for Oracle to tap into the healthy Tuxedo base. With a significant chunk of Oracle's profitability coming from maintenance, the revenue from BEA's customer base will suit its business far better than it did BEA which was suffering with its inability to grow license revenues.

This is yet another example of the bigger specialist players getting squeezed out by the industry goliaths - IBM, Microsoft, Oracle, SAP - and the open source, smaller best-of-breed players. SAP's recent acquisition of Business Objects is another example (although that did plug a few more gaps). It leaves some of the other bigger specialist players - TIBCO, SoftwareAG (and to a lesser extent Progress and Red Hat) in an interesting position. On the one hand they will be more attractive, particularly for SOA and BPM, to customers looking for an application-independent infrastructure offering. On the other, though, taking market share for those customers from BEA is one thing: taking it from Oracle quite another. Ultimately, IBM is the big beneficiary in this regard.

In summary, then, I see: the acquisition going ahead; BEA's customers looking worried as they see themselves with an application-dependent infrastructure stack; IBM looking happy at the prospect of providing those customers with an application-independent alternative; the likes of TIBCO and Software AG pondering their options; and SAP and Microsoft carrying on in there own sweet way.

Labels: BEA, BPM, ibm, Microsoft, Oracle, Progress, Red Hat, SAP, SOA, Software AG, TIBCO

posted by Neil Macehiter @ 4:57 PM 2 comments permalink

More big vs small thinking: SOA vs BPM

I was on the way back to the office from a briefing with BPM vendor Pegasystems yesterday, where we'd had an interesting discussion about the relative roles that BPM and SOA can play in business transformation for customers.

We agreed that BPM, done right, is as much of a discipline that organisations can use to transform the way that they do business, as it is a technology (or set of technologies). What's more, we observed that many BPM initiatives revolve around making big changes to the way that organisations deal with customers, partners and suppliers - and creating organisations that are really focused around delivering real service to those other parties.

Let's review that for a second: BPM thinking helps organisations get their houses in order so that they can deliver coherent and quality services (and not just disjointed experiences).

So from a business perspective, it's BPM that delivers service-orientation!

But wait - in IT we're saying that service-orientation comes from something called SOA. And in IT circles, there's a lot of discussion that positions BPM as if it's a layer of technology that sits on top of SOA technology. Kind of like we've just reinvented three-tier application design with BPM instead of business logic, and SOA instead of database access logic.

This "application architecture" lens for BPM and SOA is all wrong. It's another example of small thinking.

The more profitable way of thinking about the relationship between BPM and SOA is not to think of them as a stack of technologies: it's to think of BPM as being about "how" you do things and service-orientation being about "what" you do. Service definitions are definitions of commitments: they say what you will do, how well you can do it, and (possibly) how much it will cost you to use. Process definitions are definitions of work: they say how commitments will be fulfilled, by whom, and under what conditions.

So, BPM and SOA are interlinked - but not because one adds value to the other or because one sits on top of the other: both ideas are two sides of the same "business and IT transformation" coin.

If you do insist on thinking of the world in terms of stacks of technology tools or stacks of models or assets, think of SOA and BPM as alternating layers of concepts and practices. Services define "what" you will do at a given level; processes define "how" services will be delivered. Processes rely on foundation services to get some work done; those services in turn rely on processes of some kind.

I'm sure not everyone will agree with this, but I do hope there's one thing we can all agree on - as Sinatra once sang: "this, I tell you brother: you can't have one without the other."

Labels: architecture, BPM, Pegasystems, SOA

posted by Neil Ward-Dutton @ 3:25 PM 1 comments permalink

Early praise for SOA strategy planning tool

Just saw this great comment from Enterprise Architect blogger James McGovern.

This site is incredibly useful. It was developed by MWD so you know it is of high quality.

Thanks so much James - what an endorsement!

If you want to get to grips with your competency levels in the context of SOA, and learn how you can improve them and plan a SOA strategy, visit http://www.itstrategyplans.com/soa.

Labels: MWD, SOA, strategy planning

posted by Neil Ward-Dutton @ 11:18 AM 0 comments permalink

New SOA strategy planning tool - live from today

Today we're delighted to announce that our SOA strategy planning tool, hosted at www.itstrategyplans.com/soa, is now live. This is a new thing for us and if you're interested in SOA, we think you'll find it useful.

What we've launched is a comprehensive, interactive, and vendor-independent online tool that goes much further than IT vendors' own online SOA assessment tools - and crucially, there's nothing behind it that is going to steer you towards buying tools or technologies from a particular company.

You can use the tool, free of charge, to determine your levels of competence in six areas that are crucial to SOA success: concepts, strategy, architecture, organisation and people, governance, and technology and infrastructure. On top of that, you can quickly see how your levels of competence compare to average industry benchmark scores.

As I said above, access to these features costs nothing.

If you want to know how your competence levels affect your SOA strategy and how you can reduce the costs and risks associated with SOA, then for a modest fee (normally £200 - but if you purchase before the end of September 2007, it's £130) you can gain access to premium features. With premium access you can generate personalised action planning reports that clearly lay out actions you need to take in order to improve your competency levels - as well as providing more detailed breakdowns of how you compare to the benchmark averages (across your industry, geography and company size).

You can use premium access to generate as many reports as you like, whenever you like. So as your SOA programme progresses and you gain more experience and capability, you can revisit the tool and gain new insights.

It's been a long, long road, and we (and our partner JEMM Research) have put in months of effort to get here since we started work on this project back in November 2006. This has been a major investment for us - so why have we done it?

There are two reasons. Firstly, of course, we'll be delighted when people decide they want to buy premium access to the tool. That's easy. Just as importantly, though, we wanted to try and find new ways of learning about how companies are pursuing key IT initiatives. By building tools like this one and inviting people to use them to learn about their competence levels and how they can improve, we're also able to look at aggregated usage data from the tool to see how industries are grappling with SOA, and how the situation is changing over time.

For a group of companies that talks so much about technology innovation, we've found that the IT industry analyst community is generally pretty slow at trying to use technology in new ways. As a small company we can perhaps make bolder moves more quickly - and that's what we're trying to do here. If we see success with this tool, we'll be launching more over time, focusing on different types of IT initiative.

So - please check out the tool, give us your feedback, and let us know if there are other areas you think we should apply this approach to!

Labels: MWD, SOA, strategy planning

posted by Neil Ward-Dutton @ 5:22 PM 0 comments permalink

Microsoft's Dynamic IT: it's a start

Labels: HP, ibm, Microsoft, Oracle, SOA

posted by Neil Macehiter @ 10:22 AM 0 comments permalink

Microsoft server and tools is now part of the business division

Labels: BPM, compliance, ITSM, Microsoft, SOA

posted by Neil Macehiter @ 5:19 PM permalink

New online SOA strategy planning tool - we need beta testers!

Over the past six months we've been busy working behind the scenes with a partner, JEMM Research, on a pretty nifty online SOA strategy planning tool. We're getting near to the point of launch, and we'd really like to find some people out there who would be interested in participating in a two-week beta test phase. Are you interested?

The heart of the tool is a self-assessment model that looks at six dimensions of competence/readiness: concepts, strategy, architecture, organisation & people, governance, and technology infrastructure. You can see a small screen-shot of the (pre-beta) tool below.



Now we know there are a number of vendor-specific SOA assessment/readiness tools out there, but this one is a little different, for a couple of reasons:

1. It's vendor-neutral - it's not designed to steer you towards buying a vendor's product. It's not sponsored in any way.


2. It's more detailed - the assessment at the heart of the tool has around 150 questions, and is designed to really make you think about how well you're implementing (or able to implement) a SOA initiative that has real business value.


3. It provides some pretty cool feedback. The self-assessment, which is free of charge to use, provides you with overall scores in the six dimensions. You can go back to the tool as often as you like, and rerun the calculations to see how you've improved.


4. If you want, you can pay a modest fee to gain "premium access" to the tool. With premium access the tool will generate, based on your self-assessment answers, a detailed and personalised SOA strategy action plan report. The report benchmarks your organisation against others of the same size, in the same geography and in the same industry as you. And most importantly, the report also contains a detailed set of actions you can take to improve your critical SOA related competencies. With premium access you can run the report as often as you like, whenever you like - so you can track your progress as you make your SOA journey.

If you're interested in being a beta tester, please drop us a line. We're currently planning on running the beta from May 14th to May 25th.

Labels: MWD, SOA, strategy planning

posted by Neil Ward-Dutton @ 12:48 PM permalink

Policy interoperability - a step in the right direction

Labels: governance, HP, Layer7, policy, SOA, UDDI, webMethods

posted by Neil Macehiter @ 8:47 AM 0 comments permalink

MWD FM SOA interview: TIBCO

We're nearing the end (for now - we have more planned, but not for a little while) of a series of SOA vendor interview podcasts with this one, which we conducted recently with Rob Myer of TIBCO. Rob works in Product Management at TIBCO with responsibility for SOA.

We ask the usual four questions, and along the way swing by some interesting conversation points:

• What you need from infrastructure in order to move towards enterprise-wide SOA, and what TIBCO learned from telecoms companies' service platform requirements


• The challenges associated with the WS-Policy, WS-Management and WSDM standards


• The application of CEP (complex event processing) technology to managed service delivery in the context of SLAs.

This podcast episode is 34'28" long. The podcast episode lasts 25'34". You can download the audio here or you can subscribe to the feed.

Labels: MWD, podcast, SOA, TIBCO

posted by Neil Ward-Dutton @ 9:24 AM 0 comments permalink

Little SOA vs Big SOA

During our "off air" preamble with Miko Matsumura prior to recording our podcast earlier this week, he mentioned that he likes MWD's focus on "Big SOA".

I'd never thought of what we tell customers about SOA in this way before, but it's true that we try and get people thinking about how SOA can help them transform their organisations in the long term by pushing the boundaries of SOA and not sticking with an overly-technical, bottom-up approach that sees SOA as "EAI with standards". So Miko got me thinking about what, precisely, "Big SOA" might be and how it might be different from something that for the sake of argument we might call "little SOA".

So I came up with a picture:

As the picture shows, I think the difference between "little" and "big" SOA comes down to how you look at the "S" and the "A" of SOA.

In "little SOA" (bottom left of the picture), organisations have a software development centric view of what a "service" is. A service is basically a standalone software component with some kind of remotely addressable invocation interface (let's say a WS-* interface for now). In "little SOA" organisations have a similarly narrow view of what "architecture" is - architecture is basically software design with a corner office.

In "big SOA" (top right of the picture), organisations have a much broader view of what a "service" is. In this broader view a service isn't something you build; it's something that is experienced by a consumer. This view, therefore, is really about realising that delivering a service requires all sorts of IT competencies (design, development, integration, testing, deployment, admin, ops and change management) to be integrated together across the lifecycle of an investment that is packaged up as a service. That's a very different perspective on "service" that is not coincidentally much closer to what a business exec would think of if you asked them what a "service" is. In big SOA organisations also have a broader view of what architecture is about. Architecture isn't an inwardly-focused IT competence that seeks to make global optimisations to portfolios of software development projects. It's an outwardly-focused business-IT communication competence that seeks to further understanding of IT within business, and of business within IT.

I don't have hard quantitative data to back this up, but based on anecdotal evidence of customer conversations my feeling is that the majority of companies considering or starting out with SOA are doing "little SOA".

What do you think?

Labels: architecture, service, SOA

posted by Neil Ward-Dutton @ 2:45 PM 4 comments permalink

MWD FM SOA interview: webMethods

Here's another in our series of interviews with vendors offering SOA "solutions". This time we spoke to Miko Matsumura, head of product marketing for SOA at webMethods.

In this conversation we ask the usual four questions - and also chat about the importance of webMethods' SOA Link program, the role of the Infravio Governance Rules Engine, and SOA as an enabler of federated/cross enterprise business processes.

The podcast episode lasts 25'34". You can download the audio here or you can subscribe to the feed.

Labels: MWD, podcast, SOA, webMethods

posted by Neil Ward-Dutton @ 12:16 PM 0 comments permalink

MWD FM SOA interview: Microsoft

Here's the fourth in our series of interviews with vendors offering SOA related products and services. This time it's the turn of Kris Horrocks, who's a Technical Product Manager in the Connected Systems Division of Microsoft. (The Connected Systems Division was formed in 2005 as part of the Server and Tools business, and it brings together work on .NET, BizTalk, CardSpace and other related things).

As usual we talk through our standard four questions. In the resulting conversation we explore:

• how Microsoft deals with customers' questions about scalability and interoperability


• the importance of "high fidelity handoffs" between IT practices in quality service delivery


• how the SOA offering fits with Microsoft's Dynamic Systems Initiative (DSI) and support for "design for operations", and what this means for managing service lifecycles.

The podcast episode lasts 34'41". You can download the audio here or you can subscribe to the feed.

Labels: Microsoft, MWD, podcast, SOA

posted by Neil Ward-Dutton @ 4:33 PM 0 comments permalink

MWD FM SOA interview: HP

Here's the third in our series of interviews with SOA vendors. This week it's the turn of Roman Stanek - one of the founders of Systinet, which was bought by Mercury (which was then in turn bought by HP a few months back).

The 31'17" interview has some great stuff in it. As we ask our usual four questions about HP and Systinet SOA offerings, we swing past:

• scenarios where the standardisation and interoperability that SOA introduces are particularly important
• how SOA is about outcomes, not protocols (with reference to the SOAP vs REST debate)
• how SOA wil disappear from the IT industry's lexicon in the coming years, because it will become a standard feature of the IT landscape
• the effect that SOA has on the software development lifecycle, and how the loose coupling that it introduces into development organisations and processes brings requirements for strong management of service lifecycles and service quality.

You can download the audio here or you can subscribe to the feed.

Labels: HP, MWD, podcast, SOA

posted by Neil Ward-Dutton @ 10:47 AM 0 comments permalink

webMethods becomes SWAG

Apologies for the poor pun, it's the evening before the Easter break (public holidays on Friday and Monday here in the UK) and I'm feeling festive.

So, it's finally happened. For those of us with the dubious title of "industry watcher" it's not a particular surprise to see webMethods acquired. We'd heard rumours for some time that the company might be readying itself financially and organisationally for a sale. Clearly the 25% price premium that Software AG (SWAG) plans to pay in purchasing the company at $9.15 a share represents a good deal for its shareholders, as webMethods' stock had got stuck around $7 for quite some months.

But what is a surprise (to me at least) is the buyer.

For one thing, we're used to big industry whales swallowing small specialist minnows: this is one medium-sized software company buying another medium-sized software company. SWAG's 2006 full-year revenue totalled in the region of $500m$650m. webMethods has a different financial year to SWAG, but its revenue in the equivalent period totalled roughly $200m.

There's one particular feature of the two companies' income statements which is shared: both companies make more money from software maintenance than they do from selling new product licenses. Both companies are living more off their past success than their current success (although the balance seems as if it could be tipping the right way in SWAG's case).

Software AG is a company with a long history as a middleware company, but it's not a glorious one. It scored huge success in the 1970s with the ADABAS DBMS and 1980s with the Natural language and development environment, but none of its more recent infrastructure product developments - EntireX (object and message broker), Tamino (XML database/integration platform) and Bolero (Java app server) have penetrated far beyond its established loyal customer base. What's interesting, though, is that with new SWAG products Centrasite (SOA registry/repository) and Crossvision (BPM, ESB, legacy integration, composite application development) now on the market, the webMethods acquisition certainly doesn't look like one to be based around technology. There's a hell of a lot of technology overlap.

So this acquisition appears to be more about acquisition of customers and "mindshare" in complementary regions. Despite a US subsidiary with a long history (which started off independently in 1972, was bought by its parent in 1988, then spun off through a venture buy-out in 1997, and re-purchased in 2001), SWAG's visibility in North America is not high (any of our US readers heard of EntireX, Tamino or Bolero?). Despite webMethods' reorganisations which have cut back the size of its European operations over the past years, it seems to retain very high recognition in North America.

What will I be watching? I'll be watching to see what happens to products and people as the two companies come together. Given the huge portfolio overlap, from a product and technology - and ultimately a customer - standpoint, this will be a difficult integration to pull off.

UPDATE: John Conley, webMethods' Director of PR, got in touch with me to point out a couple of errors in the original post, which I've changed above.

Labels: acquisition, SOA, Software AG, webMethods

posted by Neil Ward-Dutton @ 8:15 PM 0 comments permalink

MWD FM SOA interview: Martin Percival, BEA

Our second SOA vendor interview was with BEA's Martin Percival yesterday. Again we followed our standard format - and in the resulting 34'30" podcast we get into discussing:

• BEA's experience of delivering "information as a service" projects within SOA initiatives
• How SOA is about more than just WS-* technology
• BEA's transition from a pure Java implementation focus to a broader focus, re-embracing its "legacy" middleware platform Tuxedo, the Microsoft expertise of its Plumtree acquisition, and also pointing to the SCA/SDO effort that it's a member of
• Why it bought Flashline (a software development repository vendor) and didn't buy a SOA registry vendor (it partners with Systinet/Mercury/HP)
• How its SOA 360 initiative will impact on the admin & management of the BEA platform.

You can download the audio here or you can subscribe to the feed.

Labels: BEA, MWD, podcast, SOA

posted by Neil Ward-Dutton @ 8:53 AM 0 comments permalink

The SOA tool pyramid

I've had a bit of a graphic spurt (as it were) and so here's another blog post based around a diagram.

I was talking to a journalist a couple of weeks back about the kinds of functionality that customers need to look for when looking for tooling for SOA initiatives, and which vendors provide which groups of functionality. It's not always easy to explain this kind of thing over the phone, so I thought I'd have a go at describing the main areas of functionality as a pyramid. Something like this:

In our assessments of SOA tool vendors' capabilities (see here for an example) we highlight nine separate areas of functionality, but this is a simpler picture that just focuses on four:

• Service enablement - this is functionality that helps you take existing IT assets (applications, databases, etc) and create service interfaces based on the capabilities they offer. A lot of vendors provide facilities in this area because in truth most of them started out as integration tools vendors.
• Orchestration and composition - this is functionality that helps you aggregate services and create "composite services" or "processes". Most vendors offer some capability along these lines, and most involve the ill-named "BPEL" in some way (but that's another story). The reason is the same as the reason above: many of the SOA tooling vendors had "pre-SOA" offerings which allowed you to aggregate and orchestrate resources from existing applications and systems.
• Lifecycle management - this is all about supporting development, integration and operations teams in linking their efforts to ensure that the consumer service experience is high-quality and consistent under potentially unpredictable circumstances. Typically the foundation of this capability is some kind of registry/repository, but ideally tools go further than this - firstly by helping to automate team workflows for implementing quality controls at design time; and secondly by helping to translate design intentions relating to operational SLAs into runtime policies which are tied into the infrastructure. Some vendors are starting to offer capabilities in this area, through acquisition (HP/Mercury/Systinet, webMethods/Infravio, BEA/Flashline (kind of)); OEM/resale agreements (Oracle/Systinet, BEA/Systinet) or in-house development (IBM, Sun).
• Service development - this is about the ability to design services "from scratch", or to design services where any existing applications/systems offer functionality which only partially fulfils a requirement. Ideally this starts "contract first" - first of all documenting what the service needs to do and the commitments the provider should make to service consumers; and only then refining that spec into a working service implementation and interface.

Most SOA tools vendors suck at this last bit, frankly. I think that TIBCO is starting to do provide some interesting supporting facilities for this broad area with ActiveMatrix, and as vendors start to implement SCA/SDO in their tools the situation might get better across the board. In the meantime if you've heard of a vendor targeting SOA specifically that really provides solid tools to help with this kind of contract first" development approach, I'd love to know.

Labels: BPEL, contracts, development, SCA, SOA, tools

posted by Neil Ward-Dutton @ 9:19 PM 0 comments permalink

BEA announces strategic partnerhsip with CA: but where does that leave AquaLogic Enterprise Security?

BEA today announced a stategic partnership with CA, which will see the latter's access and identity management solutions (SiteMinder and Identity Manager) integrated with the former's WebLogic and AquaLogic application and service infrastructure platforms.

I agree completely with Wai Wong's (BEA's executive vice president of products) statement in the press release that

Identity and Access Management is critical within SOA

not least because we have said as much in our service infrastructure assessment model and our report on identity management.

Despite this agreement, I am still left a tad confused by this partnership as it is far from clear what this means for AquaLogic Enterprise Security (ALES), which BEA describes as

a fine-grained entitlement management solution that combines centralized policy management with distributed policy decision-making and enforcement. This combination provides management and control of your critical applications

How will SiteMinder integrate with ALES? Will ALES continue to integrate with other identity and access management solutions? Does BEA plan to provide a common policy definition and enforcement framework across ALES and SiteMinder?

We point out in our assessment of BEA's service infrastructure offerings that there are some important gaps when it comes to security and identity management, which explains why BEA felt the need to establish this partnership. However, as well as answering a number of questions from potential adopters, this partnership is going to raise a few more for existing customers with an investment in ALES. I for one look forward to learning more about the two companies' plans to

validate and further extend integration between CA SiteMinder and BEA WebLogic and AquaLogic technologies

Labels: BEA, CA, identity, SOA

posted by Neil Macehiter @ 5:30 PM 3 comments permalink

First MWD FM SOA interview: David Clarke, Cape Clear

We interviewed David yesterday and asked him our standard SOA vendor questions. Considering it was the first interview, we think it went OK...

There were a couple of interesting things to come out of the interview:

• Cape Clear markets itself as an ESB vendor, but its view of what is "in" an ESB is much broader than that of most other vendors - David in particular calls out BPEL-based service orchestration


• the sweet spot for the company is really a "mainstream", "mid-market" company which may not have much in the way of deep in-house web services or SOA technology skills


• the company is currently working quite a lot in software-as-a-service (SaaS) and other commercial service delivery scenarios - helping companies in the digital service delivery business create more sophisticated and valuable services


• David also specifically calls out the need for potential SOA "customers" to make clear distinctions between management of SOA-related technology, and management of the automated processes supported by that technology. These are two separate problems with different solution needs, and they should be evaluated as such


• it's pretty obvious from the conversation, we think, that Cape Clear is very firmly a technology company selling its capability as a standards-based middleware solution to integration problems. This makes it quite different from many of the other SOA players, which position themselves almost as business change agents.

The interview lasts 32'55". The audio file containing the interview with David Clarke is here - or alternatively you can subscribe to the podcast feed.

Labels: Cape Clear, MWD, podcast, SOA

posted by Neil Ward-Dutton @ 10:47 AM 0 comments permalink

MWD FM kicks off interviews with SOA vendors

We let our podcasting get a little bit behind (well actually 3 months or so behind...) recently, and so we're trying to make up for it now. We have two initiatives underway, and this is the first (you'll find about the second one soon).

We've just recorded the first of a series of interviews with senior representatives from IT vendors working to provide SOA-related offerings - the first is with David Clarke, EVP of Products at Cape Clear.

The aim is to have a go at using the podcast medium to create a set of interviews which people considering or embarking on SOA initiatives can use as an additional reference in their selection process - kind of like an (albeit high level) set of comparable assessments in audio format... if that makes sense!

In order to make these interviews as useful as possible we're doing three particular things.

First, these are not sponsored podcasts. No-one is paying anyone for taking part in these interviews.

Second, we're keeping each interview to around 30 minutes, so everyone gets the same amount of time to talk about what they're doing.

Third, we're asking each interviewee to address the same four questions:
1) What are the commonly-occurring customer scenarios where your products and services get used?
2) How do you ensure that you "play well" with customers' existing investments - and where is this particularly important in your experience?
3) What are the main challenges that you help customers with - are you primarily helping them with service design, creation, assembly, management, etc?
4) How do you ensure that your technology is easy to deploy and manage?

Obviously there are many more questions we could ask, but given the level of confusion and FUD that currently surrounds SOA and related technologies we think these ones are pretty useful.

We've got a couple of other interviews lined up for the coming weeks, but if you've got particular vendors or groups that you'd like us to interview, please let us know!

Labels: MWD, podcast, SOA

posted by Neil Ward-Dutton @ 10:27 AM 0 comments permalink

TIBCO's ActiveMatrix and 4GL for SOA

We've been meaning to blog about TIBCO's ActiveMatrix product family for a few weeks now, since we were briefed on it in December. However it took a while for us to get to a point where we felt we had something particularly "aha" to say about it.

What makes ActiveMatrix so potentially valuable is also what makes it a challenge. As TIBCOer Rourke McNamara said in a blog entry from Gartner's Application Integration and Web Services Summit: "Explaining what ActiveMatrix does in 90 seconds isn?t easy." Actually explaining it's value is more challenging, because it's not what you might expect from TIBCO.

TIBCO talks about "service virtualisation" and describes the core element of ActiveMatrix - the ServiceGrid product - as a "network of service containers" with "embedded policy management for service governance" and "JBI and SCA support for service deployment and provisioning".

These things are all true, and they're all cool.

But what does that really *mean*, in really straightforward terms? After talking to numerous journalists and clients about ActiveMatrix over the past weeks, we've evolved our introductory explanation to this one, when talking to non-expert IT audiences:

Back in the day, a big challenge faced by large organisations trying to develop and deploy systems was how to create systems relatively quickly, that were quick to change, in complex multiplatform environments. In those days the multiplatform issues was one of 16- and 32-bit Windows, Mac and OS/2 clients; Solaris, AIX, HP-UX, OpenVMS servers; and various kinds of databases and network protocols (this was when IPX/SPX and SNA were as interesting as TCP/IP).

A whole range of "second generation 4GL" environments were created which aimed to help. They married a high-level abstract programming language (the 4GL) to a virtual machine which was ported to all the major platforms and which hid the details of the operating environment - client, server, database, network.

Now let's fast-forward to today. Java and JEE and web-based application designs helped with some of that old multiplatform nightmare, but it's not the only game in town. What's more, now we're looking at SOA, the challenge today isn't so much about building discrete systems; it's about finding a consistent way to build and integrate system elements that can talk to each other within various parts of our companies, and also across different companies and whole supply chains.

In short we have another version of the same problem, only this time the multiplatform aspect is at a higher level - and the scope and scale of the operational environment are much bigger. Now we struggle to get "plain Java", JEE, .NET and other modern application containers to talk to each other, as well as other "legacy" application platforms - and Web Services protocols and standards are only a partial answer here at best. Even if protocol interoperability gets working better we're now looking at an environment which is widely distributed, and which may not be 100% under any one party's control.

So if we're to try and address today's challenge for SOA - to enable people building and integrating services to just concentrate on the core logic of what they're doing rather than getting stuck in the weeds of environment-specifics - we need something like the old 4GL virtual machine, but with lots of extra capabilities - like service provisioning (think about telecoms service provisioning if you know that industry) , and policy-based operational and lifecycle management of services.

That's what ActiveMatrix does. It provides a virtual machine environment for the SOA age.

As an aside: Why is ESB not the answer, and why is ActiveMatrix not an ESB? Because ESBs concentrate on solving just one part of this conundrum - the operational communication between services. From the developer's perspective the ESB is pretty much invisible (intentionally), and doesn't offer any "virtual machine" facilities to that audience. ActiveMatrix works on top of, around, and underneath ESBs.

Of course there's a proprietary element - although ActiveMatrix is built around JBI / JSR 208 and elements of the SCA specification, the container design itself is unique to ActiveMatrix. Interoperability is standards-based through JBI, but of course interoperability with something else is not the same as "swappability".

So here's a couple of interesting questions. Do companies pursuing SOA know they need this, and does TIBCO have the credibility to provide it? And is there enough momentum in the SOA movement to get the mainstream of companies to the point where they need ActiveMatrix, or something like it?

The credibility question is a good one, because it certainly took me a while to work out what TIBCO was doing with ActiveMatrix. I saw it completely as an integration vendor moving into SOA, and was completely unprepared for what TIBCO was trying to tell me - for a while it just didn't compute. We're going to need some excellent customer examples, well-explained, to really help people get in the right frame of mind to hear what TIBCO's trying to say here.

The second question is concerning to me as a student of IT industry history. The truth is that the history of the industry is littered with examples of situations where "movements" were cynically abandoned by vendors in favour of newer, shinier things, just as they got to a tipping point. If you can dupe your customers well enough, you can sell them A, wait until they get familiar with it, and then tell them that A is last year's model, and sell them the virtue of A'. Example: the demoralising arrival of SOA 2.0.

Many say that SOA itself is one of these cynical reinventions, and there is a grain of truth in that (but only a grain, I'd say). I believe that SOA has real value - and that in the context of achieving that value companies will need facilities like they can get from ActiveMatrix - but my concern is that we'll get derailed by the "next big thing" before we can all realise that value.

Lastly, a disclaimer: TIBCO is an occasional client and we provided some input to help them refine their ActiveMatrix communications.

Labels: 4GL redux, SOA, TIBCO

posted by Neil Ward-Dutton @ 12:35 PM 1 comments permalink

Defeating versionitis: making things better in small ways

James Governor forwarded me a note from Duane Nickull the other day pointing to the Wikipedia entry on SOA. It seems like our anti SOA 2.0 petition - in a small way - helped to make the IT industry a less bonkers place.

We got to 480-odd "real signatories" (there were, unsurprisingly given my very hasty scripting of the petition page, a few spambot-generated entries too) and that was a hell of a lot more than I was expecting.

As a consequence the "SOA 2.0 or Advanced SOA" section on the Wikipedia SOA page seems to be tending towards a dismissal of the term "SOA 2.0" as a cynical marketing ploy. (Which of course it was).

It's a small step forward, but Wikipedia is a pretty widely read source so it'll do for me.

Now we have to think of another cause to fight for! Any ideas?

Labels: SOA, SOA 2.0, wikipedia

posted by Neil Ward-Dutton @ 2:43 PM 0 comments permalink

Sustainable SOA and "closed loop" thinking

Todd Biske of Momentum has a great blog on SOA and EA, and one of his recent posts chimed particularly with something we've been talking about for quite a while now - sustainability. This is a theme that NeilM and I and Jon (and also Dale, our partner at Freeform Dynamics and book co-author) - have been developing throughout 2006 for our book on IT-business alignment.

The idea of sustainability isn't really rocket science but it's a vital touchpoint in the process of strategic IT thinking. What it means is that it's not enough to think about technology in the context of solving a problem or addressing a need that you have today. To deliver sustainable value from IT you have to think about how technology will continue to address your needs going forward. Arguably this was a key challenge that contributed to so much disillusionment surrounding the Enterprise Application Integration (EAI) boom in the late 1990s and early 00s: EAI technology was great at fixing tactical and existing problems (how can we synchronise key customer data across these x systems? etc) but because of its sometimes esoteric (and certainly non developer friendly) nature a lot of the technology couldn't really support a strategic shift around how *new* capabilities should be developed to make integratability a "baked in" feature. A better balanced forward-looking approach to integratability (not just integration)is of course one of the things that makes SOA so interesting.

Todd's post is talking about how IT so often looks at things from the point of view of a set of discrete and disconnected events - and one particular piece grabbed my attention in the context of SOA:

"IT produces solutions, and then forgets about them unless a user complains or some alarm goes off. If an organization takes on SOA, but still operates with this mentality, the only thing that has changed is that they are producing services instead of applications."

It's more fundamental than that though.

A focus on service delivery (which for us is what SOA is really about) is a focus that is predicated on closed-loop thinking. A service is something that is experienced over a period of time by a consumer, not just a capability that you've built. I'd say, then, that if you work with the mentality Todd talks about then you're not even producing services - you're producing itty-bitty applications. The concept of "service" - a consistent experience provided to a consumer - is what underpins that evolved, closed-loop view. Without it you're not doing SOA.

This means that SOA is only possible when you consider the whole lifecycle of services over time as they are created, changed and (yes) retired. And that's where sustainability comes in. If you're not thinking ahead to how you will deliver that consistent experience you're not thinking in a sustainable way. You're thinking about point projects, point applications, point functions, and that's how we got into the mess we're in.

Importantly this shift in mindset to think about how to deliver sustainable business value from IT takes us well beyond the world of technology product procurement. It's all about process, practice, organisation and culture and nothing to do with whether you bought the blue or the red ESB.

Without this understanding at the top of your mind as you embark on SOA or indeed any other IT initiative (unless it's responding to a *very* opportunistic and short-lived requirement) entropy will always win. If you're always looking backwards then the reality of business requirements and the reality of IT capability will quickly diverge in unwelcome ways.

Labels: book, SOA, sustainability

posted by Neil Ward-Dutton @ 4:54 PM 1 comments permalink

A useful primer on SOA governance

Labels: governance, SOA

posted by Neil Macehiter @ 11:11 AM 0 comments permalink

Identity meets SOA

Labels: identity, SOA

posted by Neil Macehiter @ 7:12 PM 2 comments permalink