Archive for the ‘SOA’ Category

« Older Entries

“Good enough”: a conversation with Anne Thomas Manes on the SOA manifesto

Thursday, November 19th, 2009

When MWD started in 2005, the first topic we sunk our teeth into was SOA (it was 2005’s Cloud Computing, pretty much).We researched the technology and the practice pretty thoroughly, I think (along the way tackling “SOA 2.0” among other things), but in mid-2007 we drifted away from the topic a little. At the time it seemed that we’d said pretty much all we could about the subject – we’d done a lot of technology and architecture research (looking at the need for governance, the need to consider consumer as well as producer service lifecycles, and so on), and there were still relatively few opportunities to dig into actual case studies to start to determine real-world best practice experiences.

Over the past few months, though, I’ve started to pick away at SOA once again – it feels to me like we’re at the stage where there are some interesting real-world stories to dig into, and at the same time there’s still a lot of people looking for answers to some straightforward questions about the nature of SOA and how to deliver value from a SOA investment.

With this in mind, I was excited to see the launch of a “SOA Manifesto“, styled after the Agile Manifesto – which has had a significant impact on expanding awareness and understanding of Agile development techniques. I was intrigued to see that Anne Thomas Manes of Burton Group, of “SOA is Dead” fame, was one of the signatories – so I decided to ask her about her involvement in the drawing up of the manifesto. I know that Anne and her team do a lot of case study and best practice research on SOA, so I was interested to hear why she wanted to get involved. Below are my notes from our conversation.

NWD: What kinds of issues have you seen that made you want to get involved in the SOA manifesto?

ATM: Fundamentally, the amount of people I’ve seen who still think SOA is about technology rather than design/architecture. Though we are further along now because we have better books (like Thomas Erl’s books), I just felt that there’s still a need to show people that SOA is about designing systems differently. When I wrote the “SOA obit” blog entry, it was really a reflection of the fact that only around 15% of the projects we saw were delivering significant value; and maybe another 25% were just about breaking even. The majority of projects we’ve seen just haven’t delivered a decent return. My goal in being part of the SOA Manifesto group was really about trying to get people thinking about SOA in a different way, so they could use SOA to enable much more business-aligned portfolios of applications.

NWD: What’s the precise purpose of the manifesto, do you think? What does the group want its value to be, and to whom?

ATM: We very much admired the Agile Manifesto, and followed that path. When it was published a lot of people didn’t know what Agile meant; it really helped crystallise the role and value of Agile development. The value of the SOA Manifesto is hopefully similar – it’s in the way it establishes a set of core principles and value statements to help people think about what we think are the right things and getting SOA back on the right track.

NWD: What kind of feedback have you had on the SOA Manifesto so far?

ATM: Well, there have been a number of accusations that the content is too general-purpose – that there’s nothing that’s inherently Service-Oriented about it, and that you could apply it to many other things. You know, I think that’s true – but I don’t think that it invalidates the principles at all in the context of SOA. This kind of reflects the whole point of doing the Manifesto: the challenge we’ve had with SOA implementations is that too often, people have ignored the stuff we know works in the rush to try out a shiny new set of technologies.

NWD: Is the SOA Manifesto “group” planning to build on this initial statement? If so how?

ATM: We currently have over 300 signatories to the Manifesto [Now around 475], and all the group members are blogging/tweeting about it. We’re trying hard to make sure that we get as many signatories as possible, in other words. But, just as was the case with the Agile manifesto, there’s no group intention to directly commercialise this [in delivery of a body of knowledge, training, etc].

I’ve looked at a lot of the commentary and criticism of the SOA Manifesto out there, and a lot of it seems to be about saying “so what?”. But as Anne rightly points out above, the fact that the principles in the Manifesto are quite applicable to many types of business-technology change initiative doesn’t make them worthless. The fact is, SOA implementation *is* a business-technology change initiative.

I think the other thing to bear in mind, when looking at the SOA Manifesto and SOA analysis and commentary generally, is that no-one in their right mind is saying that SOA is always the right answer when it comes to application implementation (though in my view there’s an abstract business analysis/architecture part of SOA, reflected in OASIS’ SOA Reference Model specification work, which has general applicability across all domains). There are many technology implementation domains where SOA principles aren’t the right ones. As the Manifesto points out, SOA should be all about a set of architecture decisions that promote flexibility and agility (through things like loose coupling). In some situations, there are other factors (like extreme performance) which might trump a general desire for flexibility – or at least create interesting tensions for architects.

So – to sum up: personally I’m a fan of the SOA Manifesto. It’s not perfect, but nothing designed by committee ever is. It *is*, I think, “good enough” for what it needs to do – which is to add more weight to the argument that if you want to deliver value from SOA work, you have to continually remember to focus on  a set of IT architecture principles and avoid getting too distracted by shiny products.

Posted in SOA, SOA 2.0, SOA ROI, agile | No Comments »

Progress Software – getting past "Who"?

Wednesday, April 1st, 2009

A couple of months back I had a brief Twitter exchange with David Bressler of Progress Software (@djbressler), following a comment I’d seen from Judith Hurwitz (@jhurwitz) at Progress’ analyst day regarding the lack of brand awareness that the company has out there in industry. What I said was: “Progress is a bit like Unilever – top-level brand is vanilla, sub-brands have chops”. What I meant is that these days, there’s little knowledge of what Progress does (a typical response is either “Who?” or possibly “oh, they used to sell a 4GL and a database in the 1990s, didn’t they”) – whereas there’s much more recognition of brands like Sonic (SOA infrastructure), Actional (SOA management / governance), IONA (middleware, SOA infrastructure), Apama (event processing), DataXtend (data integration) and DataDirect (data connectivity, legacy application integration).

David replied that Progress is a technology company’s company – which is absolutely correct: Progress has a long and successful history of providing a platform for other software vendors to embed in their application offerings. And he followed up with this blog entry, saying “We’d love for the Progress brand to have some chops, and we’re trying but it’s not trivial.”

Well, for a few weeks I’d been meaning to write a blog post of my own exploring this – but in the general headlong rush that we’ve been experiencing so far this year, I’d forgotten to write that post. When I saw today’s news that there’s been a change at the top at Progress, though, I was finally prompted to write some thoughts down. (Thanks for the pointer Miko).

The main thought in my head all those weeks ago was that it’s all very well for Progress to be a bit like Unilever – with the sub-brands (Sonic, Actional, Apama, DataDirect, and so on) having much more visibility in industry than the parent brand – as long as the company doesn’t want to start pulling together broader IT and business infrastructure propositions that tie together pieces from the different brands. Unilever is well-known for owning a vast portfolio of products, many of which actually compete with others in the portfolio (Dove v Lux; or Persil v Surf, for example. The invisibility of the parent brand is fine for Unilever, but it’s bad news for Progress if it wants to really make the most of its potential within enterprises (by cross-selling or bundling its products to help customers with broader opportunities, for example).

So this is the point where the company has to undergo a pretty radical shift. As reported in PCWorld, the new Progress Software CEO (formerly the COO) has established a target of doubling the company’s annual revenue to around $1bn, by “reorienting sales towards multi-product suites, as well as aiming marketing messages more at business executives than IT workers” – that is, precisely what it’s not currently suited to doing.

This goal makes absolute sense, and in fact it has made sense for ages. The majority of the markets where Progress’ brands play are growth markets where there’s real opportunity, right now; and what’s more, the combination of the offerings could have real power, too.

The required shift will be no picnic, but there are worse times for Progress to be trying to make it happen. There’s a new man at the top with a new broom, no doubt; and what’s more, there’s still a small window of opportunity open for another medium-to-large-sized specialist infrastructure software vendor to pick up business, following BEA’s acquisition by Oracle a few months back. TIBCO and Software AG have recently been making much of BEA’s disappearance as an “independent” infrastructure software vendor, and it’s surely no coincidence that both these companies also have aspirations to reach $1bn in annual revenues (Software AG has been particularly vocal about this of late). Progress has long had the potential to join Software AG and TIBCO as a serious contender for enterprises wanting to avoid getting into bed with the MISO pack (Microsoft, IBM, SAP or Oracle) for whatever reason, but until now it just never seemed to be able to be bothered to do what was necessary.

With a new CEO at the top, it’ll be fascinating to see whether Progress can move up a gear. If it succeeds, then enterprises wanting to avoid giving too much technology supplier power to the MISO pack may well have a new choice – and in a market where consolidation has recently been rampant, more choice would be refreshing for everyone.

Posted in Progress, SOA, Software AG, TIBCO, industry, integration | 4 Comments »

Cloud computing, SaaS and SOA – the universal service network

Monday, February 23rd, 2009

Something that’s been sloshing gently around in my head for a little while came into focus the other day on reading a post by Brenda Michelson: Unintentional Cloud Watching >> Cloud Computing for Enterprise Architects. Namely, that the link between cloud computing and SOA has multiple angles.

It’s becoming clearer that, true to Tim O’Reilly’s initial Web 2.0 noodling, by providing open infrastructure services and APIs, the poster children of the Web 2.0 era – Amazon, Salesforce, Zoho, and so on – are now treading the path that companies like StrikeIron started out on in 2002.

As a result, as other commentators have noted, it looks like the bulk of the service-oriented IT that many organisations will interact with will be “stuff from outside” (commercially provided services) rather than “stuff from inside” (internally developed services). And it’s not just hosted SaaS providers who are playing here of course: there’s the issue of newer versions of on-premise commercial packaged application software products and integrations between them – SOA is coming in by the back door there, too (see SAP’s ESOA, Oracle’s Fusion Architecture). In fact, perhaps unsurprisingly, there are strong parallels here with the component-based development (CBD) hype-wave of the 1990s – a lot of the initial hype was around tools and development for enterprise IT groups, but ultimately the vast bulk of development was actually carried out by commercial software vendors, for consumption by enterprise IT teams. What we’re seeing here is a repeat of “componentware” market development, with a 21st Century twist.

Regardless of where services come from (and indeed because they will come from multiple places, creating cross-enterprise service networks), it’s increasingly the case that in order to deliver effective IT capabilities in the 21st Century, you need to understand SOA principles and build technology and management structures that really support the principles of service orientation. Much has been written about the “consumerisation of IT” and how new generations of people entering the workplace are asking difficult questions about why enterprise IT applications are so unintuitive to use. But what happens when business teams that are using SaaS-based offerings learn about the infrastructure side of the story – how easy they can be to customise, extend, and integrate with – and ask why internally-developed systems don’t exhibit the same qualities? Another slab of SOA pressure, that’s what.

Back in 2006 I was doing some research for an event that we were considering running on “IT Sourcing in the 21st Century”. As part of trying to work out what might be in scope and what might be out of scope, I drew this picture:

As the picture attempts to show, from a technology architecture point of view, the software-as-a-service (SaaS) model relates to the prior Application Service Provider (ASP) model in much the same way as SOA relates to monolithic on-premise applications – to deliver value, both SaaS and SOA need to “crack the box open” and enable IT capabilities to be customised, composed and remixed while also being shareable/reusable. There’s much I disagreed with about Anne Thomas Manes’ recent “SOA is dead” post (see Schrodinger’s SOA), but she nailed this parallel between SaaS and SOA pretty well I think.

With the increasing visibility of cloud-based infrastructure and application services in mind, anyone seriously pursuing SOA should be looking to the world of SaaS for insight, or at least inspiration. In the IT industry’s rush to SOA, many of the nuances and implications of SOA have often been condensed into simplistic advice targeted at software developers (something that’s been written about a great deal, not least by us). One of the themes we’ve returned to again and again in our SOA advice is that the concept of a “service” isn’t primarily about something you build – it’s about something you experience. If you’re going to deliver business value from your SOA efforts, you have to grasp the implications of this and make the necessary changes – not only in terms of tools and technologies, but also (crucially) in terms of your governance approach. One of the most popular posts from arch-architect Todd Biske, on ITIL and SOA, digs nicely into how it’s crucial to consider the full service lifecycle when you do SOA, and drive governance to ensure that you can deliver “real service” – not just code wrapped in XML-based interfaces.

The providers of cloud-based services should have this idea etched on their brains – and there are plenty of examples of this principle in action for any eager student of SOA. Witness the grumbles that echo when Twitter barfs, for example, or see the grumpy users trying to get to grips with Zoho’s CRM API in the face of almost non-existent documentation. These are great examples of situations where the provider’s idea of service doesn’t match the consumer’s expectation.

The rise of cloud computing and SaaS should entice us to revisit our development-centred assumptions about SOA and search for a “bigger picture” that focuses on consumer expectation and value first. The pressure to deliver business value from IT capabilities, and the increasingly diverse mix of IT capability sources, demands that we do so.

Posted in ITSM, SOA, SaaS, architecture, cloud computing, service | No Comments »

Schrödinger's SOA

Thursday, January 8th, 2009

I’ve spent a couple of days wavering over whether to jump into an ongoing blogosphere debate over the “Death of SOA”. For those who haven’t yet read any of the debate online, here’s the catalyst: SOA is Dead; Long Live Services – a fictional obituary of SOA by Anne Thomas Manes of the Burton Group. On one hand I wanted to avoid adding another voice to a growing crescendo of opinion and counter-opinion and the risk of not really shedding any extra light: but on the other hand I thought that some people might expect MWD to have something to say! Particularly given past blog posts like SOA 2.0? Stop the madness, Little SOA vs Big SOA, More big vs small thinking: SOA vs BPM, The pointless search for SOA ROI, and On SOA governance: for SOA, read CPOA?.

In the end, I couldn’t help myself. Call it New Year exuberance. So what’s our take? Is SOA dead or alive? Should anyone care?

I think there are a couple of key points to consider. First, what’s wrong with SOA today? And second, what should we do about it? I agree with Anne on the first; but I think I really disagree on the second.

When I look at Anne’s post in detail I believe she’s making 4 basic points:

  1. The majority of SOA projects have failed to deliver what they promised.
  2. Business people are disillusioned with SOA.
  3. “SOA projects” will be killed by today’s difficult economic conditions.
  4. Despite all this, the requirement for SOA is greater than ever.

I’ll happily back two of those points up – the first (UPDATE: to a degree – I have anecdotal evidence that suggests many organisations struggle with SOA, but that’s not quite the same)and the fourth. There’s one section towards the end of Anne’s post I’m particularly in agreement with:

SOA is not simply a matter of deploying new technology and building service interfaces to existing applications…it requires a massive shift in the way IT operates. … The latest shiny new technology will not make things better. Incremental integration projects will not lead to significantly reduced costs and increased agility. If you want spectacular gains, then you need to make a spectacular commitment to change.

This is pretty much what we’ve been saying about SOA for some time (see Little SOA vs Big SOA from April 2007). In short: SOA won’t succeed if you take an overly technical, product-based view of it (and indeed we’re far from alone in this – ZapThink bangs this drum a lot, too, for example).

So we agree that not all is well in the State of SOA: but where I diverge with Anne’s post is in the prognosis for SOA and the treatment for the illness.

Anne’s prognosis/suggested treatment is that SOA (or at least the term) is (or will soon be) dead. Her view appears to be that because its death is inevitable, we should accept it and move on – stopping talking about SOA and starting talking about other things instead. This appears to be bound up with her observations that “business people are disillusioned with SOA” and “SOA projects will be killed by the economic downturn”.

Before I move on – I don’t know about you, but I think that if any IT group has been trying to sell SOA directly to business people, they deserve everything they get. Regardless of economic conditions. No, no, no! You don’t sell methodologies and architectural patterns: you sell outcomes. Argh! And here’s a hint: if you work for an IT group that used to try to sell SOA to business people, and is thinking about now trying to sell “mashups”, “cloud computing”, or similar things – don’t bother, you’ll get the same result. Sell the outcome and the benefits, not the mechanism or the technology. Double Argh!

There’s another risk with changing the way we speak about what Miko Matsumura jokingly now calls “The Artist Formerly Known as SOA”: by doing so, we continue to spread the perception that the IT industry is a fashion industry unable to kick its habit of (re)inventing terms to reinvigorate markets when earlier promises go unfulfilled. Whether she knew it or not when she wrote the post, by referring to a change in terminology (away from SOA and towards “services”, “mashups”, “cloud computing”, etc) as an active and influential commentator, Anne is contributing to the fall from grace of the term.

Well, as I said here when I railed against SOA 2.0,

I sincerely believe that analysts should be good stewards of the influence they have – educating, clarifying, abstracting, comparing, acting independently, being measured, etc. It’s about filtering out hype and trying to provide practical, independent advice and insight.

Just because SOA is difficult to do, we shouldn’t start calling it something else in the hope that we can start over without anyone noticing. And it’s no surprise that SOA is tough to sell to business people – I don’t believe that was ever up for debate, and it shouldn’t be seen as any kind of broader indicator.

Let’s acknowledge that we all have more work and education to do – but let’s not jump the shark on this. “SOA is Dead” is a headline that no-one needs.

Posted in SOA, architecture, hype | 4 Comments »

On SOA governance: for SOA, read CPOA?

Friday, November 14th, 2008

A couple of weeks ago I was the happy recipient of a review copy of the excellent Todd Biske’s SOA Governance book. Todd’s “Outside the Box” blog is one of those rarities where every post is worth reading twice – so I was very interested to see whether his writing ability might stretch to something the length of a book! Todd’s clearly established himself as someone who has a lot of insight on the topic of SOA Governance, so I was pretty sure I wouldn’t be disappointed.

A number of other bloggers have posted detailed reviews of Todd’s book, so I’m not going to do exactly the same here. Take a look at the Amazon comments if you’d like to see what they said.

For me, I’ll be brief: SOA Governance is a very good book indeed, in that it does something that so many technology and business management books fail to do: it breaks a complex and hype-laden subject down into very manageable chunks, and walks through the topic clearly and at a steady pace – but it still manages to move quickly enough to prevent the reader getting bored. It’s not a perfect(*) book, but then nothing is – and if it had been, I would have been too jealous to write this. We need more technology/business management books like this, and we needed just such a book on SOA Governance. Well done Todd!

I knew this was a good book because it made me revisit some conclusions I’d already had washing around in my own head for a couple of years.

One of the things that I still find as I travel around is that when I get into discussions about SOA, there’s way too much focus on the “S” and not enough focus on the “A”. It’s almost as if we’ve been blinded by technologies and standards which have “service” somewhere in their names, and aren’t able to look at the bigger picture.

What Todd’s book reminded me is that if you want to get real value out of service orientation, then it’s the “A”rchitecture that really makes things happen. Todd’s narrative keeps coming back to his definition of Governance, which revolves around People, Policies and Processes. And it also talks a lot about the concept of “contracts” in the context of analysing how service providers and consumers should work together in order to interact. Without People, Policies and Processes in place to guide your organisation down the right path, and without the concept of “Contract” to focus on the responsibilities that need to be described and assigned when service consumers and providers interact, such an architecture effort will likely lead nowhere. You’ll end up with “just a bunch of services”.

So – and this was the thought that occurred to me after reading Todd’s book – perhaps we shouldn’t really be thinking about “service” oriented “architecture” at all. It seems to me that what architects might find more productive to focus on is policies and contracts, not “services”. Maybe “service” is better thought of as a concept that describes the outcome of this kind of architecture approach. And so maybe it’s the case that there are two things in play here, and we’re getting them mixed up: contract- and policy-oriented architecture (CPOA ;-) and service-oriented IT delivery?

What do you think?

(*) one thing I found rather strange was that despite a word at the front to reassure readers that they didn’t need to know any technology detail in order to read the book, at a number of points you’re suddenly confronted, out of nowhere, by XML fragments which (as far as I could tell) didn’t really add any value. That’s a tiny niggle though.

Posted in SOA, contracts, governance, policy, toddbiske | 2 Comments »

Software Delivery InFocus podcast – ALM challenges and direction in the real world

Friday, October 31st, 2008

Following the first Software Delivery InFocus podcast which we published in September, October sees Bola Rotibi’s second podcast episode, in which she discusses a series of questions focused on the topic of Application Lifecycle Management (ALM). Her guests are John Leegte (ICT Architect at the Dutch Tax and Customs department, Belastingdienst) and Steve Jones (Head of SOA and SaaS for Capgemini’s global outsourcing business).

Application Lifecycle Management (ALM) is a topical subject that has garnered significant column inches in recent months, as many of the leading players in the market have launched new versions of their ALM solutions, and make strategic announcements concerning future directions and customer services and support. Over the last few months we have either heard about or seen previews of products from the likes of Borland, Compuware, HP, IBM, Microsoft, MKS, Polarion and Serena, to name but a few. Software is seen by many organisations as a key enabler of business value – whether that be through improving operational efficiency, competitive differentiation or business/ product innovation. With this in mind, an ad hoc approach to software application lifecycle management cannot provide the predictability, visibility and traceability that organisations require of a process that has such a significant impact on the balance sheet. So – how relevant and applicable is ALM now and in the future, particularly in light of today’s technology and business environments, when issues such as agility are so much to the fore?

The episode is slightly longer than normal (clocking in at 41′55″). There was so much good material in the conversation, we didn’t want to cut anything! Thanks very much to both John and Steve for such a great conversation.

You can download the audio here or alternatively you can subscribe to the podcast feed to make sure you catch this and all future podcasts!

As with all the episodes in this podcast series, we’ve also published a companion report which summarises the discussion and “key takeaways”. You can find it here, and it’s free to download for all MWD’s Guest Pass research subscribers (joining is free).

Posted in SOA, agile, development, podcast | No Comments »

SOA governance and data governance – separate or one in the same?

Wednesday, September 24th, 2008
Joe McKendrick (once again!) has another post which caught my blogreader today. This time he is pondering the relationship between SOA and data governance:

If data governance is inadequate — information is outdated, out of sync, duplicated, or plain inaccurate — SOA-enabled services and applications will be delivering garbarge. That’s a formula for SOA disaster.

He goes on to reference an article by Ed Tittel, which draws the same conclusion:

Amidst all the hype and buzzwords that surround SOA nowadays, it’s still far too common for organizations that seek to integrate service-oriented architecture into their IT infrastructure to omit issues related to data integration, management and governance in their designs. As they roll out and learn to live with an SOA, however, they often discover that interoperability with other systems and solutions poses interesting problems. In fact, these problems can make interaction between systems and SOA components both vexing and time consuming.

Absolutely! When we put together our SOA Strategy Planning Tool, we explicitly acknowledged the importance of a common information model that provides:

standard representations of core information types for communication between services

Where I deviate from Joe and Ed, however, is their perspective that data governance and SOA governance are separate disciplines. Without inter-service communication there’s no SOA and so SOA governnance must encompass data governance. Furthermore, that governance needs to extend beyond service design throughout the service lifecyle.

In a subsequent post, Joe calls out this post from David Linthicum in which he noodles on the same topic. I am not so sure about David’s view that SOA initiatives:

need to start with the data first

It all depends on the scenario where a service-oriented approach is being applied. However, I agree with him that there is a need to understand:

the core purpose of the data, how it relates to other data, how the data is bound into entities, as well as security issues, integrity issues, and the binding to existing functions or transactions. I would go further and say that it’s not just about understanding those things. In the case of security and integrity issues, there is a need to ensure that what is understood is enforced. That means defining service contracts that take account of those requirements and enforcing them through policies.

Which brings me neatly back to the SOA versus data governance discussion. Policies are the lingua franca of SOA governance and policies apply as much to the data flowing in a service network as they do to the services themselves.

If you are embarking on an SOA initiative you need to ensure that those responsible for SOA governance, ideally though an SOA centre of excellence, include individuals with data management expertise. Your governance processes should enforce utlilisation of a common information model and encompass a policy-based approach to ensure that data management objectives and constraints are enforced.

Posted in SOA, governance | No Comments »

Hmm indeed Mr McKendrick – that should be "an over-simplistic definition of 'SOA'"

Wednesday, September 24th, 2008
Joe McKendrick over at ZDNet has been pondering CIO “SOA Advisor” Nicholas Petreley’s definition of SOA:

a networked subroutine

No wonder Joe’s not sure about it! Nicholas’ definition is closer to that of a web service and even that’s being generous!

Joe rightly points out that the definition completely ignores the ‘A’ of SOA and comes up with his own alternative to address that limitation:

an architecture of services

But that suggests that SOA is something that you build – an end-product. It isn’t. Architects in the real world don’t create architecture: they do it! That’s why they are called architecture practices.

Back in May 2005 in our first report on SOA, which set out to provide a big picture view of SOA, we proposed the following definition:

SOA is the disciplined approach through which an IT organisation manages the lifecycle of IT services, and assures their delivery, in a way the reflects business process priorities

As our SOA definition suggests, we see the ultimate value of SOA as shaping a service-oriented perspective on IT delivery, from top to bottom in the IT organisation. Service orientation can apply just as much to business architecture as it can to solution and technical architecture. It may not be quite as simple as Nicholas’ definition but I think it more accurately reflects how thinking about SOA has moved on since the heady days of “SOA as software development and integration based on web services”

Posted in SOA | No Comments »

IBM, Business Event Processing, and CEP: behind the bag of spanners

Friday, September 12th, 2008

Earlier this week I attended an IBM press and analyst summit on the topic of “Business Event Processing”. To coincide with this, the company made some announcements on its “BEP leadership”, with over 3700 “BEP customers”. This is fairly early days in IBM’s attempts to tell a coherent story about what it’s doing in event processing, although it’s been helping customers for many years prior to the January 2008 acquisition of AptSoft (which catalysed IBM’s use of the BEP term). However the company isn’t doing itself any favours with language like this: it’s all too easy for it to come across as disingenuous, or at the very least the result of a significant amount of Vaseline being spread on a camera lens.

Why? Well, it’s a question of boundaries and definitions. Specifically, IBM defines BEP in the following way: (1) an event is “something that happens”; (2) a business event is “an event that has relevance to the business”; and (3) Business Event Processing (BEP) is “the correlation of heterogeneous events in order to achieve smarter business outcomes”.

BEP is a term which has come about primarily through IBM’s own marketing since the AptSoft acquisition. But a similar term, Complex Event Processing (CEP), preceded this by some significant time (it was popularised by David Luckham’s 2002 book The Power of Events). With a nod to the debate that still simmers about what the CEP term should really signify (is it the processing of compound/aggregated, or “complex” events – or is it application of “complex processing” to events?), IBM’s definition of BEP is very close indeed to what seems to be the pre-eminent definition of CEP. This is no accident: on a call to discuss the AptSoft acquisition, IBM’s Sandy Carter explicitly called out her interest in trying to rename the “CEP category”.

All’s fair in love and software marketing, I suppose, even when it risks leaving everyone confused (most people are still trying to get their heads around CEP). The real problem I have with this, though, that it’s not only a simple renaming that’s at work here. When IBM then goes on to talk about the event processing work it’s doing, it tells a story that’s much broader than the AptSoft technology area, and also broader than what most event-processing folks would think of when they think of CEP. Likewise when IBM refers to “3700 customers” these aren’t AptSoft customers (apart from a tiny minority of them). As well as focusing on WebSphere Business Events (the former AptSoft technology) IBM’s view of BEP encompasses event-processing capabilities that exist in Tivoli/NetCool systems and network management products, WebSphere MQ Event Broker and MQ Low Latency Messaging, a new stream-based event processing platform called InfoSphere Streams (formerly “System S“), the SolidDB in-memory database, and many other bits and pieces as well.

In short, it looks like IBM’s taking a term that only in January 2008 it used to describe a “business friendly” event processing toolset (the AptSoft technology), and is now instead applying it to any software technology that IBM offers which involves the processing of events. A cynic would say it’s got a big plumbers’ canvas toolbag marked “BEP”, and is shoving as much into it as possible.

Now the intention of this post isn’t just to bash IBM for attempting to subvert the CEP concept. The reason I don’t want to leave things here is that there’s more to what IBM’s doing than meets the eye.

As the press and analyst day unfolded, it became clear (not through the presentations from the IBM execs, tellingly, but through Q&A and one-on-one meetings) that behind the scenes, IBM is actually trying to do something pretty interesting. It’s got a big kit-bag of event-processing technologies, yes – but behind this, it’s working to put together a common event processing technology framework that will allow its individual technology components to be assembled in different combinations to support different types of business and IT requirements. So, rather than having separate event processing stovepipes in the WebSphere, InfoSphere and Tivoli software portfolios, together with assorted miscellaneous other components, it’s enabling all of these distinct efforts to cross-fertilize. One of the first outcomes of this work is the snappily-named WebSphere Business Events eXtreme Scale, which marries the former AptSoft technology (never marketed as a platform for high-volume processing) to the WebSphere eXtreme Scale platform.

With this in mind, I think it’s a bit of a shame if IBM sticks to referring to its overall event processing architecture effort as BEP. BEP, like CEP and one of my other favourites Composite Application Development, is a “how” name – it says more about a method than its outcome. Or, in other words, although it might appeal to geeks, it says nothing about why anyone would invest in it. What would work much better, in my opinion, would be for IBM to complement (or even replace) its discussions about BEP with discussions more centred around a “what” idea that is more about the business value of event processing – what all these technologies, and the framework being built, actually enable customers to achieve. If IBM hadn’t already walked away from the term, I’d suggest something like “on-demand business”… but seeing as that’s verboten these days, I’ll keep my thinking cap on. Perhaps “event-driven business”?

Putting all the vendor marketing stuff to one side, what does all this mean for enterprises? First, try and look behind the cynical BEP and CEP marketing stuff. IBM might be mis-stepping in its marketing, but you can be sure that it’s serious about building a set of technology offerings to help customers with a variety of event-processing related problems and opportunities. Second, although algorithmic trading and other capital markets applications are where most of the technology market activity in this space has been over the past year or so, try and take a broader perspective of event processing and how it might offer value. Event processing is a useful computing approach, even when the event volumes aren’t colossal, or the processing requirement isn’t near-real-time, or the processing requirement isn’t highly complex.

Lastly: watch out for a free Guest Pass report we’ll be publishing on event processing before Christmas, where we’ll attempt to unpick the different scenarios and technology requirements in play.

Posted in BEP, CEP, SOA, ibm, industry | No Comments »

Software AG goes in an interesting direction for SOA governance

Tuesday, September 9th, 2008

As part of yesterday’s release of the latest iteration of its webMethods Insight product Software AG announced an OEM partnership with Progress Software. This announcement adds the Actional runtime SOA management and monitoring technology (which Progress acquired back in January 2006) to Software AG’s existing Centrasite design-time governance capabilities (which were bolstered by the acquisition of Infravio in September 2006) and the runtime policy enforcement provided by its webMethods X-Broker and partner Layer 7’s XML Firewall.

The incorporation of runtime SOA management and monitoring functionality into Insight is a necessary evolution of Software AG-webMethods integration strategy that we commented on just over a year ago. It’s long been our position that SOA is more than a standards-based approach to software development and integration. The business value of a service-oriented initiative depends on a recognition that software services are experienced, just like their real-world analogues. The quality of that experience depends on a governance approach that extends throughout the service lifecycle, where the contracts defined when services are designed are subsequently enforced through policies once they are deployed and running – and where runtime metrics are captured to provide insight into the service level quality that is actually exeprienced. Furthermore, those metrics can be used to inform and support change management processes, so closing the SOA lifecycle loop.

Whilst the announcement doesn’t come as any great surprise, the source of the runtime management and monitoring functionality does. When Oracle confirmed it’s intention to acquire BEA, I said:

It [the acquisition] leaves some of the other bigger specialist players – TIBCO, SoftwareAG (and to a lesser extent Progress and Red Hat) in an interesting position. On the one hand they will be more attractive, particularly for SOA and BPM, to customers looking for an application-independent infrastructure offering.

Software AG has gone to a potential competitor for the mantle of best-of-breed, specialist alternative to the likes of IBM, Microsoft and Oracle. If you had told me on Friday that Software AG was going to strike an OEM deal for SOA management and monitoring I’d have put my money on AmberPoint, which has historically been the OEM of choice for the likes of BEA and TIBCO.

I am not quite sure what to make of this decision. AmberPoint doesn’t compete with Software AG directly and has established a healthy and growing customer base, as well as partnerships with some of the leading systems integrators – and a technology partnership with Software AG! Software AG’s decision comes not long after Oracle’s decision to drop AmberPoint. As we pointed out in our analysis of Oracle’s roadmap for the BEA integration, we don’t have any hard evidence for Oracle’s claims that it had received negative feedback from BEA customers but it’s something we will continue to explore. In light of the decision to go with Actional, it will be intriguing to see how the partnership evolves and how things pan out when Software AG and Progress are in a competitive situation.

This acquisition should be welcome news to Software AG customers that have invested in the company’s SOA offerings as it will save them the time and effort of plugging the runtime governance gap that existed prior to the partnership. Those embarking on a significant SOA intiative should also give Software AG careful consideration, particularly if they are not wedded to one of the mega-platform providers.

Posted in AmberPoint, BEA, Oracle, Progress, SOA, Software AG, TIBCO, governance, ibm, webMethods | 12 Comments »

« Older Entries