3         How to Deliver and ESB

3.1       ESB Purpose - BPM, CRM and Business Benefits should be at the heart of any SOA/ESB and CIM initiative

Unless SOA, ESB’s and CIM are viewed in light of Business Process Management, Customer Relationship Management and other Business Oriented Benefits.

i.e. How they are to be used to implement/compliment BPM, CRM and add value to the Business, we don’t know what problem we’re addressing, and hence whether we’ve successfully done anything with the ESB and our SOA.

Until an ESB built using SOA principles is used by specific Business Processes and benefits relating to the Business Process are delivered, in areas such as say,

·         Business Process implementation

·         Agile Business Process Re-Engineering, change of rules etc.

·         Insight via real-time Business Activity Monitoring historical data analysis to arrive at meaningful information that can be used to improve Business Process performance

·         Customer Relationship Management

·         Improved proactive and reactive responses to important Business issues

then whether the SOA and ESB implementation was at all successful cannot even be seriously be contemplated. It just ends up being another part of the generally dizzying array of IT legacy.

The CIM argument is equally useless unless the CIM is understood by the Business users, and used by many Business Processes in order to gain from re-use of the CIM entities & schemas.

SOA should therefore, not be judged by

·         whether a particular web service or operation is re-used.

·         whether the service is using UDDI or other mechanism to look up the service endpoint.

·         how fast it runs a data transformation, or by how it has some pretty drag and drop functionality for doing data mappings.

·         how quickly a client can point to an endpoint and invoke the service.

Nor should it be directly judged by compliance to specific standards or how it uses those standards. Standards are a means to an end – they are not the end.

These are technology specific measures. They do not have any direct Business meaning or benefit.

Of course, the more of the above technology specific measures that the SOA implementation adheres to, the more amenable it is to re-use, robustness, productisation etc., so they are obviously all very important technical aspects of an SOA, ESB and CIM implementation.

They do not, however, directly make the SOA/ESB/CIM a success.

It’s whether the Business Perspective’s measures have been met that determines success, not the Technical Perspective’s measures.

3.2       ESB Implementation - Standards, Capabilities and Protocols should be at the heart of any SOA/ESB and CIM initiative

At ground zero, here are a number of reasons ESB’s go wrong:

3.2.1     This is what NOT to do in trying to implement an ESB:

·         Do NOT pick a particular Product and bet your ESB on it

·         Do NOT pick a Vendor and leave it to them

·         Do NOT put a Centralized Physical Layer in the middle of Clients/Services (aka EAI) and expect it to solve all your problems

·         Do NOT confuse Logical Models with Physical ESB Models

·         Do NOT deliver it as a single project

·         Do NOT have a central group responsible for long term delivery, maintenance of the deliverables (Software, Hardware, Network etc. of the ESB).

3.2.2     This is what you SHOULD to in order to successfully implement an ESB:

In essence, there are only a handful of relatively simple things you should do, however, they are instrumental to the success of an ESB initiative, and hence must be done well:

·         Specify what services/capabilities you want an ESB to have

o   Make up a general list of capabilities that sound right

§  (see OpenStandards Presentation - BPM IMPLEMENTATIONS USING SOA, CIM's AND ESB's for examples)

o   Use this list as inspiration to try and identify the capabilities are especially important, useful or “nice to have” in your organization

o   Have an ongoing process – this needs to live as long as the ESB lives. The Organization must never lose sight of this list of capabilities. It is the key driver of everything an ESB means to your Organization. Forget about everything else that you’ve ever heard or read about what an ESB is & what it should do etc. For your Organization, this list is ALL that matters.

o   Classify the items in terms of priority, and review them periodically as you deliver them, and as the Organization’s requirements evolve over time. Review industry best practices and add the applicable ones to the “one big day” list. This will give you an idea of what you’ve got up your sleeve if something comes out of left field. Move them up/down the priority stack as required. On the “Next To Build” list, aim to have one or two capabilities at a time.

o   Only build what you’re going to use. This ensures they are well tested. Ensure they are VERY stable. If not stable, re-visit, re-build and proceed. Unlike any other software in the Organization, the ESB implementation cannot afford to be loose. It must be the leanest, most stable, most understood part of the Organization’s Software assets. For this reason, try to keep it as simple as possible, with each capability having REAL requirements and benefits.

·         Have a Committee to manage the common capabilities list. Other features can be added by minority groups and used within their realm (or Service Domain etc.).

·         Have implementations of the above capabilities in whatever platforms are strategic for your Organization.

For example

o   .NET if you’re a .NET shop

o   Java if you’re a java shop

o   Both if you’re a both a Java and .NET shop

Anything else viable on whatever platforms you have

o   Mobile

o   Mainframe

o   …

·         Use the Internet as inspiration. Leverage as much of Internet related concepts, protocols, software etc. as possible. Don’t come up with your own implementation until you’re sure there’s no equivalent of it in the internet space. This approach helps guarantee your ESB will scale in many ways:

o   Multiple stakeholder Buy-In

o   Development, Delivery, Operations & Maintenance

o   Performance

o   Availability

o   Sustainability

o   Knowledgeable persons, useful & reusable knowledge transfer, easily available (commodity) skill sets

o   …

·         Treat the ESB as a logical (and very rich & wide) Layer 8 to the ISO’s OSI Network Model. Make it a layer by which any ESB competent node can deliver what functionality is required by the IT department of an Organization.

o   That’s what the BUS in the Enterprise Service BUS is intended for. It’s a protocol which when implemented, allows for any node to interoperate with other BUS aware nodes. This may be in a centralized or hierarchical manner, or in a peer to peer manner depending upon the capabilities required & mechanisms chosen.

o   A common mistake is to treat an ESB as a central point for applications to connect to. That model quickly encounters scalability & logistics problems. Logically, an ESB layer is one that sits between service providers and consumers. Physically, it should be a layer that allows ESB aware applications to converse. Unfortunately, the OSI model has not been extended to layer 8 as this area of IT has been dominated by application frameworks and vendor technologies. The industry has ripened and the time has come for Organizations to build their own OSI layer 8. Over time, a standard set of OSI Layer 8 specifications and implementations will prevail. The WS-* is a step in this direction, however it is too all-encompassing (focusing more again on delivering existing application level constructs into a Standards-Based set of technologies), and does not deliver on simple, free & pluggable, transparent ESB qualities that Businesses require. We still have disparate messaging, integration and data transformation technologies – because vendors can still make money here. What Businesses would like, however, is to be able to have all their systems have basic capabilities to effectively communicate. Business (especially big Organizations) would be happy to buy (usually at whatever premium) systems that have a standard basic communication capability with other systems. i.e. ESB like qualities baked into applications. Unfortunately, we are nowhere near that. One of the purposes of ESB.NET deployments is to have it sit in front of specific applications and provide this capability.

·         In each platform, start with a framework that’s very flexible. Have people that are passionate about each platform working on the ESB framework for that platform.

·         Build a platform that can be refined over time, is well understood and is easily changed over time. Try to keep it as simple as possible.

·         Have a low barrier to entry (within reason). This obviously ties in with the required capabilities

·         To start with, build a single capability in each platform at a time. Obviously, things like connectivity are the most basic and important of all. To lower the barrier to entry, pick standards based protocols as much as possible.

·         Avoid niche products, standards, protocols etc. as much as possible. Conversely, you may have to taint some standards or protocols to make them easy or useful to your requirements. Better that than to come up with something completely new.

·         Base everything on evolution. Revolution is not required. We’ve pretty much got most of the technologies we need. We just need to apply them sensibly. Undoubtedly, this will be the most difficult task.

·         Whenever using a product, search for the capabilities in the above list. Ensure the product is extensible enough to enable you to deliver on the above capabilities. 

o   Aim high

o   Deliver low

o   Deliver wide

o   Get everyone involved

§  All of IT System Aspects

·         Business Stakeholders (the Business needs to know what they’re spending their money on, and what features & capabilities they’re investing in)

·         Service Producers

·         Service Consumers

·         Business Applications/Systems representatives

o   Measure/Understand the benefits & progress

o   Repeat the above till you have the basics in place. The process should then take care of itself.

·         Encourage multiple implementations of the capabilities. Have a reference implementation for one platform and a set of conformance tests that any ESB capability producer should pass.

o   An ESB Nirvana would be that ALL systems in your Organization can natively support your ESB capabilities. To that end, aim for lightweight (or possibly even embeddable), distributed & federation capable implementations so that they can be deployed as close as possible to each system in the organization.

·         Have STRONG Governance of the ESB capabilities and teams claiming to be delivering the ESB capabilities. Although measurable conformance tests need to be passed, the principles behind the ESB should also be evaluated. Never let a single application or group bully the ESB effort by pushing a particular technology or product. Conversely, the Governance board should be open minded & platform neutral.