This is the second post of a series based on a talk I gave at the Fall CIO Summit in Baltimore on September 21st.

You can read the first part of this POST here.


Enterprise Service Bus

An enterprise service bus (ESB) is a software architecture model used for designing and implementing communication between mutually interacting software applications in a service-oriented architecture (SOA)

When explaining what an ESB is, I like to use the “Water Main Pipe” analogy. A water pipe / water distribution system receives water from different sources and distributes it to houses; when you open the faucet, you get water, you don’t really care where it is coming from as far as it complies with a pre-established protocol: H2O, odorless, transparent, flavorless and a pre-established interface: Open facet -> get water , Close facet -> stop water.

Picture1

The ESB was born out of the need to move away from point-to-point custom integrations in order to connect heterogeneous systems. Reducing the custom integration code being spread among application and providing a centralized system to monitor and troubleshoot.

REMEMBER THIS …?

2015-09-28_18-29-02

WHAT ABOUT THIS …?

2015-09-28_18-30-43

The use of a Service Bus enables a seamless integration of different systems that at the same time can be data consumers and providers. An ESB can also be a broker for different systems of record – like CRM1 and CRM2 – being this completely transparent for the data consumer.

(Systems connected to a service bus can act as data consumers and data providers at the same time).

The beauty of this is the ability to switch systems of record and underlying systems without affecting data consumers – [when correctly implemented] the service bus ensures that the protocols and the services from a consumer perspective remain the same (As if your water provider decided to source the water from somewhere else!)

FEW BENEFITS OF ESBs

SINGLE CATALOG OF SERVICES

The ESB provides an unique point of entry and catalog to your Enterprise Services. Centralization and Standardization ease the use for consumers and the driving of best practices.

ROUTING

As pipes and valves in a piping system, the ESB takes care of routing information among connected systems as needed, in a way that is transparent for the consumer. Remember the scenario where we had two CRM systems? Routing capabilities allow consumers to get the right data from both systems.

PROTOCOL CONVERSION AND MAPPING

The goal for a proper ESB implementation is that all the Services available in your catalog “speak” the same language – follow the same protocol – however, that doesn’t mean that the data providers have to “speak” that language. The ESB can act as a translator and a mapper.

2015-09-29_11-47-47

In this example, we establish that our Service Bus Services are REST/JSON , also our exposed Account object will look like :


{
AccountId : 1234,
Name : "Sebastian Hosehold",
Company : "Inspirato LLC"
}

and our exposed Bill object will look like :

{
BillId : "XYZ123",
Total : 123.23,
Items : 12
}

This is what the consumers – like the Web App – will be accessing.

However, let’s suppose that the CRM system exposes data to the ESB in REST/XML and the Account object looks like:

<Account>
<CMRFamily_ID>1234</CMRFamily_ID>
<FamilyName>Sebastian Hosehold</FamilyName>
<Company>Inspirato LLC</Company> 
</Account>

Last but not least the Billing system exposes data to the ESB with  SOAP services.

The protocol conversion an mapping allows the ESB to not only convert SOAP and REST/XML into REST/JSON, but also re map properties to new properties (i.e. CMRFamily_ID -> AccountId).

It is important to notice that when the data providers – CMR and Billing systems – act as data consumers, ideally, they access the services in the catalog with the protocol defined for the ESB, in this case REST/JSON.

QUEUING

This feature allows to build some “cushion” with the underlying systems – data providers – and also build a “retry” strategy.

SECURITY

Centralized security pattern unified to all consumers. Also centralized access control and audit.

SERVICES CHOREOGRAPHY AND ORCHESTRATION

Despite this is not necessarily a part of every ESB product, some of them offer orchestration add ons. Basically an orchestration is the coordination of data consumption and transformation that happens when a service is called. They are different theories related to this subject “should ESB orchestrations drive business processes (like BPEL)?”

PRODUCTS IN THE MARKET

This is a list of few ESB offerings. I will not be expressing any evaluation or opinion on the products this time.

TOP TIER

Picture1

MID-TIER / PAAS

Picture2

OPEN SOURCE

Picture3

OUR EXPERIENCE AT INSPIRATO

WE’VE BEEN THERE

We started with a monolithic application connecting and orchestrating 5 different systems. Luckily it was a hub-spoke design so our Metcalfe value for the network was still low.

THE PROJECT WAS TRIGGERED WHEN WE DECIDED TO BUILD A MOBILE APP
IT TAKES TIME

It took us 4 months to “service enable” our legacy systems and 2 more to orchestrate few of the services.

WE ARE NOT DONE

We finished version 1 just to learn that version 2 will have to come soon.

SOA IS PART OF OUR TECH FOOTPRINT

Every time we design a feature, we make sure that a Service Specification is part of the Acceptance Criteria.

WE DECIDED FOR REST / JSON

We wanted to keep our services lean and without much complexity and overhead looking forward to our mobile platform.

FINAL THOUGHTS

IT IS NOT ONLY ABOUT WEB SERVICES

Remember that SOA is not necessarily a synonym for Web services.

SOA IS A MEAN NOT A RESULT

SOA is a way to get where you want to be, it is an architectural guideline. It is not a product! You do not buy SOA, you do not build SOA.

PLAN BUT DO NOT OVER ENGINEER

Come up with a good strategy and spend some time thinking about your base components (like security, versioning and naming standards – you won’t be able to change that often in the future). Then be agile, fail fast and work with MVP releases.

IDENTIFY SYSTEMS OF RECORD

It doesn’t matter if you decided to cache data in multiple systems for the sake of performance but you cannot have more than one system being responsible for the same data point! Think on business terms, not technology terms, who is going to be responsible for maintaining that data and which system they are going to use.

BUSINESS FIRST THINKING

Break down business functions into macro-level services and understand the relationships between them. Don’t think about software architecture, think about business architecture: billing, ordering, distribution, etc.

ABSTRACTION, REUSABILITY, GRANULARITY AND STANDARIZATION

Do not build things for a particular consumer, build things for all the consumers (the ones you have and the ones to come). The perfect catalog of services or the perfect API is self-documented, you don’t need to explain how to code against it.

ACCEPT SOME TRADE OFFs IN LIUE OF PERFORMANCE

You will have to fight some fundamentalist software architects. Sometimes what is correct (like a canonical model) does not perform. I am not saying to go crazy and have 10 definitions for the same object, but think on ways to lighten your payloads.

VERSIONING

Version you services from day one, part of being agile means that they are going to change (sometimes too often). Salesforce for example, has never decommissioned an API version – they currently have more than 30.

SINGLE POINT OF FAILURE

The use of a service bus brings with it the implementation of a single point of failure. Make sure you have a platform that can scale with load, have a failover system in place and do not get too crazy with orchestrations – orchestrate in the ESB only what makes sense.


RESOURCES

Enterprise Service Bus – Wikipedia

TIBCO

ORACLE

MULESOFT

DELL-BOOMI

CLOUD ELEMENTS

Advertisements