Data is among the company’s most valuable resources right after humans. It is essential to the proper operation and profitability of the company.
Data collection is half the battle, as it should be used appropriately, keeping in mind that it is often distributed due to the variety of systems which support the business. In such a case one should consider the support of the ESB Enterprise Service Bus, which enables the efficient transfer of data between applications.
Where and why should you use an Enterprise Service Bus?
Enterprises often use different systems to support their business. The more complex the business, the more the need for the implementation of various parallel solutions. Just imagine the example of a sales company. It usually needs tools to support product management by PIM, sales by ERP, warehouse and freight forwarding by WMS, human resources by CRM, and often manages the website content through the CMS system or an online store through the e-commerce system. In addition, mobile applications are also more and more needed.
Of course, these systems can exist in various other combinations and configurations depending on the type of business. However, there is always the problem of data integration. If we have so many parallel systems working for one organization, we should, and even have to, make them effectively cooperate with one another, as they usually need various information coming from each of them.
However, redundancy and storing the same data in different locations is difficult to maintain and highly error-prone. Ideally, such applications or systems exchange data in a relatively quick and simple way. Being able to seamlessly integrate further systems would be a good thing as well…
How does ESB work?
Does it make sense to connect each pair of applications or systems separately for the purpose of data exchange? Let’s look at an example in our environment. Do the Internet cables run separately from the provider to each building where access to the Internet is needed? No, because it is just not economical. The line is laid out along the buildings which need to access it. Not every building needs a separate line directly from the provider. It can use one common line, simply by connecting it using a suitable connector. Over time, other buildings can also connect to such a line and use the same link.
By analogy, buildings are the equivalent of applications that need access to data exchange. They can receive and transmit data without the need for a direct cable connection to each recipient. There is no need to connect them directly in pairs, because you can use one common data exchange medium where more applications which need data exchange will be able to connect in the future. ESB (Enterprise Service Bus) meets these types of expectations.
Let’s look at two ways of data exchange to understand their operation. The first is data exchange via point-to-point connections, the other uses the data bus. You can see the difference at first glance: the complexity of processes in the case of direct connections versus simplified and minimized processes using ESB.
Figure 1: Diagram of data exchange via point-to-point connections. Source: mulesoft.org.
Figure 2: Diagram of data exchange via ESB connections. Source: mulesoft.org.
Even more benefits of an ESB
The communication between applications is homogenous in the case of an Enterprise Service Bus implementation. They do not have to contact one another directly in a variety of configurations, because that’s what a data bus is for. You do not have to integrate every application with all those with which it should interoperate. Just one integration of the application with the bus is enough to exchange data with other integrated applications.
ESB is also a great way to integrate data between desktop and mobile systems. Mobile applications can become part of the business and internal systems via the bus. On one hand, ESB can support some part of the logic, e.g. transforms, and on the other hand, the application integrated into the bus can handle a given logic. This solution enables the building of a system compatible with SOA (Service-Oriented Architecture), which meets business needs and cuts down on user intervention in the technical details of the system. Services provided by the system can operate independently, but also carry out highly complex business processes when exchanging data.
Modification, extension or appending successive system elements does not involve much time and effort, because it is enough to integrate the new application using the bus based on any data exchange protocol. New connections need to be established between applications, end-points, and the ESB data bus after the implementation of the bus. You will probably need to leave the integrated systems and external applications in the currently used form of communication due to the requirements of partners and existing integrations, although there is the option of integration via ESB.
It should be noted that the diversity of connectors and flexibility of the bus enables the connection of even archaic systems, which exist in virtually every company which does business long enough. Legacy systems, despite the lack of reasonable interfaces, can be plugged into the bus and “revived”, or allow easy transfer of data to newer systems. All this is due to the fact that ESB enables the creation of standardized APIs for applications which do not have them.
Some more things to keep in mind before implementing an Enterprise Service Bus:
Figure 3: Diagram of the architecture using ESB
The use of ESB becomes reasonable when integrating three or more applications/services (it is better to use point-to-point connectors for two).
Reckless implementation of “future-ready” solutions is not a good idea, because you may find out that nobody will use them at all.
The implementation of the ESB will bring about the greatest benefits if using multiple communication protocols.
It is worth considering whether we need the messages routing capability, such as forking and aggregating the flow of messages, or a content-based routing.
ESB usually provides a robust and scalable set of services ready for use by other applications. It is good to identify the needs in this area.
It is better to avoid the implementation of the integration with more applications from the very beginning, but start with 3-4 systems and add other ones successively. This will allow you to avoid struggling with all problems at once and solve them gradually to facilitate the implementation of further integrations.
Like any new solution, the use of Enterprise Data Bus also involves consecutive stages of the implementation process, such as:
Launching the bus with several major integrations
Bus testing, resolving errors
Extending the bus by subsequent integrations
Bus testing/resolving errors
Acceptance tests of the complete implementation
Production roll-out of the bus
– Bus maintenance – Maintenance of the bus documentation – Extensions – Changes – Integrating new applications
First stage — analysis, is particularly important because it determines the success of the entire implementation. Your plan should be as following:
Specification of system requirements
Assessment of the reasonableness of implementation
Assessment of the feasibility of implementation
Analysis and developing of the information model at the company (what information is created and where, why and how it is related, processed, and transmitted)
Determining where the original (source, reference) versions of information are and where the secondary ones (e.g. copies) are
Organizing data exchange by bringing about the hierarchical network structure (source of reference and users)
Determining the discovered old and unused applications and those with duplicate functions
Disabling unnecessary applications or excluding them from integration
Determination of connectors and the data exchange logic for all applications integrated with the bus
Keeping documentation on changes/extensions/integrations of successive systems
System Analyst at Unity Group, graduate of Computer Science at Wroclaw University of Technology. His academic interests always revolved around business information systems. Advocate of consistency and usability, Piotr specialises in analysis and design of dedicated applications. Privately, an automotive and new technologies enthusiast.