Enterprise Application Integration (EAI)

As data-warehouse technology matured, another solution for integrating enterprise-wide data appeared, called enterprise application integration (EAI). In many cases, the functioning of one enterprise system is significantly enhanced if it requests data or immediately receives newly generated data from other systems. For instance, an inventory-control system could be much more responsive if it has immediate access to point-of-sale data so that it can monitor product movement in real-time. A bank could authorize a loan to a customer while the customer is sitting with the loan officer by analyzing in real-time the customer’s current business with the bank, his credit worthiness, and the current equity in his assets. This capability requires the enhancement of various legacy enterprise systems with different hardware, operating systems, and databases so that they talk to each other and exchange information in real-time or near real-time.

Intersystem Communication with EAI

EAI provides a means for linking such systems, in some cases without having to make sweeping changes to existing applications or data structures. In effect, whenever an event occurs in an application, it is sent via a publish/subscribe mechanism to other applications in other systems that may be interested in this event. Likewise, one system requests data from another system in real-time via a request/response mechanism. As shown in Figure 5, there are three primary techniques for implementing EAI – adapters, message-oriented middleware (MOM), and data replication. These techniques all perform the transformation of data to a common format to be used by all systems - the T in ETL.

EAI Intersystem CommunicationAdapters (Figure 5a) are specialized interfaces between diverse applications that allow those applications to directly communicate. Adapter services must be defined by a common interchange data format to which all systems conform for their interapplication communication. As a consequence, the use of adapters is invasive to applications. Each application involved in communicating with other applications must be modified to use the common data format for the messages that it sends and receives. This problem is aggravated by the fact that knowledge is required of the proprietary formats used by the application vendors, information which many vendors refuse to divulge. Therefore, adapter services are often only implemented by the application vendor or in close cooperation with the application vendor.

Furthermore, each application must be modified to know how to select events to send, to which systems to send these events, how to process events it receives from remote applications, and how to respond to requests from remote applications. Adapters are highly specialized. Each application requires its own adapter that connects to the adapter network and that translates its application interface and data formats into the common formats of the adapter network. Therefore, the range of applications that participate in an EAI network interconnected with adapters is limited. These problems all serve to inhibit the widespread use of adapters for application integration.

Message-Oriented Middleware (MOM) (Figure 5b) provides a mechanism for asynchronously sending messages between applications. Messages are placed in queues and are sent to their destinations when the destinations are available. MOM is used to queue messages describing events and to send those messages to remote applications that take actions based on those events. MOM also provides the messaging facilities to send requests to other applications and to receive responses to those requests.

However, the use of MOM in EAI suffers many of the same problems that adapters face. For one, MOM is invasive to the application. As with adapters, MOM requires that a common data format be established among all systems so that they understand each other’s communications. Applications must be modified to conform to this format when they communicate with other applications. In addition, applications must be modified to know when to send messages and how to respond to messages from other applications. As a consequence, MOM has found marginal use for application integration.

Data Replication Engines (Figure 5c) exchange data between systems at the database level. This data may take the form of events generated by one system for consumption by other systems, or it may take requests from one system to other systems and the responses to those requests.

Notice that this description has used the term “system” rather than “application,” as was used in the descriptions for adapters and MOM. There is no integration with the data-replication engine at the application level. Rather, replication works at the database level. Replication occurs without any modification to the applications, and applications are typically unaware that replication is taking place. The only application modifications necessary are those required to implement new functions that make use of information from other systems.

A replication engine monitors changes to a system’s database. It is driven by a change log or by database triggers. Based on rules incorporated into the replication engine, certain database changes found in the change log are sent to other systems in the application network. Alternatively, database triggers pass critical changes to the replication engine for dissemination to other systems. These changes are placed in tables in the target systems and are used by new applications implementing new functionality for the enterprise.

In addition to the advantage of noninvasiveness described above, data replication has other advantages over adapters and MOM. For one, with adapters or MOM, the unavailability of the network or another system brings down the application because it no longer has access to the data that it needs. With replication, applications continue to function in the presence of network failures or external system outages by using the local copy of the application database.

Data replication also brings locality to the application since the application has access to a local copy of the database. It does not reach across the network for data, and performance is therefore greatly enhanced. A common interchange data format is not required when using data replication engines, since the transferred data must be put into the target tables according to specified formatting rules. The transformation of source data to target data is performed by the data-replication engine according to transformation rules specified by the user and incorporated directly into the engine.

Another advantage of data-replication engines is that security is significantly simplified. With adapters and MOM, every application is an interface and must be secured via encryption. Data-replication engines present only a single interface that must be secured. Not only is the management of security simplified, but the number of points that are attacked is also reduced. Since data replication avoids the application-specific problems associated with adapters and MOM, this mechanism has become the technique of choice for EAI.

EAI Networks

EAI networks may either be mesh networks or hub networks. In a mesh network (Figure 6a), every system is potentially connected to every other system. In a hub network (Figure 6b), all systems communicate via a central hub that passes data to the target systems.

EAI Networks

A problem with a mesh network is the number of possible connections. If there are n systems in the EAI network, there may potentially be n(n-1) connections if all systems have to communicate with all other systems. If a new system is added to the network, n new connections may need to be established, one to each of the existing systems. This number of connections creates a network complexity that rapidly becomes unmanageable as the network grows.

A hub network eliminates this problem. Each system is connected only to the hub. The hub receives messages from each system and routes each message to the appropriate target systems. A problem with a hub is that it is a single point of failure. A hub failure disrupts all functions that require data from other systems. Thus, all benefits of EAI are lost during a hub failure. Since these messages are functions that are tactical in nature, governing the real-time response of the enterprise to events as they happen, enterprise operations may be severely compromised following a hub failure. Therefore, the hub must be redundant so that it is highly available.

Another difference between mesh and hub networks is that in a mesh network, every system must know the data format of every other system. Each system must be able to transform its data format into those of the other systems. In a hub network, however, the hub is responsible for data transformation. Each system need only know the data format expected by the hub.

EAI and Business Intelligence

Using these techniques, EAI allows applications to be integrated on a real-time basis as events occur and as databases are updated without having to wait hours, days, or weeks for data from a data warehouse. Consequently, EAI supports tactical decision making very well. However, it does not provide the historical data needed for strategic decision making.

Real-Time Business Intelligence Pages