HP Reliable Transaction Router (RTR)

The 'Enterprise' Solution

Enterprise Functional Overview

The Challenges

Customer expectations have changed in todays' business environment. Corporate-Customer relationships are more complex and as a result, companies are providing an ever increasing array of products and services to accommodate them. The task of managing the vast amount of data across heterogeneous systems and networks has become a challenge.

Organizations are also under pressure to differentiate and maximize the profitability of their services while increasing market penetration through effective cross-selling. The foundation for this activity is understanding customer relationships across all lines of their business consistently, regardless of geography. Getting data from all sources, including legacy systems, in real time to an integrated enterprise-wide information management system has proven extremely difficult.

In addition to this, working in a global economy, the business day is now twenty-four hours long, seven days a week. To service this need, systems must now be integrated around the world and provide virtually zero down-time.

Oasis' and Digital's' 'Enterprise' represents the first and foremost solution to these complex business problems.

The Solution

The only long term, scaleable solution is to implement a unified, consistent, yet flexible approach to handling transactions and other messaging data across the entire organization - regardless of the source or destination. To this end, Digital and Oasis have jointly created 'Enterprise'. This is an integrated software environment allowing fault resilient transaction processing and delivery, real time and batch, between virtually any end points within your organization.

Enterprise is composed of Digital's Reliable Transaction Router (RTR) technology working in harmony with Oasis Technologies Information Switching Technology / Message Services Infrastructure (IST/MSI).

Integrating Enterprise into existing business systems is possible with Enterprise's Open Systems Architecture that can operate on your existing platforms and interface with existing transaction and message protocols. Data in legacy applications can be accessed to support improved account management and new business opportunities.

With Enterprise, transaction and data integrity is insured by using techniques such as message timers, two phase commit, disk shadowing, site shadowing and journaling. By automatically managing system failures, lost or duplicated data is avoided. Data can now be collected from all lines of business and across all geographies in accordance to the data model established.

With the Enterprise solution; in conjunction with other data mapping, modeling and integration services; Digital and Oasis help organizations avoid such problems as a lack of data integrity across multiple data-stores, inaccessible data or information relationship-blindness. Inconsistent views of data can now be eliminated.

Consistent and reliable support is provided for a full array of information management tools by focusing on the key ingredient - data - and insuring this data flows reliably and uniformly across the entire enterprise in real time or batch. Enterprise provides the means for managing organizational data effectively in order to deliver higher quality services with increased cross-selling; ultimately allowing improved profitability and customer loyalty.

IST/MSI Overview

Oasis' IST/MSI is an integrated real time and batch transaction switching and processing environment, designed specifically to handle distributed data across heterogeneous systems and networks. IST/MSI provides virtually unlimited message formats and message translation.

IST/MSI's backbone architecture is IST/Foundation, thereby taking advantage of the rich transaction processing functionality available within that Oasis product. IST/MSI also provides software for Session Management and high-performance database access.

One of the main features of IST/MSI is its flexibility and scalability. It can run on hundreds of small-end platforms residing in branches or customer call centers, for example; or it could also run on one large installation sitting in a datacentre; or any configuration in between.

In addition to the transaction processing power IST/MSI provides, many special features were built to enable the IST/MSI software to work hand-in-hand with Digital's' RTR (Reliable Transaction Router) product in order to provide the true 7/24 operations our customers' demand.

RTR Overview

RTR is just as flexible and scaleable. Where necessary, it can accompany IST/MSI into whatever configuration is required by our customers.

The DIGITAL Reliable Transaction Router (RTR) is disaster-tolerant, middleware for business applications that require the reliable execution of transactions in real time and without any interruptions of service across widely distributed systems. RTR provides reliable transactional messaging, with ACID properties, between distributed client, router and server systems. RTR takes responsibility for detecting failures in a distributed environment and providing transparent fail-over and transaction integrity for uninterrupted service. RTR insulates applications from software, network and site failures.

RTR allows the application designer to decide where to place the application data. RTR allows data to be partitioned over as many nodes and disks as the performance or business requirements dictate. The location of the application data is transparent to the applications, allowing for easier system reconfiguration and horizontal growth.

RTR applications may perform transactions, which access more than one database. RTR insures (in cooperation with the application) that transaction operations are either all applied to the databases or are all rolled back. This is known as all or nothing transaction semantics (2 Phase-Commit).

RTR lets the application designer concentrate on the application, rather than the problems of handling the network. RTR simplifies the design of applications that are distributed across the network, and preserves data-consistency even if components of the system fail (including the application themselves).

RTR provides a high degree of failure resilience and allows continued system operation in case of node or link failures. RTR supports shadow and standby operations to protect against entire site failures. RTR guarantees transaction integrity against many kinds of failures by using a transaction recovery feature.

Enterprise Overview

Together, RTR and IST/MSI comprise Enterprise.

Functionality

The ability to process financial and non-financial transactions over large heterogeneous networks;

The ability to process an unlimited number of message formats;

The ability to handle an unlimited number of devices;

The ability to process messages up to 4 gigabyte in size;

High performance transaction switching;

Seamless scalability;

Open API's so that customers can build their own customized solutions;

Plug and play capabilities with other Oasis IST products;

Parallelism; the ability to have multiple Enterprise environments operating within one hardware platform;

Open-systems architecture running under Unix and NT;

A comprehensive Data Base Management layer allowing you to connect to a range of databases (i.e. Oracle, Sybase, Informix, etc.);

'Transaction Spawning' (one-to-many) provides you a unique solution for highly complex transactions;

Proven quality and reliability;

Operability

• Provide automated and instantaneous fail-over capabilities to allow for a true 7 by 24 operating environment;
• Comply with disaster recovery strategies that call for geographically separated datacentres;
• Ensure commit order integrity across distributed and/or mirrored databases;
• Ability to work in a lights-out datacentre paradigm;

IST/MSI is based on proven technology and available products. Today, IST provides financial transaction and messaging services at over 120 sites in more than 40 countries. Citibank, MasterCard, Visa, Diners Club, Europay, Bejing Golden Card and many others are using Oasis' Information Switching Technology to route financial messages from ATMs and POS terminals, provide EFT transaction routing, provide full or partial authorizations, transaction reporting and electronic draft capture (EDC), and store-and-forward processing all in a completely secure, encrypted environment. It supports many shared-cash networks, such as VISA and MasterCard. It has been distributed across multiple machines on both Local and Wide-Area Networks.

RTR is used worldwide by some of the world's most demanding clients to provide a distributed fault-tolerant infrastructure for their transaction systems. Many of the world's stock and option exchanges use RTR; Swiss Electronic Bourse, American Stock Exchange, Hong Kong Futures Exchange, Amsterdam Stock Exchange, Australian Stock Exchange, Ced Borsa (Milan) and others. RTR can be found where transaction integrity is a must; Banking, Citibank, Bank Austria; Transportation, Polish Railways (PKP), India Railway and Information Services, Reuters. At over 250 customers RTR provides transaction reliability, every day, all day for every transaction.

Meeting Your Business Needs

Enterprise is ideally suited to help your organization integrate their transaction based systems and data; on a regional or global basis. Some typical uses for the product are :

• Customer Relationship Management
• Distributed Data Management
• Relationship Banking
• Call Center
• Financial Transaction Processing & Electronic Commerce
• Transaction Routing
• Internet Banking

Enterprise can work hand-in-hand with the entire Oasis family of products. As such, customers have the flexibility of taking advantage of these plug-and-play systems :

• IST/Switch - to handle all ATM, POS, Credit and Debit Card processing needs;
• IST/Interchange - is a fully integrated Clearing and Settlement Package;
• IST/Card - is a back-end Card Processing system for issuers and acquirers;

The Oasis Family of Products is illustrated in the following diagram.

* Digital RTR is a licensed product of the Digital Equipment Corporation.

Enterprise Architecture

The transaction processing engine of Enterprise resides within IST/MSI component. IST/MSI uses four conceptual layers to perform processing: the Communications Driver, the Agent, the Transaction Managed Processor (TMAP) and the Gateway(s). The communication driver forms the link between outside networks and IST/MSI. The Agent(s) work as translators converting data from some external format into the IST/MSI internal format. The TMAP processes and routes the transactions. The Gateways allow MSI to interact seamlessly with Digital's Reliable Transaction Router (RTR) product.

Conceptual Enterprise Architecture

IST/MSI communicates between all processes using ELF (Elementized Format) messages (based on the ASN.1 Standard). ELF is a series of containers that can hold arbitrary data fields, or groups of data fields. The result can be a simple linear set of data fields to a complex hierarchical structure. IST/MSI uses only ELF messages to communicate internally, both for transaction processing and process administration.

The following diagram shows a typical Enterprise configuration, allowing for 7/24 operation.

Conceptual Enterprise Configuration

 

Spawned Transactions

Enterprise provides for the concept of 'Spawned Transactions'. This means that a single transaction initiated at some end-point (super-transaction), may automatically spawn multiple sub-transactions within Enterprise.

Why is this an advantage? Here is an example. A customer enters a bank branch and wishes to have a complete profile of all the business that they do with this bank. This would mean all the household information, account information, brokerage information, insurance information, etc. available on that individual.

This would be a very large transaction that may have to go to multiple host and local databases in order to retrieve the desired data. By splitting the transaction into sub-transactions, the bank representative is not forced to wait for the entire transaction to complete before being able to begin serving this customers needs. As the sub-transactions complete, the representative will be able to see more and more profile information on that customer. If one sub-transaction fails, it does not mean that the entire query would fail. A diagrammatic representation of this can be found in the following section entitled "Example Enterprise Application".

 

Example Enterprise Application

In the above example, a client terminal initiates a request (1) to Enterprise. That request goes to a local database and retrieves some information (1a) and returns it back to Enterprise. Based on some of the information returned, Enterprise automatically spawns another transaction (2) which is sent to Host 1. Simultaneous to this, response 1a is sent back to the requester.

Host 1 responds with response 2a. Based on data within this reply, two additional transactions are generated (3 and 4). These are sent to Host 2 and Host 3, respectively. The first host response (2a) waits in Enterprise until a 'More' request is initiated by the client. A 'More' request is asking if there is any further data to the original request. At this point, the 2a reply is sent back to the requester.

At some point in time, Host 2 and Host 3 reply back to Enterprise. Their respective replies (3a and 4a) are suspended (held) within the system. When an additional 'More' request is initiated, those replies are sent back to the client. In this example, that original transaction is now complete.

RTR Architecture

The fault resilient capability of Enterprise is achieved through the RTR components. RTR divides the transaction system into clients (requesters), servers and the interconnection between them, the router. The client component of RTR manages the interface between the user and RTR router.

The RTR Server component manages the interface between the server and RTR router; the router manages the integrity of the interconnection between the client and the server. Together they isolate the user and the server from failures, in the network and the servers and the need to know when a client or server has moved or been added.

RTR architecture considers each RTR component as running on a node; front-end nodes, back-end nodes and router nodes as shown above.

The RTR client processes run on the front-end nodes. This layer allows computing power to be provided locally at the end-user site for transaction acquisition and presentation.

The RTR server processes run on the back-end nodes. This layer allows the application (and database) to be distributed geographically, and permit fault tolerant systems to be constructed which cope with either node or site failures.

The RTR router normally contains no application software. This layer reduces the number of logical network links required on front-end and back-end nodes. It also de-couples the back-end layer from the front-end layer so that configuration changes in the (often rapidly changing) user environment have little influence on the transaction processing and database (back-end) environment.

Although RTR is based on a three-node model, it is not necessary for all three nodes to be physically present. For systems with few front-ends, the router and front-end layers can be combined. The physical locations of the front-end, router and back-end nodes are specified using RTR configuration commands.

When configuring and managing RTR to provide the level of fault tolerance desired the user can specify a server to have a specific role.

Shadow servers are separate back-ends, which handle the same transactions in parallel on identical copies of the database. The databases are mirrors. In Figure 5 the main server (Smain) is at site 1 and the shadow server (Sshdw) are at site 2 both receive every transaction. Should site 1 fail, site 2 continues to operate, and the service is uninterrupted. When the failed back-end node restarts, the transactions, which had not been committed by the servers at the time of the back-end failure, are replayed to the servers from a disk journal.

Standby servers are "spare" servers, which automatically take over from the main back-end if it fails. This take-over is transparent to the application. The above diagram shows a possible RTR configuration. The main server (Smain) runs on one node while the standby server (Ssby) runs on the other node (both have access to the same database). The standby server is active, but RTR does not pass transactions to it. Should the first node fail, RTR starts passing transactions to the standby server. Concurrent servers are servers running on the same node. This is useful for increasing throughput, especially when the database only allows one active transaction per process.

Since the RTR Router is the heart of the fault management, multiple routers are usually configured. Front-end nodes will automatically find another router if the one being used fails. This reconnection is transparent to the application. Front-ends may also be configured to choose routers with the least loading. If the original router coordinating a transaction fails, the back-end nodes select another router that can ensure correct transaction completion.

IST/MSI Architectural Services

IST/MSI architectural services consist of five elements:

• Mailbox Manager;
• Task Manager;
• Port Manager;
• Event Manager;
• Application Manager;

Mailbox Manager

A mailbox is a basic resource that all IST applications use to communicate with other applications. It is essentially an address, or a location on the network, where processes may send and receive messages.

The mailbox manager provides the run-time message communications environment for IST/MSI applications, employing mailboxes that use semaphores and shared memory to allow inter-process communication. The IST/MSI communications components responsible for routing transaction messages through the system all utilize the mailbox system. These components each possess an inbound and outbound mailbox to simplify the direction of message traffic within the system.

Mailboxes are unique to the applications that use them. Each application may have one or more mailbox sub-systems operating on their behalf. By analyzing mailbox attributes, the Mailbox Manager will determine the appropriate message routing within the system.

Task Manager

A task is any process that is running or will be run within the system.

The Task Manager creates and monitors tasks (processes) within the system and ensures that they execute properly. If a task should fail, the Task Manager will attempt to re-initialize it.

Port Manager

A message port is an intermediary between a communications driver in IST/MSI and a physical communications link to an external network or device. It provides the actual link to the network.

The port manager starts and monitors system tasks associated with message ports. For example, it starts each communication driver in the system. The port manager is responsible for establishing, maintaining, monitoring, and shutting down the connections between tasks and the communication drivers; it is not involved in the actual transfer of transaction messages throughout the system.

The port manager also permits multi-port management and control.

Event Manager

The event manager is an IST/MSI service that handles timer events for the system as a whole. Applications responsible for message routing call upon the event manager to time transaction responses from devices, other IST/MSI applications, and hosts or external networks. Once an event is created, the event manager monitors it and notifies the requesting application, through its mailbox, when and if the event expires before the transaction processing is complete.

Application Manager (APM)

The Application Manager (APM) tracks the initialization/state of each process according to run-level. A run-level is a global stage of IST/MSI's initialization. To prevent data sharing problems, IST/MSI processes are started, run, and terminated according to run-levels, which the APM manages. The APM also receives manual requests from a command level interface, allowing the management of higher level processes.

General Platform Services (GPS)

General Platform Services (GPS) is a layer within IST/MSI that provides key services to other components of IST/MSI, such as the mailbox subsystem, task management, application management, and file service handling.

Communications Drivers

A communications driver is responsible for sending a message to the Agent. It strips the header information from the message and performs a parity check to verify the integrity of the data. Similarly, the Agent sends messages directly to a specific communications driver that has been written for a specific network. The communications driver sends the message to a network through its port.

The communications driver is programmed to handle a specific networking protocol. A switch handling multiple protocols will have multiple communications drivers. Typical examples of communication drivers would be X.25 and TCP/IP.

MSI Agents

The IST/MSI Agent is a data normalizer. It functions as a translator between message formats. It determines a message's type, source, and destination. The Agent then creates a destination message from this information and sends it to the appropriate process, via the mailbox sub-system.

The Agent transmits messages between the Communications Driver and the TMAP. Each Agent is configured to translate a specific message format and to communicate with a specific Communications Driver. However, the Agent is not limited to a specific TMAP process, and may send messages to any TMAP process. Typical examples of Agents are ISO8583 translators, CGMS translators, Visa agents, Mastercard agents, etc.

Transaction Managed Processor (TMAP)

The Transaction Managed Processor (TMAP) provides the application functionality for transaction processing. The TMAP processes transactions by following scripts contained in the Transaction Document (TDOC). The TMAP reads the instructions contained in the script and executes the Application Units (AUs) specified.

The TMAP exists in memory as a combination of the TMAP modules, which use a series of resources to process transactions. The TMAP consists of the TMAP driver, the Classifier, the Parser, and the Script Executor.

IST/MSI TMAP

Classifier

The Classifier determines message types. It looks at an ELF message, reads its data elements, and determines which script the TMAP will use to process the transaction. The Classifier then returns the ID of the script to the TMAP.

Transaction Document (TDOC)

The TDOC contains scripts that describe how to complete an application process. Each script consists of several steps. The steps of a script direct the TMAP to call Application Units (AUs) to evaluate or calculate data for the transaction being processed.

The following diagram represents a TDOC as seen in a theoretical two-dimensional view.

Sample TDOC

IST-RTR Client & Server Gateways

These processes provide the interfaces between the IST/MSI and the Digital RTR product. These gateways link the IST/MSI transaction id's to RTR channels. They also provide for parts of the protocol used for transaction execution.

TMAPR

TMAPR is a specialized TMAP which interfaces with the RTR channel model. It provides a tightly coupled scheme for ensuring that RTR transactions are equivalent to database transactions. This is because RTR considers the activity on the RTR channel to be directly indicative of the state of the database transaction.

In order to fully implement the IST/MSI Two Phase Commit protocol, a standard application skeleton is employed within TMAPR.

ELFSQL-SP

Elfsql-SP is usually used within TMAPR. It provides a high performance interface to database systems that support stored procedures. It translates the IST/MSI internal message format (ELF) directly into stored procedure arguments, and vise-versa. All execution is pre-parsed where possible to insure optimum database performance.

Application Unit

The Application Unit (AU) is the code that is executed under the direction of the TDOC script, and performs all of the transaction-related application processing for the TMAP.

Conclusion

Oasis Technology Ltd. and Digital Equipment Corporation provide a complete solution for your enterprise-wide transaction integration, switching and messaging business needs. A full range of services, including support, education, publications, and project management accompany the Enterprise solution.

The IST suite of products is a highly adaptable set of switching software and application 'building blocks'. It can be modified to suit any business based transaction processing requirement. There are no limitations to the number of devices it can control and it can operate on any UNIX or NT platform. The architecture is scaleable from small branch or regional applications to global enterprise solutions. Many of IST's features are user-configurable, allowing users to personalize and customize the product to suit specific needs.

RTR provides a geography independent delivery mechanism for fault resilient transaction processing over heterogeneous networks.

IST/MSI is a 'Universal Message Integrator', allowing our customers to integrate, process and communicate data from many disparate sources regardless of where the data is in the world.

Enterprise allows them to do it every minute of every day.

Citibank, one of the largest banks in the United States found this to be true. So will you.

How to make it work for you?

Digital and Oasis offer a range of services to help you be successful with Enterprise. Digital brings solid experience in managing large complex integration projects. As a Digital partner Oasis Technology has proven product, technical and support expertise in the field of transaction processing. As a team Digital and Oasis have a proven track record of success.

Digital and Oasis have the experienced staff to help you select the features of Enterprise you need, design the distributed Enterprise environment required to service your organization, and install your systems.

We can help you develop the applications and build your databases.

Digital has technology migration services to help you move existing applications to Enterprise.

Digital and Oasis offer educational seminars, self-paced training and classes to develop the skills of your staff to fully exploit the features of Enterprise.

Enterprise from Digital and Oasis, a solid foundation for the next millennium.