Try TT Now

Exchange Host Failure and Recovery

Overview

Several types of failures can occur between the customer’s network and the exchange. Such failures can involve network devices, the MX price and order environments, and the exchange matching engine. Of these failures, this section discusses three primary types:

  • Exchange Host and Trading Engine failure
  • MX Routing Engine server failure (HSVF)
  • MX Routing Engine server failure (SAIL)

Exchange Host and Trading Engine Failure

In the event that only the MX exchange host fails, all traders on the TT System experience the following:

  • Orders: MX rejects all submitted orders.
  • Prices: Prices go stale.

However, the TT Gateway continues to run and remains connected to the MX’s Routing Engine servers.

When the exchange host again goes live, the TT Gateway receives the appropriate HSVF and SAIL messages. Traders can begin trading normally again and prices begin updating as normal.

Routing Engine Server Failure (HSVF)

When a Routing Engine server fails that is providing HSVF prices in the MX’s environment, any Price Servers connected to it shutdown after the number of seconds set in the heartbeat time-out interval programmed in the Price Server. These Price Servers appear red (inactive) in Guardian. During the HSVF server failure, all prices remain stale.

HSVF Price Delivery Failure

When a Routing Engine server fails that is providing HSVF prices in the MX’s environment, any Price Servers connected to it shutdown after the number of seconds set in the heartbeat time-out interval programmed in the Price Server. These Price Servers appear red (inactive) in Guardian. During the HSVF server failure, all prices remain stale.

You must verify that the price feeds are receiving data. For further information, refer to Verifying Connectivity.

Routing Engine Server Failure (SAIL)

When a Routing Engine fails that is managing orders via SAIL, all Order Routers that connect to that server lose their connections. Order Routers on the MX Gateway begin cycling by attempting to connect to the exchange IP address listed in the hostinfo.cfg file.

The MX exchange host may or may not retain orders that reached the exchange prior to the failure. Traders must call MX for an accurate accounting of their orders.

For details on Order Router cycling and recovery, refer to Gateway Recovery: Order Routers.

MX Recovery Processes

Depending upon the particular type of failure, the following events occur (reconnection does not occur in any particular order):

  • When the HSVF server recovers, the Price Server reconnects to the MX exchange host. Prices go live again.
  • When the SAIL server recovers, the Order Routers on the MX Gateway reconnect to the MX exchange host. Traders can now trade.
  • Fills: Unless fills were dropped by MX, traders have access to their fills for the entire trading day. Traders see their fills for the day downloading again after the recovery.
  • Orders: Due to the possibility of orders being dropped by the MX during a SAIL server or MX exchange host crash, traders must call the exchange and verify their working orders.
  • Positions: Because the MX exchange host may drop fills or orders during a failure, traders may have incorrectly calculated positions after the gateway reconnects. To determine correct positions, traders must call MX and inquire upon all orders in their Order Book.