Disaster Recovery Documentation
Exchange Host Failure and Recovery
If an exchange host fails, all traders on all ISVs lose their connections. These traders cannot reconnect until the host is again operable. In addition, the customer experiences the following:
- Orders: All orders in the TT system at the time of exchange host failure become inactive and are dropped. Orders in transit before the failure are rejected on reaching the exchange. Furthermore, specific exchanges handle orders differently such as dropping orders or putting them on hold.
- Fills: During a host failure, the Exchange host does not send fills. If the host fails while in the process of generating or sending a fill, that fill can be dropped as well.
- Prices: During a host failure, all prices are stale.
- Spreads: At the time of failure, the Exchange host can delete all spreads.
For further information on exchange host failure, refer to the disaster recovery chapter of the SAM specific to the TT Gateway you use.
TT Gateway Cycling
When the Exchange host fails, the TT Gateway acknowledges the broken connection. At this time, the TT Gateway releases all X_TRADER® and begins cycling. As the TT Gateway cycles, it continually attempts to reconnect to the Exchange host.
TT recommends that traders document their positions at this time.
During an exchange host failure:
- Do not shut down the TT Gateway. The TT Gateway continues to cycle until it reconnects to the exchange. Barring any other issue, do not shut down or reboot the TT Gateway. On certain exchanges, shutting down the TT Gateway can cause additional problems.
- Do not close out of X_TRADER®. X_TRADER® continually attempts to connect to the TT Gateway. When the gateway reconnects to the exchange, X_TRADER® logs back in. While the Exchange connection is down, if a trader closes X_TRADER®, and attempts to log in and open his workspaces again, the X_TRADER® Login window pops up approximately every five seconds as it continually attempts to reconnect to the exchange. Traders should move the window to the edge of the screen so it does not interfere with their activities.
When the exchange host recovers, the following items occur:
- Immediately after the Exchange host recovers, a large number of traders and ISVs attempt to reconnect to the Exchange Host. Due to this traffic, it can take some time before the TT Gateway finds a connection.
- Fills: Client applications begin accepting fills again. However, during failure, the Exchange host may have dropped a fill before it could reach the TT Gateway. These fills will not be deliverable to the TT Gateway. Except for dropped fills, traders have access to all of their fills for that trading day. If the TT Gateway was rebooted during the failure, traders will see their fills for the day re-downloading.
- Orders: When a trader reconnects, the TT Gateway resends all Order Book information to X_TRADER®.
- Prices: Prices become current after traders log in and re-subscribe to specific products by opening their grids and workspaces.
- Position: Because the Exchange host can drop fills during a failure, upon reconnecting, traders may have incorrectly calculated positions and/or Order Books. If the trader’s information is incorrect, he must call the exchange to determine why any discrepancies exist. If necessary, traders can enter manual fills to correct these errors.
- Spreads: Traders may need to recreate their spreads upon reconnecting to the exchange.