Try TT Now

Disconnects, Crashes, and Restarts

What Happens to Algorithm Definitions

When you need to restart an Algo SE Server, either due to a crash or as part of general maintenance, Algo SE loads all persisted algorithm definitions and makes them available to traders. All algorithm definitions that were not specified as persistent when they were deployed will not be available to traders.

Algorithm Ownership

The most recent X_TRADER to perform an action on an algorithm is referred to as the instance owner. For example, when a user runs an algorithm, their X_TRADER is the instance owner. If another user with whom the instance owner shares an order book on both Algo SE and the destination gateway modifies the running algorithm, the X_TRADER of this other user becomes the instance owner.

What Happens to Running Algorithms

The following scenarios explain what happens to algorithms and child orders when connectivity is lost in different situations, including when:

  • Algo SE stops normally
  • Algo SE crashes
  • X_TRADER stops normally
  • X_TRADER crashes
  • Algo SE and X_TRADER lose connectivity with each other
  • Algo SE loses connectivity with a TT Gateway

Scenario 1: Algo SE Stops Normally

If you stop an Algo SE Server as part of normal operations, the Algo SE Server performs the following actions for each running or paused algorithm:

  • Stops all running algorithms.
  • Determines which of the following actions (if any) to take on child orders for which the corresponding TT Gateway is available:
    • Submit delete requests for all orders that are children of the ASE Block.
    • Submit update requests to remove the OMA indicator, i.e. the related ID field of the child order controlled by an OMA on the server, from all child orders of OMA algorithms.
    • Submit update requests to remove the OMA indicator, i.e. the related ID field of the child order controlled by an OMA on the server, from all child orders of non-OMA algorithms whose Leave Order On Cancel setting is set to true.
    • Submit delete requests for all child orders of non-OMA algorithms whose Leave Order On Cancel setting is set to false.
  • Deletes all parent algorithms regardless of the success or failure of the deletion and/or update requests on their underlying child orders.
Note

OMA is an abbreviation for Order Management Algorithm and is defined as any algorithm that takes a TT Order Key as an input variable. See the ADL User Guide for more information.

Scenario 2: Algo SE Crashes

If Algo SE crashes, X_TRADER detects that it has lost connectivity to Algo SE, which can take as long as 30 seconds.

If the instance owner of an algorithm was connected to Algo SE when it crashed, the instance owner’s X_TRADER deletes all non-ASE child orders if either of the following are true:

  • The Client Disconnect Action parameter of the parent algorithm is set to either Pause and the child order’s Leave Orders on Pause parameter is set to false.
  • The Client Disconnect Action parameter of the parent algorithm is set to either Cancel and the child order’s Leave Orders on Cancel parameter is set to false.

The instance owner’s X_TRADER will also delete all ASE child orders if the Client Disconnect Action parameter of the parent algorithm is set to either Pause or Delete.

If the instance owner of an algorithm was not connected to Algo SE when it went down, when the Algo SE Server restarts, it downloads all child orders from all TT Gateways, then performs the cleanup actions required of the server as if it were being shut down cleanly.

Scenario 3: X_TRADER Stops Normally

If a trader is the instance owner of running algorithms and tries to close X_TRADER, X_TRADER displays the following message:

If the trader chooses to exit anyway, the Algo SE Server will continue running these algorithms, but only if they do not rely on Microsoft Excel links. It will pause all algorithms that rely on Microsoft Excel links.

Scenario 4: X_TRADER Crashes

If XT crashes, Algo SE detects that it lost connectivity to X_TRADER, which can take as long as 30 seconds.

Once the Algo SE Server detects that it lost connectivity to an X_TRADER client which is the instance owner of running algorithms, Algo SE examines the value of each algorithm’s Client Disconnect Action and responds as shown in the following table.

Parameter Value

Result

Leave

Algo SE leaves the algorithm running.

Pause

Algo SE pauses the running algorithm, then checks the value of the Leave Orders on Pause parameter for each child order of this algorithm.

  • If enabled, Algo SE leaves the child order in the market.
  • If disabled, Algo SE cancels the child order.

Note: It is not permissible to enable the Leave Orders on Pause setting for ASE orders. In other words, the Algo SE Server always deletes ASE child orders when the parent algorithm is paused.

Delete

Algo SE stops the running algorithm puts it into Failed state, and then checks the value of the Leave Orders on Cancel parameter for each child order of this algorithm.

  • If enabled, Algo SE leaves the child order in the market.
  • If disabled, Algo SE cancels the child order.

Note: It is not permissible to enable the Leave Orders on Cancel parameter for ASE orders. In other words, the Algo SE Server always deletes ASE child orders when the parent algorithm is cancelled.

Note

Algo SE always cancels running algorithms whose input parameters contain Microsoft Excel links.

Scenario 5: Connection Between X_TRADER and Algo SE Is Lost (X_TRADER and Algo SE are up)

If X_TRADER and an Algo SE Server lose connectivity with each other, it can take up 30 seconds for either to detect it.

  • If Algo SE detects that it has lost connectivity to an X_TRADER client that is the instance owner of running algorithms, it performs the steps detailed in Scenario 4.
  • If X_TRADER detects that it has lost connectivity to an Algo SE Server on which it has running algorithms, it will perform the steps listed in Scenario 2.

Scenario 6: Algo SE Loses Connectivity to a TT Gateway

When an Algo SE Server loses connectivity to a TT Gateway, it pauses all algorithms that use that TT Gateway, whether or not an algorithm has any currently-working orders on the Gateway. These algorithms cannot be resumed until the Algo SE Server reconnects to the TT Gateway.

Market State Changes

During the design process, an ADL developer can specify the behavior of an algorithm when one of the markets through which it is routing orders or listening for prices closes. For each algorithm, the developer can enable or disable the Ignore Market State configuration option to instruct Algo SE whether to pause the algorithm if any of the markets for which it has an instrument closes. For more information, refer to the ADL User Guide.

When Disconnections Between Algo SE and FIX Sessions Occur

When connections between FIX sessions and Algo SE are interrupted, the following actions can occur:

  • If all FIX sessions are disconnected, and the Order Server shows as down in Guardian, Algo SE pauses all algorithms that use that TT Gateway, whether or not an algorithm has any currently-working orders on the Gateway. These algorithms cannot be resumed until the Algo SE Server reconnects to the TT Gateway.
  • If a FIX session is disconnected, but the Order Server remains active in Guardian, the Order Server rejects new orders and in the following order:
    • If order action is an add request, the algo pauses.
    • If order action is a reject to external modification, the reject is ignored and the algo keeps running.
    • If order action is a change or cancel-replace and the working quantity is greater than 0, Algo SE retries the action once and then pauses it if the retry is rejected.