Try TT Now

NIC Settings

These are explanations of the additional NIC settings that are implemented per TT’s recommendations.

NIC Setting

Description

Checksum Offload = OFF

This setting was designed to allow the NIC to carry out checksum calculations rather than the application to reduce application overheads (e.g., Is this a good packet? If not, then drop).

Intel Quote: “Checksum Offload enables the adapter to verify the TCP/IP checksum on received packets (Rx) and compute checksum on transmitted packets (Tx). Enabling this feature may improve TCP/IP performance and reduce CPU utilization. With Offloading disabled, the operating system calculates and verifies the TCP/IP checksum.

It has been shown in practice that this setting impacts the performance of the trader’s workstation and as such TT recommends that it is disabled.

Large Send Offload = OFF

This setting was designed to allow the NIC to build larger packet sizes and reduce the overhead of multiple calls down the TCP stack, utilizing the standard 1500 byte size frames which would ultimately reduce host processing. This is more efficient if working in an ‘admin’ type network environment, but it has been shown to impact trader performance so TT recommends that it is disabled.

IBM Quote: “If your primary concern is lower host CPU utilization, use this option. However, for best raw throughput, you should not enable this option because the data rate on the wire is slower with this option enabled.

TCPChimneyOffload = OFF

Microsoft Quote: “TCP Chimney Offload is a networking technology that allows the work associated with moving data across a network to be offloaded from the host computer’s CPU to the network adapter. This helps improve the processing of network data on your computer or server without the need for additional programs or any loss to manageability or security. Programs that are currently bound by network processing overhead will generally scale better when used with TCP Chimney Offload.”

In reality, this setting has proven to slow down the throughput of data packets from the wire up through the TCP stack to the application (e.g., X_TRADER® Remote) – as TT data is packet-based, not stream-based.

TX IP Checksum offload/Offload Transmit IP Checksum (Broadcom/Intel)

This setting enables the adapter to compute the IP checksum of outgoing packets.

  • With Offloading disabled, the OS verifies the IP checksum.
  • With Offloading enabled, the adapter completes the verification for the OS.

As with Large Send Offload, “for best raw throughput, you should not enable this option because the data rate on the wire is slower with this option enabled.”

RX IP Checksum offload/Offload Receive IP Checksum

This setting enables the adapter to verify the IP checksum of incoming packets.

  • With Offloading disabled, the OS verifies the IP checksum.
  • With Offloading enabled, the adapter completes the verification for the OS.

As with Large Send Offload, “for best raw throughput, you should not enable this option because the data rate on the wire is slower with this option enabled.

Receive Side Scaling = OFF

Microsoft Quote: “Receive Side Scaling enables the receive processing of networking traffic to be load balanced across multiple CPUs by leveraging new hardware in the NIC. It will dynamically balance the load as either system load or network traffic patterns vary. Any application that has significant networking traffic and runs on a multiprocessor host will benefit from RSS.

To help ensure application compatibility and security protection, TCP connection setup is not offloaded to the network adapter, only TCP data transfer. If any security policies conflict with a request to offload data transfer, the network connection will not be offloaded to the NIC, and will instead remain on the host TCP/IP stack, to ensure the system maintains robust security protection.

Quote from Intel: “When Receive Side Scaling (RSS) is enabled, all of the receive data processing for a particular TCP connection is shared across multiple processors or processor cores. Without RSS all of the processing is performed by a single processor, resulting in inefficient system cache utilization.

Note: This setting has no effect if your system has only one processing unit.

RSS has caused issues with drops, out of order and duplicate packets in 2003 SP2, and a recent article from Microsoft recommends disabling this setting.

Flow control = OFF

This setting is used to allow the NIC to manage the data flow between two machines. TT disables this and allows the application to drive the data flow through Windows Scaling, TCP Window Size, etc.

Enable PME = DISABLE

In the Adapter’s Advanced Settings, there is a setting titled “Enable PME.” When PME is enabled, you can enable Wake on Settings which explicitly allows wake up with a Magic Packet from shutdown under APM power management mode. TT does not want anything to interfere with the process of trading so this is set to be disabled.

Internal Moderate Interrupt = MINIMAL

PC Interrupts are a necessary requirement for a workstation to carry out the necessary tasks required to make it a fully functional PC. An interrupt is something that is caused by processing, for example, a request for input or output or arithmetic overflow error. In order to dedicate as much ‘processing time’ to X_TRADER, TT sets this function to minimal to stop other superfluous functions interfering with X_TRADER, therefore giving X_TRADER maximum CPU priority.

Intel Quote: “Interrupt Moderation Rate sets the rate at which the controller moderates or delays the generation of interrupts making it possible to optimize network throughput and CPU utilization. The default setting (Adaptive) adjusts the interrupt rates dynamically depending on traffic type and network usage. Choosing a different setting may improve network and system performance in certain configurations. Without interrupt moderation, CPU utilization increases at higher data rates because the system must handle a larger number of interrupts. Interrupt moderation causes the network driver to accumulate interrupts and send a single interrupt rather than a series of interrupts. At higher data rates, a high interrupt moderation setting may improve system performance. At low data rates, a lower interrupt moderation setting is preferred as delayed interrupts cause latency.

Receive Descriptors = Set to Maximum Value

Intel Quote: “Receive Descriptors defines the number of Receive Descriptors, which are data segments that enable the adapter to allocate received packets to memory. Depending on the size of the packet, each receive packet requires one or more Receive Descriptors. You might choose to increase the number of Receive Descriptors if you notice a significant decrease in the performance of received traffic. If receive performance is not an issue, use the default setting appropriate to the adapter. The default setting is 256.”

For this reason, TT recommends that this setting be set to Maximum.

Transmit Descriptors = Set to Maximum Value

Intel Quote: “Transmit Descriptors defines the number of Transmit Descriptors, which are data segments that enable the adapter to track transmit packets in the system memory. Depending on the size of the packet, each transmit packet requires one or more Transmit Descriptors. You might choose to increase the number of Transmit Descriptors if you notice a possible problem with transmit performance. Although increasing the number of Transmit Descriptors can enhance transmit performance, Transmit Descriptors do consume system memory. If transmit performance is not an issue, use the default setting of 512.”

For this reason, TT recommends that this setting be set to Maximum.

Number of Coalesce Buffer = 16

This setting specifies the number of memory buffers available to the driver in case the driver runs out of available Map Registers. This buffer area is also used when a packet consists of many fragments. If no coalesce buffers or map registers are available, the driver will be forced to queue the packet for later transmission. The preferred method of transmitting data is to use map registers because it is the most efficient method. If you are using Windows NT 3.50 and are having network performance problems, try increasing the coalesce buffers. The valid range of values for Coalesce Buffers is 16-768.TT recommends that this is set to 16

Number of RX buffers = Maximum

This setting specifies the number of buffers used by the driver when copying data to the protocol memory. The recommended value is usually 16. In high network load situations, increasing receive buffers can increase performance. The tradeoff is that this also increases the amount of system memory used by the driver. If too few receive buffers are used, network performance will suffer. If too many receive buffers are used, the driver will unnecessarily consume memory resources. The valid range of values for Receive Buffers is 80-3000. TT recommends that this is set to the maximum (2048 for Intel and 3000 for Broadcom).

Number of TX buffers = Maximum

This parameter determines the number of transmit buffers that are allocated by the driver when loaded, and thus significantly impacts the resources the driver consumes in the system (allocated memory). More transmit buffers allow more send operations to be in progress at one time, and thus potentially increases system performance – this also consumes more system resources. Setting this parameter too high can prevent the driver from loading due to insufficient system resources and runs the risk of the NIC dropping packets. The range of possible values for this parameter is 80-2048. TT recommends this is set to the maximum (2048).

Wake On = Disable Wake up Mode Cap = Disable

Wake on settings allow the user to set the adapter to wake up from a low power mode when it receives a network wake up frame. There are two wake up frames: Magic Packet and IP Frame. TT does not want anything to interfere with the process of trading and therefore we recommend that you disable this setting.

Task Offload Cap = Disable

Network adapters with appropriate NDIS 5 (and later) miniport drivers can increase the system performance by supporting hardware offload of Transmission Control Protocol/Internet Protocol (TCP/IP) CPU intensive tasks. However, we have found that enabling this setting does in fact have the potential to corrupt data at the packet level and therefore we recommend that you disable this setting.

Rx Coalescing Ticks= Disable

Configures the number of Rx Host Coalescing Ticks in microseconds. This determines the upper-bound threshold of the time interval that the device will generate an interrupt if one or more frames are received.

When an interrupt is generated, the flow of data to and/or from the NIC is temporarily disrupted. TT does not want anything to interfere with the process of trading and therefore we recommend that you disable this setting.

Tx Coalescing Ticks = Disable

Configures the number of Tx Host Coalescing Ticks in microseconds. This determines the upper-bound threshold of the time interval that the device will generate an interrupt if one or more frames are sent.

When an interrupt is generated, the flow of data to and/or from the NIC is temporarily disrupted. TT does not want anything to interfere with the process of trading and therefore we recommend that you disable this setting.

Rx Max Coalesced Frames = Disable

Configures the number of Rx Maximum Coalesced Frames parameters. This determines the upper-bound threshold of the maximum number of Rx buffer descriptors that a device processes before it will generate an interrupt.

When an interrupt is generated, the flow of data to and/or from the NIC is temporarily disrupted. TT does not want anything to interfere with the process of trading and therefore we recommend that you disable this setting.

Tx Max Coalesced Frames = Disable

Configures the number of Tx Maximum Coalesced Frames parameters. This determines the upper-bound threshold of the maximum number of Tx buffer descriptors that a device processes before it will generate an interrupt.

When an interrupt is generated, the flow of data to and/or from the NIC is temporarily disrupted. TT does not want anything to interfere with the process of trading and therefore we recommend that you disable this setting.

Jumbo Packet = Disable

In computer networking, jumbo packets are Ethernet packets with more than 1500 bytes of payload (MTU). Conventionally, jumbo packets can carry up to 9000 bytes of payload, but variations exist and some care must be taken when using the term. Many, but not all, Gigabit Ethernet switches and Gigabit Ethernet network interface cards support jumbo packets, but all Fast Ethernet switches and Fast Ethernet network interface cards support only standard-sized frames. However, we recommend disabling Jumbo Packets because we do not send packets with more than 1500 bytes of payload and therefore the larger throughput is not necessary.

Reduce Speed On Power Down = Disable

Save Power Now Enabled = Disable

The Reduce Speed On Power Down and Save Power Now Enabled settings are triggered by inactivity on a PC and actually instruct the NIC card to reduce the connection speed to which it is set. This increases the chance that problems such as latency could occur, so TT recommends that you disable these settings.