I have been dabbling with HFT for few weeks now. So, I thought to go through of the protocols such ITCH and OUCH. This post is about ITCH
Nasdaq TotalView ITCH is a direct data feed product offered by The Nasdaq Stock Market, LLC. This specification covers both the software and hardware (FPGA) versions of the feed.
Nasdaq will broadcast the TotalView ITCH FPGA feed from the U.S. primary data center facility in Carteret, New Jersey in the MoldUDP64 protocol option only. Given the unshaped network traffic, Nasdaq is requiring firms to have 10 Gb or 40 Gb network connection into the Carteret, NJ data center to obtain the TotalView ITCH FPGA feed. As with the software version of the feed, the TotalView ITCH FPGA feed will be comprised of a series of sequenced order messages. Outside of the fact that the FPGA data delivery is unthrottled or unshaped at the network level, the TotalView-•-ITCH payload will be the same for both versions of the TotalView-•-ITCH 5.0 data formats. TotalView ITCH FPGA product is guaranteed to disseminate payload messages in the same exact order as the software-•-based version of the TotalView ITCH feed.
Messages Link to heading
This is quick summary of messages. The fields are detailed in the document.
1. System Control Messages
System Event (S) – Signals system status changes like market open/close (O, S, Q, M, E, C).
2. Reference Data / Stock-Related Messages
Stock Directory (R) – Security metadata: symbol, market category, financial status.
Stock Trading Action (H) – Trading status: halted, paused, quoting, trading.
Reg SHO Indicator (Y) – Short sale restriction status (Rule 201).
Market Participant Position (L) – Status of market makers for each security.
Market-Wide Circuit Breaker (V, W) – Decline levels and status.
IPO Quoting Period Update (K) – IPO quoting schedule and pricing.
LULD Auction Collar (J) – Auction thresholds for reopening paused securities.
Operational Halt (h) – Exchange-specific trading interruptions.
3. Order Book Activity
Add Orders
Add Order – No MPID (A) – Adds order without participant ID.
Add Order with MPID (F) – Adds order with participant attribution.
Modify Orders
Order Executed (E) – Indicates execution against a posted order.
Order Executed With Price (C) – Execution at a different price than posted.
Order Cancel (X) – Partial cancellation of an order.
Order Delete (D) – Full removal of an order.
Order Replace (U) – Cancels and replaces an existing order.
4. Trade Messages
Trade (Non-Cross) (P) – Matches involving hidden orders.
Cross Trade (Q) – Bulk executions from open/close/IPO crosses.
Broken Trade (B) – Execution canceled due to error.
5. Auction/Cross Support
Net Order Imbalance Indicator (NOII) (I) – Pre-cross imbalance and indicative prices.
Retail Price Improvement Indicator (RPII) (N) – Retail interest indication for buy/sell side.
Direct Listing Capital Raise (O) – Price discovery info for direct listing with capital raise.
Order messages Link to heading
The order book keeps track of buy and sell order coming in ( A and F), when order executed comes in, the order book is updated to reflect the incoming execution. Same for other order modification messages such as cancel/delete/replace.
Trade messages Link to heading
The trade messages (P) is interesting one as it seems indicate trade happening for hidden orders (orders are not in order book). So, It’s important to keep track of those as well.
Implementation Link to heading
I asked cline to generate message packer and unpacker using ITCH format in ITCH documentation. The generated code is pretty good as it generated ItchMessage
and several classes for different message types.
import struct
from typing import Iterator, BinaryIO
class ItchMessage:
def __init__(self, message_type: str, payload: bytes):
self.message_type = message_type # 1 char
self.payload = payload # variable length
def pack(self) -> bytes:
"""
Packs the message into bytes.
Format: 1 byte message_type + payload
"""
return self.message_type.encode('ascii') + self.payload
@classmethod
def unpack(cls, data: bytes):
"""
Unpacks bytes into an ItchMessage.
Assumes first byte is message_type, rest is payload.
"""
if len(data) < 1:
raise ValueError("Data too short to unpack")
message_type = data[0:1].decode('ascii')
payload = data[1:]
return cls(message_type, payload)
class SystemEventMessage(ItchMessage):
"""
System Event Message Type
Format:
Name Offset Length Value Notes
Message Type 0 1 "S" System Event Message
Stock Locate 1 2 Integer Always 0
Tracking Number 3 2 Integer Nasdaq internal tracking number
Timestamp 5 6 Integer Nanoseconds since midnight
Event Code 11 1 Alpha System Event Code
Event Codes:
Code Explanation
"O" Start of Messages. First message sent in any trading day (after time stamp messages)
"S" Start of System hours. NASDAQ is open and ready to start accepting orders
"Q" Start of Market hours. Market Hours orders are available for execution
"M" End of Market hours. Market Hours orders are no longer available for execution
"E" End of System hours. NASDAQ is closed and will not accept new orders
"C" End of Messages. Last message sent in any trading day
"""
# Valid event codes and their descriptions
EVENT_CODES = {
'O': 'Start of Messages',
'S': 'Start of System hours',
'Q': 'Start of Market hours',
'M': 'End of Market hours',
'E': 'End of System hours',
'C': 'End of Messages'
}
def __init__(self, stock_locate: int, tracking_number: int, timestamp: int, event_code: str):
if event_code not in self.EVENT_CODES:
raise ValueError(f"Invalid event code '{event_code}'. Must be one of: {', '.join(self.EVENT_CODES.keys())}")
super().__init__("S", b'') # Initialize parent with type "S"
self.stock_locate = stock_locate
self.tracking_number = tracking_number
self.timestamp = timestamp
self.event_code = event_code
# Override payload with packed binary data
# Pack 6-byte timestamp as two parts: 4-byte prefix and 2-byte suffix
timestamp_hi = timestamp >> 16
timestamp_lo = timestamp & 0xFFFF
self.payload = struct.pack("!HHIHc",
stock_locate, # H = 2 byte unsigned short
tracking_number, # H = 2 byte unsigned short
timestamp_hi, # I = 4 byte timestamp prefix
timestamp_lo, # H = 2 byte timestamp suffix
event_code.encode('ascii') # c = char
)
@classmethod
def unpack(cls, data: bytes):
"""
Unpacks bytes into a SystemEventMessage.
Expects 12 bytes total (1 type + 11 payload)
"""
if len(data) < 12:
raise ValueError("Data too short to unpack SystemEventMessage")
message_type = data[0:1].decode('ascii')
if message_type != "S":
raise ValueError(f"Invalid message type {message_type} for SystemEventMessage")
# Unpack the 6-byte timestamp as two parts and combine them
stock_locate, tracking_number, timestamp_hi, timestamp_lo, event_code = struct.unpack("!HHIHc", data[1:])
timestamp = (timestamp_hi << 16) | timestamp_lo
return cls(
stock_locate,
tracking_number,
timestamp,
event_code.decode('ascii')
)