I have been dabbling with HFT for few weeks now. So, I thought to go through of the protocols such ITCH and OUCH. This post is about ITCH

Nasdaq TotalView ITCH is a direct data feed product offered by The Nasdaq Stock Market, LLC. This specification covers both the software and hardware (FPGA) versions of the feed.

Nasdaq will broadcast the TotalView ITCH FPGA feed from the U.S. primary data center facility in Carteret, New Jersey in the MoldUDP64 protocol option only. Given the unshaped network traffic, Nasdaq is requiring firms to have 10 Gb or 40 Gb network connection into the Carteret, NJ data center to obtain the TotalView ITCH FPGA feed. As with the software version of the feed, the TotalView ITCH FPGA feed will be comprised of a series of sequenced order messages. Outside of the fact that the FPGA data delivery is unthrottled or unshaped at the network level, the TotalView-•-ITCH payload will be the same for both versions of the TotalView-•-ITCH 5.0 data formats. TotalView ITCH FPGA product is guaranteed to disseminate payload messages in the same exact order as the software-•-based version of the TotalView ITCH feed.

Messages Link to heading

This is quick summary of messages. The fields are detailed in the document.

1. System Control Messages
System Event (S) – Signals system status changes like market open/close (O, S, Q, M, E, C).

2. Reference Data / Stock-Related Messages
Stock Directory (R) – Security metadata: symbol, market category, financial status.
Stock Trading Action (H) – Trading status: halted, paused, quoting, trading.
Reg SHO Indicator (Y) – Short sale restriction status (Rule 201).
Market Participant Position (L) – Status of market makers for each security.
Market-Wide Circuit Breaker (V, W) – Decline levels and status.
IPO Quoting Period Update (K) – IPO quoting schedule and pricing.
LULD Auction Collar (J) – Auction thresholds for reopening paused securities.
Operational Halt (h) – Exchange-specific trading interruptions.

3. Order Book Activity

Add Orders

Add Order – No MPID (A) – Adds order without participant ID.
Add Order with MPID (F) – Adds order with participant attribution.

Modify Orders

Order Executed (E) – Indicates execution against a posted order.
Order Executed With Price (C) – Execution at a different price than posted.
Order Cancel (X) – Partial cancellation of an order.
Order Delete (D) – Full removal of an order.
Order Replace (U) – Cancels and replaces an existing order.

4. Trade Messages
Trade (Non-Cross) (P) – Matches involving hidden orders.
Cross Trade (Q) – Bulk executions from open/close/IPO crosses.
Broken Trade (B) – Execution canceled due to error.

5. Auction/Cross Support
Net Order Imbalance Indicator (NOII) (I) – Pre-cross imbalance and indicative prices.
Retail Price Improvement Indicator (RPII) (N) – Retail interest indication for buy/sell side.
Direct Listing Capital Raise (O) – Price discovery info for direct listing with capital raise.

Order messages Link to heading

The order book keeps track of buy and sell order coming in ( A and F), when order executed comes in, the order book is updated to reflect the incoming execution. Same for other order modification messages such as cancel/delete/replace.

Trade messages Link to heading

The trade messages (P) is interesting one as it seems indicate trade happening for hidden orders (orders are not in order book). So, It’s important to keep track of those as well.

Implementation Link to heading

I asked cline to generate message packer and unpacker using ITCH format in ITCH documentation. The generated code is pretty good as it generated ItchMessage and several classes for different message types.

import struct
from typing import Iterator, BinaryIO

class ItchMessage:
    def __init__(self, message_type: str, payload: bytes):
        self.message_type = message_type  # 1 char
        self.payload = payload            # variable length

    def pack(self) -> bytes:
        """
        Packs the message into bytes.
        Format: 1 byte message_type + payload
        """
        return self.message_type.encode('ascii') + self.payload

    @classmethod
    def unpack(cls, data: bytes):
        """
        Unpacks bytes into an ItchMessage.
        Assumes first byte is message_type, rest is payload.
        """
        if len(data) < 1:
            raise ValueError("Data too short to unpack")
        message_type = data[0:1].decode('ascii')
        payload = data[1:]
        return cls(message_type, payload)
class SystemEventMessage(ItchMessage):
    """
    System Event Message Type

    Format:
    Name            Offset  Length  Value   Notes
    Message Type    0       1       "S"     System Event Message
    Stock Locate    1       2       Integer Always 0
    Tracking Number 3       2       Integer Nasdaq internal tracking number
    Timestamp       5       6       Integer Nanoseconds since midnight
    Event Code      11      1       Alpha   System Event Code

    Event Codes:
    Code    Explanation
    "O"     Start of Messages. First message sent in any trading day (after time stamp messages)
    "S"     Start of System hours. NASDAQ is open and ready to start accepting orders
    "Q"     Start of Market hours. Market Hours orders are available for execution
    "M"     End of Market hours. Market Hours orders are no longer available for execution
    "E"     End of System hours. NASDAQ is closed and will not accept new orders
    "C"     End of Messages. Last message sent in any trading day
    """

    # Valid event codes and their descriptions
    EVENT_CODES = {
        'O': 'Start of Messages',
        'S': 'Start of System hours',
        'Q': 'Start of Market hours',
        'M': 'End of Market hours',
        'E': 'End of System hours',
        'C': 'End of Messages'
    }
    
    def __init__(self, stock_locate: int, tracking_number: int, timestamp: int, event_code: str):
        if event_code not in self.EVENT_CODES:
            raise ValueError(f"Invalid event code '{event_code}'. Must be one of: {', '.join(self.EVENT_CODES.keys())}")
        
        super().__init__("S", b'')  # Initialize parent with type "S"
        self.stock_locate = stock_locate
        self.tracking_number = tracking_number
        self.timestamp = timestamp
        self.event_code = event_code
        # Override payload with packed binary data
        # Pack 6-byte timestamp as two parts: 4-byte prefix and 2-byte suffix
        timestamp_hi = timestamp >> 16
        timestamp_lo = timestamp & 0xFFFF
        self.payload = struct.pack("!HHIHc", 
            stock_locate,           # H = 2 byte unsigned short
            tracking_number,        # H = 2 byte unsigned short
            timestamp_hi,           # I = 4 byte timestamp prefix
            timestamp_lo,           # H = 2 byte timestamp suffix
            event_code.encode('ascii')  # c = char
        )

    @classmethod
    def unpack(cls, data: bytes):
        """
        Unpacks bytes into a SystemEventMessage.
        Expects 12 bytes total (1 type + 11 payload)
        """
        if len(data) < 12:
            raise ValueError("Data too short to unpack SystemEventMessage")
        
        message_type = data[0:1].decode('ascii')
        if message_type != "S":
            raise ValueError(f"Invalid message type {message_type} for SystemEventMessage")
        
        # Unpack the 6-byte timestamp as two parts and combine them
        stock_locate, tracking_number, timestamp_hi, timestamp_lo, event_code = struct.unpack("!HHIHc", data[1:])
        timestamp = (timestamp_hi << 16) | timestamp_lo
        return cls(
            stock_locate,
            tracking_number,
            timestamp,
            event_code.decode('ascii')
        )