Position Reconciliation

Position Reconciliation - Research Notes

Experiment Overview

Context

Live trading systems maintain internal position state that can drift from reality:

• Orders execute but callback/update fails
• Manual interventions via broker UI
• Partial fills not properly tracked
• Network issues during order submission
• Broker-side corporate actions (splits, mergers)

Without reconciliation, the system may try to close positions it doesn't have or miss positions it should manage.

Verified Workflow

1. Reconciliation Function

def reconcile_positions(
    broker: Broker,
    states: Dict[str, PositionState],
    logger: logging.Logger,
) -> Dict[str, PositionState]:
    """
    Reconcile internal position state with broker's actual positions.

    Broker is source of truth - any discrepancies update internal state.
    """
    try:
        broker_positions = broker.get_positions()
        broker_symbols = set()

        for pos in broker_positions:
            symbol = pos.get('symbol', '')
            if not symbol:
                continue
            broker_symbols.add(symbol)

            # Parse broker position
            qty = float(pos.get('qty', 0))
            side = 1 if qty > 0 else (-1 if qty < 0 else 0)
            avg_entry = float(pos.get('avg_entry_price', 0))
            unrealized_pl = float(pos.get('unrealized_pl', 0))

            # Check if we have internal state
            if symbol in states:
                internal = states[symbol]

                # Check for quantity or side mismatch
                if abs(abs(qty) - internal.shares) > 0.001 or internal.side != side:
                    logger.warning(
                        f"RECONCILE {symbol}: Internal={internal.side}x{internal.shares:.4f} vs "
                        f"Broker={side}x{abs(qty):.4f} - updating to broker"
                    )
                    states[symbol] = PositionState(
                        side=side,
                        shares=abs(qty),
                        entry_price=avg_entry,
                        unrealized_pnl=unrealized_pl,
                        trade_count=internal.trade_count,  # Preserve history
                    )
            else:
                # Broker has position we don't track
                if qty != 0:
                    logger.warning(
                        f"RECONCILE {symbol}: Found untracked broker position "
                        f"{side}x{abs(qty):.4f} @ {avg_entry:.2f} - adding to state"
                    )
                    states[symbol] = PositionState(
                        side=side,
                        shares=abs(qty),
                        entry_price=avg_entry,
                        unrealized_pnl=unrealized_pl,
                        trade_count=0,
                    )

        # Check for internal positions broker doesn't have
        for symbol, internal in list(states.items()):
            if internal.side != 0 and symbol not in broker_symbols:
                logger.warning(
                    f"RECONCILE {symbol}: Internal shows {internal.side}x{internal.shares:.4f} "
                    f"but broker has no position - clearing internal state"
                )
                states[symbol] = PositionState(trade_count=internal.trade_count + 1)

        return states

    except Exception as e:
        logger.error(f"Position reconciliation failed: {e}")
        return states  # Return unchanged on error

2. Integration in Trading Loop

# Run reconciliation periodically (every 30 min = 120 loops at 15s interval)
RECONCILIATION_INTERVAL = 120

while running:
    loop_count += 1

    # ... trading logic ...

    # Position reconciliation
    if loop_count % RECONCILIATION_INTERVAL == 0:
        logger.info("Running position reconciliation with broker...")
        states = reconcile_positions(broker, states, logger)

    time.sleep(15)

3. Broker API Response Format (Alpaca)

# Example broker.get_positions() response
[
    {
        'symbol': 'AAPL',
        'qty': '10.5',           # Supports fractional
        'avg_entry_price': '150.25',
        'current_price': '152.00',
        'unrealized_pl': '18.375',
        'market_value': '1596.00',
        'side': 'long'
    },
    ...
]

Failed Attempts (Critical)

Final Parameters

# Recommended reconciliation settings
RECONCILIATION_INTERVAL = 120  # loops (30 min at 15s interval)
QUANTITY_TOLERANCE = 0.001     # shares - for floating point comparison
LOG_LEVEL = logging.WARNING    # Log discrepancies prominently

Key Insights

• Broker is ALWAYS source of truth - Never override broker with internal state
• Run periodically, not just on startup - State can drift anytime
• Preserve metadata on reconcile - Keep trade_count, strategy attribution
• Handle three cases: Mismatch, untracked broker position, phantom internal position
• Use tolerance for float comparison - Exact equality fails with fractional shares
• Log discrepancies prominently - RECONCILE prefix for easy grep
• Don't raise on errors - Log and continue with existing state
• Reconcile BEFORE trading logic - Ensure state is accurate before decisions

Edge Cases to Handle

# 1. Partial fills
#    Internal: 100 shares, Broker: 75 shares
#    -> Update internal to 75, order may still be working

# 2. Stock splits
#    Internal: 10 shares @ $100, Broker: 40 shares @ $25
#    -> Trust broker, update shares and price

# 3. Manual close via broker UI
#    Internal: 50 shares, Broker: 0 shares
#    -> Clear internal state, increment trade_count

# 4. Zero quantity edge case
#    Broker returns qty=0 for recently closed position
#    -> Don't add to tracking, just skip

Testing Checklist

# Verify these scenarios:
1. [ ] Internal 100 shares, broker 100 shares -> No change
2. [ ] Internal 100 shares, broker 50 shares -> Update to 50
3. [ ] Internal 0 shares, broker 100 shares -> Add new position
4. [ ] Internal 100 shares, broker 0 shares -> Clear internal
5. [ ] Internal long, broker short -> Flip side
6. [ ] Broker API fails -> Return unchanged state
7. [ ] Fractional shares (10.5) handled correctly

References

• Alpaca API: client.get_all_positions()
• Best practice: Reconcile every 15-60 minutes in production
• Related: Order fill tracking, execution confirmation