Message Threading System

Message Threading System

Overview

Version: 0.233.208
Implemented: December 4, 2025

This feature implements a linked-list threading system for chat messages that establishes proper relationships between user messages, system messages, AI responses, image generations, and file uploads. Messages are now ordered by thread chains rather than just timestamps, ensuring proper conversation flow and message association.

Purpose

The threading system solves several key problems:

• Message Association: Links user messages to their corresponding AI responses and system augmentations
• Proper Ordering: Ensures messages are displayed in logical conversation order, not just temporal order
• File Upload Tracking: Properly sequences uploaded files within the conversation flow
• Image Generation Tracking: Associates generated images with the messages that requested them
• Legacy Support: Gracefully handles existing messages without thread information

Thread Fields

Each message now includes four new fields:

thread_id

• Type: String (UUID)
• Purpose: Unique identifier for this message in the thread chain
• Generated: For every new message (user, system, assistant, image, file)

previous_thread_id

• Type: String (UUID) or None
• Purpose: Links to the previous message's thread_id
• Value: None for the first message in a conversation or when following a legacy message

active_thread

• Type: Boolean
• Purpose: Indicates if this thread is currently active
• Value: Always True in current implementation (reserved for future retry/edit functionality)

thread_attempt

• Type: Integer
• Purpose: Tracks the attempt number for retries or edits
• Value: Always 1 in current implementation (reserved for future retry functionality)

Message Flow Examples

Standard Chat Interaction

User Message (Thread 1)
├─ thread_id: "abc-123"
├─ previous_thread_id: None
├─ active_thread: True
└─ thread_attempt: 1
   │
   ↓
System Message (Thread 2) [Optional - if RAG/search enabled]
├─ thread_id: "def-456"
├─ previous_thread_id: "abc-123"
├─ active_thread: True
└─ thread_attempt: 1
   │
   ↓
AI Response (Thread 3)
├─ thread_id: "ghi-789"
├─ previous_thread_id: "def-456" (or "abc-123" if no system message)
├─ active_thread: True
└─ thread_attempt: 1

Image Generation

User Message (Thread 1)
├─ thread_id: "aaa-111"
├─ previous_thread_id: None
├─ active_thread: True
└─ thread_attempt: 1
   │
   ↓
Image Message (Thread 2)
├─ thread_id: "bbb-222"
├─ previous_thread_id: "aaa-111"
├─ active_thread: True
├─ thread_attempt: 1
└─ role: "image"

File Upload

(Previous conversation messages...)
   │
   ↓
File Upload (New Thread)
├─ thread_id: "ccc-333"
├─ previous_thread_id: "zzz-999" (last message in conversation)
├─ active_thread: True
├─ thread_attempt: 1
├─ role: "image" (for images) or "file" (for documents)
└─ filename: "document.pdf"

Implementation Details

Modified Files

1. functions_chat.py

   • Added sort_messages_by_thread() function
   • Implements linked-list traversal algorithm
   • Handles both legacy (timestamp-ordered) and threaded messages

2. route_backend_chats.py

   • Updated chat_api() endpoint
   • Updated chat_stream_api() endpoint
   • Added threading to user messages, system messages, and assistant messages
   • Added threading to generated images (chunked and non-chunked)
   • Queries last message's thread_id before creating new messages

3. route_frontend_chats.py

   • Updated file upload handler (/upload)
   • Added threading to uploaded images (chunked and non-chunked)
   • Added threading to uploaded files

4. route_frontend_conversations.py

   • Updated get_conversation_messages() endpoint
   • Applies sort_messages_by_thread() before returning messages

5. config.py

   • Updated version to 0.233.208

Sorting Algorithm

The sort_messages_by_thread() function:

1. Separates messages into legacy (no thread_id) and threaded messages
2. Sorts legacy messages by timestamp
3. Builds thread chain:
   • Creates a map of thread_id → message
   • Creates a map of previous_thread_id → children
4. Finds root messages: Messages with no previous_thread_id or where previous_thread_id doesn't exist in current set
5. Traverses chains: Recursively follows the linked list structure
6. Returns ordered list: Legacy messages first, then threaded messages in chain order

Thread Chain Establishment

When creating a new message:

# Query for the last message's thread_id
last_msg_query = """
    SELECT TOP 1 c.thread_id 
    FROM c 
    WHERE c.conversation_id = '{conversation_id}' 
    ORDER BY c.timestamp DESC
"""
last_msgs = list(cosmos_messages_container.query_items(
    query=last_msg_query,
    partition_key=conversation_id
))
previous_thread_id = last_msgs[0].get('thread_id') if last_msgs else None

# Generate new thread_id and create message
current_thread_id = str(uuid.uuid4())
message = {
    'thread_id': current_thread_id,
    'previous_thread_id': previous_thread_id,
    'active_thread': True,
    'thread_attempt': 1,
    # ... other message fields
}

Legacy Message Support

The system is fully backward compatible:

• Existing messages without thread_id are sorted by timestamp
• Legacy messages are placed before threaded messages
• No migration required - threading applies only to new messages
• Gradual adoption - conversations naturally transition to threaded ordering

Frontend Impact

No frontend changes required. The frontend continues to:

• Fetch messages from the backend
• Display them in the order received
• The backend now returns messages in thread order instead of timestamp order

Performance Considerations

Database Queries

• Single additional query per message creation (to get last thread_id)
• Query is optimized: SELECT TOP 1 with ORDER BY timestamp DESC
• Uses partition key for efficient lookup

Sorting Performance

• O(n log n) for legacy message sorting (timestamp-based)
• O(n) for building thread chain maps
• O(n) for traversing chains
• Overall complexity: O(n log n) where n = number of messages

Indexing Recommendations

Consider adding indexes for:

• conversation_id + timestamp (DESC) - for last message lookup
• conversation_id + thread_id - for thread chain traversal

Future Enhancements

Retry/Edit Support

The thread_attempt field enables future retry functionality:

User Message (Thread 1, Attempt 1) - active_thread: False
   │
   ↓
Assistant Response (Thread 2, Attempt 1) - active_thread: False
   │
   ↓
User Message (Thread 1, Attempt 2) - active_thread: True
   │
   ↓
Assistant Response (Thread 2, Attempt 2) - active_thread: True

Branching Conversations

The linked-list structure supports conversation branches:

• Multiple messages can share the same previous_thread_id
• UI could display conversation tree
• Users could explore different conversation paths

Thread Metadata

Additional thread-level metadata could include:

• Thread creation timestamp
• Thread type (chat, image generation, file upload)
• Thread tags or labels
• Thread-level citations or sources

Testing

Functional Tests Needed

1. New conversation - verify first message has previous_thread_id = None
2. Multi-turn conversation - verify thread chain is established
3. Image generation - verify image links to user message
4. File upload - verify file links to previous message
5. Legacy messages - verify old messages sort by timestamp
6. Mixed conversation - verify legacy + threaded messages sort correctly
7. Message retrieval - verify frontend receives correctly ordered messages

Test Scenarios

• Create new conversation → verify threading
• Upload file mid-conversation → verify threading
• Generate image → verify threading
• Load conversation with 50+ messages → verify performance
• Load legacy conversation → verify backward compatibility

Troubleshooting

Messages Out of Order

Symptom: Messages appear in wrong order
Check: Verify sort_messages_by_thread() is called before returning messages
Solution: Ensure all message retrieval endpoints apply sorting

Broken Thread Chain

Symptom: Messages missing or duplicated
Check: Verify previous_thread_id references exist in database
Solution: Check thread chain integrity with query:

SELECT m.thread_id, m.previous_thread_id, m.timestamp, m.role
FROM c m
WHERE m.conversation_id = '{conversation_id}'
ORDER BY m.timestamp ASC

Performance Issues

Symptom: Slow message loading
Check: Number of messages in conversation
Solution:

• Add database indexes
• Implement pagination for large conversations
• Cache sorted message lists

Configuration

No configuration changes required. The feature is enabled by default for all new messages.

Security Considerations

• Thread IDs use UUIDs - not guessable
• Thread relationships maintained per conversation
• No cross-conversation thread linking
• Thread fields included in normal message access control

Related Documentation

• Message Management Architecture
• Conversation Metadata
• Message Masking

References

• Implementation: functions_chat.py::sort_messages_by_thread()
• Usage: route_backend_chats.py, route_frontend_chats.py, route_frontend_conversations.py
• Version: config.py::VERSION