Memory T1 Temporal

Overview

Memory-T1 addresses a critical failure mode in long-context language models: inaccurate temporal reasoning when processing multi-session dialogues. As conversation history grows, models conflate events from different time periods. This framework adds explicit temporal reasoning via RL-trained memory retrieval.

Core Technique

The key insight is that temporal reasoning requires multi-level supervision: accuracy, evidence grounding, and temporal consistency.

Coarse-to-Fine Retrieval Strategy: Two-phase filtering manages computational complexity while preserving accuracy.

# Two-phase candidate selection for temporal reasoning
class TemporalMemoryRetrieval:
    def __init__(self):
        self.temporal_predictor = TemporalFilter()
        self.bm25_ranker = BM25Ranker()
        self.fine_grained_selector = RLPolicyModel()

    def retrieve_with_temporal_filtering(self, query, dialogue_history):
        """
        Phase 1: Coarse filtering narrows candidate pool
        Phase 2: RL-based fine selection with temporal rewards
        """
        # Phase 1a: Temporal filtering
        # Predict time range of query
        query_time_range = self.temporal_predictor(query)
        # Filter dialogue to sessions within time range
        candidate_sessions = filter_by_timerange(
            dialogue_history, query_time_range
        )

        # Phase 1b: Relevance filtering (BM25)
        relevant_candidates = self.bm25_ranker.rank(
            query, candidate_sessions, top_k=50
        )

        # Phase 2: Fine-grained RL selection
        selected_evidence = self.fine_grained_selector.select_with_policy(
            query, relevant_candidates
        )

        return selected_evidence

Multi-Level Temporal Consistency Reward: Three complementary reward signals enable dense supervision on temporal dimensions.

def compute_temporal_consistency_rewards(
    selected_evidence, correct_evidence, query_time
):
    """
    Three-level reward structure for temporal reasoning.
    """
    rewards = {}

    # Reward 1: Accuracy (Answer correctness)
    answer_correct = evaluate_answer_correctness(
        selected_evidence, query_time
    )
    rewards['accuracy'] = float(answer_correct)

    # Reward 2: Evidence Grounding (Right sessions cited)
    # Jaccard similarity between predicted and correct sessions
    predicted_sessions = extract_sessions(selected_evidence)
    correct_sessions = extract_sessions(correct_evidence)
    session_overlap = jaccard_similarity(predicted_sessions, correct_sessions)
    rewards['grounding'] = session_overlap

    # Reward 3: Temporal Consistency (Novel - dual level)
    # 3a. Session-level: chronological proximity to query
    session_temporal_distance = compute_temporal_distance(
        predicted_sessions, query_time
    )
    rewards['temporal_proximity'] = 1.0 / (1.0 + session_temporal_distance)

    # 3b. Utterance-level: time expressions align with range
    utterance_temporal_alignment = compute_utterance_alignment(
        selected_evidence, query_time
    )
    rewards['utterance_alignment'] = utterance_temporal_alignment

    # Composite reward
    total_reward = (
        0.4 * rewards['accuracy'] +
        0.3 * rewards['grounding'] +
        0.15 * rewards['temporal_proximity'] +
        0.15 * rewards['utterance_alignment']
    )

    return total_reward, rewards

RL Training with Sparse-to-Dense Reward: GRPO training uses the multi-level reward structure for robust learning.

def train_temporal_memory_policy(
    model, dialogue_dataset, num_epochs
):
    """
    RL training with structured temporal rewards.
    """
    optimizer = torch.optim.AdamW(model.parameters())

    for epoch in range(num_epochs):
        total_loss = 0

        for batch in dialogue_dataset:
            query = batch['query']
            dialogue_history = batch['history']
            correct_evidence = batch['correct_evidence']

            # Forward: retrieve evidence via policy
            selected = model.retrieve(query, dialogue_history)

            # Compute multi-level rewards
            reward, reward_breakdown = compute_temporal_consistency_rewards(
                selected, correct_evidence, batch['query_time']
            )

            # GRPO loss: maximize weighted reward
            log_prob = model.log_probability(selected)
            loss = -reward * log_prob

            total_loss += loss
            loss.backward()

        optimizer.step()
        optimizer.zero_grad()

        print(f"Epoch {epoch}: Loss={total_loss:.4f}")

When to Use This Technique

Use Memory-T1 when:

• Processing long multi-session dialogues
• Temporal reasoning is critical
• Distinguishing between similar events across time periods
• Extended context (128k+ tokens) with temporal structure

When NOT to Use This Technique

Avoid this approach if:

• Single-session conversations (temporal reasoning unnecessary)
• No temporal dimension in data
• Real-time inference requires fast retrieval
• Dialogue lacks clear time markers

Implementation Notes

The framework requires:

• Temporal filter to predict query time ranges
• BM25 ranker for relevance pre-filtering
• RL policy model for fine-grained selection
• Multi-level reward computation infrastructure
• GRPO training loop with composite rewards

Key Performance

• Handles 128k-token contexts with temporal coherence
• Disambiguates time expressions robustly
• Maintains consistency across multi-session dialogues
• Baseline model collapse on long contexts prevented

References

• Coarse-to-fine memory retrieval for efficiency
• Multi-level temporal consistency rewards
• Temporal filtering and relevance ranking
• RL-based fine-grained selection
• Dense supervision from multi-dimensional rewards