# Message Metadata Feature
> The Message Metadata feature introduces comprehensive tracking and display of detailed information about each message in conversations, including timestamps, token usage, model information, processing time, and other…
- Skill: `tools-only/message-metadata-feature` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add tools-only/message-metadata-feature`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tools-only/message-metadata-feature/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: tools-only (https://skillmd.com/u/tools-only)
- Updated: 2026-09-29
- Page: https://skillmd.com/skills/tools-only/message-metadata-feature
---
# Message Metadata Feature
**Version Implemented:** 0.229.001
## Overview
The Message Metadata feature introduces comprehensive tracking and display of detailed information about each message in conversations, including timestamps, token usage, model information, processing time, and other relevant metadata that enhances transparency and debugging capabilities.
## Purpose
Message Metadata enables users and administrators to:
- Track token usage and costs for each message exchange
- Monitor processing times and performance metrics
- Identify which AI models were used for specific responses
- Debug conversation issues with detailed technical information
- Analyze conversation patterns and efficiency
- Provide transparency in AI interactions
## Technical Specifications
### Architecture Overview
- **Real-time Collection**: Metadata captured during message processing
- **Structured Storage**: Metadata stored alongside message content in Cosmos DB
- **UI Integration**: Expandable metadata display in chat interface
- **Performance Tracking**: Processing time and token usage monitoring
### Database Schema
```json
{
"message_id": "msg-uuid-12345",
"conversation_id": "conv-uuid-67890",
"user_id": "user-uuid-abcde",
"content": "What is machine learning?",
"role": "user",
"timestamp": "2025-09-12T10:30:00Z",
"metadata": {
"processing_time_ms": 1250,
"token_usage": {
"prompt_tokens": 45,
"completion_tokens": 187,
"total_tokens": 232
},
"model_info": {
"deployment_name": "gpt-4o",
"model_version": "2024-11-30",
"temperature": 0.7,
"max_tokens": 4000
},
"search_metadata": {
"documents_searched": 12,
"relevant_chunks": 3,
"search_time_ms": 340,
"search_query": "machine learning definition"
},
"safety_check": {
"content_safety_enabled": true,
"safety_check_time_ms": 156,
"safety_result": "safe"
},
"response_metadata": {
"finish_reason": "stop",
"created_at": "2025-09-12T10:30:01.250Z",
"response_id": "resp-uuid-fghij"
},
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)...",
"ip_address": "192.168.1.100", // Hashed for privacy
"session_id": "session-uuid-klmno"
}
}
```
### Metadata Collection System
```javascript
class MessageMetadataCollector {
constructor() {
this.startTime = null;
this.endTime = null;
this.metadata = {};
}
startCollection(messageData) {
this.startTime = performance.now();
this.metadata = {
message_id: messageData.message_id,
user_id: messageData.user_id,
conversation_id: messageData.conversation_id,
timestamp: new Date().toISOString(),
user_agent: navigator.userAgent,
session_id: this.getSessionId(),
client_metadata: {
viewport_size: {
width: window.innerWidth,
height: window.innerHeight
},
browser_info: this.getBrowserInfo(),
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone
}
};
}
addTokenUsage(tokenData) {
this.metadata.token_usage = {
prompt_tokens: tokenData.prompt_tokens,
completion_tokens: tokenData.completion_tokens,
total_tokens: tokenData.total_tokens,
estimated_cost: this.calculateCost(tokenData)
};
}
addModelInfo(modelData) {
this.metadata.model_info = {
deployment_name: modelData.deployment,
model_version: modelData.version,
temperature: modelData.temperature,
max_tokens: modelData.max_tokens,
provider: modelData.provider || 'azure_openai'
};
}
addSearchMetadata(searchData) {
this.metadata.search_metadata = {
documents_searched: searchData.total_documents,
relevant_chunks: searchData.returned_chunks,
search_time_ms: searchData.processing_time,
search_query: searchData.query,
search_filters: searchData.filters
};
}
finishCollection(responseData) {
this.endTime = performance.now();
this.metadata.processing_time_ms = Math.round(this.endTime - this.startTime);
this.metadata.response_metadata = {
finish_reason: responseData.finish_reason,
created_at: responseData.created_at,
response_id: responseData.id
};
return this.metadata;
}
calculateCost(tokenData) {
// Token cost calculation based on model pricing
const costPerThousand = {
'gpt-4o': { input: 0.005, output: 0.015 },
'gpt-35-turbo': { input: 0.0015, output: 0.002 }
};
const modelCost = costPerThousand[this.metadata.model_info?.deployment_name] ||
costPerThousand['gpt-4o'];
const inputCost = (tokenData.prompt_tokens / 1000) * modelCost.input;
const outputCost = (tokenData.completion_tokens / 1000) * modelCost.output;
return {
input_cost: inputCost,
output_cost: outputCost,
total_cost: inputCost + outputCost,
currency: 'USD'
};
}
}
```
## Configuration Options
### Admin Settings
```html
```
## Usage Instructions
### For End Users
#### Viewing Message Metadata
1. **Access Metadata**: Click the info icon (ⓘ) next to any message
2. **Expandable Display**: Click "Show Details" to expand full metadata
3. **Copy Information**: Use copy buttons to copy specific metadata values
4. **Performance Insights**: View response times and token usage
#### Metadata Display Interface
```html
```
### For Administrators
#### Metadata Analytics Dashboard
```javascript
class MetadataAnalytics {
async generateUsageReport(dateRange) {
const report = {
totalMessages: 0,
totalTokens: 0,
totalCost: 0,
averageResponseTime: 0,
modelUsage: {},
performanceMetrics: {
fastest_response: null,
slowest_response: null,
average_search_time: 0
}
};
const messages = await this.fetchMessagesInRange(dateRange);
messages.forEach(msg => {
if (msg.metadata) {
report.totalMessages++;
report.totalTokens += msg.metadata.token_usage?.total_tokens || 0;
report.totalCost += msg.metadata.token_usage?.estimated_cost?.total_cost || 0;
// Model usage tracking
const model = msg.metadata.model_info?.deployment_name;
if (model) {
report.modelUsage[model] = (report.modelUsage[model] || 0) + 1;
}
// Performance tracking
const responseTime = msg.metadata.processing_time_ms;
if (responseTime) {
if (!report.performanceMetrics.fastest_response ||
responseTime < report.performanceMetrics.fastest_response) {
report.performanceMetrics.fastest_response = responseTime;
}
if (!report.performanceMetrics.slowest_response ||
responseTime > report.performanceMetrics.slowest_response) {
report.performanceMetrics.slowest_response = responseTime;
}
}
}
});
report.averageResponseTime = this.calculateAverageResponseTime(messages);
return report;
}
}
```
#### Metadata Export Functionality
```python
import csv
import json
from datetime import datetime, timedelta
class MetadataExporter:
def export_metadata_csv(self, date_range, include_sensitive=False):
"""Export message metadata to CSV format"""
messages = self.get_messages_with_metadata(date_range)
csv_data = []
for message in messages:
metadata = message.get('metadata', {})
row = {
'message_id': message['message_id'],
'timestamp': message['timestamp'],
'processing_time_ms': metadata.get('processing_time_ms'),
'total_tokens': metadata.get('token_usage', {}).get('total_tokens'),
'estimated_cost': metadata.get('token_usage', {}).get('estimated_cost', {}).get('total_cost'),
'model_deployment': metadata.get('model_info', {}).get('deployment_name'),
'model_version': metadata.get('model_info', {}).get('model_version'),
'documents_searched': metadata.get('search_metadata', {}).get('documents_searched'),
'search_time_ms': metadata.get('search_metadata', {}).get('search_time_ms')
}
# Include sensitive data only if requested and authorized
if include_sensitive:
row['user_id'] = message['user_id']
row['conversation_id'] = message['conversation_id']
row['ip_address'] = metadata.get('ip_address')
csv_data.append(row)
return csv_data
```
## Integration Points
### Chat Interface Integration
```javascript
// Add metadata display to each message
function renderMessageWithMetadata(message) {
const messageElement = document.createElement('div');
messageElement.className = 'chat-message';
// Message content
const contentElement = document.createElement('div');
contentElement.className = 'message-content';
contentElement.innerHTML = message.content;
// Metadata toggle
const metadataToggle = document.createElement('button');
metadataToggle.className = 'btn btn-sm btn-outline-secondary metadata-toggle';
metadataToggle.innerHTML = '';
metadataToggle.onclick = () => toggleMetadata(message.message_id);
// Metadata display
const metadataDisplay = document.createElement('div');
metadataDisplay.className = 'message-metadata';
metadataDisplay.id = `metadata-${message.message_id}`;
metadataDisplay.style.display = 'none';
metadataDisplay.innerHTML = renderMetadataDetails(message.metadata);
messageElement.appendChild(contentElement);
messageElement.appendChild(metadataToggle);
messageElement.appendChild(metadataDisplay);
return messageElement;
}
```
### Performance Monitoring Integration
```javascript
class PerformanceMonitor {
constructor() {
this.thresholds = {
response_time_warning: 5000, // 5 seconds
response_time_critical: 10000, // 10 seconds
token_usage_warning: 4000,
token_usage_critical: 8000
};
}
analyzeMessagePerformance(metadata) {
const alerts = [];
// Check response time
if (metadata.processing_time_ms > this.thresholds.response_time_critical) {
alerts.push({
type: 'critical',
message: `Response time (${metadata.processing_time_ms}ms) exceeded critical threshold`
});
} else if (metadata.processing_time_ms > this.thresholds.response_time_warning) {
alerts.push({
type: 'warning',
message: `Response time (${metadata.processing_time_ms}ms) exceeded warning threshold`
});
}
// Check token usage
const totalTokens = metadata.token_usage?.total_tokens;
if (totalTokens > this.thresholds.token_usage_critical) {
alerts.push({
type: 'critical',
message: `Token usage (${totalTokens}) exceeded critical threshold`
});
} else if (totalTokens > this.thresholds.token_usage_warning) {
alerts.push({
type: 'warning',
message: `Token usage (${totalTokens}) exceeded warning threshold`
});
}
return alerts;
}
}
```
## Privacy and Security Considerations
### Data Privacy
```python
import hashlib
import hmac
class MetadataPrivacy:
def __init__(self, secret_key):
self.secret_key = secret_key
def hash_ip_address(self, ip_address):
"""Hash IP addresses for privacy while maintaining uniqueness"""
return hmac.new(
self.secret_key.encode(),
ip_address.encode(),
hashlib.sha256
).hexdigest()[:16]
def anonymize_user_agent(self, user_agent):
"""Remove potentially identifying information from user agent"""
# Remove version numbers and specific browser versions
import re
anonymized = re.sub(r'\d+\.\d+\.\d+\.\d+', 'x.x.x.x', user_agent)
anonymized = re.sub(r'Chrome/\d+\.\d+\.\d+\.\d+', 'Chrome/xxx', anonymized)
return anonymized
def sanitize_metadata_for_export(self, metadata, user_role):
"""Remove sensitive information based on user role"""
sanitized = metadata.copy()
if user_role != 'Admin':
# Remove sensitive fields for non-admin users
sanitized.pop('ip_address', None)
sanitized.pop('session_id', None)
sanitized.pop('user_agent', None)
return sanitized
```
### Data Retention
```python
from datetime import datetime, timedelta
class MetadataRetention:
def cleanup_old_metadata(self, retention_days):
"""Remove metadata older than specified retention period"""
cutoff_date = datetime.utcnow() - timedelta(days=retention_days)
# Query messages older than cutoff date
old_messages = self.cosmos_client.query_items(
container=self.conversations_container,
query="SELECT * FROM c WHERE c.timestamp < @cutoff_date",
parameters=[{"name": "@cutoff_date", "value": cutoff_date.isoformat()}]
)
for message in old_messages:
# Remove detailed metadata but keep basic info
if 'metadata' in message:
# Keep only essential metadata
essential_metadata = {
'processing_time_ms': message['metadata'].get('processing_time_ms'),
'total_tokens': message['metadata'].get('token_usage', {}).get('total_tokens'),
'model_deployment': message['metadata'].get('model_info', {}).get('deployment_name')
}
message['metadata'] = essential_metadata
# Update the document
self.cosmos_client.upsert_item(
container=self.conversations_container,
body=message
)
```
## Testing and Validation
### Functional Testing
- Verify metadata collection for all message types (user, assistant, system)
- Test metadata display toggle functionality in chat interface
- Validate token usage calculations for different models
- Confirm processing time measurements are accurate
### Performance Testing
- Measure impact of metadata collection on message processing time
- Test metadata storage and retrieval performance with large datasets
- Validate memory usage during extended conversations with metadata
### Privacy Testing
- Verify IP address hashing works correctly
- Test data retention cleanup functionality
- Confirm sensitive data removal for non-admin users
- Validate export functionality respects privacy settings
## Known Limitations
- Metadata collection adds slight overhead to message processing
- Some metadata fields may not be available for all message types
- Token usage calculations are estimates based on known model pricing
- Processing time includes network latency and may vary significantly
## Future Enhancements
- Real-time performance alerting for slow responses
- Advanced analytics dashboard with trend analysis
- Integration with external monitoring systems
- Custom metadata fields for organization-specific tracking
- Automated performance optimization recommendations based on metadata analysis