GROUP TABLE AUTO-REFRESH FIX

GROUP TABLE AUTO-REFRESH FIX

Fixed/Implemented in version: 0.230.057

Overview

This fix ensures that when the Control Center data refresh completes, the groups table automatically updates with fresh data without requiring a page refresh. This provides a seamless user experience where storage account sizes, AI search sizes, and last upload dates are immediately visible after refresh.

Problem Addressed

Previously, after clicking "Refresh Data" in the Control Center, users had to manually reload the page or navigate away and back to see updated group metrics. The storage account calculation wasn't working for groups, and the frontend wasn't automatically refreshing the groups table after a data refresh.

Root Cause Analysis

Backend Issues

1. Storage Account Calculation: The storage calculation code for groups was nested inside an except block, meaning it only ran when there was an error in document metrics calculation
2. Missing Storage Fallback: Groups didn't have proper fallback estimation when Azure Storage client wasn't available

Frontend Issues

1. Missing Integration: The ControlCenter class didn't have a loadGroups() method to integrate with the refresh workflow
2. Refresh Workflow Gap: The refreshActiveTabContent() function called window.controlCenter.loadGroups() but this method didn't exist

Technical Details

Files Modified

Backend Changes

• route_backend_control_center.py: Fixed group storage account calculation logic
• config.py: Updated version to 0.230.057

Frontend Changes

• static/js/control-center.js: Added loadGroups() method to ControlCenter class

Storage Account Calculation Fix

Before: Storage calculation code was inside except block

except Exception as doc_e:
    debug_print(f"❌ [GROUP DOCUMENT DEBUG] Error calculating document metrics...")
    
    # Get actual storage account size... (WRONG LOCATION)
    debug_print(f"💾 [GROUP STORAGE DEBUG] Enhanced citation enabled...")

After: Storage calculation moved outside try/except blocks

except Exception as doc_e:
    debug_print(f"❌ [GROUP DOCUMENT DEBUG] Error calculating document metrics...")

# Get actual storage account size if enhanced citation is enabled (CORRECT LOCATION)
debug_print(f"💾 [GROUP STORAGE DEBUG] Enhanced citation enabled: {app_enhanced_citations}")

Frontend Integration Fix

Added to ControlCenter class:

async loadGroups() {
    console.log('ControlCenter.loadGroups() called');
    
    try {
        // Call GroupManager's loadGroups method if it exists
        if (typeof GroupManager !== 'undefined' && GroupManager.loadGroups) {
            console.log('Calling GroupManager.loadGroups()');
            await GroupManager.loadGroups();
        } else {
            console.log('GroupManager.loadGroups not available, skipping group refresh');
        }
    } catch (error) {
        console.error('Error loading groups:', error);
    }
}

Refresh Workflow

Complete Flow

1. User Action: User clicks "Refresh Data" button in Control Center
2. API Call: Frontend calls /api/admin/control-center/refresh (POST)
3. Backend Processing: Backend refreshes all user and group metrics with force_refresh=True
4. Metrics Calculation:
   • Document counts and AI search sizes recalculated
   • Storage account sizes calculated from Azure Storage (with fallback estimation)
   • Last upload dates formatted correctly (MM/DD/YYYY)
5. Backend Response: Returns success with counts of refreshed items
6. Frontend Refresh: Calls refreshActiveTabContent()
7. Tab Detection: Determines active tab and calls appropriate refresh method
8. Groups Refresh: If groups tab active, calls window.controlCenter.loadGroups()
9. API Fetch: GroupManager.loadGroups() calls /api/admin/control-center/groups
10. Data Display: Backend returns groups with cached metrics, frontend updates table

Data Consistency

• Refresh Phase: Uses force_refresh=True to recalculate all metrics
• Display Phase: Uses force_refresh=False to return cached calculated data
• Storage Calculation: Uses Azure Storage client with fallback to page-based estimation
• Date Formatting: Consistent MM/DD/YYYY format across all group displays

Testing

Functional Tests

• test_group_document_metrics_fix.py: Validates group metrics calculation
• test_group_table_auto_refresh.py: Validates refresh integration workflow

Manual Testing Steps

1. Open Control Center → Groups tab
2. Note current storage sizes and last upload dates
3. Click "Refresh Data" button
4. Observe that groups table automatically updates with fresh data
5. Verify storage sizes reflect actual Azure Storage usage
6. Confirm last upload dates are properly formatted

User Experience Improvements

Before Fix

• ❌ Groups showed 0 KB storage even when files existed
• ❌ Users had to refresh page manually after data refresh
• ❌ Inconsistent data display between refresh operations
• ❌ Storage calculation errors not handled gracefully

After Fix

• ✅ Groups show accurate storage account sizes
• ✅ Groups table updates automatically after refresh
• ✅ Consistent data across all refresh operations
• ✅ Fallback estimation when Azure Storage unavailable
• ✅ Proper error handling and debug logging

Performance Considerations

Optimizations

• Cached Metrics: Groups store calculated metrics in database for quick retrieval
• Batch Processing: Document calculations processed efficiently
• Selective Refresh: Only recalculates when explicitly requested
• Pagination Support: Groups API supports pagination for large datasets

Resource Usage

• Storage Queries: Optimized Azure Storage blob listing with prefixes
• Database Queries: Separate queries prevent MultipleAggregates issues
• Index Avoidance: No ORDER BY clauses to avoid composite index requirements

Future Enhancements

Potential Improvements

1. Real-time Updates: WebSocket integration for live metric updates
2. Progressive Refresh: Refresh individual groups as they complete
3. Background Processing: Queue-based metric calculation for large datasets
4. Caching Strategy: Redis integration for faster metric retrieval

Monitoring

• Debug logging available for troubleshooting refresh operations
• Error tracking for storage calculation failures
• Performance metrics for refresh operation timing

Validation

Success Criteria

✅ Storage account calculation runs outside error handling blocks
✅ Groups show accurate storage sizes after refresh
✅ Frontend automatically updates groups table after refresh
✅ No page reload required to see fresh data
✅ Proper error handling and fallback mechanisms
✅ Consistent date formatting (MM/DD/YYYY)
✅ Integration with existing refresh workflow

Debug Information

Enhanced debug logging provides visibility into:

• Storage calculation process and results
• Group metrics calculation steps
• Frontend refresh workflow execution
• API call success/failure status
• Data transformation and caching operations