Issue Description

AI_SEARCH_INDEX_MANAGEMENT_AND_AGENT_SETTINGS_FIX

Version Implemented: 0.229.001

Issue Description

Users experienced multiple issues when trying to manage AI Search indexes and when agents were disabled:

1. 404 Error on Agent Settings: The JavaScript was trying to load agent settings (/api/admin/agent/settings) even when agents were disabled, causing 404 errors
2. 500 Error on Index Field Checking: The API call to check AI Search index fields (/api/admin/settings/check_index_fields) was failing with 500 errors when Azure AI Search wasn't configured or indexes didn't exist
3. No Way to Create Indexes: Users had no interface to create missing AI Search indexes through the application

Fixed in version: 0.227.013 (Extended with public workspace index support)

Root Cause Analysis

Agent Settings Issue

• The loadAgentSettings() function in JavaScript was being called unconditionally
• When agents are disabled (enable_semantic_kernel = false), the agent settings endpoint doesn't exist or returns 404
• This caused console errors and poor user experience

AI Search Index Issues

• The check_index_fields endpoint had minimal error handling
• When Azure AI Search wasn't configured, the function would throw unhandled exceptions
• When indexes didn't exist, the error wasn't user-friendly
• No way to create missing indexes through the UI

Technical Details

Files Modified

• route_backend_settings.py - Enhanced error handling and added index creation endpoint
• static/js/admin/admin_settings.js - Fixed conditional loading and improved error handling
• config.py - Version update

Backend Improvements

Enhanced check_index_fields Endpoint

@app.route('/api/admin/settings/check_index_fields', methods=['POST'])
@login_required
@admin_required
def check_index_fields():
    try:
        # Validate input
        if not idx_type or idx_type not in ['user', 'group', 'public']:
            return jsonify({'error': 'Invalid indexType'}), 400
        
        # Check if Azure AI Search is configured
        if not settings.get("azure_ai_search_endpoint"):
            return jsonify({
                'error': 'Azure AI Search not configured',
                'needsConfiguration': True
            }), 400
        
        # Handle index not found vs other errors
        try:
            client = get_index_client()
            current = client.get_index(expected['name'])
            # Check for missing fields...
        except Exception as search_error:
            if "not found" in str(search_error).lower():
                return jsonify({
                    'error': f'Index "{expected["name"]}" does not exist yet',
                    'indexExists': False,
                    'needsCreation': True
                }), 404
    except Exception as e:
        app.logger.error(f"Error in check_index_fields: {str(e)}")
        return jsonify({'error': f'Unexpected error: {str(e)}'}), 500

New create_index Endpoint

@app.route('/api/admin/settings/create_index', methods=['POST'])
@login_required
@admin_required
def create_index():
    """Create an AI Search index from scratch using the JSON schema."""
    try:
        # Load JSON schema and validate
        with open(json_path, 'r') as f:
            index_definition = json.load(f)
        
        # Create index using Azure SDK
        from azure.search.documents.indexes.models import SearchIndex
        index = SearchIndex.deserialize(index_definition)
        result = client.create_index(index)
        
        return jsonify({
            'status': 'success',
            'message': f'Successfully created index "{result.name}"',
            'indexName': result.name,
            'fieldsCount': len(result.fields)
        }), 200
    except Exception as e:
        return jsonify({'error': f'Failed to create index: {str(e)}'}), 500

Frontend Improvements

Conditional Agent Settings Loading

// Before (Always called)
loadAgentSettings();

// After (Conditional)
if (typeof settings !== 'undefined' && settings && settings.enable_semantic_kernel) {
    loadAgentSettings();
}

Enhanced Index Management JavaScript

// Improved error handling for index checking
.then(response => {
    if (response.missingFields && response.missingFields.length > 0) {
        // Show missing fields
        fixBtn.textContent = `Fix ${type} Index Fields`;
    } else if (response.indexExists) {
        // Index is complete
        console.log(`${type} index is properly configured`);
    }
})
.catch(err => {
    if (err.message.includes('does not exist yet')) {
        // Index doesn't exist - show create option
        fixBtn.textContent = `Create ${type} Index`;
        fixBtn.dataset.action = 'create';
    } else if (err.message.includes('not configured')) {
        // Azure AI Search not configured
        missingSpan.textContent = 'Azure AI Search not configured';
    }
});

// Dynamic button handling for create vs fix
fixBtn.addEventListener('click', () => {
    const action = fixBtn.dataset.action || 'fix';
    const endpoint = action === 'create' ? 
        '/api/admin/settings/create_index' : 
        '/api/admin/settings/fix_index_fields';
    
    // Handle different actions appropriately
});

Benefits

1. Improved Error Handling

• Clear, actionable error messages instead of generic 500 errors
• Different handling for "not configured" vs "index not found" scenarios
• Proper logging for debugging

2. Better User Experience

• No more 404 errors when agents are disabled
• Clear indication of what needs to be done (configure Azure AI Search, create index, fix fields)
• Dynamic button text that reflects the available action

3. Index Creation Capability

• Users can now create missing indexes directly through the admin interface
• No need to manually run scripts or use external tools
• Proper validation and error handling for the creation process

4. Conditional Loading

• JavaScript only makes API calls when features are enabled
• Cleaner console output with no unnecessary errors
• Better performance by avoiding pointless API calls

User Interface Behavior

When Azure AI Search is Not Configured

• Shows: "Azure AI Search not configured"
• Action: Fix button is hidden, user needs to configure Azure AI Search settings first

When Azure AI Search is Configured but Index Doesn't Exist

• Shows: Index "user" does not exist yet
• Action: Button shows "Create user Index" and creates the index from JSON schema

When Index Exists but Missing Fields

• Shows: List of missing field names
• Action: Button shows "Fix user Index Fields" and adds missing fields

When Index is Complete

• Shows: No warning message
• Action: No button displayed, everything is properly configured

Integration Notes

Required Files

The solution depends on these JSON schema files being present:

• static/json/ai_search-index-user.json
• static/json/ai_search-index-group.json

These files define the complete index structure including fields, vector configurations, and other properties.

Configuration Requirements

• Azure AI Search endpoint must be configured in settings
• Proper authentication (managed identity or API key) must be set up
• User must have admin privileges to create/modify indexes

Testing and Validation

The fix includes comprehensive functional testing that validates:

• ✅ Agent settings only load when agents are enabled
• ✅ Proper error handling for all AI Search scenarios
• ✅ Index creation endpoint functionality
• ✅ JavaScript error handling and user feedback
• ✅ Index schema file integrity
• ✅ Version consistency

Public Index Support Enhancement (v0.227.013)

What Was Added

Extended the AI Search index management system to support public workspace indexes:

1. Schema File Addition: Added ai_search-index-public.json to static/json/ directory
2. Backend Support: Updated validation to accept 'public' as a valid index type
3. Frontend Integration: Enhanced JavaScript to process user, group, and public indexes
4. HTML Template: Added public index warning div and fix button elements

Implementation Details

• Backend: Modified route_backend_settings.py to validate ['user', 'group', 'public'] index types
• Frontend: Updated admin_settings.js array from ['user','group'] to ['user','group','public']
• Template: Added index-warning-public div and fix-public-index-btn button to admin_settings.html
• Schema: Copied ai_search-index-public.json from artifacts to static/json/ directory

How It Works

The system now automatically:

1. Checks all three index types (user, group, public) on admin page load
2. Shows warnings for missing fields or missing indexes for each type
3. Provides "Create Index" or "Fix Index Fields" buttons for each type
4. Handles errors gracefully with type-specific messages

This completes the comprehensive AI Search index management system for all workspace types.

Troubleshooting Guide

If you see "Azure AI Search not configured"

1. Go to admin settings
2. Configure azure_ai_search_endpoint
3. Configure authentication (managed identity or API key)
4. Refresh the page

If you see "Index does not exist yet"

1. Click the "Create [type] Index" button
2. Wait for the success message
3. Refresh the page to verify

If you see missing fields listed

1. Click the "Fix [type] Index Fields" button
2. Wait for the success message
3. Refresh the page to verify

If creation/fixing fails

1. Check that Azure AI Search is properly configured
2. Verify your user has permissions to modify the search service
3. Check the browser console and server logs for detailed error messages