Orchardcore AI Chat Analytics

Orchard Core AI Chat Analytics - Prompt Templates

Configure AI Chat Analytics

You are an Orchard Core expert. Generate code, configuration, and recipes for adding AI chat session analytics and reporting to an Orchard Core application using CrestApps modules.

Guidelines

• The AI Chat Analytics feature (CrestApps.OrchardCore.AI.Chat.Analytics) tracks chat session metrics and provides dashboard reporting with extensible display drivers.
• It depends on both the AI Chat feature (CrestApps.OrchardCore.AI.Chat) and the AI Chat Core feature (CrestApps.OrchardCore.AI.Chat.Core).
• Metrics are collected automatically when analytics is enabled on an AI profile.
• Per-profile analytics settings control whether session metrics, AI resolution detection, and conversion metrics are active.
• Dashboard reports are rendered as display drivers on AIChatAnalyticsReport, making them extensible through Orchard Core's display management.
• Filters are rendered as display drivers on AIChatAnalyticsFilter, supporting date range and profile filtering.
• Session data is stored as AIChatSessionEvent documents using YesSql with the AIChatSessionMetricsIndex for efficient queries.
• Install CrestApps packages in the web/startup project.

Feature ID

Dependencies

NuGet Package

Install CrestApps.OrchardCore.AI.Chat in the web/startup project. The analytics feature is included in this module.

Enabling AI Chat Analytics

{
  "steps": [
    {
      "name": "Feature",
      "enable": [
        "CrestApps.OrchardCore.AI",
        "CrestApps.OrchardCore.AI.Chat",
        "CrestApps.OrchardCore.AI.Chat.Analytics",
        "CrestApps.OrchardCore.OpenAI"
      ],
      "disable": []
    }
  ]
}

Metrics Tracked

The analytics feature tracks the following metrics per chat session:

Dashboard Reports

The analytics dashboard at Admin → Artificial Intelligence → Reports → AI Chat Session Analytics provides seven report sections, each rendered by its own display driver:

Filtering Reports

Reports support two filters rendered as display drivers on AIChatAnalyticsFilter:

CSV Export

The dashboard includes an Export action that generates a CSV file with session-level data including SessionId, ProfileId, VisitorId, UserId, IsAuthenticated, SessionStartedUtc, SessionEndedUtc, MessageCount, HandleTimeSeconds, and IsResolved.

Permissions

Both permissions are granted to the Administrator role by default.

Admin Menu Entries

Enabling the analytics feature adds four entries under Artificial Intelligence → Reports:

1. AI Chat Session Analytics
2. AI Chat Extracted Data
3. AI Usage Analytics
4. AI Chat Conversion Goals

Per-Profile Analytics Settings

Analytics behavior is configured on each AI profile through the AnalyticsMetadata property:

Configuring Analytics on a Profile via Code

public sealed class AnalyticsProfileMigrations : DataMigration
{
    private readonly IAIProfileManager _profileManager;

    public AnalyticsProfileMigrations(IAIProfileManager profileManager)
    {
        _profileManager = profileManager;
    }

    public async Task<int> CreateAsync()
    {
        var profile = await _profileManager.NewAsync();

        profile.Name = "support-chat";
        profile.DisplayText = "Support Chat";
        profile.Type = AIProfileType.Chat;

        profile.Put(new AIProfileMetadata
        {
            SystemMessage = "You are a helpful support assistant.",
            Temperature = 0.3f,
            MaxTokens = 4096,
            PastMessagesCount = 10,
        });

        profile.Put(new AnalyticsMetadata
        {
            EnableSessionMetrics = true,
            EnableAIResolutionDetection = true,
            EnableConversionMetrics = true,
            ConversionGoals =
            [
                new ConversionGoal
                {
                    Name = "issue-resolved",
                    Description = "The user's issue was fully resolved by the assistant.",
                    MinScore = 0,
                    MaxScore = 100,
                },
            ],
        });

        await _profileManager.SaveAsync(profile);

        return 1;
    }
}

Key Services

Extending Reports with Custom Display Drivers

Reports are extensible through Orchard Core's display driver pattern. Register a custom display driver on AIChatAnalyticsReport to add a new report section:

public sealed class Startup : StartupBase
{
    public override void ConfigureServices(IServiceCollection services)
    {
        services.AddDisplayDriver<AIChatAnalyticsReport, CustomAnalyticsDisplayDriver>();
    }
}

Extending Filters with Custom Display Drivers

Add custom filters by registering a display driver on AIChatAnalyticsFilter:

public sealed class Startup : StartupBase
{
    public override void ConfigureServices(IServiceCollection services)
    {
        services.AddDisplayDriver<AIChatAnalyticsFilter, CustomAnalyticsFilterDisplayDriver>();
    }
}

The custom filter driver can append query conditions to AIChatAnalyticsFilter.Conditions to restrict the result set.

Usage Analytics

The AI Usage Analytics page (separate from session analytics) shows aggregated token usage data grouped by user, client, and model. It displays total API calls, sessions, chat interactions, and tokens with average latency per group.