Process Task Mining

Process and Task Mining — Microsoft Log Sources

This skill provides end-to-end knowledge for extracting event logs from Microsoft systems, constructing process models, analyzing performance, checking conformance, and producing structured reports for process improvement.

1. Integration Context Contract

Required Permissions per Source

Licensing Requirements

Always check tenant license during setup — running a query against logs older than the retention window silently returns zero results.

Concurrent Query Limits (Graph Audit Log API)

• Maximum 10 concurrent audit query jobs per admin
• Maximum 1 unfiltered query at a time
• Query results remain available for 30 days after job completion
• Always use async pattern: POST query → poll status → GET results

2. Base URLs

Power Platform API:   https://api.powerplatform.com/powerautomate/environments/{envId}
M365 Audit (Graph):   https://graph.microsoft.com/beta/security/auditLog/queries
O365 Mgmt API:        https://manage.office.com/api/v1.0/{tenantId}/activity/feed/subscriptions
Power Platform Logs:  https://manage.office.com/api/v1.0/{tenantId}/activity/feed/audit/PowerPlatformAdminActivity
Dataverse:            https://{org}.{region}.dynamics.com/api/data/v9.2/audits
Azure Monitor CLI:    az monitor log-analytics query --workspace {id} --analytics-query "..."

3. API Endpoints

3.1 Power Automate — Flow Runs

List runs for a specific flow:

GET https://api.powerplatform.com/powerautomate/environments/{envId}/flowRuns
  ?workflowId={flowId}
  &api-version=2022-03-01-preview
  &$filter=startTime ge {ISO8601} and startTime le {ISO8601}
  &$top=1000

Response fields used for event log:

CaseId strategy for Power Automate: Use correlation.clientTrackingId if set; otherwise use the flow run name (GUID). For flows triggered by SharePoint/Dataverse items, extract the item ID from the trigger output and use it as caseId to correlate multiple flow runs on the same business object.

Activity name mapping (flow-level granularity):

For action-level granularity, retrieve individual action run details:

GET https://api.powerplatform.com/powerautomate/environments/{envId}/flowRuns/{runId}/actions
  ?api-version=2022-03-01-preview

3.2 M365 Unified Audit Log — Graph Beta (Async Pattern)

Step 1 — Create query job:

POST https://graph.microsoft.com/beta/security/auditLog/queries
Content-Type: application/json

{
  "displayName": "process-mining-{timestamp}",
  "filterStartDateTime": "2026-02-01T00:00:00Z",
  "filterEndDateTime": "2026-03-01T00:00:00Z",
  "recordTypeFilters": ["SharePoint", "SharePointFileOperation", "MicrosoftTeams"],
  "operationFilters": ["FileAccessed", "FileModified", "FileUploaded", "FileDeleted"]
}

Supported recordTypeFilters (process mining relevant):

Step 2 — Poll until succeeded:

GET https://graph.microsoft.com/beta/security/auditLog/queries/{queryId}

Poll every 10–30 seconds until status = succeeded.

Step 3 — Retrieve results (paginated):

GET https://graph.microsoft.com/beta/security/auditLog/queries/{queryId}/records
  ?$top=1000
  &$skiptoken={token}

Key response fields:

3.3 Office 365 Management Activity API (Power Platform Logs)

Power Platform activity (flow runs, connector usage, DLP events) is surfaced through the Office 365 Management Activity API, not a dedicated PA API.

List available subscriptions:

curl -H "Authorization: Bearer {token}" \
  "https://manage.office.com/api/v1.0/{tenantId}/activity/feed/subscriptions/list"

Start subscription (one-time per content type):

curl -X POST -H "Authorization: Bearer {token}" \
  "https://manage.office.com/api/v1.0/{tenantId}/activity/feed/subscriptions/start?contentType=Audit.General"

Fetch available content blobs:

curl -H "Authorization: Bearer {token}" \
  "https://manage.office.com/api/v1.0/{tenantId}/activity/feed/audit/Audit.General
   ?startTime=2026-02-01T00:00:00&endTime=2026-03-01T00:00:00"

Each blob URL then returns an array of audit records. Filter by Workload = "PowerAutomate" or Workload = "PowerPlatform".

3.4 Azure Monitor — Log Analytics KQL

az monitor log-analytics query \
  --workspace {workspaceId} \
  --analytics-query "
    AzureActivity
    | where TimeGenerated between (datetime(2026-02-01) .. datetime(2026-03-01))
    | where ResourceGroup == '{rg}'
    | project TimeGenerated, Caller, OperationNameValue, ResourceId, ResultType, CorrelationId
    | order by TimeGenerated asc
  " \
  --output json

Common KQL tables for process mining:

CaseId strategy for Azure Monitor: Use CorrelationId for ARM operations; use a business-key dimension (resource ID segment, ticket ID extracted from tags) where available.

3.5 Dataverse Audit Log

GET https://{org}.{region}.dynamics.com/api/data/v9.2/audits
  ?$filter=createdon ge {ISO8601} and createdon le {ISO8601}
  &$select=auditid,createdon,operation,objecttypecode,_userid_value,attributemask
  &$expand=regardingobjectid_systemuser($select=fullname,internalemailaddress)
  &$orderby=createdon asc
  &$top=5000

Dataverse operation codes:

CaseId strategy for Dataverse: Use the _regardingobjectid_value (the record GUID that was acted on) as caseId. This groups all audit events for the same business record into one process instance.

4. Native PA Process Mining Format

Microsoft's official PA Process Mining ingestion format (CSV import via Power Automate Process Mining in Power Platform):

Reference: https://learn.microsoft.com/en-us/power-automate/process-mining-processes-and-data

The log-extract command output is a superset of this format — it can be directly imported into PA Process Mining for native visualization.

5. Unified Event Log Schema

The plugin's canonical schema used between log-extract and all analysis commands:

CSV column order: caseId,activityName,timestamp,resource,lifecycle,duration_ms,sourceSystem,rawEventId

6. Source Field Mapping Tables

Power Automate → Unified Schema

For action-level: use {flowName} - {actionName} as activityName.

M365 Audit → Unified Schema

Azure Monitor → Unified Schema

Dataverse → Unified Schema

7. Process Mining Algorithms

7.1 Directly-Follows Graph (DFG) Construction

A DFG captures which activity directly follows which other activity, and how often.

Algorithm:

1. Sort events by caseId, then by timestamp within each case
2. For each case, iterate through sorted activities: record pair (A → B) where B directly follows A
3. Count frequency of each (A → B) pair across all cases
4. Identify START (first activity in each case) and END (last activity in each case)
5. Calculate edge weight = count / total cases = relative frequency

Output format (Mermaid flowchart LR):

flowchart LR
    START([Start]) -->|1124| A[Flow Started]
    A -->|1124| B[Validation Step]
    B -->|1089| C[Approval Requested]
    C -->|901| D[Approved]
    C -->|188| E[Rejected]
    D -->|901| F[Order Created]
    E -->|188| END([End])
    F -->|901| END([End])

Edge labels show absolute counts. Add % suffix for relative frequency on large diagrams.

7.2 Process Variants

A variant is the unique ordered sequence of activities in a case.

Algorithm:

1. For each case, concatenate sorted activity names into a sequence string: "A → B → C → D"
2. Count cases per unique sequence → variant frequency
3. Rank by frequency descending
4. Calculate cumulative coverage: top-N variants covering X% of all cases

Variant table output:

Rule of thumb: Top 3 variants covering >80% of cases defines the "happy path". Variants with frequency=1 are candidates for data quality issues.

7.3 Throughput Time

• Case throughput time = max(timestamp) − min(timestamp) for all events in a case
• Mean throughput time = average across all cases
• Median throughput time = 50th percentile (less sensitive to outliers)
• P90 throughput time = 90th percentile — the SLA breach threshold for most orgs

Report in human-readable units: minutes if < 60 min, hours if < 48 hours, days otherwise.

7.4 Waiting Time vs Processing Time

When both start and complete lifecycle events exist for an activity:

• Processing time = complete.timestamp − start.timestamp (actual work duration)
• Waiting time = next_activity.start.timestamp − current_activity.complete.timestamp

When only point-in-time events exist (M365 audit, Dataverse): set processing time = 0, waiting time = gap between consecutive events in the case.

7.5 Bottleneck Detection

1. For each activity, calculate mean and P90 waiting time before that activity starts
2. Activities with P90 waiting time > threshold (default: P90 of all activities × 2) are bottleneck candidates
3. Also flag activities with rework — cases where the same activity appears more than once
4. Report: bottleneck activity name, mean wait, P90 wait, % of cases affected, # rework instances

7.6 Conformance Checking

Approach: token-based replay against a reference process

Reference process is a sequence or graph of allowed activity orderings, provided as a simple spec file:

START → [Flow Started] → [Validation Step] → [Approval Requested] → {[Approved] | [Rejected]} → END

Deviations to detect:

1. Missing activity — expected activity not found in case
2. Extra activity — unexpected activity found in case
3. Wrong order — activity occurs before its required predecessor
4. Repeated activity — activity occurs more than allowed

Conformance metrics:

• Fitness = cases with zero deviations / total cases (0–1)
• Per-case deviation score = number of deviations found in that case
• Per-activity deviation rate = % of cases where this activity deviates

8. People / Resource Analysis Algorithms

8.1 Workload Distribution

For each unique resource (user UPN or service):

1. Count total activities performed
2. Calculate percentage of total activities
3. Calculate mean throughput time for cases they participated in
4. Flag users more than 2σ above mean workload as overloaded

Output table:

8.2 Handover Network

A handover occurs when one user performs activity A and a different user performs the immediately following activity B in the same case.

Algorithm:

1. For each consecutive activity pair (A → B) in each case:
   • If resource(A) ≠ resource(B), record handover (resource(A) → resource(B))
2. Count handover frequency per pair
3. Calculate mean delay between handover events (time from A.complete to B.start)
4. Flag pairs with mean delay > P75 of all handovers as handover bottlenecks

Output (Mermaid):

flowchart LR
    alice -->|324 handovers, avg 0.4h| bob
    bob -->|298 handovers, avg 1.2h| carol
    carol -->|187 handovers, avg 4.5h ⚠️| dave

8.3 Per-User Conformance

Using the same conformance algorithm (§7.6), calculate per user:

• Cases where user participated with zero deviations / total cases with user participation
• Rank users by deviation rate descending

8.4 Role / Authorization Check

Given an --expected-roles mapping file (JSON: {activityName: [authorized_upn_or_group]}):

• For each activity event in the log, check if resource is in the authorized list
• Flag events where resource is NOT authorized — include: case ID, activity, actual user, expected users
• Summarize: total unauthorized actions, unique unauthorized users, most violated activities

9. Output Report Templates

Every analysis command must produce a structured report:

## Executive Summary
- [3-5 bullets: most important findings for a non-technical stakeholder]

## Key Metrics
| Metric | Value |
|---|---|
| Total Cases | N |
| Total Events | N |
| Mean Throughput Time | X hrs |
| ... | ... |

## Findings
### Finding 1: [Title]
**Evidence**: Specific numbers and examples from the log
**Impact**: What this means for the business process
**Recommendation**: Specific, actionable step to address it

## Process / Flow Diagram
[Mermaid DFG or handover network diagram]

## Action Items
| Priority | Action | Owner | Effort |
|---|---|---|---|
| High | ... | Process Owner | Low |

## Data Quality Notes
[Caveats: missing data, skipped events, sampling, time zone assumptions]

## Next Steps
[Which command to run next — e.g., "Run /performance-analyze to quantify bottleneck impact"]

10. Decision Tree

1. First time setup / don't know where to start? → /mining-setup
2. Have a setup context, need raw event data? → /log-extract
3. Have an event log CSV, need to see what process actually runs? → /process-discover
4. Need to find bottlenecks, slow cases, rework? → /performance-analyze
5. Need to compare actual vs. intended process? → /conformance-check
6. Need to understand who is doing what, who is overloaded? → /resource-analyze
7. End-to-end flow: /mining-setup → /log-extract → /process-discover → /performance-analyze → /conformance-check → /resource-analyze

11. Official Documentation References

Progressive Disclosure — Reference Files