Workflow Triaging

AEM Workflow Triaging — 6.5 LTS / AMS

Classify workflow issues, determine what logs and data to gather, and map to the correct runbook or log search. Optimized for production support on AEM 6.5 LTS and Adobe Managed Services (AMS).

Variant Scope

• This skill is 6.5-lts-only (includes AMS).
• Log access via direct filesystem, AMS log access, or Splunk.
• JMX available for workflow counts, queue metrics, and remediation.

When to use this skill

• User asks: "Workflow errors on <host> for the past X hours", "Workflow activity on <host>", "Why did workflow X fail?", "What should I collect to debug this workflow ticket?"
• User needs: Symptom classification, log patterns to search, Splunk queries, or required inputs for a runbook.
• Context: AEM 6.5 LTS / AMS (author/publish hostname format).

Step 1: Classify symptom (symptom_id)

Map the user's description to a symptom_id and runbook.

Step 2: Required inputs for triage

Before suggesting a runbook or Splunk search, try to obtain:

If the user only provides host + time, respond with the generic workflow error searches and note that narrowing by model/instance ID will improve accuracy.

Step 3: Log patterns and Splunk (what to search)

Logs on 6.5 / AMS are accessible via direct filesystem (crx-quickstart/logs/error.log), AMS log access, or Splunk (if indexed).

Example Splunk searches (replace index/sourcetype/field names as needed):

• All workflow step errors (last 24h): index=aem sourcetype=aem:error "Error executing workflow step" | table _time host message | sort - _time
• Process not registered: index=aem "getProcess for" "failed" | table _time host message
• By workflow model or instance: index=aem ("Error executing workflow step" OR WorkflowException) (message=*<modelName>* OR message=*<instanceId>*) | sort - _time
• Lock contention: index=aem "wait for a lock" OR "refreshing the session since we had to wait" | table _time host message

Step 4: JMX-based diagnostics

On 6.5 / AMS, use JMX for metrics that are not available from logs alone.

Always use dryRun=true first before executing destructive operations.

Step 5: Example triage prompts and responses

Step 6: What logs and JMX can and cannot answer

Can answer (logs + JMX on 6.5 / AMS):

• Step failures: exception type, message, stack (by host, time, model, step).
• Process not registered: which process.label is missing (logs + Felix Console).
• Stuck: step errors, getProcess failures, lock wait, payload/path errors.
• Stale: JMX countStaleWorkflows + "Cannot archive workitem" in logs.
• Queue metrics: JMX returnWorkflowQueueInfo, returnSystemJobInfo.
• Running instance counts: JMX countRunningWorkflows.
• Throughput: lock wait, session refresh, JobHandler volume.
• Permission: Terminate/Resume/Suspend failed, AccessControlException.
• Payload/launcher: PathNotFoundException, launcher config errors.
• Purge: JMX purgeCompleted(dryRun) + "Workflow purge …" in logs.
• Thread pool state: Configuration status ZIP → 039_Sling_Thread_Pools.txt.
• Config state: Felix Console or config status ZIP → 003_Configurations.txt.

Cannot answer directly:

• Console state (e.g. "is there a current work item?"). Use Workflow Console UI.
• Runtime process step code behavior. Requires code review + log correlation.

Always pair log-based triage with JMX diagnostics and the appropriate runbook.

References (in repo)

• Machine-readable index: aem-agent-marketplace-workflow-knowledge-base/docs/debugging-index.md
• Decision guide: runbooks/runbook-decision-guide.md
• Splunk scenarios and queries: Workflow-docs/splunk-workflow-triaging.md
• Error patterns: docs/error-patterns.md