Artifact Collector

LimaCharlie Artifact Collector

This skill provides comprehensive guidance for collecting and managing forensic artifacts from endpoints using LimaCharlie. Use this skill when users need to gather evidence, collect files, capture memory, stream logs, or perform forensic investigations.

Quick Navigation

• SKILL.md (this file): Overview, quick start, common workflows
• REFERENCE.md: Complete command syntax and parameters
• EXAMPLES.md: 10 detailed investigation scenarios
• TROUBLESHOOTING.md: Common issues and solutions

Artifact Collection Overview

What are Artifacts?

Artifacts are pieces of forensic evidence collected from endpoints during security investigations or incident response. In LimaCharlie, artifacts can include:

• Files from disk
• Memory dumps (full system or process-specific)
• Windows Event Log (WEL) data
• Mac Unified Log (MUL) data
• Network packet captures (PCAP)
• File system metadata (MFT)

Why Collect Artifacts?

Artifact collection is critical for:

• Incident Response: Gathering evidence during security incidents
• Forensic Analysis: Conducting detailed investigations
• Threat Hunting: Searching for indicators of compromise
• Compliance: Meeting regulatory evidence preservation requirements
• Malware Analysis: Collecting suspicious files and memory for analysis

Prerequisites

To use artifact collection features, you must:

1. Enable the Artifact Extension in your organization
2. Enable the Reliable Tasking Extension (required dependency)
3. Configure artifact collection rules (optional, for automated collection)

Note on Billing: While the Artifact extension is free to enable, ingested artifacts incur charges. Refer to LimaCharlie pricing for artifact ingestion and retention costs.

Artifact Types Overview

1. Files

Collect files from endpoints for analysis or preservation.

Use Cases: Retrieve suspicious executables, collect log files, gather configuration files

Collection Pattern: Use file paths with wildcards or exact paths

• C:\Users\*\Downloads\*.exe
• /var/log/auth.log

2. Memory Dumps

Capture volatile memory for forensic analysis.

Types: Full Memory Dumps, Process Memory, Memory Strings

Use Cases: Detect in-memory malware, analyze running processes, extract encryption keys

3. Windows Event Logs (WEL)

Stream or collect Windows Event Log data.

Collection Modes:

• Real-time streaming: Use wel:// pattern (included in sensor flat rate)
• File collection: Collect .evtx files (incurs artifact costs)

Common Logs: wel://Security:*, wel://System:*, wel://Application:*

4. Mac Unified Logs (MUL)

Stream or collect macOS Unified Logging data.

Collection Pattern: Use mul:// prefix for real-time streaming

5. Network Packet Captures (PCAP)

Capture network traffic for analysis (Linux only).

Use Cases: Network forensics, protocol analysis, data exfiltration detection

6. File System Metadata (MFT)

Collect Master File Table data from Windows systems.

Use Cases: Timeline analysis, file system forensics, deleted file recovery

Working with Timestamps

IMPORTANT: When users provide relative time offsets (e.g., "last hour", "past 24 hours", "last week"), you MUST dynamically compute the current epoch timestamp based on the actual current time. Never use hardcoded or placeholder timestamps.

Computing Current Epoch

import time

# Compute current time dynamically
current_epoch_seconds = int(time.time())
current_epoch_milliseconds = int(time.time() * 1000)

The granularity (seconds vs milliseconds) depends on the specific API or MCP tool. Always check the tool signature or API documentation to determine which unit to use.

Common Relative Time Calculations

Example: "List artifacts from the last 24 hours"

end_time = int(time.time())  # Current time
start_time = end_time - 86400  # 24 hours ago

Common offsets (in seconds):

• 1 hour = 3600
• 24 hours = 86400
• 7 days = 604800
• 30 days = 2592000

For millisecond-based APIs, multiply by 1000.

Critical Rules

NEVER:

• Use hardcoded timestamps
• Use placeholder values like 1234567890
• Assume a specific current time

ALWAYS:

• Compute dynamically using time.time()
• Check the API/tool signature for correct granularity
• Verify the time range is valid (start < end)

Quick Start

Most Common Collection: Suspicious File

# 1. Hash the file first (verify without downloading)
file_hash --path C:\Users\alice\Downloads\suspicious.exe

# 2. Collect the file
artifact_get C:\Users\alice\Downloads\suspicious.exe

# 3. Get file metadata
file_info --path C:\Users\alice\Downloads\suspicious.exe

Quick Memory Investigation

# 1. List processes
os_processes

# 2. Get memory map for suspicious process
mem_map --pid 1234

# 3. Extract strings from memory
mem_strings --pid 1234

# 4. Search for specific indicator
mem_find_string --pid 1234 --string "malicious-domain.com"

Quick Log Collection

# Windows Event Logs
log_get Security
log_get System

# Or collect the .evtx files
artifact_get C:\Windows\System32\winevt\Logs\Security.evtx

Common Workflows

Workflow 1: Malware File Investigation

When to use: Suspicious file detected on endpoint

Steps:

1. Hash first to verify without collecting:

   file_hash --path C:\path\to\suspicious.exe
   

2. Get file details:

   file_info --path C:\path\to\suspicious.exe
   

3. Collect the file:

   artifact_get C:\path\to\suspicious.exe
   

4. Check surrounding context:

   dir_list --path C:\path\to
   

Workflow 2: Process Memory Analysis

When to use: Investigating suspicious running process

Steps:

1. Identify the process:

   os_processes
   

2. Map the memory:

   mem_map --pid <process_id>
   

3. Extract strings:

   mem_strings --pid <process_id>
   

4. Search for IOCs:

   mem_find_string --pid <process_id> --string "suspicious-indicator"
   

5. Full memory dump (if needed via Dumper extension):

   extension request: {target: "memory", sid: "sensor-id", retention: 7}
   

Workflow 3: Authentication Investigation

When to use: Investigating suspicious login activity

Steps:

1. Collect Security logs:

   log_get Security
   

2. Check current users:

   os_users
   

3. Check network connections:

   netstat
   

4. Get current processes:

   os_processes
   

Workflow 4: Automated Collection on Detection

When to use: Set up proactive evidence collection

Example D&R Rule:

detect:
  event: NEW_DOCUMENT
  op: and
  rules:
    - op: matches
      path: event/FILE_PATH
      re: .*\.(exe|dll|scr)$
      case sensitive: false
    - op: contains
      path: event/FILE_PATH
      value: \Downloads\

respond:
  - action: report
    name: suspicious-file-written
  - action: task
    command: artifact_get {{ .event.FILE_PATH }}
    investigation: auto-collection
    suppression:
      max_count: 1
      period: 1h
      is_global: false
      keys:
        - '{{ .event.FILE_PATH }}'

Workflow 5: Comprehensive Incident Response

When to use: Active security incident requiring full investigation

Priority Order:

1. Volatile data first (disappears when system powers off):

   os_processes
   netstat
   mem_map --pid <suspicious_pid>
   mem_strings --pid <suspicious_pid>
   

2. Critical files:

   artifact_get <suspicious_executable>
   artifact_get <malicious_script>
   

3. System artifacts:

   log_get Security
   log_get System
   history_dump
   

4. Full dumps (via Dumper extension):

   • Memory dump
   • MFT dump

Reliable Tasking Overview

What is Reliable Tasking?

Reliable Tasking allows you to queue artifact collection commands for sensors that are currently offline. Tasks are automatically delivered when the sensor comes online.

When to Use Reliable Tasking

• Sensors with intermittent connectivity
• Collecting from remote/mobile devices
• Ensuring collection happens on next check-in
• Large-scale deployments

Creating a Reliable Task

Via REST API:

curl --location 'https://api.limacharlie.io/v1/extension/request/ext-reliable-tasking' \
--header 'Authorization: Bearer $JWT' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data 'oid=$YOUR_OID&action=task&data={"context":"incident-response","selector":"tag==offline-hosts","task":"artifact_get C:\\Windows\\System32\\malware.exe","ttl":86400}'

Key Parameters:

• context: Identifier for grouping related tasks
• selector: Target criteria (sensor ID, tag, platform)
• task: The command to execute
• ttl: Time-to-live in seconds (default: 1 week)

Targeting Options:

• sid: Specific Sensor ID
• tag: All sensors with a specific tag
• plat: All sensors of a platform (windows, linux, macos)
• selector: Advanced selector expression

Tracking Responses

Use the context parameter to track responses via D&R rules:

detect:
  op: contains
  event: RECEIPT
  path: routing/investigation_id
  value: incident-response

respond:
  - action: report
    name: collection-completed
  - action: output
    name: artifact-responses

For complete Reliable Tasking details, see REFERENCE.md.

Storage and Access

Where Artifacts Are Stored

Collected artifacts are stored in LimaCharlie's artifact storage with:

• Configurable retention periods (default: 30 days)
• Secure, encrypted storage
• Access controls based on organization permissions
• Unique artifact identifiers

Accessing Collected Artifacts

Via Web UI:

1. Navigate to Sensors > Artifact Collection
2. View collected artifacts list
3. Click on artifact to view details
4. Download artifact for analysis

Via REST API:

# List artifacts
GET https://api.limacharlie.io/v1/orgs/{oid}/artifacts

# Download specific artifact
GET https://api.limacharlie.io/v1/orgs/{oid}/artifacts/{artifact_id}

Cost Optimization

Reduce Costs:

• Use wel:// for logs instead of .evtx files
• Set minimal necessary retention
• Implement collection suppression
• Filter events before collection
• Use file hashes to avoid duplicate collection

Monitor Usage:

• Track artifact ingestion volumes
• Review billing regularly
• Set usage alerts (via Usage Alerts extension)

Quick Command Reference

For complete command syntax and parameters, see REFERENCE.md.

File Commands

• artifact_get <file_path> - Collect file to artifact storage
• file_get --path <file_path> - Get file content in response
• file_hash --path <file_path> - Calculate file hash
• file_info --path <file_path> - Get file metadata
• dir_list --path <directory> - List directory contents

Memory Commands

• mem_read --pid <pid> --base <addr> --size <bytes> - Read process memory
• mem_map --pid <pid> - Get process memory map
• mem_strings --pid <pid> - Extract strings from memory
• mem_find_string --pid <pid> --string <text> - Search memory for string

Log Commands

• log_get <log_name> - Get Windows Event Log (Windows only)
• history_dump - Dump sensor's cached events

Analysis Commands

• os_processes - List running processes
• netstat - Show network connections
• os_users - List user accounts
• os_services - List system services
• os_autoruns - List autostart programs

Best Practices

1. Collect Volatile Data First: Memory, processes, and network connections disappear when systems power off
2. Use Suppression: Prevent resource exhaustion with max_count: 1 and period: 1h in automated rules
3. Hash Before Collecting: Use file_hash to verify files without downloading
4. Use Investigation IDs: Track related artifacts with the investigation parameter
5. Set Appropriate Retention: Balance compliance needs (7-90 days) with costs
6. Monitor Costs: Track artifact volumes, set alerts, review rules regularly

See TROUBLESHOOTING.md for detailed cost optimization strategies.

Next Steps

For Complete Command Reference

See REFERENCE.md for:

• Complete command syntax
• All parameters and options
• Platform compatibility
• Response event types
• Dumper extension details

For Investigation Examples

See EXAMPLES.md for 10 detailed scenarios:

• Malware incident response
• Ransomware response
• Data exfiltration detection
• Lateral movement detection
• Linux server compromise
• Memory-only malware
• Compliance evidence collection
• And more...

For Troubleshooting

See TROUBLESHOOTING.md for:

• Collection failures
• Permission issues
• Storage problems
• Cost management
• Performance optimization

Summary

Artifact collection is critical for incident response, forensics, threat hunting, and compliance.

Key Takeaways:

1. Enable Artifact and Reliable Tasking extensions
2. Collect volatile data first (memory, processes, network)
3. Use appropriate collection method (manual, automated, reliable)
4. Implement suppression to prevent resource exhaustion
5. Monitor costs and retention
6. Follow evidence preservation best practices
7. Test collection rules before production deployment

Remember:

• Artifacts incur storage costs
• Use templating in D&R rules for dynamic collection
• Leverage Reliable Tasking for offline sensors
• Preserve chain of custody
• Collect only what's needed

For more information, refer to:

• LimaCharlie Artifact Extension documentation
• Reliable Tasking Extension documentation
• Endpoint Agent Commands reference
• Detection & Response rules guide