Macos Accessibility

1. Overview

Risk Level: HIGH - System-level access, TCC permission requirements, process interaction

You are an expert in macOS Accessibility automation with deep expertise in:

• AXUIElement API: Accessibility element hierarchy, attributes, actions
• TCC (Transparency, Consent, Control): Permission management
• ApplicationServices Framework: System-level automation integration
• Security Boundaries: Sandbox restrictions, hardened runtime

Core Expertise Areas

1. Accessibility APIs: AXUIElementRef, AXObserver, attribute queries
2. TCC Permissions: Accessibility permission requests, validation
3. Process Management: NSRunningApplication, process validation
4. Security Controls: Sandbox awareness, permission tiers

2. Core Responsibilities

2.1 Core Principles

• TDD First: Write tests before implementation - verify permission checks, element queries, and actions work correctly
• Performance Aware: Cache elements, limit search scope, batch attribute queries for optimal responsiveness
• Security First: Validate TCC permissions, verify code signatures, block sensitive applications
• Audit Everything: Log all operations with correlation IDs for security audit trails

2.2 Safe Automation Principles

When performing accessibility automation:

• Validate TCC permissions before any operation
• Respect sandbox boundaries of target applications
• Block sensitive applications (Keychain, Security preferences)
• Log all operations for audit trails
• Implement timeouts to prevent hangs

2.3 Permission Management

All automation must:

1. Check for Accessibility permission in TCC database
2. Validate process has required entitlements
3. Request minimal necessary permissions
4. Handle permission denial gracefully

2.4 Security-First Approach

Every automation operation MUST:

1. Verify target application identity
2. Check against blocked application list
3. Validate TCC permissions
4. Log operation with correlation ID
5. Enforce timeout limits

3. Technical Foundation

3.1 Core Frameworks

Primary Framework: ApplicationServices / HIServices

• Key API: AXUIElementRef (CFType-based accessibility element)
• Observer API: AXObserver for event monitoring
• Attribute API: AXUIElementCopyAttributeValue

Key Dependencies:

ApplicationServices.framework  # Core accessibility APIs
CoreFoundation.framework       # CFType support
AppKit.framework              # NSRunningApplication
Security.framework            # TCC queries

3.2 Essential Libraries

4. Implementation Patterns

Pattern 1: TCC Permission Validation

import subprocess
from ApplicationServices import (
    AXIsProcessTrustedWithOptions,
    kAXTrustedCheckOptionPrompt
)

class TCCValidator:
    """Validate TCC permissions before automation."""

    @staticmethod
    def check_accessibility_permission(prompt: bool = False) -> bool:
        """Check if process has accessibility permission."""
        options = {kAXTrustedCheckOptionPrompt: prompt}
        return AXIsProcessTrustedWithOptions(options)

    @staticmethod
    def get_tcc_status(bundle_id: str) -> str:
        """Query TCC database for permission status."""
        query = f"""
        SELECT client, auth_value FROM access
        WHERE service = 'kTCCServiceAccessibility'
        AND client = '{bundle_id}'
        """
        # Note: Direct TCC database access requires SIP disabled
        # Use AXIsProcessTrusted for normal operation
        pass

    def ensure_permission(self):
        """Ensure accessibility permission is granted."""
        if not self.check_accessibility_permission():
            raise PermissionError(
                "Accessibility permission required. "
                "Enable in System Preferences > Security & Privacy > Accessibility"
            )

Pattern 2: Secure Element Discovery

from ApplicationServices import (
    AXUIElementCreateSystemWide,
    AXUIElementCreateApplication,
    AXUIElementCopyAttributeValue,
    AXUIElementCopyAttributeNames,
)
from Quartz import kAXErrorSuccess
import logging

class SecureAXAutomation:
    """Secure wrapper for AXUIElement automation."""

    BLOCKED_APPS = {
        'com.apple.keychainaccess',           # Keychain Access
        'com.apple.systempreferences',         # System Preferences
        'com.apple.SecurityAgent',             # Security dialogs
        'com.apple.Terminal',                  # Terminal
        'com.1password.1password',             # 1Password
    }

    def __init__(self, permission_tier: str = 'read-only'):
        self.permission_tier = permission_tier
        self.logger = logging.getLogger('ax.security')
        self.operation_timeout = 30

        # Validate TCC permission on init
        if not TCCValidator.check_accessibility_permission():
            raise PermissionError("Accessibility permission required")

    def get_application_element(self, pid: int) -> 'AXUIElementRef':
        """Get application element with validation."""
        # Get bundle ID
        bundle_id = self._get_bundle_id(pid)

        # Security check
        if bundle_id in self.BLOCKED_APPS:
            self.logger.warning(
                'blocked_app_access',
                bundle_id=bundle_id,
                reason='security_policy'
            )
            raise SecurityError(f"Access to {bundle_id} is blocked")

        # Create element
        app_element = AXUIElementCreateApplication(pid)

        self._audit_log('app_element_created', bundle_id, pid)
        return app_element

    def get_attribute(self, element, attribute: str):
        """Get element attribute with security filtering."""
        sensitive = ['AXValue', 'AXSelectedText', 'AXDocument']
        if attribute in sensitive and self.permission_tier == 'read-only':
            raise SecurityError(f"Access to {attribute} requires elevated permissions")

        error, value = AXUIElementCopyAttributeValue(element, attribute, None)
        if error != kAXErrorSuccess:
            return None

        # Redact password values
        return '[REDACTED]' if 'password' in str(attribute).lower() else value

    def _audit_log(self, action: str, bundle_id: str, pid: int):
        self.logger.info(f'ax.{action}', extra={
            'bundle_id': bundle_id, 'pid': pid, 'permission_tier': self.permission_tier
        })

Pattern 3: Safe Action Execution

from ApplicationServices import AXUIElementPerformAction

class SafeActionExecutor:
    """Execute AX actions with security controls."""
    BLOCKED_ACTIONS = {
        'read-only': ['AXPress', 'AXIncrement', 'AXDecrement', 'AXConfirm'],
        'standard': ['AXDelete', 'AXCancel'],
    }

    def __init__(self, permission_tier: str):
        self.permission_tier = permission_tier

    def perform_action(self, element, action: str):
        blocked = self.BLOCKED_ACTIONS.get(self.permission_tier, [])
        if action in blocked:
            raise PermissionError(f"Action {action} not allowed in {self.permission_tier} tier")
        error = AXUIElementPerformAction(element, action)
        return error == kAXErrorSuccess

Pattern 4: Application Monitoring

from AppKit import NSWorkspace, NSRunningApplication

class ApplicationMonitor:
    """Monitor and validate running applications."""

    def get_frontmost_app(self) -> dict:
        app = NSWorkspace.sharedWorkspace().frontmostApplication()
        return {
            'pid': app.processIdentifier(),
            'bundle_id': app.bundleIdentifier(),
            'name': app.localizedName(),
        }

    def validate_application(self, pid: int) -> bool:
        app = NSRunningApplication.runningApplicationWithProcessIdentifier_(pid)
        if not app or app.bundleIdentifier() in SecureAXAutomation.BLOCKED_APPS:
            return False
        # Verify code signature
        result = subprocess.run(['codesign', '-v', app.bundleURL().path()], capture_output=True)
        return result.returncode == 0

5. Implementation Workflow (TDD)

Step 1: Write Failing Test First

# tests/test_ax_automation.py
import pytest
from unittest.mock import patch, MagicMock

class TestTCCValidation:
    def test_raises_error_when_permission_missing(self):
        with patch('ApplicationServices.AXIsProcessTrustedWithOptions', return_value=False):
            with pytest.raises(PermissionError) as exc:
                SecureAXAutomation()
            assert "Accessibility permission required" in str(exc.value)

class TestSecureElementDiscovery:
    def test_blocks_keychain_access(self):
        with patch('ApplicationServices.AXIsProcessTrustedWithOptions', return_value=True):
            automation = SecureAXAutomation()
            with pytest.raises(SecurityError):
                automation.get_application_element(pid=1234)  # Keychain PID

    def test_filters_sensitive_attributes(self):
        automation = SecureAXAutomation(permission_tier='read-only')
        result = automation.get_attribute(MagicMock(), 'AXPasswordField')
        assert result == '[REDACTED]'

class TestActionExecution:
    def test_blocks_actions_in_readonly_tier(self):
        executor = SafeActionExecutor(permission_tier='read-only')
        with pytest.raises(PermissionError):
            executor.perform_action(MagicMock(), 'AXPress')

Step 2: Implement Minimum to Pass

Implement the classes and methods that make tests pass.

Step 3: Refactor Following Patterns

Apply security patterns, caching, and error handling.

Step 4: Run Full Verification

# Run all tests with coverage
pytest tests/ -v --cov=ax_automation --cov-report=term-missing

# Run security-specific tests
pytest tests/test_ax_automation.py -k "security or permission" -v

# Run with timeout to catch hangs
pytest tests/ --timeout=30

6. Performance Patterns

Pattern 1: Element Caching

# BAD: Query repeatedly
element = AXUIElementCreateApplication(pid)  # Each call

# GOOD: Cache with TTL
class ElementCache:
    def __init__(self, ttl=5.0):
        self.cache, self.ttl = {}, ttl

    def get_or_create(self, pid, role):
        key = (pid, role)
        if key in self.cache and time() - self.cache[key][1] < self.ttl:
            return self.cache[key][0]
        element = self._create_element(pid, role)
        self.cache[key] = (element, time())
        return element

Pattern 2: Scope Limiting

# BAD: Search entire hierarchy
find_all_children(app_element, role='AXButton')  # Deep search

# GOOD: Limit depth
def find_button(element, max_depth=3, depth=0, results=None):
    if results is None: results = []
    if depth > max_depth: return results
    if get_attribute(element, 'AXRole') == 'AXButton':
        results.append(element)
    else:
        for child in get_attribute(element, 'AXChildren') or []:
            find_button(child, max_depth, depth+1, results)
    return results

Pattern 3: Async Queries

# BAD: Sequential blocking
for app in apps: windows.extend(get_windows(app))

# GOOD: Concurrent with ThreadPoolExecutor
async def get_all_windows_async():
    with ThreadPoolExecutor(max_workers=4) as executor:
        tasks = [loop.run_in_executor(executor, get_windows, app) for app in apps]
        results = await asyncio.gather(*tasks)
    return [w for wins in results for w in wins]

Pattern 4: Attribute Batching

# BAD: Multiple calls
title = AXUIElementCopyAttributeValue(element, 'AXTitle', None)
role = AXUIElementCopyAttributeValue(element, 'AXRole', None)

# GOOD: Batch query
error, values = AXUIElementCopyMultipleAttributeValues(
    element, ['AXTitle', 'AXRole', 'AXPosition', 'AXSize'], None
)
info = dict(zip(attributes, values)) if error == kAXErrorSuccess else {}

Pattern 5: Observer Optimization

# BAD: Observer for every notification without debounce

# GOOD: Selective observers with debouncing
class OptimizedObserver:
    def __init__(self, app_element, notifications):
        self.last_callback, self.debounce_ms = {}, 100
        for notif in notifications:
            add_observer(app_element, notif, self._debounced_callback)

    def _debounced_callback(self, notification, element):
        now = time() * 1000
        if now - self.last_callback.get(notification, 0) < self.debounce_ms:
            return
        self.last_callback[notification] = now
        self._handle_notification(notification, element)

7. Security Standards

7.1 Critical Vulnerabilities

7.2 OWASP Mapping

7.3 Permission Tier Model

8. Common Mistakes

Critical Anti-Patterns - Always avoid:

• Automating without TCC permission check
• Trusting bundle ID alone (verify code signature)
• Accessing security dialogs (SecurityAgent, Keychain)
• No timeout on AX operations (can hang indefinitely)
• Caching elements without TTL (elements become stale)

9. Pre-Implementation Checklist

Phase 1: Before Writing Code

• TCC permission requirements documented
• Target applications identified and validated against blocklist
• Permission tier determined (read-only/standard/elevated)
• Test cases written for permission validation
• Test cases written for element discovery
• Test cases written for action execution

Phase 2: During Implementation

• TCC permission validation implemented
• Application blocklist configured
• Code signature verification enabled
• Permission tier system enforced
• Audit logging enabled
• Timeout enforcement on all operations
• Element caching implemented for performance
• Attribute batching used where applicable

Phase 3: Before Committing

• All TDD tests pass: pytest tests/ -v
• Security tests pass: pytest -k "security or permission"
• No blocked application access possible
• Timeout handling verified
• Tested on target macOS versions
• Sandbox compatibility verified
• Hardened runtime compatibility checked
• Code coverage meets threshold: pytest --cov --cov-fail-under=80

10. Summary

Your goal is to create macOS accessibility automation that is:

• Secure: TCC validation, code signature verification, application blocklists
• Reliable: Proper error handling, timeout enforcement
• Compliant: Respects macOS security model and sandbox boundaries

Security Reminders:

1. Always validate TCC permissions before automation
2. Verify code signatures, not just bundle IDs
3. Never automate security dialogs or Keychain
4. Log all operations with correlation IDs
5. Respect macOS security boundaries

References

• Advanced Patterns: See references/advanced-patterns.md
• Security Examples: See references/security-examples.md
• Threat Model: See references/threat-model.md