Td Change Point

Teradata Change Point Detection

Core Capabilities

• MLE table-operator implementation with PARTITION BY and ORDER BY support
• Scalable time series analysis for millions of products or billions of IoT sensors
• Multiple segmentation methods including normal distribution modeling
• Production-ready SQL generation with proper MLE ON...USING syntax
• Comprehensive error handling and data validation
• Business-focused interpretation of analytical results
• Flexible search methods (binary, linear) for change point detection

Machine Learning Engine (MLE) Overview

The Machine Learning Engine is Teradata's framework for advanced analytics using table operators:

• Table-operator pattern: SELECT * FROM Function(ON table PARTITION BY ... ORDER BY ... USING ...)
• Parallel execution across AMPs via PARTITION BY
• Built-in functions for classification, regression, time series, and more
• SQL-native integration with standard Teradata workflows

Table Analysis Workflow

This skill automatically analyzes your time series data to generate optimized MLE workflows:

1. Time Series Structure Analysis

• Temporal Column Detection: Identifies time/date columns for ordering
• Value Column Classification: Distinguishes between numeric time series values
• Series ID Detection: Identifies grouping columns for PARTITION BY
• Frequency Analysis: Determines sampling frequency and intervals

2. MLE-Specific Recommendations

• Partition Strategy: Configures PARTITION BY for parallel processing
• Parameter Optimization: Suggests optimal parameters for ChangePointDetection
• Search Method Selection: Recommends binary vs linear search
• Segmentation Method: Suggests appropriate statistical model

3. SQL Generation Process

• MLE Syntax Generation: Creates proper ON...PARTITION BY...ORDER BY...USING SQL
• Parameter Configuration: Sets function-specific parameters
• Result Storage: Generates CREATE TABLE AS patterns for persisting results

How to Use This Skill

1. Provide Your Time Series Data:

   "Analyze time series table: database.sensor_data with timestamp column and value columns"
   

2. The Skill Will:

   • Analyze temporal structure and sampling frequency
   • Identify optimal function parameters
   • Generate complete ChangePointDetection workflow
   • Provide performance optimization recommendations

Input Requirements

Data Requirements

• Time series table: Teradata table with temporal data
• Time column: Time/date column for ORDER BY
• Value column: Numeric column to analyze for change points
• Series ID column: Column to PARTITION BY (grouping key for parallel processing)

Technical Requirements

• Teradata Vantage with Machine Learning Engine (MLE) enabled
• MLE License: Access to analytic table operators
• Database permissions: CREATE, DROP, SELECT on working database
• Function access: ChangePointDetection

Output Formats

Generated Results

• Change point locations with timestamps
• Segment boundaries identifying structural breaks
• Statistical metrics for each detected change point
• Diagnostic information for result validation

SQL Scripts

• Complete MLE workflows ready for execution
• Parameterized queries optimized for your data structure
• Result tables with proper schema for downstream analysis

Time Series Analysis Use Cases Supported

1. Structural break detection: Identify regime changes in time series
2. Change point analysis: Detect shifts in statistical properties
3. Regime changes: Find transitions between different data-generating processes
4. Anomaly detection: Locate abrupt changes in sensor or financial data

Key Parameters for ChangePointDetection

• TargetColumn: The numeric column to analyze for change points
• SegmentationMethod: Statistical model for segments (e.g., 'normal_distribution')
• SearchMethod: Algorithm for finding change points ('binary' or 'linear')
• MaxChangeNum: Maximum number of change points to detect
• Penalty: Information criterion for model selection ('BIC', 'AIC', etc.)
• OutputOption: Type of output ('changepoint', 'segment', 'verbose')

MLE Best Practices Applied

• PARTITION BY optimization for parallel processing across series
• ORDER BY with proper temporal column for time series ordering
• Parameter tuning specific to ChangePointDetection
• Result persistence using CREATE TABLE AS patterns
• Error handling for MLE-specific scenarios
• Scalability considerations for production workloads

Example Usage

-- Example ChangePointDetection workflow
-- Replace parameters with your specific requirements

-- 1. Execute ChangePointDetection
SELECT * FROM ChangePointDetection (
    ON your_database.your_timeseries_table AS InputTable
    PARTITION BY series_id
    ORDER BY timestamp_col
    USING
    TargetColumn('value_col')
    SegmentationMethod('normal_distribution')
    SearchMethod('binary')
    MaxChangeNum(10)
    Penalty('BIC')
    OutputOption('changepoint')
) AS dt;

Scripts Included

Core MLE Scripts

• mle_data_preparation.sql: Data preparation for MLE processing
• td_change_point_workflow.sql: Complete ChangePointDetection implementation
• table_analysis.sql: Time series structure analysis
• parameter_optimization.sql: Function parameter tuning

Integration Scripts

• mle_pipeline_template.sql: Multi-step analytical workflows
• performance_monitoring.sql: Execution monitoring
• result_interpretation.sql: Output analysis and visualization

Industry Applications

Supported Domains

• Economic forecasting and financial analysis
• Sales forecasting and demand planning
• Manufacturing process monitoring and quality control
• IoT sensor data analysis and alerting
• Network monitoring and traffic analysis
• Energy load and consumption pattern analysis

Limitations and Considerations

• MLE licensing: Requires proper Teradata MLE licensing
• Data ordering: Time column must support meaningful ORDER BY
• Computational complexity: Large datasets with many partitions may be resource-intensive
• Data quality: Results depend on clean, well-structured time series data
• Parameter sensitivity: Function performance depends on proper parameter tuning
• Temporal consistency: Irregular sampling may affect detection quality

Quality Checks

Automated Validations

• Time series structure verification
• Partition key distribution checks
• Parameter validation for ChangePointDetection
• Result quality assessment

Manual Review Points

• Parameter selection appropriateness
• Result interpretation accuracy
• Performance optimization opportunities
• Integration with existing workflows

Updates and Maintenance

• MLE compatibility: Tested with latest Teradata Vantage releases
• Performance optimization: Regular MLE-specific optimizations
• Best practices: Updated with Teradata community recommendations
• Documentation: Maintained with latest MLE features

This skill provides production-ready time series change point detection using Teradata's Machine Learning Engine ChangePointDetection function with industry best practices for scalable analytics.