Teradata Word Embeddings

Teradata Word Embeddings

Core Capabilities

• Automated table structure analysis via DBC.ColumnsV
• Dynamic SQL generation for TD_WordEmbeddings
• Complete workflow from data preparation to results interpretation
• Data quality validation and preprocessing guidance
• Parameter optimization and tuning

Key Parameters

• TextColumn: Column containing text data
• Operation: 'TOKEN-EMBEDDING','DOC-EMBEDDING','TOKEN2TOKEN-SIMILARITY','DOC2DOC-SIMILARITY'
• EmbeddingModelTable: Pre-trained embedding model (e.g., GloVe) stored as a table
• Accumulate: Columns to pass through

Use Cases

1. Document similarity analysis
2. Text classification feature extraction
3. Semantic search
4. Content recommendation
5. Sentiment analysis preprocessing

Example Usage

-- TD_WordEmbeddings execution
SELECT * FROM TD_WordEmbeddings (
    ON {USER_DATABASE}.{TEXT_TABLE} AS InputTable
    ON {USER_DATABASE}.{EMBEDDING_MODEL_TABLE} AS EmbeddingModelTable DIMENSION
    USING
    TextColumn ('{TEXT_COLUMN}')
    Operation ('DOC-EMBEDDING')        -- 'TOKEN-EMBEDDING','DOC-EMBEDDING',
                                       -- 'TOKEN2TOKEN-SIMILARITY','DOC2DOC-SIMILARITY'
    Accumulate ('{DOC_ID}')
) AS dt;

Scripts Included

Core Analytics Scripts

• table_analysis.sql: Automatic table structure discovery
• preprocessing.sql: Data preparation and feature engineering
• model_training.sql: TD_WordEmbeddings execution
• evaluation.sql: Results analysis and metrics
• complete_workflow_template.sql: End-to-end workflow

Utility Scripts

• data_quality_checks.sql: Comprehensive data validation
• parameter_tuning.sql: Parameter optimization
• diagnostic_queries.sql: Results diagnostics and interpretation

Best Practices

• Always run table_analysis.sql first to understand your data structure
• Validate data quality before executing the analytical function
• Use parameter_tuning.sql to find optimal configuration
• Review diagnostic_queries.sql output for model/results validation

Limitations

• Requires Teradata Vantage 17.20+ with ClearScape Analytics
• Input data must meet function-specific requirements
• Results depend on data quality and parameter configuration

Teradata Word Embeddings - ClearScape Analytics skill for Teradata Vantage