Overview and Purpose

COMPREHENSIVE_TABLE_SUPPORT

Version: 0.229.058

Overview and Purpose

SimpleChat now supports comprehensive table rendering from multiple input formats, ensuring that tables generated by AI agents or pasted by users are properly displayed as styled HTML tables regardless of their original format.

Version implemented: 0.229.005
Dependencies: Marked.js 15.0.7, DOMPurify 3.1.3, Bootstrap 5.1.3

Problem Statement

AI agents generate tables in various formats that weren't being properly rendered:

• Unicode box-drawing tables (┌─┬─┐ style)
• Markdown tables wrapped in code blocks
• Pipe-separated values (PSV) in code blocks
• Standard markdown tables (worked but needed pipeline integration)

Users were seeing raw text instead of properly formatted tables, reducing readability and user experience.

Technical Specifications

Architecture Overview

The solution implements a preprocessing pipeline that detects and converts various table formats to standard markdown tables before the markdown parser processes them.

Processing Pipeline:

1. Content cleaning and citation parsing
2. Unwrap markdown tables from code blocks
3. Convert Unicode box-drawing tables to markdown
4. Convert pipe-separated values to markdown tables
5. Parse with Marked.js (GFM tables enabled)
6. Sanitize with DOMPurify
7. Apply Bootstrap styling

File Structure

application/single_app/static/js/chat/
├── chat-messages.js          # Main implementation
functional_tests/
├── test_comprehensive_table_support.py     # Validation script
├── test_unicode_table_conversion.html      # Unicode test page
└── test_psv_table_conversion.html         # PSV test page

API Integration Points

No new API endpoints required. The table processing happens entirely on the client side during message rendering.

Implementation Details

1. unwrapTablesFromCodeBlocks()

Purpose: Detects markdown tables wrapped in code blocks and unwraps them for proper parsing.

Pattern: /```[\w]*\n((?:[^\n]*\|[^\n]*\n)+)```/g

Logic:

• Finds code blocks containing pipe-separated content
• Validates that content has multiple lines with pipes
• Removes code block wrapper, preserving table content

2. convertUnicodeTableToMarkdown()

Purpose: Converts Unicode box-drawing tables to standard markdown format.

Pattern: /┌[─┬┐]*┐[\s\S]*?└[─┴┘]*┘/g

Logic:

• Identifies complete Unicode table structures
• Extracts data rows (ignoring border/separator lines)
• Splits cells by │ character
• Builds markdown table with header separator

Example transformation:

┌─────────┬─────────┐
│ Name    │ Status  │
├─────────┼─────────┤
│ App1    │ Active  │
└─────────┴─────────┘

Becomes:
| Name | Status |
| --- | --- |
| App1 | Active |

3. convertPSVCodeBlockToMarkdown()

Purpose: Converts pipe-separated values in code blocks to markdown tables.

Pattern: /```[\w]*\n((?:[^|\n]+\|[^|\n]*(?:\|[^|\n]*)*\n)+)```/g

Logic:

• Detects code blocks with pipe-separated content
• Handles both header-only and header+data scenarios
• Limits display to 50 data rows (plus header) for performance
• Adds truncation notice for large datasets

Row Limit: First 50 rows displayed, with "... and X more rows" notice for larger datasets.

Bootstrap Integration

All generated tables automatically receive Bootstrap classes:

• table - Base table styling
• table-striped - Alternating row colors
• table-hover - Row highlighting on hover
• table-bordered - Cell borders

DOMPurify Configuration

Table elements explicitly allowed through sanitization:

ALLOWED_TAGS: ['table', 'thead', 'tbody', 'tr', 'th', 'td', ...]

Usage Instructions

For Users

No configuration required. The feature works automatically:

1. Paste any table format in a message
2. AI-generated tables are automatically converted
3. Tables display with professional Bootstrap styling

For Developers

Adding new table formats:

1. Create detection regex pattern
2. Implement conversion function following existing patterns
3. Add to processing pipeline in chat-messages.js
4. Create test cases in functional tests

Pipeline integration:

const withUnwrappedTables = unwrapTablesFromCodeBlocks(withInlineCitations);
const withMarkdownTables = convertUnicodeTableToMarkdown(withUnwrappedTables);
const withPSVTables = convertPSVCodeBlockToMarkdown(withMarkdownTables);
const htmlContent = DOMPurify.sanitize(marked.parse(withPSVTables));

Testing and Validation

Functional Tests

• test_comprehensive_table_support.py - Integration validation
• test_unicode_table_conversion.html - Browser testing for Unicode tables
• test_psv_table_conversion.html - Browser testing for PSV format

Test Coverage

✅ Unicode box-drawing table conversion
✅ Standard markdown table preservation
✅ Pipe-separated values in code blocks
✅ Markdown tables wrapped in code blocks
✅ Mixed content with multiple table formats
✅ Bootstrap styling application
✅ DOMPurify sanitization compatibility
✅ Large dataset handling (50+ row tables)

Performance Considerations

• Regex optimization: Patterns designed for efficient matching
• Row limiting: PSV tables limited to 50 visible rows
• Memory usage: Processing happens during render, no storage impact
• Load time: Minimal impact on message rendering speed

Examples and Screenshots

Example 1: Unicode Table Input

┌─────────────┬─────────┬────────────┐
│ Application │ Version │ Status     │
├─────────────┼─────────┼────────────┤
│ Simple Chat │ 0.229   │ Active     │
│ ESAM Agent  │ 1.2.3   │ Testing    │
└─────────────┴─────────┴────────────┘

Renders as: Professional HTML table with Bootstrap styling

Example 2: PSV Code Block Input

Application Name|Version|Environment|Status Simple Chat|0.229.005|Production|Active ESAM Agent|1.2.3|Development|Testing

Renders as: Formatted table with headers and data rows

Example 3: Mixed Format Support

Messages can contain multiple table formats simultaneously, and all will be properly converted and displayed.

Known Limitations

1. Complex Unicode tables with merged cells not supported
2. Very wide tables may require horizontal scrolling on mobile
3. Nested tables are not supported (by design)
4. Custom table styling beyond Bootstrap classes not preserved

Integration Notes

CSS Dependencies

Requires Bootstrap 5.x for optimal table styling. Tables will render without Bootstrap but with basic browser default styling.

JavaScript Dependencies

• Marked.js 15.0.7+ (GFM tables support)
• DOMPurify 3.1.3+ (HTML sanitization)

Browser Compatibility

Works in all modern browsers supporting ES6+ features. Tested in:

• Chrome 90+
• Firefox 88+
• Safari 14+
• Edge 90+

Future Enhancements

Potential Improvements

1. Excel table paste support - Direct paste from spreadsheets
2. CSV import handling - File upload table conversion
3. Table editing capabilities - Inline cell editing
4. Export functionality - Download tables as CSV/Excel
5. Advanced styling options - Theme customization
6. Column sorting - Interactive table features

Extensibility

The pipeline architecture allows easy addition of new table formats by:

1. Adding new conversion functions
2. Inserting into the processing pipeline
3. Creating corresponding test cases

Version History

• 0.229.005 - Added comprehensive table support (Unicode, PSV, wrapped markdown)
• 0.229.004 - Initial Unicode table conversion
• 0.229.003 - Table preprocessing foundation

Related Documentation

• Feature implementation: chat-messages.js lines 150-300
• Test validation: functional_tests/test_comprehensive_table_support.py
• HTML examples: functional_tests/test_*_table_conversion.html