Ecoli Periodicity Detection Eval

ecoli-periodicity-detection-eval

Periodicity Detection of Outlier Sequences Using Constraint Based Pattern Tree with MAD — Archana N. et al. (2015) (arXiv:1507.01685, 2015)

What this evaluates

Evaluates the ability to detect periodic outlier patterns in protein sequence time-series data. It measures the statistical significance and reliability of discovered patterns compared to a baseline algorithm.

Datasets

• E.Coli — total ?; splits: test (-1)

Metrics

• Surprise score (primary) — range: [0, 1]
  • Evaluates detected periodic outlier patterns based on their statistical Surprise (Surp) and Confidence (Conf) values. Higher values indicate more significant and reliable patterns.
• Execution time — range: seconds
  • Wall-clock time required to process the sequence and generate patterns.

Input / output format

Input: E. coli protein sequence data provided as a time-series string.

Output: A list of detected periodic outlier patterns, each containing Count, Period, Pattern sequence, Start Position, End Position, Confidence (Conf), and Surprise (Surp) score.

Scoring recipe

def evaluate(predictions):
    n_patterns = len(predictions)
    if n_patterns == 0:
        return {'n_patterns': 0, 'avg_surprise': 0, 'avg_confidence': 0}
    avg_surprise = sum(p['Surp'] for p in predictions) / n_patterns
    avg_confidence = sum(p['Conf'] for p in predictions) / n_patterns
    return {'n_patterns': n_patterns, 'avg_surprise': avg_surprise, 'avg_confidence': avg_confidence}

Common pitfalls

• No ground-truth labels or standard train/val/test splits are provided; evaluation is purely comparative against a baseline.
• Accuracy is not measured via standard precision/recall/F1, but via the count and statistical scores (Surprise/Confidence) of discovered patterns.
• Time performance results are implementation-dependent and not normalized for hardware or algorithmic optimizations.

Evidence (verbatim from paper)

The parameters of the surprising patterns obtained when experimenting using both the algorithms are shown below in Table II and III respectively... Suffix tree with mean gives less accurate results when compared to consensus based-FP Tree with MAD i.e. more surprising patterns obtained in the proposed system.

Citation

@misc{archana2015periodicity,
  title={Periodicity Detection of Outlier Sequences Using Constraint Based Pattern Tree with MAD},
  author={Archana N. et al. (2015)},
  year={2015},
  note={arXiv:1507.01685}
}

• arXiv: 1507.01685