SkipClass

SkipClass Skill

Purpose

Convert course recordings into structured English lecture notes by running the provided scripts (video -> audio -> transcript), matching transcripts to slide PDFs, and summarizing the content into a fixed note template that highlights key points, notifications, TODOs, and attendance checks.

Use this skill to operationalize a repeatable workflow for course note generation in this repository.

Core Workflow (High Level)

1. Verify the workspace layout and required data folders.
2. Place lecture videos and slide PDFs in the correct folders.
3. Run the scripts to produce audio and transcript files.
4. Match transcripts to slides for context.
5. Generate structured notes using transcript + slide context.
6. Record artifacts in data/notes/ for traceability.

Repository Layout and Data Folders

Maintain the following folder structure under data/:

• data/video/ - Raw lecture recordings (mp4/mov/etc.)
• data/audio/ - Extracted audio files (wav)
• data/transcripts/ - Text transcripts from audio
• data/class_slides/ - Lecture slide PDFs
• data/notes/ - Final lecture notes and slide matches

Create missing folders before running scripts. Keep file names consistent across stages (video -> audio -> transcript) to simplify matching.

Scripts and Their Roles

• scripts/video2audio.py - Convert videos to audio with optional silence removal.
• scripts/audio2text.py - Transcribe audio using faster-whisper; supports chunking.
• scripts/find_matched_slides.py - Retrieve top-matching slide PDFs for a transcript.

Step-by-Step Workflow

1) Inspect and Prepare the Workspace

Check that data/ exists and has required subfolders. Create any missing folders.

Confirm that dependencies are available:

• ffmpeg and ffprobe on PATH.
• Python packages: faster-whisper, torch (if using Silero VAD), pymupdf, sentence-transformers.

2) Place Input Files

• Put lecture videos in data/video/.
• Put lecture slides (PDFs) in data/class_slides/.
• If using a pre-existing transcript, place it in data/transcripts/ (or data/rawtexts/ for legacy data).

Prefer consistent naming, e.g. week2_lecture.mp4 -> week2_lecture.wav -> week2_lecture.txt.

3) Run Video -> Audio

Convert lecture videos to audio files:

uv run scripts/video2audio.py \
  --input-dir data/video \
  --output-dir data/audio \
  --silence-backend silero

4) Run Audio -> Text

Transcribe audio into text:

uv run scripts/audio2text.py \
  --input-dir data/audio \
  --output-dir data/transcripts \
  --model base \
  --chunk-minutes 30 \
  --chunk-overlap-minutes 0.2 \
  --parallel-workers 2 \
  --language en \
  --device cpu \
  --compute-type int8

5) Verify Transcript Quality

Open the generated transcript in data/transcripts/ and scan for:

• Missing segments (check for long gaps or repeated phrases).
• Poor language detection (wrong language).
• Excessive silence removal (too short transcript).

If needed, re-run with adjusted parameters:

• Increase --min-silence-duration or lower --silence-threshold-db.
• Change --model to a larger Whisper model.
• Increase --chunk-minutes for long lectures.

6) Match Slides to Transcript

Use the slide matching script:

uv run scripts/find_matched_slides.py \
  --mode hybrid \
  --hybrid-weight 0.7 \
  --transcript data/transcripts/week2_lecture.txt \
  --slides-dir data/class_slides \
  --top-k 5

Record the results in data/notes/ for reference and use the top results as context when summarizing.

7) Generate Lecture Notes

Combine the transcript and the most relevant slides into a concise English note that follows the structure and tone of example_note.md. Keep the output short, factual, and actionable. Use markdown headings and bullet points where appropriate.

Required Output Structure

1. Lesson Recap

   • Primarily summarize the slides content.
   • Organize by sections/topics as in the slides (use numbered sections or subheadings if helpful).
   • Include key points, definitions, examples, and important formulas/processes from slides.

2. Key Takeaways By Teacher

   • Synthesize Lesson Recap with the teacher’s spoken emphasis from the transcript.
   • Extract higher-level insights, rationale, or cautions the teacher stressed.
   • Avoid repeating the Lesson Recap verbatim; focus on distilled, teacher-driven emphasis.

3. Important Notification

   • Exams/quiz arrangements.
   • Assignment details and deadlines (include explicit dates).
   • Project milestones or schedule changes.

4. TODO

   • Direct instructions for a student who skipped class.
   • Required readings, problem sets, or code tasks.
   • Mention how to verify understanding (e.g. "work through slide X").

5. Attendance Check

   • State whether attendance was checked.
   • If unknown, state "Not mentioned in recording."

8) Save the Final Output

Write the final notes to data/notes/<lecture_name>_notes.md. Optionally store slide matches to data/notes/<lecture_name>_slides.txt.

Notes Quality Rules

• Keep each section concise and scannable.
• Prefer concrete terms over generic summaries.
• Avoid fabricating deadlines or announcements; if not found, state "Not mentioned."
• Convert relative time phrases into absolute dates when possible.

Troubleshooting

• ffmpeg/ffprobe not found: Install ffmpeg and ensure it is on PATH.
• faster-whisper import error: Install faster-whisper and its dependencies.
• Slide matching slow: Reduce PDF count or use --mode keyword.
• Slide matching inaccurate: Use --mode hybrid and tune --hybrid-weight.

Additional Resources

Scripts

• scripts/video2audio.py - Video to audio conversion and silence removal.
• scripts/audio2text.py - Faster-whisper transcription with chunking.
• scripts/find_matched_slides.py - Slide matching with vector/keyword/hybrid retrieval.