Communication Multimodal Analysis
Overview
This Skill treats modality as evidence, not preprocessing debris. It defines what is
lost when image, sound, speech, motion, layout, or cross-modal relations are transformed
into model features.
Required start
- State the communication construct and whether it resides in one modality or in the
relation among modalities.
- Define sampling, unitization, temporal segmentation, and provenance with
references/multimodal-protocol.md.
- Separate extraction quality (OCR/ASR/detection) from substantive coding validity.
Method routing
- OCR and ASR recover text/speech but require error audits by language, accent, noise,
typography, and platform format;
- object/face/logo detection measures specified visual entities, not meanings;
- visual/audio embeddings support similarity or retrieval with reference cases;
- scene, shot, speaker, gesture, prosody, and temporal features require explicit
segmentation rules;
- early/late/joint multimodal fusion must match the theoretical cross-modal relation;
- LLM/VLM coding requires versioned prompts and a human reference set.
Validation
Validate unit boundaries, extraction error, annotation reliability, subgroup/model bias,
known cases, negative controls, cross-platform compression, and robustness across
models, thresholds, frames, and transcript variants. Preserve original media locators
and transformation provenance.
Output shape
Return the multimodal corpus contract, unit/segment definitions, feature or codebook
lineage, extraction and substantive validation results, bias/ethics review, robustness
plan, and claim boundaries.
Hard boundaries
- Detected objects, faces, emotions, or topics are measures, not ground truth.
- A transcript cannot support claims about visual rhetoric, timing, or prosody.
- Frame sampling can miss transitions and short-lived content.
- Model similarity does not establish shared meaning.
- Biometric or identity inference requires necessity, lawful basis, and heightened review.
1---2name: communication-multimodal-analysis3description: Design and audit image, video, audio, OCR, ASR, computer-vision, embedding, and multimodal communication analysis with explicit units and human validation. Use for 视觉传播/视频分析/音频分析/多模态/计算机视觉/OCR/ASR. Do not reduce visuals or speech to extracted text when form is part of the claim.4---56# Communication Multimodal Analysis78## Overview910This Skill treats modality as evidence, not preprocessing debris. It defines what is11lost when image, sound, speech, motion, layout, or cross-modal relations are transformed12into model features.1314## Required start15161. State the communication construct and whether it resides in one modality or in the17 relation among modalities.182. Define sampling, unitization, temporal segmentation, and provenance with19 [references/multimodal-protocol.md](references/multimodal-protocol.md).203. Separate extraction quality (OCR/ASR/detection) from substantive coding validity.2122## Method routing2324- OCR and ASR recover text/speech but require error audits by language, accent, noise,25 typography, and platform format;26- object/face/logo detection measures specified visual entities, not meanings;27- visual/audio embeddings support similarity or retrieval with reference cases;28- scene, shot, speaker, gesture, prosody, and temporal features require explicit29 segmentation rules;30- early/late/joint multimodal fusion must match the theoretical cross-modal relation;31- LLM/VLM coding requires versioned prompts and a human reference set.3233## Validation3435Validate unit boundaries, extraction error, annotation reliability, subgroup/model bias,36known cases, negative controls, cross-platform compression, and robustness across37models, thresholds, frames, and transcript variants. Preserve original media locators38and transformation provenance.3940## Output shape4142Return the multimodal corpus contract, unit/segment definitions, feature or codebook43lineage, extraction and substantive validation results, bias/ethics review, robustness44plan, and claim boundaries.4546## Hard boundaries4748- Detected objects, faces, emotions, or topics are measures, not ground truth.49- A transcript cannot support claims about visual rhetoric, timing, or prosody.50- Frame sampling can miss transitions and short-lived content.51- Model similarity does not establish shared meaning.52- Biometric or identity inference requires necessity, lawful basis, and heightened review.
haochengw372-hash