MarkItDown Testing
Quick Reference
Test File Layout
packages/<pkg>/tests/
├── __init__.py
├── _test_vectors.py # FileTestVector dataclass + test vector lists
├── test_module_vectors.py # Parametrized tests for convert_local/stream/uri
├── test_module_misc.py # Non-vector tests (LLM, exiftool, specific formats)
├── test_cli_vectors.py # CLI tests (stdout, file output, stdin)
├── test_cli_misc.py # CLI version flag, error handling
└── test_files/ # Test input files
├── test.docx
├── test.pdf
└── expected_outputs/ # Expected conversion outputs
FileTestVector
@dataclasses.dataclass(frozen=True, kw_only=True)
class FileTestVector:
filename: str # Relative to test_files/
mimetype: str | None # Expected MIME type
charset: str | None # Expected charset
url: str | None # Optional URL for remote tests
must_include: List[str] # Strings that MUST appear in output
must_not_include: List[str] # Strings that MUST NOT appear
Parametrized Test Pattern
import pytest
from ._test_vectors import MY_TEST_VECTORS
@pytest.mark.parametrize("test_vector", MY_TEST_VECTORS)
def test_convert_local(test_vector):
markitdown = MarkItDown()
result = markitdown.convert(
os.path.join(TEST_FILES_DIR, test_vector.filename)
)
for s in test_vector.must_include:
assert s in result.markdown
for s in test_vector.must_not_include:
assert s not in result.markdown
Plugin Test Pattern
def test_converter():
"""Test the converter directly."""
with open("test_files/test.rtf", "rb") as f:
converter = MyConverter()
result = converter.convert(
file_stream=f,
stream_info=StreamInfo(mimetype="text/rtf", extension=".rtf"),
)
assert "expected text" in result.text_content
def test_markitdown():
"""Test that MarkItDown loads the plugin."""
md = MarkItDown(enable_plugins=True)
result = md.convert("test_files/test.rtf")
assert "expected text" in result.text_content
Running Tests
# All tests in a package
cd packages/markitdown
hatch test
# Direct pytest
pytest
# Specific test file
pytest tests/test_module_vectors.py
# Specific test function
pytest tests/test_module_vectors.py::test_convert_local -k "test.docx"
# With coverage
hatch test -- --cov=markitdown --cov-report=term
# Type checking
hatch run types:check
Test Conventions
- Remote tests: Skip in CI via
os.environ.get("GITHUB_ACTIONS") - LLM tests: Skip without
OPENAI_API_KEY - Exiftool tests: Skip if
exiftoolnot on PATH - Test files: Go in
test_files/, expected outputs intest_files/expected_outputs/ - Vector naming:
GENERAL_TEST_VECTORS,DATA_URI_TEST_VECTORS, etc. - Import guard: Use
if __name__ == "__main__":pattern for relative imports in test files