irahardianto Data Serialization and Interchange Principles
Validate at System Boundaries
All data entering system must be validated:
- API requests, file uploads, message queue messages
- Validate type, format, range, required fields
- Fail fast on invalid data (don't process partially valid data)
- Return clear validation errors to client
Handle Encoding Explicitly
Default to UTF-8:
- UTF-8 for all text data (API responses, file contents, database strings)
- Specify encoding explicitly when reading/writing files
- Handle encoding errors gracefully (replacement characters or error)
Encoding errors:
- Invalid UTF-8 sequences (malformed bytes)
- Mixing encodings (reading UTF-8 as ISO-8859-1)
- Always validate and normalize encoding
Serialization Format Selection
JSON:
- Human-readable, widely supported, language-agnostic
- Good for: APIs, configuration files, web applications
- Limitations: No binary support, larger size than binary formats
Protocol Buffers:
- Compact binary format, fast serialization/deserialization
- Schema evolution (backward/forward compatibility)
- Good for: Internal microservices, high-throughput systems
- Limitations: Not human-readable, requires schema definition
MessagePack:
- Binary JSON-like format, faster and more compact than JSON
- Good for: Internal APIs, when JSON too slow but readability still desired
- Limitations: Less widely supported than JSON
XML:
- Verbose, legacy systems, document-oriented
- Good for: Enterprise systems, SOAP APIs, RSS/Atom feeds
- Limitations: Verbosity, complexity, security issues (XXE attacks)
YAML:
- Human-friendly, good for configuration files
- Good for: Config files, Infrastructure as Code (Kubernetes, CI/CD)
- Limitations: Complex parsing, performance, security issues (arbitrary code execution)
Security Considerations
Validate before deserialization:
- Prevent deserialization attacks (arbitrary code execution)
- Set size limits on payloads (prevent memory exhaustion)
- Whitelist allowed types/classes for deserialization
Disable dangerous features:
- XML: Disable external entity processing (XXE prevention)
- YAML: Disable unsafe constructors
- Python pickle: Never deserialize untrusted data
Related Principles
- Error Handling Principles @.claude/rules/error-handling-principles.md
- Security Mandate @.claude/rules/security-mandate.md
- Security Principles @.claude/rules/security-principles.md
- API Design Principles @.claude/skills/api-design-principles/SKILL.md