related-skills: cncf-aws-dynamodb, cncf-aws-ecr, cncf-aws-rds, cncf-aws-s3
TiKV in Cloud-Native Engineering
Category: Database
Status: Active
Stars: 10,000
Last Updated: 2026-04-22
Primary Language: Rust
Documentation: Distributed transactional key-value database inspired by Google Spanner
Purpose and Use Cases
TiKV is a core component of the cloud-native ecosystem, serving as Google Spanner
What Problem Does It Solve?
TiKV addresses the challenge of distributed transactional key-value storage with strong consistency. It provides horizontal scalability, ACID transactions, and cloud-native architecture.
When to Use This Project
Use TiKV when need distributed storage, require strong consistency, or manage large-scale data. Not ideal for simple deployments or when distributed database requirements, transactional workloads, or horizontal scaling needs.
Key Use Cases
- Distributed Database Storage
- Transaction Processing
- High Availability Storage
- Multi-Region Deployments
- Hybrid Cloud Storage
Architecture Design Patterns
Core Components
- PD: Placement Driver for cluster management
- TiKV Node: Storage node
- Raft: Consensus protocol
- Transaction: Transaction engine
- Region: Data partition unit
Component Interactions
- Client → TiKV: Client reads/writes to TiKV
- TiKV → PD: PD manages region distribution
- TiKV → TiKV: Raft replication between nodes
- PD → TiKV: PD schedules regions
Data Flow Patterns
- Write Flow: Client writes → Raft leader → Raft followers → Ack → Response
- Read Flow: Client reads → Raft leader → Response (or follower)
- Region Split: Region grows → Split → New regions
- Region Balance: PD schedules → Region moved → Balance restored
Design Principles
- Consistency: Strong consistency with Raft
- Scalability: Horizontal scalability
- Availability: High availability with replication
- Transaction Support: ACID transactions
Integration Approaches
Integration with Other CNCF Projects
- TiDB: SQL layer
- TiFlash: Columnar storage
- TiCDC: Change data capture
- PD: Placement Driver
API Patterns
- KV API: Key-value operations
- PD API: Cluster management API
- Raft API: Raft consensus API
- Transaction API: Transaction operations
Configuration Patterns
- TiKV Config: TiKV configuration
- PD Config: Placement Driver config
- Raft Config: Raft configuration
- Transaction Config: Transaction settings
Extension Mechanisms
- Custom Storage Engines: Add storage engines
- Custom PD Schedulers: Custom scheduling logic
- Custom Transaction Logic: Custom transaction handling
Common Pitfalls and How to Avoid Them
Misconfigurations
- Disk Space: Disk space exhaustion
- How to Avoid: Monitor disk space, configure disk quota, add nodes
- Replication Issues: Replication lag
- How to Avoid: Monitor replication, check network, tune settings
Performance Issues
- Raft Issues: Raft consensus problems
- How to Avoid: Monitor Raft health, check node availability
- PD Issues: PD cluster problems
- How to Avoid: Monitor PD health, scale PD cluster
Operational Challenges
- Memory Pressure: High memory usage
- How to Avoid: Configure memory limits, tune cache settings
- Upgrade Issues: Upgrade failures
- How to Avoid: Test upgrades, follow upgrade path
Security Pitfalls
Coding Practices
Idiomatic Configuration
- Transaction Design: Design efficient transactions
- Key Design: Design keys for even distribution
- Region Management: Manage region size
API Usage Patterns
- tikv-ctl: TiKV control utility
- KV API: Key-value operations
- PD API: PD management API
- gRPC: TiKV gRPC API
Observability Best Practices
- TiKV Metrics: Monitor TiKV performance
- PD Metrics: Monitor PD health
- Raft Metrics: Track Raft operations
Testing Strategies
- Integration Tests: Test TiKV functionality
- Failover Tests: Test cluster failover
- Performance Tests: Validate performance
Development Workflow
- Local Development: Use tikv-server locally
- Debug Commands: Check TiKV and PD logs
- Test Environment: Set up test cluster
- CI/CD Integration: Automate testing
- Monitoring Setup: Configure observability
- Documentation: Maintain documentation
Fundamentals
Essential Concepts
- Region: Data partition unit
- Raft Group: Replica group
- PD: Placement Driver
- Transaction: Transaction engine
- ** MVCC**: Multi-version concurrency control
- Lock: Lock mechanism
- Write Batch: Batch write operations
- Snapshot: Snapshot isolation
Terminology Glossary
- Region: Data partition
- Raft Group: Replica group
- PD: Placement Driver
- Transaction: Transaction engine
- MVCC: Multi-version concurrency control
Data Models and Types
- Region: Data partition
- Raft Log: Raft log entries
- Write Batch: Batch write data
- Transaction State: Transaction state
Lifecycle Management
- Write Flow: Client writes → Raft leader → Replicate → Response
- Read Flow: Client reads → Read from leader or follower
- Region Split: Region grows → Split → New regions created
- Region Balance: PD schedules → Region moved → Balance restored
State Management
- Region State: Available, splitting, or merging
- Replica State: Leader, follower, or learner
- Transaction State: Active, committed, or aborted
- PD State: Leader, follower, or offline
Scaling and Deployment Patterns
Horizontal Scaling
- Region Scaling: Split regions as data grows
- Node Scaling: Add TiKV nodes
- PD Scaling: Scale PD cluster
- Replica Scaling: Adjust replica count
High Availability
- Region HA: Multiple replicas per region
- Raft HA: Raft consensus with quorum
- PD HA: PD cluster with leader election
- Node HA: Distribute regions across nodes
Production Deployments
- Cluster Setup: Deploy TiKV cluster
- Network Configuration: Configure network
- Security Setup: Enable TLS, authentication
- Monitoring Setup: Configure metrics
- Logging Setup: Centralize logs
- Backup Strategy: Configure backups
- Resource Quotas: Set resource limits
- Performance Tuning: Optimize settings
Upgrade Strategies
- TiKV Upgrade: Upgrade TiKV nodes
- PD Upgrade: Upgrade PD cluster
- Rolling Upgrade: Rolling node upgrade
- Testing: Verify functionality
Resource Management
- CPU Resources: CPU limits
- Memory Resources: Memory limits
- Storage Resources: Storage configuration
- Network Resources: Network configuration
Additional Resources
- Official Documentation: https://tikv.org/docs/
- GitHub Repository: Check the project's official documentation for repository link
- CNCF Project Page: cncf.io/projects/cncf-tikv/
- Community: Check the official documentation for community channels
- Versioning: Refer to project's release notes for version-specific features
Troubleshooting
Common Issues
Deployment Failures
- Check pod logs for errors
- Verify configuration values
- Ensure network connectivity
Performance Issues
- Monitor resource usage
- Adjust resource limits
- Check for bottlenecks
Configuration Errors
- Validate YAML syntax
- Check required fields
- Verify environment-specific settings
Integration Problems
- Verify API compatibility
- Check dependency versions
- Review integration documentation
Getting Help
- Check official documentation
- Search GitHub issues
- Join community channels
- Review logs and metrics
Content generated automatically. Verify against official documentation before production use.
Examples
TiKV Cluster Configuration
# TiKV configuration for a production cluster
[tikv]
# Server configuration
addr = "192.168.1.101:20160"
advertise-addr = "192.168.1.101:20160"
status-addr = "192.168.1.101:20180"
# PD configuration
pd-endpoints = ["192.168.1.101:2379","192.168.1.102:2379","192.168.1.103:2379"]
# Storage configuration
storage.data-dir = "/data/tikv"
storage.engine-addr = "192.168.1.101:20160"
storage.block-cache.capacity = "8GB"
# Logging configuration
log-level = "info"
log-file = "/var/log/tikv/tikv.log"
# Coprocessor configuration
coprocessor.split-region-on-table = true
coprocessor.batch-split-limit = 10
coprocessor.region-max-size = "96MB"
coprocessor.region-split-size = "64MB"
TiKV with TiFlash Configuration
# TiKV configuration with TiFlash replication
[tikv]
addr = "192.168.1.101:20160"
advertise-addr = "192.168.1.101:20160"
status-addr = "192.168.1.101:20180"
pd-endpoints = ["192.168.1.101:2379"]
storage.data-dir = "/data/tikv"
storage.engine-addr = "192.168.1.101:20160"
[server]
engine-addr = "192.168.1.101:20160"
status-addr = "192.168.1.101:20180"
tidb-status-addr = "192.168.1.101:10080"
[raftstore]
apply-pool-size = 4
store-pool-size = 4
raft-entry-max-size = 8
raft-snapshot-count = 10000
[coprocessor]
region-max-keys = 100000
region-split-keys = 50000
TiKV Backup and Restore Configuration
# TiKV backup configuration for BR (Backup & Restore)
[br]
# Backup configuration
backup-concurrency = 16
checksum-concurrency = 4
send-credentials-to-tikv = true
send-stores-to-tikv = true
[restore]
# Restore configuration
restore-concurrency = 16
data-concurrency = 4
index-concurrency = 4
region-merge-concurrency = 4
[pd]
pd-endpoints = ["192.168.1.101:2379"]
[tikv]
# Ensure sufficient resources for backup/restore
storage.write-thread-num = 4
storage.backup-thread-num = 4
coprocessorThreadPoolSize = 4
When to Use
Use this skill when:
- Integrating a CNCF project into Kubernetes infrastructure — You need to configure, deploy, or troubleshoot a cloud-native tool within a cluster
- Designing cloud-native architecture — You are selecting and integrating CNCF tools to solve specific infrastructure challenges
- Resolving operational issues — A CNCF component is misbehaving, underperforming, or needs configuration changes
Core Workflow
Assess Requirements — Understand the use case, scale, integration needs, and existing infrastructure. Checkpoint: Document requirements, constraints, and success criteria.
Design Architecture — Plan component interactions, data flow, and deployment strategy using cloud-native best practices. Checkpoint: Verify the architecture addresses all requirements and follows CNCF conventions.
Implement & Configure — Create manifests, configurations, and deployment scripts. Include resource limits, health checks, and observability hooks. Checkpoint: Validate all YAML against schema and test in a staging environment.
Deploy & Monitor — Apply manifests to the cluster, verify component health, and confirm observability is working. Checkpoint: Confirm all pods/services are running, probes passing, and metrics/alerts configured.
Constraints
MUST DO
- Include at least one complete working YAML manifest example
- Note when content is auto-generated vs. manually verified
- Reference relevant CNCF project documentation
MUST NOT DO
- Deploy manifests without testing in a staging environment first
- Use deprecated API versions (e.g., apps/v1beta1)
- Omit resource limits and requests in Kubernetes manifests
1---2name: tikv3description: "TiKV in Distributed transactional key-value database inspired by Google" Spanner4license: MIT5---678910 related-skills: cncf-aws-dynamodb, cncf-aws-ecr, cncf-aws-rds, cncf-aws-s3111213# TiKV in Cloud-Native Engineering1415**Category:** Database 16**Status:** Active 17**Stars:** 10,000 18**Last Updated:** 2026-04-22 19**Primary Language:** Rust 20**Documentation:** [Distributed transactional key-value database inspired by Google Spanner](https://tikv.org/docs/) 2122---2324## Purpose and Use Cases2526TiKV is a core component of the cloud-native ecosystem, serving as Google Spanner2728### What Problem Does It Solve?2930TiKV addresses the challenge of distributed transactional key-value storage with strong consistency. It provides horizontal scalability, ACID transactions, and cloud-native architecture.3132### When to Use This Project3334Use TiKV when need distributed storage, require strong consistency, or manage large-scale data. Not ideal for simple deployments or when distributed database requirements, transactional workloads, or horizontal scaling needs.3536### Key Use Cases3738- Distributed Database Storage39- Transaction Processing40- High Availability Storage41- Multi-Region Deployments42- Hybrid Cloud Storage4344---4546## Architecture Design Patterns4748### Core Components4950- **PD**: Placement Driver for cluster management51- **TiKV Node**: Storage node52- **Raft**: Consensus protocol53- **Transaction**: Transaction engine54- **Region**: Data partition unit5556### Component Interactions57581. **Client → TiKV**: Client reads/writes to TiKV591. **TiKV → PD**: PD manages region distribution601. **TiKV → TiKV**: Raft replication between nodes611. **PD → TiKV**: PD schedules regions6263### Data Flow Patterns64651. **Write Flow**: Client writes → Raft leader → Raft followers → Ack → Response661. **Read Flow**: Client reads → Raft leader → Response (or follower)671. **Region Split**: Region grows → Split → New regions681. **Region Balance**: PD schedules → Region moved → Balance restored6970### Design Principles7172- **Consistency**: Strong consistency with Raft73- **Scalability**: Horizontal scalability74- **Availability**: High availability with replication75- **Transaction Support**: ACID transactions7677---7879## Integration Approaches8081### Integration with Other CNCF Projects8283- **TiDB**: SQL layer84- **TiFlash**: Columnar storage85- **TiCDC**: Change data capture86- **PD**: Placement Driver8788### API Patterns8990- **KV API**: Key-value operations91- **PD API**: Cluster management API92- **Raft API**: Raft consensus API93- **Transaction API**: Transaction operations9495### Configuration Patterns9697- **TiKV Config**: TiKV configuration98- **PD Config**: Placement Driver config99- **Raft Config**: Raft configuration100- **Transaction Config**: Transaction settings101102### Extension Mechanisms103104- **Custom Storage Engines**: Add storage engines105- **Custom PD Schedulers**: Custom scheduling logic106- **Custom Transaction Logic**: Custom transaction handling107108---109110## Common Pitfalls and How to Avoid Them111112### Misconfigurations113114- **Disk Space**: Disk space exhaustion115 - **How to Avoid**: Monitor disk space, configure disk quota, add nodes116- **Replication Issues**: Replication lag117 - **How to Avoid**: Monitor replication, check network, tune settings118119### Performance Issues120121- **Raft Issues**: Raft consensus problems122 - **How to Avoid**: Monitor Raft health, check node availability123- **PD Issues**: PD cluster problems124 - **How to Avoid**: Monitor PD health, scale PD cluster125126### Operational Challenges127128- **Memory Pressure**: High memory usage129 - **How to Avoid**: Configure memory limits, tune cache settings130- **Upgrade Issues**: Upgrade failures131 - **How to Avoid**: Test upgrades, follow upgrade path132133### Security Pitfalls134135136---137138## Coding Practices139140### Idiomatic Configuration141142- **Transaction Design**: Design efficient transactions143- **Key Design**: Design keys for even distribution144- **Region Management**: Manage region size145146### API Usage Patterns147148- **tikv-ctl**: TiKV control utility149- **KV API**: Key-value operations150- **PD API**: PD management API151- **gRPC**: TiKV gRPC API152153### Observability Best Practices154155- **TiKV Metrics**: Monitor TiKV performance156- **PD Metrics**: Monitor PD health157- **Raft Metrics**: Track Raft operations158159### Testing Strategies160161- **Integration Tests**: Test TiKV functionality162- **Failover Tests**: Test cluster failover163- **Performance Tests**: Validate performance164165### Development Workflow166167- **Local Development**: Use tikv-server locally168- **Debug Commands**: Check TiKV and PD logs169- **Test Environment**: Set up test cluster170- **CI/CD Integration**: Automate testing171- **Monitoring Setup**: Configure observability172- **Documentation**: Maintain documentation173174---175176## Fundamentals177178### Essential Concepts179180- **Region**: Data partition unit181- **Raft Group**: Replica group182- **PD**: Placement Driver183- **Transaction**: Transaction engine184- ** MVCC**: Multi-version concurrency control185- **Lock**: Lock mechanism186- **Write Batch**: Batch write operations187- **Snapshot**: Snapshot isolation188189### Terminology Glossary190191- **Region**: Data partition192- **Raft Group**: Replica group193- **PD**: Placement Driver194- **Transaction**: Transaction engine195- **MVCC**: Multi-version concurrency control196197### Data Models and Types198199- **Region**: Data partition200- **Raft Log**: Raft log entries201- **Write Batch**: Batch write data202- **Transaction State**: Transaction state203204### Lifecycle Management205206- **Write Flow**: Client writes → Raft leader → Replicate → Response207- **Read Flow**: Client reads → Read from leader or follower208- **Region Split**: Region grows → Split → New regions created209- **Region Balance**: PD schedules → Region moved → Balance restored210211### State Management212213- **Region State**: Available, splitting, or merging214- **Replica State**: Leader, follower, or learner215- **Transaction State**: Active, committed, or aborted216- **PD State**: Leader, follower, or offline217218---219220## Scaling and Deployment Patterns221222### Horizontal Scaling223224- **Region Scaling**: Split regions as data grows225- **Node Scaling**: Add TiKV nodes226- **PD Scaling**: Scale PD cluster227- **Replica Scaling**: Adjust replica count228229### High Availability230231- **Region HA**: Multiple replicas per region232- **Raft HA**: Raft consensus with quorum233- **PD HA**: PD cluster with leader election234- **Node HA**: Distribute regions across nodes235236### Production Deployments237238- **Cluster Setup**: Deploy TiKV cluster239- **Network Configuration**: Configure network240- **Security Setup**: Enable TLS, authentication241- **Monitoring Setup**: Configure metrics242- **Logging Setup**: Centralize logs243- **Backup Strategy**: Configure backups244- **Resource Quotas**: Set resource limits245- **Performance Tuning**: Optimize settings246247### Upgrade Strategies248249- **TiKV Upgrade**: Upgrade TiKV nodes250- **PD Upgrade**: Upgrade PD cluster251- **Rolling Upgrade**: Rolling node upgrade252- **Testing**: Verify functionality253254### Resource Management255256- **CPU Resources**: CPU limits257- **Memory Resources**: Memory limits258- **Storage Resources**: Storage configuration259- **Network Resources**: Network configuration260261---262263## Additional Resources264265- **Official Documentation:** https://tikv.org/docs/266- **GitHub Repository:** Check the project's official documentation for repository link267- **CNCF Project Page:** [cncf.io/projects/cncf-tikv/](https://www.cncf.io/projects/cncf-tikv/)268- **Community:** Check the official documentation for community channels269- **Versioning:** Refer to project's release notes for version-specific features270271---272273## Troubleshooting274275### Common Issues2762771. **Deployment Failures**278 - Check pod logs for errors279 - Verify configuration values280 - Ensure network connectivity2812822. **Performance Issues**283 - Monitor resource usage284 - Adjust resource limits285 - Check for bottlenecks2862873. **Configuration Errors**288 - Validate YAML syntax289 - Check required fields290 - Verify environment-specific settings2912924. **Integration Problems**293 - Verify API compatibility294 - Check dependency versions295 - Review integration documentation296297### Getting Help298299- Check official documentation300- Search GitHub issues301- Join community channels302- Review logs and metrics303*Content generated automatically. Verify against official documentation before production use.*304305## Examples306307### TiKV Cluster Configuration308309310```yaml311# TiKV configuration for a production cluster312[tikv]313# Server configuration314addr = "192.168.1.101:20160"315advertise-addr = "192.168.1.101:20160"316status-addr = "192.168.1.101:20180"317318# PD configuration319pd-endpoints = ["192.168.1.101:2379","192.168.1.102:2379","192.168.1.103:2379"]320321# Storage configuration322storage.data-dir = "/data/tikv"323storage.engine-addr = "192.168.1.101:20160"324storage.block-cache.capacity = "8GB"325326# Logging configuration327log-level = "info"328log-file = "/var/log/tikv/tikv.log"329330# Coprocessor configuration331coprocessor.split-region-on-table = true332coprocessor.batch-split-limit = 10333coprocessor.region-max-size = "96MB"334coprocessor.region-split-size = "64MB"335```336337### TiKV with TiFlash Configuration338339340```yaml341# TiKV configuration with TiFlash replication342[tikv]343addr = "192.168.1.101:20160"344advertise-addr = "192.168.1.101:20160"345status-addr = "192.168.1.101:20180"346347pd-endpoints = ["192.168.1.101:2379"]348349storage.data-dir = "/data/tikv"350storage.engine-addr = "192.168.1.101:20160"351352[server]353engine-addr = "192.168.1.101:20160"354status-addr = "192.168.1.101:20180"355tidb-status-addr = "192.168.1.101:10080"356357[raftstore]358apply-pool-size = 4359store-pool-size = 4360raft-entry-max-size = 8361raft-snapshot-count = 10000362363[coprocessor]364region-max-keys = 100000365region-split-keys = 50000366```367368### TiKV Backup and Restore Configuration369370371```yaml372# TiKV backup configuration for BR (Backup & Restore)373[br]374# Backup configuration375backup-concurrency = 16376checksum-concurrency = 4377send-credentials-to-tikv = true378send-stores-to-tikv = true379380[restore]381# Restore configuration382restore-concurrency = 16383data-concurrency = 4384index-concurrency = 4385region-merge-concurrency = 4386387[pd]388pd-endpoints = ["192.168.1.101:2379"]389390[tikv]391# Ensure sufficient resources for backup/restore392storage.write-thread-num = 4393storage.backup-thread-num = 4394coprocessorThreadPoolSize = 4395```396397---398399## When to Use400401Use this skill when:402403- **Integrating a CNCF project into Kubernetes infrastructure** — You need to configure, deploy, or troubleshoot a cloud-native tool within a cluster404- **Designing cloud-native architecture** — You are selecting and integrating CNCF tools to solve specific infrastructure challenges405- **Resolving operational issues** — A CNCF component is misbehaving, underperforming, or needs configuration changes406---407408## Core Workflow4094101. **Assess Requirements** — Understand the use case, scale, integration needs, and existing infrastructure. **Checkpoint:** Document requirements, constraints, and success criteria.4114122. **Design Architecture** — Plan component interactions, data flow, and deployment strategy using cloud-native best practices. **Checkpoint:** Verify the architecture addresses all requirements and follows CNCF conventions.4134143. **Implement & Configure** — Create manifests, configurations, and deployment scripts. Include resource limits, health checks, and observability hooks. **Checkpoint:** Validate all YAML against schema and test in a staging environment.4154164. **Deploy & Monitor** — Apply manifests to the cluster, verify component health, and confirm observability is working. **Checkpoint:** Confirm all pods/services are running, probes passing, and metrics/alerts configured.417418---419420## Constraints421422### MUST DO423- Include at least one complete working YAML manifest example424- Note when content is auto-generated vs. manually verified425- Reference relevant CNCF project documentation426427### MUST NOT DO428- Deploy manifests without testing in a staging environment first429- Use deprecated API versions (e.g., apps/v1beta1)430- Omit resource limits and requests in Kubernetes manifests