TiKV

 Distributed transactional key-value database, originally created to complement TiDB.

https://tikv.org/

Website | Documentation | Community Chat

  

TiKV is an open-source, distributed, and transactional key-value database. Unlike other traditional NoSQL systems, TiKV not only provides classical key-value APIs, but also transactional APIs with ACID compliance. Built in Rust and powered by Raft, TiKV was originally created to complement TiDB, a distributed HTAP database compatible with the MySQL protocol.

The design of TiKV ('Ti' stands for titanium) is inspired by some great distributed systems from Google, such as BigTable, Spanner, and Percolator, and some of the latest achievements in academia in recent years, such as the Raft consensus algorithm.

If you're interested in contributing to TiKV, or want to build it from source, see CONTRIBUTING.md.

TiKV is a graduated project of the Cloud Native Computing Foundation (CNCF). If you are an organization that wants to help shape the evolution of technologies that are container-packaged, dynamically-scheduled and microservices-oriented, consider joining the CNCF. For details about who's involved and how TiKV plays a role, read the CNCF announcement.

---

With the implementation of the Raft consensus algorithm in Rust and consensus state stored in RocksDB, TiKV guarantees data consistency. Placement Driver (PD), which is introduced to implement auto-sharding, enables automatic data migration. The transaction model is similar to Google's Percolator with some performance improvements. TiKV also provides snapshot isolation (SI), snapshot isolation with lock (SQL: SELECT ... FOR UPDATE), and externally consistent reads and writes in distributed transactions.

TiKV has the following key features:

• Geo-Replication

  TiKV uses Raft and the Placement Driver to support Geo-Replication.

• Horizontal scalability

  With PD and carefully designed Raft groups, TiKV excels in horizontal scalability and can easily scale to 100+ TBs of data.

• Consistent distributed transactions

  Similar to Google's Spanner, TiKV supports externally-consistent distributed transactions.

• Coprocessor support

  Similar to HBase, TiKV implements a coprocessor framework to support distributed computing.

• Cooperates with TiDB

  Thanks to the internal optimization, TiKV and TiDB can work together to be a compelling database solution with high horizontal scalability, externally-consistent transactions, support for RDBMS, and NoSQL design patterns.

Governance

See Governance.

Documentation

For instructions on deployment, configuration, and maintenance of TiKV,see TiKV documentation on our website. For more details on concepts and designs behind TiKV, see Deep Dive TiKV.

Note:

We have migrated our documentation from the TiKV's wiki page to the official website. The original Wiki page is discontinued. If you have any suggestions or issues regarding documentation, offer your feedback here.

TiKV adopters

You can view the list of TiKV Adopters.

from https://github.com/tikv/tikv

-----

Building and setting up a development workspace of tikv

TiKV is mostly written in Rust, but has components written in C++ (RocksDB, gRPC). We are currently using the Rust nightly toolchain. To provide consistency, we use linters and automated formatting tools.

Prerequisites

To build TiKV you'll need to at least have the following installed:

• git - Version control
• rustup - Rust installer and toolchain manager
• make - Build tool (run common workflows)
• cmake - Build tool (required for gRPC)
• awk - Pattern scanning/processing language
• C++ compiler - gcc 4.9+ (required for gRPC)

If you are targeting platforms other than x86_64 linux, you'll also need:

• llvm and clang - Used to generate bindings for different platforms and build native libraries (required for grpcio, rocksdb).

Getting the repository

git clone https://github.com/tikv/tikv.git
cd tikv
# Future instructions assume you are in this directory

Configuring your Rust toolchain

rustup is the official toolchain manager for Rust, similar to rvm or rbenv from the Ruby world.

TiKV is pinned to a version of Rust using a rust-toolchain file. rustup and cargo will automatically use this file. We also use the rustfmt and clippy components, to install those:

rustup component add rustfmt
rustup component add clippy

Building and testing

TiKV includes a Makefile that has common workflows and sets up a standard build environment. You can also use cargo, as you would in many other Rust projects. It can help to run a command in the same environment as the Makefile: this can avoid re-compilations due to environment changes. This is done by prefixing a command with scripts/env, for example: ./scripts/env cargo build

You can build TiKV:

make build

During interactive development, you may prefer using cargo check, which will parse, borrow check, and lint your code, but not actually compile it:

cargo check --all

It is particularly handy alongside cargo-watch, which runs a command each time you change a file.

cargo install cargo-watch
cargo watch -s "cargo check --all"

When you're ready to test out your changes, use the dev task. It will format your codebase, build with clippy enabled, and run tests. This should run without failure before you create a PR. Unfortunately, some tests will fail intermittently and others don't pass on all platforms. If you're unsure, just ask!

make dev

You can run the test suite alone, or just run a specific test:

# Run the full suite
make test
# Run a specific test
./scripts/test $TESTNAME -- --nocapture

TiKV follows the Rust community coding style. We use Rustfmt and Clippy to automatically format and lint our code. Using these tools is checked in our CI. These are as part of make dev, you can also run them alone:

# Run Rustfmt
cargo fmt
# Run Clippy (note that some lints are ignored, so `cargo clippy` will give many false positives)
make clippy

See the style doc and the API guidelines for details on the conventions.

Please follow this style to make TiKV easy to review, maintain, and develop.

Build issues

To reduce compilation time, TiKV builds do not include full debugging information by default — release and bench builds include no debuginfo; dev and test builds include line numbers only. The easiest way to enable debuginfo is to precede build commands with RUSTFLAGS=-Cdebuginfo=1 (for line numbers), or RUSTFLAGS=-Cdebuginfo=2 (for full debuginfo). For example,

RUSTFLAGS=-Cdebuginfo=2 make dev
RUSTFLAGS=-Cdebuginfo=2 cargo build

When building with make, cargo will automatically use pipelined compilation to increase the parallelism of the build. To turn on pipelining while using cargo directly, set CARGO_BUILD_PIPELINING=true.

Running TiKV

To run TiKV as an actual key-value store, you will need to run it as a cluster (a cluster can have just one node, which is useful for testing). You can do this on a single machine or on multiple machines. You need to use PD to manage the cluster (even if there is just one node on a single machine). Instructions are in our docs (if you build TiKV from source, then you don't need to download the binary).

Configuration

Read our configuration guide to learn about various configuration options. There is also a configuration template.

from https://github.com/tikv/tikv/blob/master/CONTRIBUTING.md

-----

https://github.com/pingcap/tidb-helper

https://github.com/pingcap/docs-cn

https://github.com/pingcap/tidb-ctl