CockroachDB - the scalable, survivable, strongly-consistent SQL database
Go JavaScript TypeScript Protocol Buffer Yacc C++ Other
Latest commit 9ead8a7 Apr 24, 2017 @benesch benesch committed on GitHub Merge pull request #15293 from benesch/restore-segfault
sqlccl: fix segfault with RESTORE
Permalink
Failed to load latest commit information.
.github Correct command for getting first lines of log Mar 31, 2017
build build: upgrade crosstool-ng and stop setting CC/CXX in builder image Apr 22, 2017
c-deps c-deps: patch rocksdb to avoid jemalloc bug Apr 20, 2017
cloud *: add github.com/google/pprof and update instructions Mar 15, 2017
docs Merge pull request #14968 from mjibson/rfc-syntax Apr 24, 2017
githooks githooks: reintroduce post-rewrite Dec 12, 2016
monitoring Add grafana dashboards. Fix typo Dec 14, 2016
pkg Merge pull request #15293 from benesch/restore-segfault Apr 24, 2017
scripts build: drop dependency on XZ Utils where possible Apr 21, 2017
vendor @ 5414638 build: minimize cgo's role in the build Apr 18, 2017
.editorconfig Add `.editorconfig` and update style notes. Nov 25, 2015
.gitattributes build,ui: add support for protobuf WKTs Mar 27, 2017
.gitignore build: minimize cgo's role in the build Apr 18, 2017
.gitmodules *: add vendor submodule Nov 23, 2016
.go-version Update .go-version Mar 10, 2017
.mailmap Populate a .mailmap to enhance the output of git blame/shortlog/etc. May 5, 2016
APL.txt cockroach: describe the two LICENSEs Jan 19, 2017
CONTRIBUTING.md build: drop dependency on XZ Utils where possible Apr 21, 2017
LICENSE ccl: move the CCL text to top-level LICENSE Jan 24, 2017
Makefile build: avoid non-standard 'find -not' construction Apr 21, 2017
README.md README improvements; fixes cockroachdb/docs#1189 Mar 16, 2017
STYLE.md *: Update STYLE.md to recommend fmt.Errorf for user errors Oct 21, 2016
glide.lock build: minimize cgo's role in the build Apr 18, 2017
glide.yaml build: minimize cgo's role in the build Apr 18, 2017
main.go main: fix comment typo Mar 17, 2017

README.md

CockroachDB

CockroachDB is a scalable, survivable, strongly-consistent SQL database.

TeamCity CI GoDoc Project Status Gitter

What is CockroachDB?

CockroachDB is a distributed SQL database built on a transactional and strongly-consistent key-value store. It scales horizontally; survives disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention; supports strongly-consistent ACID transactions; and provides a familiar SQL API for structuring, manipulating, and querying data.

For more details, see our FAQ and original design document.

Status

CockroachDB is currently in beta. See our 1.0 Roadmap and Issues for a list of features planned or in development.

Docs

For guidance on installation, development, deployment, and administration, see our User Documentation.

Quickstart

  1. Install CockroachDB.

  2. Start a local cluster and talk to it via the built-in SQL client.

  3. Secure the cluster with TLS encryption.

  4. Learn more about CockroachDB SQL.

  5. Explore core features, such as data replication and fault tolerance and recovery.

Client Drivers

CockroachDB supports the PostgreSQL wire protocol, so you can use any available PostgreSQL client drivers to connect from various languages.

Deployment

  • Manual Deployment - Steps to deploy a CockroachDB cluster manually on multiple machines.

  • Cloud Deployment - Guides for deploying CockroachDB on various cloud platforms.

  • Orchestration - Guides for running CockroachDB with popular open-source orchestration systems.

Need Help?

  • CockroachDB Forum - Ask questions, find answers, and help other users.

  • Join us on Gitter - This is the most immediate way to connect with CockroachDB engineers.

  • For filing bugs, suggesting improvements, or requesting new features, help us out by opening an issue.

Contributing

We're an open source project and welcome contributions.

  1. See CONTRIBUTING.md to get your local environment set up.

  2. Take a look at our open issues, in particular those with the helpwanted label.

  3. Review our style guide and follow our code reviews to learn about our style and conventions.

  4. Make your changes according to our code review workflow.

Design

This is an overview. For an in-depth discussion of the design and architecture, see the full design doc.

For another quick design overview, see the CockroachDB tech talk slides.

Design Goals

CockroachDB is a distributed SQL database built on top of a transactional and consistent key:value store.

The primary design goals are support for ACID transactions, horizontal scalability and survivability, hence the name.

It aims to tolerate disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention.

CockroachDB nodes (RoachNodes) are symmetric; a design goal is homogeneous deployment (one binary) with minimal configuration.

How it Works in a Nutshell

CockroachDB implements a single, monolithic sorted map from key to value where both keys and values are byte strings (not unicode).

The map is composed of one or more ranges and each range is backed by data stored in RocksDB (a variant of LevelDB), and is replicated to a total of three or more CockroachDB servers. This enables CockroachDB to scale linearly — theoretically up to 4 exabytes (4E) of logical data.

Ranges are defined by start and end keys. Ranges are merged and split to maintain total byte size within a globally configurable min/max size interval. Range sizes default to target 64M in order to facilitate quick splits and merges and to distribute load at hotspots within a key range. Range replicas are intended to be located in disparate datacenters for survivability (e.g. { US-East, US-West, Japan }, { Ireland, US-East, US-West} , { Ireland, US-East, US-West, Japan, Australia }).

Single mutations to ranges are mediated via an instance of a distributed consensus algorithm to ensure consistency. We’ve chosen to use the Raft consensus algorithm. All consensus state is also stored in RocksDB.

A single logical mutation may affect multiple key/value pairs. Logical mutations have ACID transactional semantics. If all keys affected by a logical mutation fall within the same range, atomicity and consistency are guaranteed by Raft; this is the fast commit path. Otherwise, a non-locking distributed commit protocol is employed between affected ranges.

CockroachDB provides snapshot isolation (SI) and serializable snapshot isolation (SSI) semantics, allowing externally consistent, lock-free reads and writes--both from an historical snapshot timestamp and from the current wall clock time. SI provides lock-free reads and writes but still allows write skew. SSI eliminates write skew, but introduces a performance hit in the case of a contentious system. SSI is the default isolation; clients must consciously decide to trade correctness for performance. CockroachDB implements a limited form of linearalizability, providing ordering for any observer or chain of observers.

Similar to Spanner directories, CockroachDB allows configuration of arbitrary zones of data. This allows replication factor, storage device type, and/or datacenter location to be chosen to optimize performance and/or availability. Unlike Spanner, zones are monolithic and don’t allow movement of fine grained data on the level of entity groups.

Comparison with Other Databases

To see how key features of CockroachDB stack up against other databases, visit the CockroachDB in Comparison page on our website.

See Also