Dhruba Borthakur 96e2c2c031 Geo spatial support.

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

2014-03-28 16:12:50 -07:00

Rocksdb Change Log

Unreleased

Removed arena.h from public header files.
By default, checksums are verified on every read from database
Added is_manual_compaction to CompactionFilter::Context
Added "virtual void WaitForJoin()" in class Env. Default operation is no-op.
Removed BackupEngine::DeleteBackupsNewerThan() function
Added new option -- verify_checksums_in_compaction
Chagned Options.prefix_extractor from raw pointer to shared_ptr (take ownership) Changed HashSkipListRepFactory and HashLinkListRepFactory constructor to not take SliceTransform object (use Options.prefix_extractor implicitly)
Added Env::GetThreadPoolQueueLen(), which returns the waiting queue length of thread pools
Added a command "checkconsistency" in ldb tool, which checks if file system state matches DB state (file existence and file sizes)
CompactionFilter::Context is now CompactionFilterContext. It is shared by CompactionFilter and CompactionFilterV2

If we find one truncated record at the end of the MANIFEST or WAL files, we will ignore it. We assume that writers of these records were interrupted and that we can safely ignore it.
Now compaction filter has a V2 interface. It buffers the kv-pairs sharing the same key prefix, process them in batches, and return the batched results back to DB.
Geo-spatial support for locations and radial-search.

Renamed StackableDB::GetRawDB() to StackableDB::GetBaseDB().
Renamed WriteBatch::Data() const std::string& Data() const.
Renamed class TableStats to TableProperties.
Deleted class PrefixHashRepFactory. Please use NewHashSkipListRepFactory() instead.
Supported multi-threaded EnableFileDeletions() and DisableFileDeletions().
Added DB::GetOptions().
Added DB::GetDbIdentity().

Added BackupableDB
Implemented TailingIterator, a special type of iterator that doesn't create a snapshot (can be used to read newly inserted data) and is optimized for doing sequential reads.
Added property block for table, which allows (1) a table to store its metadata and (2) end user to collect and store properties they are interested in.
Enabled caching index and filter block in block cache (turned off by default).
Supported error report when doing manual compaction.
Supported additional Linux platform flavors and Mac OS.
Put with SliceParts - Variant of Put() that gathers output like writev(2)
Bug fixes and code refactor for compatibility with upcoming Column Family feature.

Huge benchmark performance improvements by multiple efforts. For example, increase in readonly QPS from about 530k in 2.6 release to 1.1 million in 2.7 [1]
Speeding up a way RocksDB deleted obsolete files - no longer listing the whole directory under a lock -- decrease in p99
Use raw pointer instead of shared pointer for statistics: 5b825d -- huge increase in performance -- shared pointers are slow
Optimized locking for Get() -- 1fdb3f -- 1.5x QPS increase for some workloads
Cache speedup - e8d40c3
Implemented autovector, which allocates first N elements on stack. Most of vectors in RocksDB are small. Also, we never want to allocate heap objects while holding a mutex. -- c01676e4
Lots of efforts to move malloc, memcpy and IO outside of locks