andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
04b2c16f9b
rocksdb/file
History
Akanksha Mahajan 04b2c16f9b Fix bug in rocksdb internal automatic prefetching (#9234) 
Summary:
After introducing adaptive_readahead, the original flow got
broken. Readahead size was set to 0 because of which rocksdb wasn't be
able to do automatic prefetching which it enables after seeing
sequential reads. This PR fixes it.

----------------------------------------------------------------------------------------------------

Before this patch:
b_bench -use_existing_db=true -db=/tmp/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_size=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
RocksDB:    version 6.27
Date:       Tue Nov 30 11:56:50 2021
CPU:        24 * Intel Core Processor (Broadwell)
CPUCache:   16384 KB
Keys:       32 bytes each (+ 0 bytes user-defined timestamp)
Values:     512 bytes each (256 bytes after compression)
Entries:    5000000
Prefix:    0 bytes
Keys per prefix:    0
RawSize:    2594.0 MB (estimated)
FileSize:   1373.3 MB (estimated)
Write rate: 0 bytes/second
Read rate: 0 ops/second
Compression: Snappy
Compression sampling rate: 0
Memtablerep: SkipListFactory
Perf Level: 1
WARNING: Assertions are enabled; benchmarks unnecessarily slow
------------------------------------------------
DB path: [/tmp/prefix_scan]

seekrandom   : 5356367.174 micros/op 0 ops/sec;   29.4 MB/s (23 of 23 found)

----------------------------------------------------------------------------------------------------

After the patch:
./db_bench -use_existing_db=true -db=/tmp/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_size=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
RocksDB:    version 6.27
Date:       Tue Nov 30 14:38:33 2021
CPU:        24 * Intel Core Processor (Broadwell)
CPUCache:   16384 KB
Keys:       32 bytes each (+ 0 bytes user-defined timestamp)
Values:     512 bytes each (256 bytes after compression)
Entries:    5000000
Prefix:    0 bytes
Keys per prefix:    0
RawSize:    2594.0 MB (estimated)
FileSize:   1373.3 MB (estimated)
Write rate: 0 bytes/second
Read rate: 0 ops/second
Compression: Snappy
Compression sampling rate: 0
Memtablerep: SkipListFactory
Perf Level: 1
WARNING: Assertions are enabled; benchmarks unnecessarily slow
------------------------------------------------
DB path: [/tmp/prefix_scan]
seekrandom   :  456504.277 micros/op 2 ops/sec;  359.8 MB/s (264 of 264 found)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9234

Test Plan:
Ran ./db_bench -db=/data/mysql/rocksdb/prefix_scan
-benchmarks="fillseq" -key_size=32 -value_size=512 -num=5000000 -use_d
irect_io_for_flush_and_compaction=true -target_file_size_base=16777216
and then ./db_bench -use_existing_db=true
-db=/data/mysql/rocksdb/prefix_scan -benchmarks="seekrandom"
-key_size=32 -value_siz
e=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680
-duration=120 -ops_between_duration_checks=1
 and compared the results.

Reviewed By: anand1976

Differential Revision: D32743965

Pulled By: akankshamahajan15

fbshipit-source-id: b950fba68c91963b7deb5c20acdf471bc60251f5
2021-11-30 22:53:10 -08:00
..
delete_scheduler_test.cc Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
delete_scheduler.cc Skip directory fsync for filesystem btrfs (#8903) 2021-11-03 12:21:27 -07:00
delete_scheduler.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
file_prefetch_buffer.cc Fix bug in rocksdb internal automatic prefetching (#9234) 2021-11-30 22:53:10 -08:00
file_prefetch_buffer.h Fix bug in rocksdb internal automatic prefetching (#9234) 2021-11-30 22:53:10 -08:00
file_util.cc Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
file_util.h Allow WAL dir to change with db dir (#8582) 2021-07-30 12:16:44 -07:00
filename.cc Skip directory fsync for filesystem btrfs (#8903) 2021-11-03 12:21:27 -07:00
filename.h Add (Live)FileStorageInfo API (#8968) 2021-10-16 10:04:32 -07:00
line_file_reader.cc Replace Status with IOStatus in the backupable_db (#8820) 2021-09-15 15:09:48 -07:00
line_file_reader.h Replace Status with IOStatus in the backupable_db (#8820) 2021-09-15 15:09:48 -07:00
prefetch_test.cc Fix bug in rocksdb internal automatic prefetching (#9234) 2021-11-30 22:53:10 -08:00
random_access_file_reader_test.cc use the pointer directly (#8095) 2021-03-26 21:31:16 -07:00
random_access_file_reader.cc Add listener API that notifies on IOError (#9177) 2021-11-18 17:11:19 -08:00
random_access_file_reader.h Add listener API that notifies on IOError (#9177) 2021-11-18 17:11:19 -08:00
read_write_util.cc Move old files to warm tier in FIFO compactions (#8310) 2021-08-09 12:51:14 -07:00
read_write_util.h Refactor: add LineFileReader and Status::MustCheck (#8026) 2021-03-09 20:12:38 -08:00
readahead_file_info.h Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
readahead_raf.cc Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
readahead_raf.h Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
sequence_file_reader.cc Add file operation callbacks to SequentialFileReader (#8982) 2021-10-05 10:51:59 -07:00
sequence_file_reader.h Add listener API that notifies on IOError (#9177) 2021-11-18 17:11:19 -08:00
sst_file_manager_impl.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
sst_file_manager_impl.h Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
writable_file_writer.cc Add listener API that notifies on IOError (#9177) 2021-11-18 17:11:19 -08:00
writable_file_writer.h Add listener API that notifies on IOError (#9177) 2021-11-18 17:11:19 -08:00