Commit Graph

17 Commits

Author SHA1 Message Date
Sagar Vemuri
eb912a927e Remove disableDataSync option
Summary:
Remove disableDataSync, and another similarly named disable_data_sync options.
This is being done to simplify options, and also because the performance gains of this feature can be achieved by other methods.
Closes https://github.com/facebook/rocksdb/pull/1859

Differential Revision: D4541292

Pulled By: sagar0

fbshipit-source-id: 5b3a6ca
2017-02-13 11:09:13 -08:00
Mark Callaghan
4041903ecd Enhance db_bench write rate limit
Summary:
1) changes tools/{benchmark,run_flash_bench}.sh to optionally use the write rate limit
2) removes code for --writes_per_second and switches the 'background' write rate limit
to use --benchmark_write_rate_limit

Replaces https://reviews.facebook.net/D49113

Task ID: #9555881

Blame Rev:

Test Plan:
tools/run_flash_bench.sh

Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D52485
2016-01-04 12:01:27 -08:00
Igor Canadi
8de151bb99 Add db_bench with lots of column families to regression tests
Summary:
That way we can see when this graph goes up and be happy.

Couple of changes:
1. title
2. fix db_bench to delete column families before deleting the DB. this was asserting when compiled in debug mode
3. don't sync manifest when disableDataSync. We discussed this offline. I can move it to separate diff if you'd like

Test Plan: ran it

Reviewers: sdong, yhchiang, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D22815
2014-09-05 14:20:18 -07:00
Igor Canadi
d4a8423334 Remove seek compaction
Summary:
As discussed in our internal group, we don't get much use of seek compaction at the moment, while it's making code more complicated and slower in some cases.

This diff removes seek compaction and (hopefully) all code that was introduced to support seek compaction.

There is one test case that relied on didIO information. I'll try to find another way to implement it.

Test Plan: make check

Reviewers: sdong, haobo, yhchiang, ljin, dhruba

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19161
2014-06-20 10:23:02 +02:00
Lei Jin
578cf84ddf give correct metric name for grep in regression script
Summary: as title

Test Plan: jenkin

Reviewers: igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19005
2014-06-10 16:32:09 -07:00
Lei Jin
c4b3817d7c fix regression test
Summary: as title

Test Plan: push

Reviewers: igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18999
2014-06-10 11:24:24 -07:00
Lei Jin
e2d3101cf1 collect metrics for in memory workload get/seek
Summary:
collect in-memory workload get/seek metrics so that we can alert on
regression

Test Plan: ran locally

Reviewers: igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18969
2014-06-10 09:59:16 -07:00
Igor Canadi
457bae6911 Fix regression test
Summary:
388d2054c7
added extra line to db_bench output, breaking regression tests. This diff makes it more robust and fixes the issue

Test Plan: ran it

Reviewers: ljin, sdong

Reviewed By: sdong

Subscribers: sdong, leveldb

Differential Revision: https://reviews.facebook.net/D18897
2014-06-04 09:59:44 -07:00
Igor Canadi
1560bb913e Readrandom with tailing iterator
Summary:
Added an option for readrandom benchmark to run with tailing iterator instead of Get. Benefit of tailing iterator is that it doesn't require locking DB mutex on access.

I also have some results when running on my machine. The results highly depend on number of cache shards. With our current benchmark setting of 4 table cache shards and 6 block cache shards, I don't see much improvements of using tailing iterator. In that case, we're probably seeing cache mutex contention.

Here are the results for different number of shards

    cache shards       tailing iterator        get
       6                      1.38M           1.16M
      10                      1.58M           1.15M

As soon as we get rid of cache mutex contention, we're seeing big improvements in using tailing iterator vs. ordinary get.

Test Plan: ran regression test

Reviewers: dhruba, haobo, ljin, kailiu, sding

Reviewed By: haobo

CC: tnovak

Differential Revision: https://reviews.facebook.net/D15867
2014-02-07 09:47:47 -08:00
Igor Canadi
d0406675c2 readwhilewriting benchmark
Summary:
Added readwhilewriting benchmark to our regression tests.
Changed block cache shards from 16 to 64, as Mark found that cache mutex contention is a big bottleneck.

Test Plan: Ran it.

Reviewers: dhruba, haobo, MarkCallaghan, xjin

Reviewed By: MarkCallaghan

CC: leveldb

Differential Revision: https://reviews.facebook.net/D15075
2014-01-08 17:44:58 -08:00
Igor Canadi
f5f5c645a8 Add readrandom with both memtable and sst regression test
Summary: @MarkCallaghan's tests indicate that performance with 8k rows in memtable is much worse than empty memtable. I wanted to add a regression tests that measures this effect, so we could optimize it. However, current config shows 634461 QPS on my devbox. Mark, any idea why this is so much faster than your measurements?

Test Plan: Ran the regression test.

Reviewers: MarkCallaghan, dhruba, haobo

Reviewed By: MarkCallaghan

CC: leveldb, MarkCallaghan

Differential Revision: https://reviews.facebook.net/D14511
2013-12-11 13:51:20 -08:00
Igor Canadi
26bc40a89a Fixing git branch detection in Jenkins
Branch detection did not work in Jenkins. I realized that it set
GIT_BRANCH env variable to point to the current branch, so let's try
using this for branch detection.
2013-12-09 10:36:39 -08:00
Igor Canadi
07c8448845 Enable regression tests to be run on other branches
Summary: When running regression tests on other branches, this will push values to entity rocksdb_build.$git_branch

Test Plan: Ran regression test on regression branch, observed values send to ODS in entity rocksdb_build.regression

Reviewers: kailiu

Reviewed By: kailiu

CC: leveldb

Differential Revision: https://reviews.facebook.net/D14493
2013-12-06 16:22:38 -08:00
Igor Canadi
fe754fe7e3 Readrandom with small block cache
Summary: Added readrandom benchmark with 300MB block cache, while database has 1GB of data

Test Plan: Ran it

Reviewers: dhruba, MarkCallaghan

Reviewed By: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D14373
2013-11-27 13:33:14 -08:00
Igor Canadi
5ebc6b0f0b [rocksdb] Regression tests
Summary:
* Fixed regression test params by @dhruba's suggestion
* Added p50, p75 and p99 to regression metrics

Test Plan: build_tools/build_regression_test.sh

Reviewers: dhruba, emayanke

Reviewed By: dhruba

CC: leveldb, dhruba, reconnect.grayhat

Differential Revision: https://reviews.facebook.net/D14355
2013-11-26 16:27:31 -08:00
Igor Canadi
e37221f56b Memtable regression test
Summary: The old regression tests didn't cover memtable part at all. This is an atempt to also measure memtable performance in regression tests.

Test Plan: Ran regression_build_test.sh

Reviewers: dhruba, haobo, kailiu, emayanke

Reviewed By: dhruba

CC: leveldb, reconnect.grayhat

Differential Revision: https://reviews.facebook.net/D14325
2013-11-25 15:34:23 -08:00
Kai Liu
457dcc605a Clean up the Makefile and the build scripts
Summary: As Aaron suggested, there are quite some problems with our Makefile and scripts. So in this diff I did some cleanup for them and revise some part of the scripts/makefile to help people better understand some mysterious parts.

Test Plan:
Ran make in several modes;
Ran the updated scripts.

Reviewers: dhruba, emayanke, akushner

Differential Revision: https://reviews.facebook.net/D12285
2013-08-15 12:59:45 -07:00