Commit Graph

3403 Commits

Author SHA1 Message Date
Igor Canadi
216a9e16f4 Fix compile
Summary: I was pretty sure I compiled this before landing, sorry :/

Test Plan: compiles

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34173
2015-02-27 14:21:16 -08:00
Igor Canadi
b9ff6b050d Fix a bug in ReadOnlyBackupEngine
Summary:
This diff fixes a bug introduced by D28521. Read-only backup engine can delete a backup that is later than the latest -- we never check the condition.

I also added a bunch of logging that will help with debugging cases like this in the future.

See more discussion at t6218248.

Test Plan: Added a unit test that was failing before the change. Also, see new LOG file contents: https://phabricator.fb.com/P19738984

Reviewers: benj, sanketh, sumeet, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33897
2015-02-27 14:03:56 -08:00
Islam AbdelRahman
afa8156af9 adding stdlib to fbson
Summary: adding stdlib to fbson to fix FreeBSD compilation

Test Plan: 4cfccf14f4

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34143
2015-02-27 10:27:14 -08:00
Igor Canadi
f9c14a42e8 Fix compile on Mac 2015-02-27 09:31:29 -08:00
Igor Canadi
a9f0e2b5b6 Fix compile
Summary: as title, we have unused variables. this is a short-term solution

Test Plan: compiles

Reviewers: IslamAbdelRahman, sdong, rven

Reviewed By: rven

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34125
2015-02-26 16:56:39 -08:00
Sameet Agarwal
e7c434c364 Add columnfamily option optimize_filters_for_hits to optimize for key hits only
Summary:
    Summary:
    Added a new option to ColumnFamllyOptions  - optimize_filters_for_hits. This option can be used in the case where most
    accesses to the store are key hits and we dont need to optimize performance for key misses.
    This is useful when you have a very large database and most of your lookups succeed.  The option allows the store to
     not store and use filters in the last level (the largest level which contains data). These filters can take a large amount of
     space for large databases (in memory and on-disk). For the last level, these filters are only useful for key misses and not
     for key hits. If we are not optimizing for key misses, we can choose to not store these filters for that level.

    This option is only provided for BlockBasedTable. We skip the filters when we are compacting

Test Plan:
1. Modified db_test toalso run tests with an additonal option (skip_filters_on_last_level)
 2. Added another unit test to db_test which specifically tests that filters are being skipped

Reviewers: rven, igor, sdong

Reviewed By: sdong

Subscribers: lgalanis, yoshinorim, MarkCallaghan, rven, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33717
2015-02-26 16:25:56 -08:00
Islam AbdelRahman
ba9d1737a8 RocksDB on FreeBSD support
Summary:
This patch will update the Makefile and source code so that we can build RocksDB successfully on FreeBSD 10 and 11 (64-bit and 32-bit)
I have also encountered some problems when running tests on FreeBSD, I will try to fix them individually in different diffs

Notes:

  - FreeBSD uses clang as it's default compiler (http://lists.freebsd.org/pipermail/freebsd-current/2012-September/036480.html)
  - GNU C++ compiler have C++ 11 problems on FreeBSD (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193528)
  - make is not gmake on FreeBSD (http://www.khmere.com/freebsd_book/html/ch01.html)

Test Plan:
Using VMWare Fusion Create 4 VM machines (FreeBSD 11 64-bit, FreeBSD 11 32-bit, FreeBSD 10 64-bit, FreeBSD 10 32-bit)

  - pkg install git gmake gflags archivers/snappy
  - git clone https://github.com/facebook/rocksdb.git
  - apply this patch
  - setenv CXX c++
  - setenv CPATH /usr/local/include/
  - setenv LIBRARY_PATH  /usr/local/lib/
  - gmake db_bench
  - make sure compilation is successful and db_bench is running
  - gmake all
  - make sure compilation is successful

Reviewers: sdong, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33891
2015-02-26 15:19:17 -08:00
Igor Sugak
4ba119df5c rocksdb: Update path to dev clang in fbcode
Summary: Current clang path in fbcode points to clang 3.5. This diff updates clang path to clang 3.7.

Test Plan:
Make sure clang 3.7 is in use and all tests are passing.
```lang=bash
% USE_CLANG make check
```

Reviewers: meyering, sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34119
2015-02-26 15:12:08 -08:00
Venkatesh Radhakrishnan
8984e5f848 Fix race in sync point.
Summary:
The LoadDependency function does not take a lock when it runs
and it could be modifying data structures while other threads are
accessing it.

Test Plan: Run TSAN.

Reviewers: igor, sdong

Reviewed By: igor, sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34095
2015-02-26 15:11:50 -08:00
Igor Sugak
03b432d4b8 rocksdb: Fix uninitialized use error
Summary:
When using latest clang (3.6 or 3.7/trunck) rocksdb is failing with many errors. Some errors are uninitialized use errors.

```
...
  CC       db/log_test.o
util/ldb_cmd.cc:394:16: error: base class 'rocksdb::LDBCommand' is uninitialized when used here to access 'rocksdb::LDBCommand::BuildCmdLineOptions' [-Werror,-Wuninitialized]
               BuildCmdLineOptions({ARG_FROM, ARG_TO, ARG_HEX, ARG_KEY_HEX,
               ^
...
```

```lang=c++
CompactorCommand::CompactorCommand(const vector<string>& params,
      const map<string, string>& options, const vector<string>& flags) :
    LDBCommand(options, flags, false,
               BuildCmdLineOptions({ARG_FROM, ARG_TO, ARG_HEX, ARG_KEY_HEX,
                                    ARG_VALUE_HEX, ARG_TTL})),
    null_from_(true), null_to_(true) {
. . .
}
```
For the fourth parameter of the base constructor (`LDBCommand`) we call `BuildCmdLineOptions`, which is a private non-static method of `LDBCommand` base class.

This diff adds missing `static` keyword for `LDBCommand::BuildCmdLineOptions` method.

Test Plan:
Build with trunk clang and make sure all tests are passing.
```lang=bash
% # Have trunk clang present in path.
% ROCKSDB_NO_FBCODE=1 CC=clang CXX=clang++ make check
``

Reviewers: meyering, sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34083
2015-02-26 14:19:51 -08:00
Adam Retter
ccef2a7660 Merge pull request #518 from fyrz/RocksJava-Native-Library-Loader-Fix
[RocksJava] RocksJava Testcases
2015-02-26 21:47:13 +00:00
fyrz
9fcf1a7b00 [RocksJava] RocksJava Testcases
- NativeLibraryLoader Test
- EnvironmentTest Bugfix
2015-02-26 22:35:26 +01:00
stash93
03bbf718cb Return fbson
Summary: mac compile is fixed in fbson, so it can be returned back from 7ce1b2c

Test Plan:
make all check
make valgrind_check

Reviewers: golovachalexander, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33855
2015-02-27 00:04:14 +03:00
Igor Sugak
62247ffa3b rocksdb: Add missing override
Summary:
When using latest clang (3.6 or 3.7/trunck) rocksdb is failing with many errors. Almost all of them are missing override errors. This diff adds missing override keyword. No manual changes.

Prerequisites: bear and clang 3.5 build with extra tools

```lang=bash
% USE_CLANG=1 bear make all # generate a compilation database http://clang.llvm.org/docs/JSONCompilationDatabase.html
% clang-modernize -p . -include . -add-override
% make format
```

Test Plan:
Make sure all tests are passing.
```lang=bash
% #Use default fb code clang.
% make check
```
Verify less error and no missing override errors.
```lang=bash
% # Have trunk clang present in path.
% ROCKSDB_NO_FBCODE=1 CC=clang CXX=clang++ make
```

Reviewers: igor, kradhakrishnan, rven, meyering, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34077
2015-02-26 11:28:41 -08:00
Mark Callaghan
1e06a40685 Support builds for MongoDB+RocksDB
Summary:
This changes the RocksDB build to share headers provided by MongoDB. It is invoked as:
  ROCKSDB_FOR_MONGO=/path/to/mongodb/root make static_lib

Task ID: #

Blame Rev:

Test Plan:
Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34071
2015-02-26 10:50:03 -08:00
Adam Retter
b74ad66320 Merge pull request #508 from fyrz/RocksJava-Final-Patch
[RocksJava] Final usage correction
2015-02-26 13:40:05 +00:00
krad
d9f4875e52 Disable pre-fetching of index and filter blocks for sst_dump_tool.
Summary:
BlockBasedTable pre-fetches the filter and index blocks on Open call.
This is an optimistic optimization targeted for runtime scenario. The
optimization is unnecessary for sst_dump_tool

- Added a provision to disable pre-fetching of index and filter blocks
  in BlockBasedTable
- Disabled pre-fetching for the sst_dump tool

Stack for reference :

#01  0x00000000005ed944 in snappy::InternalUncompress<snappy::SnappyArrayWriter> () from /home/engshare/third-party2/snappy/1.0.3/src/snappy-1.0.3/snappy.cc:148
#02  0x00000000005edeee in snappy::RawUncompress () from /home/engshare/third-party2/snappy/1.0.3/src/snappy-1.0.3/snappy.cc:947
#03  0x00000000004e0b4d in rocksdb::UncompressBlockContents () from /data/users/paultuckfield/rocksdb/./util/compression.h:69
#04  0x00000000004e145c in rocksdb::ReadBlockContents () from /data/users/paultuckfield/rocksdb/table/format.cc:334
#05  0x00000000004ca424 in rocksdb::(anonymous namespace)::ReadBlockFromFile () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:70
#06  0x00000000004cccad in rocksdb::BlockBasedTable::CreateIndexReader () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:173
#07  0x00000000004d17e5 in rocksdb::BlockBasedTable::Open () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:553
#08  0x00000000004c8184 in rocksdb::BlockBasedTableFactory::NewTableReader () from /data/users/paultuckfield/rocksdb/table/block_based_table_factory.cc:51
#09  0x0000000000598463 in rocksdb::SstFileReader::NewTableReader () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:69
#10  0x00000000005986c2 in rocksdb::SstFileReader::SstFileReader () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:26
#11  0x0000000000599047 in rocksdb::SSTDumpTool::Run () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:332
#12  0x0000000000409b06 in main () from /data/users/paultuckfield/rocksdb/tools/sst_dump.cc:12

Test Plan:
- Added a unit test to trigger the code.
- Also did some manual verification.
- Passed all unit tests

task #6296048

Reviewers: igor, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34041
2015-02-25 16:34:26 -08:00
Mark Callaghan
182b4ceacd Limit key range to number of keys, not number of writes
Summary:
An old commit (482401) changed DoWrite to use the value of --writes rather
than --num to determine the range for keys. This restores the old and correct
behavior which is to limit it using --num.

Task ID: #6353043

Blame Rev:

Test Plan:
run db_bench

Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34065
2015-02-25 15:53:45 -08:00
Igor Canadi
a360bb61b5 Merge pull request #516 from fyrz/RocksJava-Update-Statistics-To-3.10
[RocksJava] Updated TickerTypes and Histogram to 3.10
2015-02-25 14:40:15 -08:00
fyrz
8c76844742 [RocksJava] Updated TickerTypes and Histogram to 3.10
Updated tickers and histogram types to 3.10.
2015-02-25 23:34:10 +01:00
fyrz
7f0c77cb31 [RocksJava] Integrated changes from D33417 2015-02-25 21:55:32 +01:00
fyrz
819e787bb9 [RocksJava] Final usage correction
Summary:
Introduced final keyword to parameters with immutable values and classes
which should not be derived.

Test Plan:
make rocksdbjava
make jtest

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33417
2015-02-25 21:55:31 +01:00
Igor Canadi
5139e678b0 Upgrade compiler in Travis 2015-02-24 17:19:58 -08:00
Venkatesh Radhakrishnan
4ade89962d Fix compile error on MacOS.
Summary:
In a release build, a member was not being accessed. This
member was only being accessed in a debug build. We now add an accessor
function for this member and the buid succeeds.

Test Plan: build release/unity/debug on linux/mac

Reviewers: sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34035
2015-02-24 16:24:53 -08:00
Igor Canadi
ace3d85068 Revert "Unused managed iterator"
This reverts commit bd339a9798.

Conflicts:
	db/managed_iterator.cc
2015-02-24 13:27:41 -08:00
Igor Canadi
7b8f348e56 Attempt at fixing travis issue 2015-02-24 12:20:43 -08:00
Adam Retter
d85993998c Merge pull request #506 from fyrz/RocksJava-Raw-Use
[RocksJava] Raw use of parametrized class
2015-02-24 20:11:42 +00:00
Adam Retter
30e93c9b9c Merge pull request #505 from fyrz/RocksJava-Redundant-Modifier
[RocksJava] Redundant access-modifier in interfaces
2015-02-24 20:03:00 +00:00
Igor Canadi
217854dc4a Introduce DISABLE_WARNING_AS_ERROR in Makefile
Summary: See https://www.facebook.com/groups/rocksdb.dev/permalink/718536351578215/. We should enable people to run without warning-as-error, since we break this so often for different compilers.

Test Plan: introduce an warning, compile sucessfully

Reviewers: yhchiang, meyering, igor.sugak, rven, sdong

Reviewed By: sdong

Subscribers: adamretter, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33963
2015-02-24 09:52:28 -08:00
Igor Canadi
bd339a9798 Unused managed iterator
Summary: This causes warnings on OS X

Test Plan: compiles

Reviewers: rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33969
2015-02-24 09:51:52 -08:00
stash93
174a79c993 LevelDb include guard replaced with #pragma once
Summary: Replaced LevelDb include guards with #pragma once

Test Plan: make all check

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33939
2015-02-24 09:05:06 +03:00
Igor Sugak
6fdda8ac4d rocksdb: changes to support 'make analyze' in Jenkins
Summary:
* Updated Makefile to exit `make analyze` with status 1 if scan-build detected any bugs.
* scan-build automatically detects which c++ compiler to use, and some times is uses wrong ones (from $CPP). Added implicit parameters to use $CC and $CXX.
* Added `scan_build_report` directory to .gitignore file.
* Added `scan_build_report` directory to clean target.

Test Plan:
Run `make analyze` and verify that exit status is 1, if there are scan-build bugs detected.
Run `make clean` and verify that files in `scan_build_report` directory are deleted.
After running `make analyze; git status` and verify that no untracked files in `scan_build_report` directory.

Reviewers: meyering, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33831
2015-02-23 17:45:25 -08:00
Jinfu Leng
96d989f70d catch config errors with L0 file count triggers
Test Plan: Run "make clean && make all check"

Reviewers: rven, igor, yhchiang, kradhakrishnan, MarkCallaghan, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33627
2015-02-23 16:08:27 -08:00
Igor Sugak
62f7a1be4f rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings
Summary:
This diff contains trivial fixes for 6 scan-build warnings:

**db/c_test.c**
`db` variable is never read. Removed assignment.
scan-build report:
http://home.fburl.com/~sugak/latest20/report-9b77d2.html#EndPath

**db/db_iter.cc**
`skipping` local variable is assigned to false. Then in the next switch block the only "non return" case assign `skipping` to true, the rest cases don't use it and all do return.
scan-build report:
http://home.fburl.com/~sugak/latest20/report-13fca7.html#EndPath

**db/log_reader.cc**
In `bool Reader::SkipToInitialBlock()` `offset_in_block` local variable is assigned to 0 `if (offset_in_block > kBlockSize - 6)` and then never used. Removed the assignment and renamed it to `initial_offset_in_block` to avoid confusion.
scan-build report:
http://home.fburl.com/~sugak/latest20/report-a618dd.html#EndPath

In `bool Reader::ReadRecord(Slice* record, std::string* scratch)` local variable `in_fragmented_record` in switch case `kFullType` block is assigned to false and then does `return` without use. In the other switch case `kFirstType` block the same `in_fragmented_record` is assigned to false, but later assigned to true without prior use. Removed assignment for both cases.
scan-build reprots:
http://home.fburl.com/~sugak/latest20/report-bb86b0.html#EndPath
http://home.fburl.com/~sugak/latest20/report-a975be.html#EndPath

**table/plain_table_key_coding.cc**
Local variable `user_key_size` is assigned when declared. But then in both places where it is used assigned to `static_cast<uint32_t>(key.size() - 8)`. Changed to initialize the variable to the proper value in declaration.
scan-build report:
http://home.fburl.com/~sugak/latest20/report-9e6b86.html#EndPath

**tools/db_stress.cc**
Missing `break` in switch case block. This seems to be a bug. Added missing `break`.

Test Plan:
Make sure all tests are passing and scan-build does not report 'Dead assignment' and 'Dead initialization' bugs.
```lang=bash
% make check
% make analyze
```

Reviewers: meyering, igor, kradhakrishnan, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33795
2015-02-23 14:10:09 -08:00
Igor Canadi
5636c4a80b Verbose build in travis 2015-02-23 12:32:31 -08:00
krad
a047409ae9 Fixed a bug in the test case
Summary:
The unit test was supposed to check that the old file and the new file contains
the header message.

Test Plan: Run the unit test.

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33705
2015-02-23 11:59:39 -08:00
Jim Meyering
4f514a53d6 build: enable more compiler warnings
Summary:
* Makefile (WARNING_FLAGS): Add -W and -Wextra, and at least for now,
-Wno-unused-parameter.  Without that latter one, there would be
many errors/warnings about unused parameters.

Test Plan:
run these and ensure there is no warning:
  make clean && make
  make clean && USE_CLANG=1 make

Reviewers: ljin, sdong, igor.sugak, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33819
2015-02-20 15:02:40 -08:00
Jim Meyering
a2b911b63f inputs: restore "const" attribute removed by D33759
Summary:
The "const" attribute applies to the type, and placing it
before that return type retains the desired semantics,
yet avoids the compiler error/warning.

Test Plan: Run make

Reviewers: ljin, sdong, igor.sugak, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33789
2015-02-20 11:52:20 -08:00
Jim Meyering
1b4082581c mark as unused some variables with cpp-derived names
Summary: as above

Test Plan:
  Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors.

Reviewers: ljin, sdong, igor.sugak, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33753
2015-02-20 11:07:21 -08:00
Jim Meyering
c6d54b5037 fix erroneous assert: cast kBlockSize (of type unsigned int) to "int"
Summary:
Otherwise, we would assert that an unsigned expression is always >= 0.
The intent was to form a possibly negative number, and to assert that
that value is always >= 0, but since one variable in the computation
was unsigned, the result was guaranteed to be unsigned, too, rendering
the assertion useless.

Cast that unsigned variable to "int", so that all operands
are signed, and thus so that the result can be negative.

Test Plan:
  Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors.

Reviewers: ljin, sdong, igor.sugak, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33771
2015-02-20 11:07:17 -08:00
Jim Meyering
aa5d8e6d95 table_test.cc: add missing 5th arg in TestArgs initializer
Summary:
Adding -W and -Wextra to CXXFLAGS provoked this failure:

	  table/table_test.cc:1854:56: error: missing initializer for member ‘rocksdb::TestArgs::format_version’ [-Werror=missing-field-initializers]
	     TestArgs args = { DB_TEST, false, 16, kNoCompression };
	                                                          ^

Add the missing, 5th value (format_version).

Test Plan:
  Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors.

Reviewers: ljin, sdong, igor.sugak, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33765
2015-02-20 11:07:11 -08:00
Jim Meyering
c37937a9ce maint: remove extraneous "const" attribute from return type
Summary:
The "const" attribute does not make sense on a return type,
and provokes a warning/error from gcc -W -Wextra.

Test Plan:
  Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors.

Reviewers: ljin, sdong, igor.sugak, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33759
2015-02-20 11:07:07 -08:00
Jim Meyering
9283c7afd2 build: remove always-true assertions
Summary:
Remove some always-true assertions.
They provoke these compilation failures:

  table/plain_table_key_coding.cc:279:20: error: comparison of unsigned expression >= 0 is always true [-Werror=type-limits]
  db/version_set.cc:336:15: error: comparison of unsigned expression >= 0 is always true [-Werror=type-limits]

* table/plain_table_key_coding.cc (rocksdb): Remove assertion that
unsigned type variable is >= 0.
* db/version_set.cc (DoGenerateLevelFilesBrief): Likewise.

Test Plan:
  Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors.

Reviewers: ljin, sdong, igor.sugak, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33747
2015-02-20 11:07:03 -08:00
Islam AbdelRahman
06a766de56 Adding Flush to AutoRollLogger
Summary:
During running AutoRollLoggerTest on FreeBSD we have found out that AutoRollLogger is not flushing correctly (test fails on FreeBSD)
This diff add Flush to AutoRollLogger to fix this problem

Test Plan:
[My machine] make all check ( all tests pass)
[FreeBSD VM] running AutoRollLoggerTest ( all tests pass )

Reviewers: sdong, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33633
2015-02-20 10:58:29 -08:00
Igor Canadi
92416fa7f2 Fix mac build 2015-02-19 19:26:38 -08:00
Igor Canadi
96ab15d306 GetOptionsFromString + fixes to block_based_table_options
Summary:
In mongo, we currently have a single column family and I'd like to support setting rocksdb::Options from string. This diff provides an option to GetOptionsFromString()

There's one more problem. Currently GetColumnFamilyOptionsFromString() overwrites block_based options. In mongo I set default values for block_cache and some other values of BlockBasedTableOptions and I don't want them reset to default with GetOptionsFromString().

Test Plan: added unit test

Reviewers: yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33729
2015-02-19 19:11:14 -08:00
Igor Sugak
73711f956c rocksdb: Fix scan-build bug 'Memory leak' in db/db_bench.cc
Summary:
The bug is detected by scan-build.

In `void WriteSeqSeekSeq(ThreadState* thread)` memory is allocated in line 3118 `Slice key = AllocateKey();` but `Slice` is not responsible deleting `Slice::data()`.

Added `std::unique_ptr<const char[]>*` parameter to ` AllocateKey()`, so that it requires caller to not forget about Slice::data() management.

scan-build bug report: http://home.fburl.com/~sugak/latest6/report-6e9754.html#EndPath

Test Plan:
Make sure scan-build does not report 'Memory leak' in db/db_bench.cc and all tests are passing.
```lang=bash
% make analyze
% make check
```

Reviewers: lgalanis, igor, meyering, sdong

Reviewed By: meyering, sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33501
2015-02-19 14:27:48 -08:00
Igor Sugak
98870c7b9c rocksdb: Fix scan-build memory warning in table/block_based_table_reader.cc
Summary:
scan-build is reporting two memory leak bugs in `table/block_based_table_reader.cc`. They are both false positives. In both cases we allocate memory in `ReadBlockFromFile` if `s.ok()`. Then after the function `ReadBlockFromFile` returns we check for the same variable if `s.ok()` and then use the memory that was allocated. The bugs reported by scan-build is if `ReadBlockFromFile` allocates memory and returns, but for some reason status `s` is not the same and `s.ok() != true`.

In this case scan-build is concerned that memory owner transfer is not explicit. I modified `ReadBlockFromFile` to accept `std::unique_ptr<Block>*` as a parameter, instead of raw pointer.

scan-build reports:
http://home.fburl.com/~sugak/latest2/report-a4b3fa.html#EndPath
http://home.fburl.com/~sugak/latest2/report-29adbf.html#EndPath

Test Plan:
Make sure scan-build does not report these bugs and all tests are passing.
```lang=bash
% make check
% make analyze
```

Reviewers: sdong, lgalanis, meyering, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33681
2015-02-19 14:07:38 -08:00
Jim Meyering
a42324e370 build: do not relink every single binary just for a timestamp
Summary:
Prior to this change, "make check" would always waste a lot of
time relinking 60+ binaries. With this change, it does that
only when the generated file, util/build_version.cc, changes,
and that happens only when the date changes or when the
current git SHA changes.

This change makes some other improvements: before, there was no
rule to build a deleted util/build_version.cc. If it was somehow
removed, any attempt to link a program would fail.
There is no longer any need for the separate file,
build_tools/build_detect_version.  Its functionality is
now in the Makefile.

* Makefile (DEPFILES): Don't filter-out util/build_version.cc.
No need, and besides, removing that dependency was wrong.
(date, git_sha, gen_build_version): New helper variables.
(util/build_version.cc): New rule, to create this file
and update it only if it would contain new information.
* build_tools/build_detect_platform: Remove file.
* db/db_impl.cc: Now, print only date (not the time).
* util/build_version.h (rocksdb_build_compile_time): Remove
declaration.  No longer used.

Test Plan:
- Run "make check" twice, and note that the second time no linking is performed.
- Remove util/build_version.cc and ensure that any "make"
command regenerates it before doing anything else.
- Run this: strings librocksdb.a|grep _build_.
That prints output including the following:

  rocksdb_build_git_date:2015-02-19
  rocksdb_build_git_sha:2.8.fb-1792-g3cb6cc0

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33591
2015-02-19 13:11:10 -08:00
sdong
d45a6a4002 Add rocksdb.num-live-versions: number of live versions
Summary: Add a DB property about live versions. It can be helpful to figure out whether there are files not live but not yet deleted, in some use cases.

Test Plan: make all check

Reviewers: rven, yhchiang, igor

Reviewed By: igor

Subscribers: yoshinorim, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33327
2015-02-19 13:10:37 -08:00