Commit Graph

123 Commits

Author SHA1 Message Date
Igor Canadi
c89369f57c Move dockerbuild.sh to build_tools/
Summary: That's where we keep build tools :)

Test Plan: none

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D39741
2015-06-17 14:09:12 -07:00
sdong
46296cc869 Cygwin build not to use -fPIC
Summary:
Cygwin doesn't support -fPIC. Remove it.
Not sure whether we can build shared library in Cygwin but at least it can build without warning.

Test Plan: Build under Cygwin

Reviewers: yhchiang, rven, kradhakrishnan, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D40077
2015-06-12 13:59:59 -07:00
sdong
98a44559d5 Build for CYGWIN
Summary:
Make it build for CYGWIN.
Need to define "-std=gnu++11" instead of "-std=c++11" and use some replacement functions.

Test Plan: Build it and run some unit tests in CYGWIN

Reviewers: yhchiang, rven, anthony, kradhakrishnan, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D37605
2015-04-23 21:33:44 -07:00
Igor Canadi
fd7a357318 Enable open source users to use jemalloc (github issue #438)
Summary: Currently open source rocksdb only builds with tcmalloc. This diff first checks if jemalloc is available. If it is, it compiles with jemalloc. If it isn't, it checks for tcmalloc.

Test Plan: Tried this out on my Ubuntu virtual machine and confirms that jemalloc is correctly detected and compiled.

Reviewers: MarkCallaghan, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: adamretter, meyering, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36789
2015-04-23 17:48:18 -07:00
Mark Callaghan
6e359419fe Add rpath for production builds
Summary:
This lets the production toolchain libraries get used on devservers and
in production.

Task ID: #6849362

Blame Rev:

Test Plan:
Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D37533
2015-04-22 17:17:50 -07:00
Igor Canadi
6a5ffee0cc Fix gflags Makefile
Summary: `echo` correctly interpretes \n on mac, but not on linux. On linux you have to give it `-e` to interpret \n. Unfortunately, `-e` options is not available on Mac. Go back to old way of checking gflags

Test Plan: build_tools/build_detect_platform on mac and linux

Reviewers: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D37515
2015-04-22 12:50:28 -07:00
Igor Canadi
7d136994c9 Get rid of error output
Summary: We should send error output to /dev/null

Test Plan: none

Reviewers: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D37449
2015-04-20 19:44:32 -07:00
Igor Canadi
742fa9e316 Fix compile with two gflags
Summary:
If the system has gflags with both `google` and `gflags` namespaces installed, we try to define GFLAGS as two things. This breaks the compile.

Fix: Use `else if` -- try compiling with `google` namespace only if compile with `gflags` failed

Test Plan: build_tools/build_detect_platform correctly identifies gflags

Reviewers: lgalanis

Reviewed By: lgalanis

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D37389
2015-04-20 10:55:17 -07:00
Igor Canadi
98ef21d2ff Merge pull request #584 from pshareghi/rocksdb-3.10-falloch
Added falloc.h in build_detect_platform
2015-04-15 14:56:36 -07:00
Pooya Shareghi
e8808b9128 Added falloc.h in build_detect_platform
On Centos 6, you need to explicitely include linux/falloc.h which is
whele the  FALLOC_FL_* flags are defined. Otherwise, the fallocate()
support test defined in build_detect_platform will fail.

Signed-off-by: Pooya Shareghi <shareghi@gmail.com>
2015-04-13 17:56:12 -07:00
Igor Canadi
91df4e969d Remove use of whole-archive to include jemalloc
Summary: I don't think we need to use whole-archive to include jemalloc. This change only affects our development builds -- it does not affect our open source builds (which don't support jemalloc) or our fbcode third-party2 builds (which use open-source build codepaths).

Test Plan:
make
verify that jemalloc is running by running `MALLOC_CONF="prof:true" ./cache_test` and observing that file was created

Reviewers: MarkCallaghan

Reviewed By: MarkCallaghan

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36783
2015-04-09 15:10:53 -07:00
Igor Canadi
c66483c132 Fix github issue #563
Summary:
As described in https://github.com/facebook/rocksdb/issues/563, we should add minor version to SONAME, since we break ABI with minor releases.

I also turned PLATFORM_SHARED_VERSIONED to true by default. This is true in LevelDB and it was switched to false by D15117 for no apparent reason. It should only be false for iOS.

Test Plan: `make shared_lib` produced librocksdb.dylib.3.10.0

Reviewers: sdong, yhchiang, meyering

Reviewed By: meyering

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36573
2015-04-07 13:22:22 -07:00
Igor Canadi
de22c7bd1f Integrate Jenkins with Phabricator
Summary:
After this diff, when a user submits a diff from Facebook's VPN
network, we'll automatically trigger a jenkins test. Once jenkins test
is done, we'll update the diff with test results.

Test Plan:
Made sure that jenkins build is triggered on `arc diff` and
that result is reflected back on the diff

Reviewers: sdong, rven, kradhakrishnan, anthony, yhchiang

Reviewed By: anthony

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D36555
2015-04-07 11:56:29 -07:00
Igor Canadi
df71c6b9ed Script to trigger jenkins test
Summary: After you run `arc diff`, just run `build_tools/trigger_jenkins_test.sh` and Jenkins will test your diff!

Test Plan: Triggered a build to jenkins

Reviewers: sdong, rven, IslamAbdelRahman, anthony, yhchiang, meyering

Reviewed By: meyering

Subscribers: meyering, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36021
2015-03-31 08:56:40 -07:00
Igor Sugak
67d995808a rocksdb: fix make unity
Summary:
changed make unity target to use $LIB_SOURCES as a source of library source code. In the old way (using find) table/mock_table.h was added to a list of library objects and this was a course of `make unity` break.

`build_tools/unity` contains some redundant code, I deleted it and moved the functionality in the Makefile.

Test Plan:
Make sure unity completes with no errors.
```lang=bash
% make unity
```

Reviewers: sdong, rven, igor, meyering

Reviewed By: igor, meyering

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35385
2015-03-18 19:12:23 -07:00
Igor Sugak
bd4963e64d rocksdb: update reference to the latest version of clang dev in fbcode
Summary: Updated reference to the latest clang dev in fbcode. Since RocksDB already uses recent version of clang dev no code changes need.

Test Plan:
Make sure can be build with clang.
```lang=bash
% USE_CLANG=1 make all
```

Reviewers: meyering, sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35121
2015-03-16 17:08:05 -07:00
Venkatesh Radhakrishnan
480b284766 Fix make_new_version.sh
Summary:
One of the commands in the script make_new_version.sh was
incorrect. Fixed it.

Test Plan: Try the script out to verify that it works.

Reviewers: igor, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34791
2015-03-10 17:24:33 -07:00
Jim Meyering
ebc647de87 build: fix missing dependency problems
Summary:
Any time one would modify a dependent of any *test*.cc file,
"make" would fail to rebuild the affected test binaries,
e.g., db_test.  That was due to the fact that we deliberately
excluded those test-related files from the definition of SOURCES
and only $(SOURCES) was used to create the automatically-generated
.d dependency files.  The fix is to generate a .d file for every
source file.
* src.mk: New file.  Defines LIB_SOURCES, MOCK_SOURCES
and TEST_BENCH_SOURCES.
* Makefile: Include src.mk.
Reflect s/SOURCES/LIB_SOURCES/ renaming.
* build_tools/build_detect_platform: Remove the code
that was used to generate SOURCES= and MOCK_SOURCES=
definitions in make_config.mk. Those lists of files
are now hard-coded in src.mk. Hard-coding this list of
sources is desirable, because without that, one risks
including stray .cc files in a build.  Not reproducible.

Test Plan:
Touch a file used by db_test's dependent .o files and ensure that
they are all recompiled.  Before, none would be:

  $ touch db/db_impl.h && make db_test
    CC       db/db_test.o
    CC       db/column_family.o
    CC       db/db_filesnapshot.o
    CC       db/db_impl.o
    CC       db/db_impl_debug.o
    CC       db/db_impl_readonly.o
    CC       db/forward_iterator.o
    CC       db/internal_stats.o
    CC       db/managed_iterator.o
    CC       db/repair.o
    CC       db/write_batch.o
    CC       utilities/compacted_db/compacted_db_impl.o
    CC       utilities/ttl/db_ttl_impl.o
    CC       util/ldb_cmd.o
    CC       util/ldb_tool.o
    CC       util/sst_dump_tool.o
    CC       util/xfunc.o
    CCLD     db_test

Reviewers: ljin, igor.sugak, igor, rven, sdong

Reviewed By: sdong

Subscribers: yhchiang, adamretter, fyrz, dhruba

Differential Revision: https://reviews.facebook.net/D33849
2015-03-06 10:55:11 -08:00
Igor Canadi
f9c14a42e8 Fix compile on Mac 2015-02-27 09:31:29 -08:00
Islam AbdelRahman
ba9d1737a8 RocksDB on FreeBSD support
Summary:
This patch will update the Makefile and source code so that we can build RocksDB successfully on FreeBSD 10 and 11 (64-bit and 32-bit)
I have also encountered some problems when running tests on FreeBSD, I will try to fix them individually in different diffs

Notes:

  - FreeBSD uses clang as it's default compiler (http://lists.freebsd.org/pipermail/freebsd-current/2012-September/036480.html)
  - GNU C++ compiler have C++ 11 problems on FreeBSD (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193528)
  - make is not gmake on FreeBSD (http://www.khmere.com/freebsd_book/html/ch01.html)

Test Plan:
Using VMWare Fusion Create 4 VM machines (FreeBSD 11 64-bit, FreeBSD 11 32-bit, FreeBSD 10 64-bit, FreeBSD 10 32-bit)

  - pkg install git gmake gflags archivers/snappy
  - git clone https://github.com/facebook/rocksdb.git
  - apply this patch
  - setenv CXX c++
  - setenv CPATH /usr/local/include/
  - setenv LIBRARY_PATH  /usr/local/lib/
  - gmake db_bench
  - make sure compilation is successful and db_bench is running
  - gmake all
  - make sure compilation is successful

Reviewers: sdong, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33891
2015-02-26 15:19:17 -08:00
Igor Sugak
4ba119df5c rocksdb: Update path to dev clang in fbcode
Summary: Current clang path in fbcode points to clang 3.5. This diff updates clang path to clang 3.7.

Test Plan:
Make sure clang 3.7 is in use and all tests are passing.
```lang=bash
% USE_CLANG make check
```

Reviewers: meyering, sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34119
2015-02-26 15:12:08 -08:00
Mark Callaghan
1e06a40685 Support builds for MongoDB+RocksDB
Summary:
This changes the RocksDB build to share headers provided by MongoDB. It is invoked as:
  ROCKSDB_FOR_MONGO=/path/to/mongodb/root make static_lib

Task ID: #

Blame Rev:

Test Plan:
Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34071
2015-02-26 10:50:03 -08:00
Jim Meyering
a42324e370 build: do not relink every single binary just for a timestamp
Summary:
Prior to this change, "make check" would always waste a lot of
time relinking 60+ binaries. With this change, it does that
only when the generated file, util/build_version.cc, changes,
and that happens only when the date changes or when the
current git SHA changes.

This change makes some other improvements: before, there was no
rule to build a deleted util/build_version.cc. If it was somehow
removed, any attempt to link a program would fail.
There is no longer any need for the separate file,
build_tools/build_detect_version.  Its functionality is
now in the Makefile.

* Makefile (DEPFILES): Don't filter-out util/build_version.cc.
No need, and besides, removing that dependency was wrong.
(date, git_sha, gen_build_version): New helper variables.
(util/build_version.cc): New rule, to create this file
and update it only if it would contain new information.
* build_tools/build_detect_platform: Remove file.
* db/db_impl.cc: Now, print only date (not the time).
* util/build_version.h (rocksdb_build_compile_time): Remove
declaration.  No longer used.

Test Plan:
- Run "make check" twice, and note that the second time no linking is performed.
- Remove util/build_version.cc and ensure that any "make"
command regenerates it before doing anything else.
- Run this: strings librocksdb.a|grep _build_.
That prints output including the following:

  rocksdb_build_git_date:2015-02-19
  rocksdb_build_git_sha:2.8.fb-1792-g3cb6cc0

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33591
2015-02-19 13:11:10 -08:00
Igor Sugak
4d98e29352 rocksdb: Enable scan-build static analysis
Summary:
Added new target ##make analyze## into Makefile. This command runs clang static analyzer and builds the sources as ##make all##. The result report is put into ##$(RocksDbSourceRoot)/can_build_report/##

If the development environment is a Facebook devserver and ##ROCKSDB_NO_FBCODE## is not set, then scan-build is used from fbcode. If it is run not on a Facebook devserver, scan-build should be available in ##$PATH##. I'll add details to wiki how to install scan-build on a non Facebook devserver environment.

Test Plan:
Run the fallowing commands on a Facebook devserver and Mac OS, and ensure no build or test errors.

```
% make all check -j32
% make clean
% USE_CLANG=1 make all -j32
% make analyze
% USE_CLANG=1 make analyze
```

Reviewers: sdong, lgalanis, leveldb, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32799
2015-02-03 21:43:06 -08:00
Igor Canadi
2fd8f750ab Compile MemEnv with standard RocksDB library
Summary: This was a feature request by osquery. See task t5617758

Test Plan: compiles and memenv_test runs

Reviewers: yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32115
2015-01-29 16:33:11 -08:00
Yueh-Hsuan Chiang
c4fb83441c Update the comment for the removal of mac-install-gflags.sh
Summary: Update the comment for the removal of mac-install-gflags.sh

Test Plan: n/a

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32295
2015-01-27 10:30:35 -08:00
Igor Canadi
58f34edfce Fix valgrind 2015-01-26 13:17:28 -08:00
Igor Canadi
910186c278 Return the build with 4.8.1
Summary: We need this because we build MySQL with 4.8.1.

Test Plan: ROCKSDB_FBCODE_BUILD_WITH_481=1 make check

Reviewers: sdong, yhchiang, rven, yoshinorim

Reviewed By: yoshinorim

Subscribers: jonahcohen, yoshinorim, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32073
2015-01-23 14:51:27 -08:00
Igor Canadi
401d4205ef Add thread sanitizer
Summary: When you compile with COMPILE_WITH_TSAN=1, we will compile the code with -fsanitize=thread. This will resolve bunch of data race issues we might have.

Test Plan: COMPILE_WITH_TSAN=1 m db_test

Reviewers: yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32019
2015-01-23 11:22:20 -08:00
Igor Canadi
b068f0a673 Upgrade our compilers
Summary:
Upgrade gcc to 4.9.1 and clang to dev.

With new compilers I succeeded to run thread sanitizer, too. I'll post output (doesn't look good) and fix some things in separate diffs.

Test Plan: compiles with both g++ and clang

Reviewers: yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32007
2015-01-23 11:12:37 -08:00
Igor Canadi
155bec4cf2 fallocate also tests FALLOC_FL_KEEP_SIZE 2015-01-15 14:15:43 -08:00
Igor Canadi
a3001b1d3d Remove -mtune=native because it's redundant 2014-12-19 09:06:45 -08:00
Igor Canadi
06eed650a0 Optimize default compile to compilation platform by default
Summary:
This diff changes compile to optimize for native platform by default. This will automatically turn on crc32 optimizations for modern processors, which greatly improves rocksdb's performance.

I also did some more changes to compilation documentation.

Test Plan:
compile with `make`, observe -march=native
compile with `PORTABLE=1 make`, observe no -march=native

Reviewers: sdong, rven, yhchiang, MarkCallaghan

Reviewed By: MarkCallaghan

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D30225
2014-12-15 11:29:41 +01:00
Igor Canadi
767777c2bd Turn on -Wshorten-64-to-32 and fix all the errors
Summary:
We need to turn on -Wshorten-64-to-32 for mobile. See D1671432 (internal phabricator) for details.

This diff turns on the warning flag and fixes all the errors. There were also some interesting errors that I might call bugs, especially in plain table. Going forward, I think it makes sense to have this flag turned on and be very very careful when converting 64-bit to 32-bit variables.

Test Plan: compiles

Reviewers: ljin, rven, yhchiang, sdong

Reviewed By: yhchiang

Subscribers: bobbaldwin, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28689
2014-11-11 16:47:22 -05:00
Igor Canadi
5c93090530 Turn on -Wshadow
Summary:
Only one more try, I promise.

I talked to Jim and he mentioned that if we include our system includes with -isystem rather than with -I, that signals to the compile that those are system includes and thus no warnings are issued. So I turned our glibc includes into system includes and now we no longer get the warning from there, making us shadow-warning-free!

Test Plan: compiles with both clang and gcc

Reviewers: sdong, yhchiang, rven, ljin

Reviewed By: ljin

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28479
2014-11-07 12:57:08 -08:00
Igor Canadi
30ca3752ba Revamp our build tools
Summary:
This diff is revamping our build tools:
1) Use third-party2 instead of third-party
2) consolidate clang and gcc scripts together, lots of duplication there
3) remove hdfs libs, we never compile rocksdb with them

clang compilation doesn't work yet. It doesn't work in master either. I plan to fix it soon, but I just spent 2 hours trying to make it work and failed. I'll ask experts.

Test Plan: compiles with gcc

Reviewers: ljin, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28047
2014-11-03 14:00:45 -08:00
Lei Jin
8db24f4b35 exclude mock test file from MOCK_SOURCES
Summary: as title

Test Plan: build with mock_env_test.cc

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28107
2014-10-31 15:07:27 -07:00
Igor Canadi
c082853340 Include all the mocks 2014-10-28 17:55:08 -07:00
Igor Canadi
abac3d6476 TableMock + framework for mock classes
Summary:
This diff replaces BlockBasedTable in flush_job_test with TableMock, making it depend on less things and making it closer to an unit test than integration test.

It also introduces a framework to compile mock classes -- Any file named *mock.cc will not be compiled into the build. It will only get compiled into the tests. What way we can mock out most other classes, Version, VersionSet, DBImpl, etc.

Test Plan: flush_job_test

Reviewers: ljin, rven, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27681
2014-10-28 17:52:32 -07:00
Igor Canadi
48842ab316 Deprecate AtomicPointer
Summary: RocksDB already depends on C++11, so we might as well all the goodness that C++11 provides. This means that we don't need AtomicPointer anymore. The less things in port/, the easier it will be to port to other platforms.

Test Plan: make check + careful visual review verifying that NoBarried got memory_order_relaxed, while Acquire/Release methods got memory_order_acquire and memory_order_release

Reviewers: rven, yhchiang, ljin, sdong

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D27543
2014-10-27 14:50:21 -07:00
fyrz
69d4c5123e Cross-platform fix version.sh
version.sh now works also on linux properly.
2014-10-04 11:17:06 +02:00
Igor Canadi
079a612b5e Fix build_tools/version.sh 2014-10-02 14:03:49 -07:00
Igor Canadi
0908ddcea5 Don't keep managing two rocksdb version
Summary:
Before this diff, there are two places with rocksdb versions. After the diff:
1. we only have one source of truth for rocksdb version
2. we have a script that we can use to get the version that we can use in other compilations (java, go, etc).

Test Plan: make

Reviewers: yhchiang, sdong, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24333
2014-10-02 11:59:22 -07:00
mike@arpaia.co
ee1f3ccb06 Package generation for Ubuntu and CentOS
Summary:
I put together a script to assist in the generation of deb's and
rpm's. I've tested that this works on ubuntu via vagrant. I've included the
Vagrantfile here, but I can remove it if it's not useful. The package.sh
script should work on any ubuntu or centos machine, I just added a bit of
logic in there to allow a base Ubuntu or Centos machine to be able to build
RocksDB from scratch.

Example output on Ubuntu 14.04:

```
root@vagrant-ubuntu-trusty-64:/vagrant# ./tools/package.sh
[+] g++-4.7 is already installed. skipping.
[+] libgflags-dev is already installed. skipping.
[+] ruby-all-dev is already installed. skipping.
[+] fpm is already installed. skipping.
Created package {:path=>"rocksdb_3.5_amd64.deb"}
root@vagrant-ubuntu-trusty-64:/vagrant# dpkg --info rocksdb_3.5_amd64.deb
 new debian package, version 2.0.
 size 17392022 bytes: control archive=1518 bytes.
     275 bytes,    11 lines      control
    2911 bytes,    38 lines      md5sums
 Package: rocksdb
 Version: 3.5
 License: BSD
 Vendor: Facebook
 Architecture: amd64
 Maintainer: rocksdb@fb.com
 Installed-Size: 83358
 Section: default
 Priority: extra
 Homepage: http://rocksdb.org/
 Description: RocksDB is an embeddable persistent key-value store for fast storage.
 ```

 Example output on CentOS 6.5:

 ```
 [root@localhost vagrant]# rpm -qip rocksdb-3.5-1.x86_64.rpm
 Name        : rocksdb                      Relocations: /usr
 Version     : 3.5                               Vendor: Facebook
 Release     : 1                             Build Date: Mon 29 Sep 2014 01:26:11 AM UTC
 Install Date: (not installed)               Build Host: localhost
 Group       : default                       Source RPM: rocksdb-3.5-1.src.rpm
 Size        : 96231106                         License: BSD
 Signature   : (none)
 Packager    : rocksdb@fb.com
 URL         : http://rocksdb.org/
 Summary     : RocksDB is an embeddable persistent key-value store for fast storage.
 Description :
 RocksDB is an embeddable persistent key-value store for fast storage.
 ```

Test Plan:
How this gets used is really up to the RocksDB core team. If you
want to actually get this into mainline, you might have to change `make
install` such that it install the RocksDB shared object file as well, which
would require you to link against gflags (maybe?) and that would require some
potential modifications to the script here (basically add a depends on that
package).

Currently, this will install the headers and a pre-compiled statically linked
object file. If that's what you want out of life, than this requires no
modifications.

Reviewers: ljin, yhchiang, igor

Reviewed By: igor

Differential Revision: https://reviews.facebook.net/D24141
2014-09-29 16:09:46 -07:00
Igor Canadi
3b897cddd7 Enable no-fbcode RocksDB build
Summary: I want to use open source build rather than fbcode one. This enables me to run `ROCKSDB_NO_FBCODE=1 make` and run it with my system g++.

Test Plan:
ROCKSDB_NO_FBCODE=1 make
make

Reviewers: sdong, ljin, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23613
2014-09-19 09:27:16 -07:00
Igor Canadi
8de151bb99 Add db_bench with lots of column families to regression tests
Summary:
That way we can see when this graph goes up and be happy.

Couple of changes:
1. title
2. fix db_bench to delete column families before deleting the DB. this was asserting when compiled in debug mode
3. don't sync manifest when disableDataSync. We discussed this offline. I can move it to separate diff if you'd like

Test Plan: ran it

Reviewers: sdong, yhchiang, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D22815
2014-09-05 14:20:18 -07:00
miguelportilla
93e6b5e9d9 Changes to support unity build:
* Script for building the unity.cc file via Makefile
* Unity executable Makefile target for testing builds
* Source code changes to fix compilation of unity build
2014-08-11 13:22:47 -04:00
Yueh-Hsuan Chiang
ae7743f226 Fixed some make and linking issues of RocksDBJava
Summary:
Fixed some make and linking issues of RocksDBJava. Specifically:
* Add JAVA_LDFLAGS, which does not include gflags
* rocksdbjava library now uses JAVA_LDFLAGS instead of LDFLAGS
* java/Makefile now includes build_config.mk
* rearrange make rocksdbjava workflow to ensure the library file is correctly
  included in the jar file.

Test Plan:
make rocksdbjava
make jdb_bench
java/jdb_bench.sh

Reviewers: dhruba, swapnilghike, zzbennett, rsumbaly, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20289
2014-07-21 22:41:54 -07:00
Radheshyam Balasundaram
f0660d5253 Adding NUMA support to db_bench tests
Summary:
Changes:
- Adding numa_aware flag to db_bench.cc
- Using numa.h library to bind memory and cpu of threads to a fixed NUMA node
Result: There seems to be no significant change in the micros/op time with numa_aware enabled. I also tried this with other implementations, including a combination of pthread_setaffinity_np, sched_setaffinity and set_mempolicy methods. It'd be great if someone could point out where I'm going wrong and if we can achieve a better micors/op.

Test Plan:
Ran db_bench tests using following command:
./db_bench --db=/mnt/tmp --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_type=none --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=134217728 --max_bytes_for_level_base=1073741824 --disable_wal=0 --wal_dir=/mnt/tmp --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --duration=300 --benchmarks=readwhilewriting --use_existing_db=1 --num=157286400 --threads=24 --writes_per_second=10240 --numa_aware=[False/True]

The tests were run in private devserver with 24 cores and the db was prepopulated using filluniquerandom test. The tests resulted in 0.145 us/op with numa_aware=False and 0.161 us/op with numa_aware=True.

Reviewers: sdong, yhchiang, ljin, igor

Reviewed By: ljin, igor

Subscribers: igor, leveldb

Differential Revision: https://reviews.facebook.net/D19353
2014-07-07 10:53:31 -07:00
nawu
fb54eef744 escaped the special characters and added a dot 2014-06-24 00:14:02 -05:00