andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
df4197ca6e
rocksdb/fuzz/Makefile

62 lines
2.6 KiB
Makefile
Raw Normal View History

Fuzzing RocksDB (#7685) Summary: This is the initial PR to support adding fuzz tests to RocksDB. It includes the necessary build infrastructure, and includes an example fuzzer. There is also a README serving as the tutorial for how to add more tests. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7685 Test Plan: Manually build and run the fuzz test according to README. Reviewed By: pdillinger Differential Revision: D25013847 Pulled By: cheng-chang fbshipit-source-id: c91e3b337398d7f4d8f769fd5091cd080487b171
2020-11-17 21:55:27 +01:00
# Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
# This source code is licensed under both the GPLv2 (found in the
# COPYING file in the root directory) and Apache 2.0 License
# (found in the LICENSE.Apache file in the root directory).
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
ROOT_DIR = $(abspath $(shell pwd)/../)
include $(ROOT_DIR)/make_config.mk
Fuzzing RocksDB (#7685) Summary: This is the initial PR to support adding fuzz tests to RocksDB. It includes the necessary build infrastructure, and includes an example fuzzer. There is also a README serving as the tutorial for how to add more tests. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7685 Test Plan: Manually build and run the fuzz test according to README. Reviewed By: pdillinger Differential Revision: D25013847 Pulled By: cheng-chang fbshipit-source-id: c91e3b337398d7f4d8f769fd5091cd080487b171
2020-11-17 21:55:27 +01:00
PROTOBUF_CFLAGS = `pkg-config --cflags protobuf`
PROTOBUF_LDFLAGS = `pkg-config --libs protobuf`
PROTOBUF_MUTATOR_CFLAGS = `pkg-config --cflags libprotobuf-mutator`
PROTOBUF_MUTATOR_LDFLAGS = `pkg-config --libs libprotobuf-mutator`
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
ROCKSDB_INCLUDE_DIR = $(ROOT_DIR)/include
ROCKSDB_LIB_DIR = $(ROOT_DIR)
Fuzzing RocksDB (#7685) Summary: This is the initial PR to support adding fuzz tests to RocksDB. It includes the necessary build infrastructure, and includes an example fuzzer. There is also a README serving as the tutorial for how to add more tests. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7685 Test Plan: Manually build and run the fuzz test according to README. Reviewed By: pdillinger Differential Revision: D25013847 Pulled By: cheng-chang fbshipit-source-id: c91e3b337398d7f4d8f769fd5091cd080487b171
2020-11-17 21:55:27 +01:00
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
PROTO_IN = $(ROOT_DIR)/fuzz/proto
PROTO_OUT = $(ROOT_DIR)/fuzz/proto/gen
Fuzzing RocksDB (#7685) Summary: This is the initial PR to support adding fuzz tests to RocksDB. It includes the necessary build infrastructure, and includes an example fuzzer. There is also a README serving as the tutorial for how to add more tests. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7685 Test Plan: Manually build and run the fuzz test according to README. Reviewed By: pdillinger Differential Revision: D25013847 Pulled By: cheng-chang fbshipit-source-id: c91e3b337398d7f4d8f769fd5091cd080487b171
2020-11-17 21:55:27 +01:00
OSS-Fuzz integration and db_fuzzer (#7674) Summary: This PR adds a fuzzer to the project and infrastructure to integrate Rocksdb with OSS-Fuzz. OSS-Fuzz is a service run by Google that performs continuous fuzzing of important open source projects. The LevelDB project is also in being fuzzed by OSS-Fuzz (https://github.com/google/oss-fuzz/tree/master/projects/leveldb). Essentially, OSS-Fuzz will perform the fuzzing for you and email you bug reports, coverage reports etc. All we need is a set of email addresses that will receive this information. For cross-referencing, the PR that adds the OSS-Fuzz logic is here: https://github.com/google/oss-fuzz/pull/4642 The `db_fuzzer` of the PR performs stateful fuzzing of Rocksdb by calling a sequence of Rockdb's APIs with random input in each fuzz iteration. Each fuzz iteration, thus, creates a new instance of Rocksdb and operates on this given instance. The goal is to test diverse states of Rocksdb and ensure no state lead to error conditions, e.g. memory corruption vulnerabilities. The fuzzer is similar (although more complex) to the fuzzer that is currently being used to analyse Leveldb (https://github.com/google/oss-fuzz/blob/master/projects/leveldb/fuzz_db.cc) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7674 Reviewed By: pdillinger Differential Revision: D25238536 Pulled By: cheng-chang fbshipit-source-id: 610331c49a77eb68d3b1d7d5ef1b0ce230ac0630
2020-12-07 23:01:05 +01:00
ifneq ($(FUZZ_ENV), ossfuzz)
Fuzzing RocksDB (#7685) Summary: This is the initial PR to support adding fuzz tests to RocksDB. It includes the necessary build infrastructure, and includes an example fuzzer. There is also a README serving as the tutorial for how to add more tests. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7685 Test Plan: Manually build and run the fuzz test according to README. Reviewed By: pdillinger Differential Revision: D25013847 Pulled By: cheng-chang fbshipit-source-id: c91e3b337398d7f4d8f769fd5091cd080487b171
2020-11-17 21:55:27 +01:00
CC = clang++
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
CCFLAGS += -Wall -fsanitize=address,fuzzer
CFLAGS += $(PLATFORM_CXXFLAGS) $(PROTOBUF_CFLAGS) $(PROTOBUF_MUTATOR_CFLAGS) -I$(PROTO_OUT) -I$(ROCKSDB_INCLUDE_DIR) -I$(ROCKSDB_LIB_DIR)
LDFLAGS += $(PLATFORM_LDFLAGS) $(PROTOBUF_LDFLAGS) $(PROTOBUF_MUTATOR_LDFLAGS) -L$(ROCKSDB_LIB_DIR) -lrocksdb
OSS-Fuzz integration and db_fuzzer (#7674) Summary: This PR adds a fuzzer to the project and infrastructure to integrate Rocksdb with OSS-Fuzz. OSS-Fuzz is a service run by Google that performs continuous fuzzing of important open source projects. The LevelDB project is also in being fuzzed by OSS-Fuzz (https://github.com/google/oss-fuzz/tree/master/projects/leveldb). Essentially, OSS-Fuzz will perform the fuzzing for you and email you bug reports, coverage reports etc. All we need is a set of email addresses that will receive this information. For cross-referencing, the PR that adds the OSS-Fuzz logic is here: https://github.com/google/oss-fuzz/pull/4642 The `db_fuzzer` of the PR performs stateful fuzzing of Rocksdb by calling a sequence of Rockdb's APIs with random input in each fuzz iteration. Each fuzz iteration, thus, creates a new instance of Rocksdb and operates on this given instance. The goal is to test diverse states of Rocksdb and ensure no state lead to error conditions, e.g. memory corruption vulnerabilities. The fuzzer is similar (although more complex) to the fuzzer that is currently being used to analyse Leveldb (https://github.com/google/oss-fuzz/blob/master/projects/leveldb/fuzz_db.cc) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7674 Reviewed By: pdillinger Differential Revision: D25238536 Pulled By: cheng-chang fbshipit-source-id: 610331c49a77eb68d3b1d7d5ef1b0ce230ac0630
2020-12-07 23:01:05 +01:00
else
# OSS-Fuzz sets various environment flags that are used for compilation.
# These environment flags depend on which type of sanitizer build is being
# used, however, an ASan build would set the environment flags as follows:
# CFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only \
-DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address \
-fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link"
# CXXFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only \
-DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address \
-fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link \
-stdlib=libc++"
# LIB_FUZZING_ENGINE="-fsanitize=fuzzer"
CC = $(CXX)
CCFLAGS = $(CXXFLAGS)
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
CFLAGS += $(PROTOBUF_CFLAGS) $(PROTOBUF_MUTATOR_CFLAGS) -I$(PROTO_OUT) -I$(ROCKSDB_INCLUDE_DIR) -I$(ROCKSDB_LIB_DIR)
LDFLAGS += $(PLATFORM_LDFLAGS) $(LIB_FUZZING_ENGINE) $(PROTOBUF_MUTATOR_LDFLAGS) $(PROTOBUF_LDFLAGS) -L$(ROCKSDB_LIB_DIR) -lrocksdb
OSS-Fuzz integration and db_fuzzer (#7674) Summary: This PR adds a fuzzer to the project and infrastructure to integrate Rocksdb with OSS-Fuzz. OSS-Fuzz is a service run by Google that performs continuous fuzzing of important open source projects. The LevelDB project is also in being fuzzed by OSS-Fuzz (https://github.com/google/oss-fuzz/tree/master/projects/leveldb). Essentially, OSS-Fuzz will perform the fuzzing for you and email you bug reports, coverage reports etc. All we need is a set of email addresses that will receive this information. For cross-referencing, the PR that adds the OSS-Fuzz logic is here: https://github.com/google/oss-fuzz/pull/4642 The `db_fuzzer` of the PR performs stateful fuzzing of Rocksdb by calling a sequence of Rockdb's APIs with random input in each fuzz iteration. Each fuzz iteration, thus, creates a new instance of Rocksdb and operates on this given instance. The goal is to test diverse states of Rocksdb and ensure no state lead to error conditions, e.g. memory corruption vulnerabilities. The fuzzer is similar (although more complex) to the fuzzer that is currently being used to analyse Leveldb (https://github.com/google/oss-fuzz/blob/master/projects/leveldb/fuzz_db.cc) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7674 Reviewed By: pdillinger Differential Revision: D25238536 Pulled By: cheng-chang fbshipit-source-id: 610331c49a77eb68d3b1d7d5ef1b0ce230ac0630
2020-12-07 23:01:05 +01:00
endif
Fuzzing RocksDB (#7685) Summary: This is the initial PR to support adding fuzz tests to RocksDB. It includes the necessary build infrastructure, and includes an example fuzzer. There is also a README serving as the tutorial for how to add more tests. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7685 Test Plan: Manually build and run the fuzz test according to README. Reviewed By: pdillinger Differential Revision: D25013847 Pulled By: cheng-chang fbshipit-source-id: c91e3b337398d7f4d8f769fd5091cd080487b171
2020-11-17 21:55:27 +01:00
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
.PHONY: gen_proto
Fuzzing RocksDB (#7685) Summary: This is the initial PR to support adding fuzz tests to RocksDB. It includes the necessary build infrastructure, and includes an example fuzzer. There is also a README serving as the tutorial for how to add more tests. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7685 Test Plan: Manually build and run the fuzz test according to README. Reviewed By: pdillinger Differential Revision: D25013847 Pulled By: cheng-chang fbshipit-source-id: c91e3b337398d7f4d8f769fd5091cd080487b171
2020-11-17 21:55:27 +01:00
gen_proto:
mkdir -p $(PROTO_OUT)
protoc \
--proto_path=$(PROTO_IN) \
--cpp_out=$(PROTO_OUT) \
$(PROTO_IN)/*.proto
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
db_fuzzer: db_fuzzer.cc
OSS-Fuzz integration and db_fuzzer (#7674) Summary: This PR adds a fuzzer to the project and infrastructure to integrate Rocksdb with OSS-Fuzz. OSS-Fuzz is a service run by Google that performs continuous fuzzing of important open source projects. The LevelDB project is also in being fuzzed by OSS-Fuzz (https://github.com/google/oss-fuzz/tree/master/projects/leveldb). Essentially, OSS-Fuzz will perform the fuzzing for you and email you bug reports, coverage reports etc. All we need is a set of email addresses that will receive this information. For cross-referencing, the PR that adds the OSS-Fuzz logic is here: https://github.com/google/oss-fuzz/pull/4642 The `db_fuzzer` of the PR performs stateful fuzzing of Rocksdb by calling a sequence of Rockdb's APIs with random input in each fuzz iteration. Each fuzz iteration, thus, creates a new instance of Rocksdb and operates on this given instance. The goal is to test diverse states of Rocksdb and ensure no state lead to error conditions, e.g. memory corruption vulnerabilities. The fuzzer is similar (although more complex) to the fuzzer that is currently being used to analyse Leveldb (https://github.com/google/oss-fuzz/blob/master/projects/leveldb/fuzz_db.cc) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7674 Reviewed By: pdillinger Differential Revision: D25238536 Pulled By: cheng-chang fbshipit-source-id: 610331c49a77eb68d3b1d7d5ef1b0ce230ac0630
2020-12-07 23:01:05 +01:00
$(CC) $(CCFLAGS) -o db_fuzzer db_fuzzer.cc $(CFLAGS) $(LDFLAGS)
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
db_map_fuzzer: gen_proto db_map_fuzzer.cc proto/gen/db_operation.pb.cc
Add a new db_map_fuzzer (#7762) Summary: Execute randomly generated operations on both a DB and a std::map, then reopen the DB and make sure that iterating the DB produces the same key-value pairs as iterating through the std::map. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7762 Test Plan: cd fuzz && make db_map_fuzzer && ./db_map_fuzzer Reviewed By: pdillinger Differential Revision: D25437485 Pulled By: cheng-chang fbshipit-source-id: 3a93f7efd046b194193e45d2ab1ad81565510781
2020-12-10 01:23:52 +01:00
$(CC) $(CCFLAGS) -o db_map_fuzzer db_map_fuzzer.cc proto/gen/db_operation.pb.cc $(CFLAGS) $(LDFLAGS)
Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7761 Test Plan: cd fuzz && make sst_file_writer_fuzzer && ./sst_file_writer_fuzzer Reviewed By: pdillinger Differential Revision: D25430802 Pulled By: cheng-chang fbshipit-source-id: 01436307df6f4c434bb608f44e1c8e4a1119f94f
2020-12-12 01:07:48 +01:00
sst_file_writer_fuzzer: gen_proto sst_file_writer_fuzzer.cc proto/gen/db_operation.pb.cc
OSS-Fuzz integration and db_fuzzer (#7674) Summary: This PR adds a fuzzer to the project and infrastructure to integrate Rocksdb with OSS-Fuzz. OSS-Fuzz is a service run by Google that performs continuous fuzzing of important open source projects. The LevelDB project is also in being fuzzed by OSS-Fuzz (https://github.com/google/oss-fuzz/tree/master/projects/leveldb). Essentially, OSS-Fuzz will perform the fuzzing for you and email you bug reports, coverage reports etc. All we need is a set of email addresses that will receive this information. For cross-referencing, the PR that adds the OSS-Fuzz logic is here: https://github.com/google/oss-fuzz/pull/4642 The `db_fuzzer` of the PR performs stateful fuzzing of Rocksdb by calling a sequence of Rockdb's APIs with random input in each fuzz iteration. Each fuzz iteration, thus, creates a new instance of Rocksdb and operates on this given instance. The goal is to test diverse states of Rocksdb and ensure no state lead to error conditions, e.g. memory corruption vulnerabilities. The fuzzer is similar (although more complex) to the fuzzer that is currently being used to analyse Leveldb (https://github.com/google/oss-fuzz/blob/master/projects/leveldb/fuzz_db.cc) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7674 Reviewed By: pdillinger Differential Revision: D25238536 Pulled By: cheng-chang fbshipit-source-id: 610331c49a77eb68d3b1d7d5ef1b0ce230ac0630
2020-12-07 23:01:05 +01:00
$(CC) $(CCFLAGS) -o sst_file_writer_fuzzer sst_file_writer_fuzzer.cc proto/gen/db_operation.pb.cc $(CFLAGS) $(LDFLAGS)
Reference in New Issue Copy Permalink