From efd035164b443e0ae552a82ad8b47a8048e652ca Mon Sep 17 00:00:00 2001 From: Peter Dillinger Date: Wed, 13 Apr 2022 07:34:01 -0700 Subject: [PATCH] Meta-internal folly integration with F14FastMap (#9546) Summary: Especially after updating to C++17, I don't see a compelling case for *requiring* any folly components in RocksDB. I was able to purge the existing hard dependencies, and it can be quite difficult to strip out non-trivial components from folly for use in RocksDB. (The prospect of doing that on F14 has changed my mind on the best approach here.) But this change creates an optional integration where we can plug in components from folly at compile time, starting here with F14FastMap to replace std::unordered_map when possible (probably no public APIs for example). I have replaced the biggest CPU users of std::unordered_map with compile-time pluggable UnorderedMap which will use F14FastMap when USE_FOLLY is set. USE_FOLLY is always set in the Meta-internal buck build, and a simulation of that is in the Makefile for public CI testing. A full folly build is not needed, but checking out the full folly repo is much simpler for getting the dependency, and anything else we might want to optionally integrate in the future. Some picky details: * I don't think the distributed mutex stuff is actually used, so it was easy to remove. * I implemented an alternative to `folly::constexpr_log2` (which is much easier in C++17 than C++11) so that I could pull out the hard dependencies on `ConstexprMath.h` * I had to add noexcept move constructors/operators to some types to make F14's complainUnlessNothrowMoveAndDestroy check happy, and I added a macro to make that easier in some common cases. * Updated Meta-internal buck build to use folly F14Map (always) No updates to HISTORY.md nor INSTALL.md as this is not (yet?) considered a production integration for open source users. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9546 Test Plan: CircleCI tests updated so that a couple of them use folly. Most internal unit & stress/crash tests updated to use Meta-internal latest folly. (Note: they should probably use buck but they currently use Makefile.) Example performance improvement: when filter partitions are pinned in cache, they are tracked by PartitionedFilterBlockReader::filter_map_ and we can build a test that exercises that heavily. Build DB with ``` TEST_TMPDIR=/dev/shm/rocksdb ./db_bench -benchmarks=fillrandom -num=10000000 -disable_wal=1 -write_buffer_size=30000000 -bloom_bits=16 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=0 -partition_index_and_filters ``` and test with (simultaneous runs with & without folly, ~20 times each to see convergence) ``` TEST_TMPDIR=/dev/shm/rocksdb ./db_bench_folly -readonly -use_existing_db -benchmarks=readrandom -num=10000000 -bloom_bits=16 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=0 -partition_index_and_filters -duration=40 -pin_l0_filter_and_index_blocks_in_cache ``` Average ops/s no folly: 26229.2 Average ops/s with folly: 26853.3 (+2.4%) Reviewed By: ajkr Differential Revision: D34181736 Pulled By: pdillinger fbshipit-source-id: ffa6ad5104c2880321d8a1aa7187e00ab0d02e94 --- .circleci/config.yml | 18 +- .gitignore | 1 + CMakeLists.txt | 31 +- Makefile | 52 +- TARGETS | 4 +- buckifier/buckify_rocksdb.py | 5 +- build_tools/build_detect_platform | 4 +- build_tools/dependencies_platform009.sh | 1 + build_tools/fbcode_config.sh | 2 - build_tools/fbcode_config_platform009.sh | 4 +- cache/cache_entry_roles.cc | 6 +- cache/cache_entry_roles.h | 4 +- cache/cache_reservation_manager_test.cc | 9 +- db/column_family.h | 11 +- db/db_impl/db_impl.cc | 9 +- db/internal_stats.cc | 3 +- db/internal_stats.h | 5 +- db/memtable.h | 4 +- db/version_set.h | 6 +- memory/memory_usage.h | 10 + port/lang.h | 4 + src.mk | 11 +- table/block_based/block_based_table_reader.cc | 2 +- table/block_based/block_based_table_reader.h | 5 +- table/block_based/cachable_entry.h | 41 +- table/block_based/filter_policy.cc | 6 +- table/block_based/partitioned_filter_block.h | 4 +- table/block_based/partitioned_index_reader.cc | 2 +- table/block_based/partitioned_index_reader.h | 3 +- table/plain/plain_table_bloom.h | 11 +- third-party/folly/folly/CPortability.h | 27 - third-party/folly/folly/ConstexprMath.h | 45 - third-party/folly/folly/Indestructible.h | 166 -- third-party/folly/folly/Optional.h | 570 ------ third-party/folly/folly/Portability.h | 99 - third-party/folly/folly/ScopeGuard.h | 54 - third-party/folly/folly/Traits.h | 152 -- third-party/folly/folly/Unit.h | 59 - third-party/folly/folly/Utility.h | 141 -- third-party/folly/folly/chrono/Hardware.h | 33 - third-party/folly/folly/container/Array.h | 74 - third-party/folly/folly/detail/Futex-inl.h | 117 -- third-party/folly/folly/detail/Futex.cpp | 259 --- third-party/folly/folly/detail/Futex.h | 96 - third-party/folly/folly/functional/Invoke.h | 40 - third-party/folly/folly/hash/Hash.h | 29 - third-party/folly/folly/lang/Align.h | 144 -- third-party/folly/folly/lang/Bits.h | 30 - third-party/folly/folly/lang/Launder.h | 51 - third-party/folly/folly/portability/Asm.h | 28 - .../folly/folly/portability/SysSyscall.h | 10 - .../folly/folly/portability/SysTypes.h | 26 - .../synchronization/AtomicNotification-inl.h | 138 -- .../synchronization/AtomicNotification.cpp | 23 - .../synchronization/AtomicNotification.h | 57 - .../folly/synchronization/AtomicUtil-inl.h | 260 --- .../folly/folly/synchronization/AtomicUtil.h | 52 - .../folly/folly/synchronization/Baton.h | 328 ---- .../synchronization/DistributedMutex-inl.h | 1704 ----------------- .../synchronization/DistributedMutex.cpp | 16 - .../folly/synchronization/DistributedMutex.h | 304 --- .../DistributedMutexSpecializations.h | 39 - .../folly/synchronization/ParkingLot.cpp | 26 - .../folly/folly/synchronization/ParkingLot.h | 318 --- .../folly/synchronization/WaitOptions.cpp | 12 - .../folly/folly/synchronization/WaitOptions.h | 57 - .../detail/InlineFunctionRef.h | 219 --- .../detail/ProxyLockable-inl.h | 207 -- .../synchronization/detail/ProxyLockable.h | 164 -- .../folly/synchronization/detail/Sleeper.h | 57 - .../folly/folly/synchronization/detail/Spin.h | 77 - .../test/DistributedMutexTest.cpp | 1145 ----------- util/hash_containers.h | 51 + util/hash_test.cc | 9 + util/math.h | 11 + utilities/cache_dump_load_impl.h | 4 +- .../lock/point/point_lock_manager.cc | 7 +- .../lock/point/point_lock_manager.h | 3 +- 78 files changed, 228 insertions(+), 7588 deletions(-) delete mode 100644 third-party/folly/folly/CPortability.h delete mode 100644 third-party/folly/folly/ConstexprMath.h delete mode 100644 third-party/folly/folly/Indestructible.h delete mode 100644 third-party/folly/folly/Optional.h delete mode 100644 third-party/folly/folly/Portability.h delete mode 100644 third-party/folly/folly/ScopeGuard.h delete mode 100644 third-party/folly/folly/Traits.h delete mode 100644 third-party/folly/folly/Unit.h delete mode 100644 third-party/folly/folly/Utility.h delete mode 100644 third-party/folly/folly/chrono/Hardware.h delete mode 100644 third-party/folly/folly/container/Array.h delete mode 100644 third-party/folly/folly/detail/Futex-inl.h delete mode 100644 third-party/folly/folly/detail/Futex.cpp delete mode 100644 third-party/folly/folly/detail/Futex.h delete mode 100644 third-party/folly/folly/functional/Invoke.h delete mode 100644 third-party/folly/folly/hash/Hash.h delete mode 100644 third-party/folly/folly/lang/Align.h delete mode 100644 third-party/folly/folly/lang/Bits.h delete mode 100644 third-party/folly/folly/lang/Launder.h delete mode 100644 third-party/folly/folly/portability/Asm.h delete mode 100644 third-party/folly/folly/portability/SysSyscall.h delete mode 100644 third-party/folly/folly/portability/SysTypes.h delete mode 100644 third-party/folly/folly/synchronization/AtomicNotification-inl.h delete mode 100644 third-party/folly/folly/synchronization/AtomicNotification.cpp delete mode 100644 third-party/folly/folly/synchronization/AtomicNotification.h delete mode 100644 third-party/folly/folly/synchronization/AtomicUtil-inl.h delete mode 100644 third-party/folly/folly/synchronization/AtomicUtil.h delete mode 100644 third-party/folly/folly/synchronization/Baton.h delete mode 100644 third-party/folly/folly/synchronization/DistributedMutex-inl.h delete mode 100644 third-party/folly/folly/synchronization/DistributedMutex.cpp delete mode 100644 third-party/folly/folly/synchronization/DistributedMutex.h delete mode 100644 third-party/folly/folly/synchronization/DistributedMutexSpecializations.h delete mode 100644 third-party/folly/folly/synchronization/ParkingLot.cpp delete mode 100644 third-party/folly/folly/synchronization/ParkingLot.h delete mode 100644 third-party/folly/folly/synchronization/WaitOptions.cpp delete mode 100644 third-party/folly/folly/synchronization/WaitOptions.h delete mode 100644 third-party/folly/folly/synchronization/detail/InlineFunctionRef.h delete mode 100644 third-party/folly/folly/synchronization/detail/ProxyLockable-inl.h delete mode 100644 third-party/folly/folly/synchronization/detail/ProxyLockable.h delete mode 100644 third-party/folly/folly/synchronization/detail/Sleeper.h delete mode 100644 third-party/folly/folly/synchronization/detail/Spin.h delete mode 100644 third-party/folly/folly/synchronization/test/DistributedMutexTest.cpp create mode 100644 util/hash_containers.h diff --git a/.circleci/config.yml b/.circleci/config.yml index 3232c9530..be32203c2 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -368,7 +368,7 @@ jobs: - run: CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 CLANG_ANALYZER="/usr/bin/clang++-10" CLANG_SCAN_BUILD=scan-build-10 USE_CLANG=1 make V=1 -j32 analyze # aligned new doesn't work for reason we haven't figured out. For unknown, reason passing "clang++-10" as CLANG_ANALYZER doesn't work, and we need a full path. - post-steps - build-linux-cmake: + build-linux-cmake-with-folly: machine: image: ubuntu-2004:202111-02 resource_class: 2xlarge @@ -376,10 +376,11 @@ jobs: - pre-steps - install-gflags - upgrade-cmake - - run: (mkdir build && cd build && cmake -DWITH_GFLAGS=1 .. && make V=1 -j20 && ctest -j20) + - run: make checkout_folly + - run: (mkdir build && cd build && cmake -DUSE_FOLLY=1 -DWITH_GFLAGS=1 .. && make V=1 -j20 && ctest -j20) - post-steps - build-linux-cmake-ubuntu-20: + build-linux-cmake-with-benchmark: machine: image: ubuntu-2004:202111-02 resource_class: 2xlarge @@ -401,14 +402,15 @@ jobs: - run: make V=1 -j8 -k check-headers # could be moved to a different build - post-steps - build-linux-gcc-7: + build-linux-gcc-7-with-folly: machine: image: ubuntu-2004:202111-02 resource_class: 2xlarge steps: - pre-steps - run: sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test && sudo apt-get update -y && sudo apt-get install gcc-7 g++-7 libgflags-dev - - run: CC=gcc-7 CXX=g++-7 V=1 make -j32 check + - run: make checkout_folly + - run: USE_FOLLY=1 CC=gcc-7 CXX=g++-7 V=1 make -j32 check - post-steps build-linux-gcc-8-no_test_run: @@ -799,8 +801,8 @@ workflows: - build-linux build-linux-cmake: jobs: - - build-linux-cmake - - build-linux-cmake-ubuntu-20 + - build-linux-cmake-with-folly + - build-linux-cmake-with-benchmark build-linux-encrypted-env: jobs: - build-linux-encrypted-env @@ -871,7 +873,7 @@ workflows: jobs: - build-linux-clang-no_test_run - build-linux-clang-13-no_test_run - - build-linux-gcc-7 + - build-linux-gcc-7-with-folly - build-linux-gcc-8-no_test_run - build-linux-gcc-10-cxx20-no_test_run - build-linux-gcc-11-no_test_run diff --git a/.gitignore b/.gitignore index 663181cca..7d6f61aff 100644 --- a/.gitignore +++ b/.gitignore @@ -95,3 +95,4 @@ fuzz/proto/gen/ fuzz/crash-* cmake-build-* +third-party/folly/ diff --git a/CMakeLists.txt b/CMakeLists.txt index 31d2eaa4b..1923d1707 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -78,19 +78,6 @@ if ($ENV{CIRCLECI}) add_definitions(-DCIRCLECI) endif() -# third-party/folly is only validated to work on Linux and Windows for now. -# So only turn it on there by default. -if(CMAKE_SYSTEM_NAME MATCHES "Linux|Windows") - if(MSVC AND MSVC_VERSION LESS 1910) - # Folly does not compile with MSVC older than VS2017 - option(WITH_FOLLY_DISTRIBUTED_MUTEX "build with folly::DistributedMutex" OFF) - else() - option(WITH_FOLLY_DISTRIBUTED_MUTEX "build with folly::DistributedMutex" ON) - endif() -else() - option(WITH_FOLLY_DISTRIBUTED_MUTEX "build with folly::DistributedMutex" OFF) -endif() - if( NOT DEFINED CMAKE_CXX_STANDARD ) set(CMAKE_CXX_STANDARD 17) endif() @@ -594,8 +581,9 @@ endif() include_directories(${PROJECT_SOURCE_DIR}) include_directories(${PROJECT_SOURCE_DIR}/include) -if(WITH_FOLLY_DISTRIBUTED_MUTEX) +if(USE_FOLLY) include_directories(${PROJECT_SOURCE_DIR}/third-party/folly) + add_definitions(-DUSE_FOLLY -DFOLLY_NO_CONFIG) endif() find_package(Threads REQUIRED) @@ -977,13 +965,12 @@ else() env/io_posix.cc) endif() -if(WITH_FOLLY_DISTRIBUTED_MUTEX) +if(USE_FOLLY) list(APPEND SOURCES - third-party/folly/folly/detail/Futex.cpp - third-party/folly/folly/synchronization/AtomicNotification.cpp - third-party/folly/folly/synchronization/DistributedMutex.cpp - third-party/folly/folly/synchronization/ParkingLot.cpp - third-party/folly/folly/synchronization/WaitOptions.cpp) + third-party/folly/folly/container/detail/F14Table.cpp + third-party/folly/folly/lang/SafeAssert.cpp + third-party/folly/folly/lang/ToAscii.cpp + third-party/folly/folly/ScopeGuard.cpp) endif() set(ROCKSDB_STATIC_LIB rocksdb${ARTIFACT_SUFFIX}) @@ -1379,10 +1366,6 @@ if(WITH_TESTS) ) endif() - if(WITH_FOLLY_DISTRIBUTED_MUTEX) - list(APPEND TESTS third-party/folly/folly/synchronization/test/DistributedMutexTest.cpp) - endif() - set(TESTUTIL_SOURCE db/db_test_util.cc monitoring/thread_status_updater_debug.cc diff --git a/Makefile b/Makefile index e9ef93b63..b251fdc16 100644 --- a/Makefile +++ b/Makefile @@ -400,6 +400,10 @@ ifndef DISABLE_JEMALLOC ifdef JEMALLOC PLATFORM_CXXFLAGS += -DROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE PLATFORM_CCFLAGS += -DROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE + ifeq ($(USE_FOLLY),1) + PLATFORM_CXXFLAGS += -DUSE_JEMALLOC + PLATFORM_CCFLAGS += -DUSE_JEMALLOC + endif endif ifdef WITH_JEMALLOC_FLAG PLATFORM_LDFLAGS += -ljemalloc @@ -410,8 +414,8 @@ ifndef DISABLE_JEMALLOC PLATFORM_CCFLAGS += $(JEMALLOC_INCLUDE) endif -ifndef USE_FOLLY_DISTRIBUTED_MUTEX - USE_FOLLY_DISTRIBUTED_MUTEX=0 +ifndef USE_FOLLY + USE_FOLLY=0 endif ifndef GTEST_THROW_ON_FAILURE @@ -431,8 +435,12 @@ else PLATFORM_CXXFLAGS += -isystem $(GTEST_DIR) endif -ifeq ($(USE_FOLLY_DISTRIBUTED_MUTEX),1) - FOLLY_DIR = ./third-party/folly +# This provides a Makefile simulation of a Meta-internal folly integration. +# It is not validated for general use. +ifeq ($(USE_FOLLY),1) + ifeq (,$(FOLLY_DIR)) + FOLLY_DIR = ./third-party/folly + endif # AIX: pre-defined system headers are surrounded by an extern "C" block ifeq ($(PLATFORM), OS_AIX) PLATFORM_CCFLAGS += -I$(FOLLY_DIR) @@ -441,6 +449,8 @@ ifeq ($(USE_FOLLY_DISTRIBUTED_MUTEX),1) PLATFORM_CCFLAGS += -isystem $(FOLLY_DIR) PLATFORM_CXXFLAGS += -isystem $(FOLLY_DIR) endif + PLATFORM_CCFLAGS += -DUSE_FOLLY -DFOLLY_NO_CONFIG + PLATFORM_CXXFLAGS += -DUSE_FOLLY -DFOLLY_NO_CONFIG endif ifdef TEST_CACHE_LINE_SIZE @@ -527,7 +537,7 @@ LIB_OBJECTS += $(patsubst %.c, $(OBJ_DIR)/%.o, $(LIB_SOURCES_C)) LIB_OBJECTS += $(patsubst %.S, $(OBJ_DIR)/%.o, $(LIB_SOURCES_ASM)) endif -ifeq ($(USE_FOLLY_DISTRIBUTED_MUTEX),1) +ifeq ($(USE_FOLLY),1) LIB_OBJECTS += $(patsubst %.cpp, $(OBJ_DIR)/%.o, $(FOLLY_SOURCES)) endif @@ -562,11 +572,6 @@ ALL_SOURCES += $(ROCKSDB_PLUGIN_SOURCES) TESTS = $(patsubst %.cc, %, $(notdir $(TEST_MAIN_SOURCES))) TESTS += $(patsubst %.c, %, $(notdir $(TEST_MAIN_SOURCES_C))) -ifeq ($(USE_FOLLY_DISTRIBUTED_MUTEX),1) - TESTS += folly_synchronization_distributed_mutex_test - ALL_SOURCES += third-party/folly/folly/synchronization/test/DistributedMutexTest.cc -endif - # `make check-headers` to very that each header file includes its own # dependencies ifneq ($(filter check-headers, $(MAKECMDGOALS)),) @@ -791,7 +796,7 @@ endif # PLATFORM_SHARED_EXT .PHONY: blackbox_crash_test check clean coverage crash_test ldb_tests package \ release tags tags0 valgrind_check whitebox_crash_test format static_lib shared_lib all \ dbg rocksdbjavastatic rocksdbjava gen-pc install install-static install-shared uninstall \ - analyze tools tools_lib check-headers \ + analyze tools tools_lib check-headers checkout_folly \ blackbox_crash_test_with_atomic_flush whitebox_crash_test_with_atomic_flush \ blackbox_crash_test_with_txn whitebox_crash_test_with_txn \ blackbox_crash_test_with_best_efforts_recovery \ @@ -1306,11 +1311,6 @@ trace_analyzer: $(OBJ_DIR)/tools/trace_analyzer.o $(ANALYZE_OBJECTS) $(TOOLS_LIB block_cache_trace_analyzer: $(OBJ_DIR)/tools/block_cache_analyzer/block_cache_trace_analyzer_tool.o $(ANALYZE_OBJECTS) $(TOOLS_LIBRARY) $(LIBRARY) $(AM_LINK) -ifeq ($(USE_FOLLY_DISTRIBUTED_MUTEX),1) -folly_synchronization_distributed_mutex_test: $(OBJ_DIR)/third-party/folly/folly/synchronization/test/DistributedMutexTest.o $(TEST_LIBRARY) $(LIBRARY) - $(AM_LINK) -endif - cache_bench: $(OBJ_DIR)/cache/cache_bench.o $(CACHE_BENCH_OBJECTS) $(LIBRARY) $(AM_LINK) @@ -2383,6 +2383,22 @@ commit_prereq: false # J=$(J) build_tools/precommit_checker.py unit clang_unit release clang_release tsan asan ubsan lite unit_non_shm # $(MAKE) clean && $(MAKE) jclean && $(MAKE) rocksdbjava; +# For public CI runs, checkout folly in a way that can build with RocksDB. +# This is mostly intended as a test-only simulation of Meta-internal folly +# integration. +checkout_folly: + if [ -e third-party/folly ]; then \ + cd third-party/folly && git fetch origin; \ + else \ + cd third-party && git clone https://github.com/facebook/folly.git; \ + fi + @# Pin to a particular version for public CI, so that PR authors don't + @# need to worry about folly breaking our integration. Update periodically + cd third-party/folly && git reset --hard 98b9b2c1124e99f50f9085ddee74ce32afffc665 + @# A hack to remove boost dependency. + @# NOTE: this hack is not needed if using FBCODE compiler config + perl -pi -e 's/^(#include > "$OUTPUT" fi echo "LUA_PATH=$LUA_PATH" >> "$OUTPUT" -if test -n "$USE_FOLLY_DISTRIBUTED_MUTEX"; then - echo "USE_FOLLY_DISTRIBUTED_MUTEX=$USE_FOLLY_DISTRIBUTED_MUTEX" >> "$OUTPUT" +if test -n "$USE_FOLLY"; then + echo "USE_FOLLY=$USE_FOLLY" >> "$OUTPUT" fi if test -n "$PPC_LIBC_IS_GNU"; then echo "PPC_LIBC_IS_GNU=$PPC_LIBC_IS_GNU" >> "$OUTPUT" diff --git a/build_tools/dependencies_platform009.sh b/build_tools/dependencies_platform009.sh index 323fb771f..427b62da4 100644 --- a/build_tools/dependencies_platform009.sh +++ b/build_tools/dependencies_platform009.sh @@ -19,3 +19,4 @@ BINUTILS_BASE=/mnt/gvfs/third-party2/binutils/08634589372fa5f237bfd374e8c644a836 VALGRIND_BASE=/mnt/gvfs/third-party2/valgrind/6ae525939ad02e5e676855082fbbc7828dbafeac/3.15.0/platform009/7f3b187 LUA_BASE=/mnt/gvfs/third-party2/lua/162efd9561a3d21f6869f4814011e9cf1b3ff4dc/5.3.4/platform009/a6271c4 BENCHMARK_BASE=/mnt/gvfs/third-party2/benchmark/30bf49ad6414325e17f3425b0edcb64239427ae3/1.6.1/platform009/7f3b187 +BOOST_BASE=/mnt/gvfs/third-party2/boost/201b7d74941e54b436dfa364a063aa6d2cd7de4c/1.69.0/platform009/8a7ffdf diff --git a/build_tools/fbcode_config.sh b/build_tools/fbcode_config.sh index 2f56daa08..060c88ad8 100644 --- a/build_tools/fbcode_config.sh +++ b/build_tools/fbcode_config.sh @@ -162,6 +162,4 @@ else LUA_LIB=" $LUA_PATH/lib/liblua_pic.a" fi -USE_FOLLY_DISTRIBUTED_MUTEX=1 - export CC CXX AR CFLAGS CXXFLAGS EXEC_LDFLAGS EXEC_LDFLAGS_SHARED VALGRIND_VER JEMALLOC_LIB JEMALLOC_INCLUDE CLANG_ANALYZER CLANG_SCAN_BUILD LUA_PATH LUA_LIB diff --git a/build_tools/fbcode_config_platform009.sh b/build_tools/fbcode_config_platform009.sh index e9cae8684..89ae4f098 100644 --- a/build_tools/fbcode_config_platform009.sh +++ b/build_tools/fbcode_config_platform009.sh @@ -58,6 +58,8 @@ CFLAGS+=" -DGFLAGS=gflags" BENCHMARK_INCLUDE=" -I $BENCHMARK_BASE/include/" BENCHMARK_LIBS=" $BENCHMARK_BASE/lib/libbenchmark${MAYBE_PIC}.a" +BOOST_INCLUDE=" -I $BOOST_BASE/include/" + # location of jemalloc JEMALLOC_INCLUDE=" -I $JEMALLOC_BASE/include/" JEMALLOC_LIB=" $JEMALLOC_BASE/lib/libjemalloc${MAYBE_PIC}.a" @@ -89,7 +91,7 @@ BINUTILS="$BINUTILS_BASE/bin" AR="$BINUTILS/ar" AS="$BINUTILS/as" -DEPS_INCLUDE="$SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $ZSTD_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE $TBB_INCLUDE $LIBURING_INCLUDE $BENCHMARK_INCLUDE" +DEPS_INCLUDE="$SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $ZSTD_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE $TBB_INCLUDE $LIBURING_INCLUDE $BENCHMARK_INCLUDE $BOOST_INCLUDE" STDLIBS="-L $GCC_BASE/lib64" diff --git a/cache/cache_entry_roles.cc b/cache/cache_entry_roles.cc index ea6bfe3fb..19af4996c 100644 --- a/cache/cache_entry_roles.cc +++ b/cache/cache_entry_roles.cc @@ -43,12 +43,12 @@ namespace { struct Registry { std::mutex mutex; - std::unordered_map role_map; + UnorderedMap role_map; void Register(Cache::DeleterFn fn, CacheEntryRole role) { std::lock_guard lock(mutex); role_map[fn] = role; } - std::unordered_map Copy() { + UnorderedMap Copy() { std::lock_guard lock(mutex); return role_map; } @@ -65,7 +65,7 @@ void RegisterCacheDeleterRole(Cache::DeleterFn fn, CacheEntryRole role) { GetRegistry().Register(fn, role); } -std::unordered_map CopyCacheDeleterRoleMap() { +UnorderedMap CopyCacheDeleterRoleMap() { return GetRegistry().Copy(); } diff --git a/cache/cache_entry_roles.h b/cache/cache_entry_roles.h index acc3e4d4a..d46abf207 100644 --- a/cache/cache_entry_roles.h +++ b/cache/cache_entry_roles.h @@ -9,9 +9,9 @@ #include #include #include -#include #include "rocksdb/cache.h" +#include "util/hash_containers.h" namespace ROCKSDB_NAMESPACE { @@ -78,7 +78,7 @@ void RegisterCacheDeleterRole(Cache::DeleterFn fn, CacheEntryRole role); // * This is suitable for preparing for batch operations, like with // CacheEntryStatsCollector. // * The number of mappings should be sufficiently small (dozens). -std::unordered_map CopyCacheDeleterRoleMap(); +UnorderedMap CopyCacheDeleterRoleMap(); // ************************************************************** // // An automatic registration infrastructure. This enables code diff --git a/cache/cache_reservation_manager_test.cc b/cache/cache_reservation_manager_test.cc index 0ed00a008..87af653bc 100644 --- a/cache/cache_reservation_manager_test.cc +++ b/cache/cache_reservation_manager_test.cc @@ -47,13 +47,14 @@ TEST_F(CacheReservationManagerTest, GenerateCacheKey) { // Next unique Cache key CacheKey ckey = CacheKey::CreateUniqueForCacheLifetime(cache.get()); - // Back it up to the one used by CRM (using CacheKey implementation details) - using PairU64 = std::pair; + // Get to the underlying values + using PairU64 = std::array; auto& ckey_pair = *reinterpret_cast(&ckey); - ckey_pair.second--; + // Back it up to the one used by CRM (using CacheKey implementation details) + ckey_pair[1]--; // Specific key (subject to implementation details) - EXPECT_EQ(ckey_pair, PairU64(0, 2)); + EXPECT_EQ(ckey_pair, PairU64({0, 2})); Cache::Handle* handle = cache->Lookup(ckey.AsSlice()); EXPECT_NE(handle, nullptr) diff --git a/db/column_family.h b/db/column_family.h index b854633e2..c37430366 100644 --- a/db/column_family.h +++ b/db/column_family.h @@ -9,10 +9,10 @@ #pragma once -#include -#include -#include #include +#include +#include +#include #include "db/memtable_list.h" #include "db/table_cache.h" @@ -25,6 +25,7 @@ #include "rocksdb/env.h" #include "rocksdb/options.h" #include "trace_replay/block_cache_tracer.h" +#include "util/hash_containers.h" #include "util/thread_local.h" namespace ROCKSDB_NAMESPACE { @@ -705,8 +706,8 @@ class ColumnFamilySet { // * when reading, at least one condition needs to be satisfied: // 1. DB mutex locked // 2. accessed from a single-threaded write thread - std::unordered_map column_families_; - std::unordered_map column_family_data_; + UnorderedMap column_families_; + UnorderedMap column_family_data_; uint32_t max_column_family_; const FileOptions file_options_; diff --git a/db/db_impl/db_impl.cc b/db/db_impl/db_impl.cc index e4b275e56..9ec1677b8 100644 --- a/db/db_impl/db_impl.cc +++ b/db/db_impl/db_impl.cc @@ -101,6 +101,7 @@ #include "util/compression.h" #include "util/crc32c.h" #include "util/defer.h" +#include "util/hash_containers.h" #include "util/mutexlock.h" #include "util/stop_watch.h" #include "util/string_util.h" @@ -2000,7 +2001,7 @@ std::vector DBImpl::MultiGet( SequenceNumber consistent_seqnum; - std::unordered_map multiget_cf_data( + UnorderedMap multiget_cf_data( column_family.size()); for (auto cf : column_family) { auto cfh = static_cast_with_check(cf); @@ -2012,13 +2013,13 @@ std::vector DBImpl::MultiGet( } std::function::iterator&)> + UnorderedMap::iterator&)> iter_deref_lambda = - [](std::unordered_map::iterator& + [](UnorderedMap::iterator& cf_iter) { return &cf_iter->second; }; bool unref_only = - MultiCFSnapshot>( + MultiCFSnapshot>( read_options, nullptr, iter_deref_lambda, &multiget_cf_data, &consistent_seqnum); diff --git a/db/internal_stats.cc b/db/internal_stats.cc index 68e7ab5ee..a3d8113aa 100644 --- a/db/internal_stats.cc +++ b/db/internal_stats.cc @@ -27,6 +27,7 @@ #include "rocksdb/system_clock.h" #include "rocksdb/table.h" #include "table/block_based/cachable_entry.h" +#include "util/hash_containers.h" #include "util/string_util.h" namespace ROCKSDB_NAMESPACE { @@ -405,7 +406,7 @@ const std::string DB::Properties::kTotalBlobFileSize = const std::string DB::Properties::kLiveBlobFileSize = rocksdb_prefix + live_blob_file_size; -const std::unordered_map +const UnorderedMap InternalStats::ppt_name_to_info = { {DB::Properties::kNumFilesAtLevelPrefix, {false, &InternalStats::HandleNumFilesAtLevel, nullptr, nullptr, diff --git a/db/internal_stats.h b/db/internal_stats.h index 9e100c12f..dbf15d98e 100644 --- a/db/internal_stats.h +++ b/db/internal_stats.h @@ -18,6 +18,7 @@ #include "cache/cache_entry_roles.h" #include "db/version_set.h" #include "rocksdb/system_clock.h" +#include "util/hash_containers.h" class ColumnFamilyData; @@ -387,7 +388,7 @@ class InternalStats { SystemClock* clock) const; private: - std::unordered_map role_map_; + UnorderedMap role_map_; uint64_t GetLastDurationMicros() const; }; @@ -482,7 +483,7 @@ class InternalStats { // Store a mapping from the user-facing DB::Properties string to our // DBPropertyInfo struct used internally for retrieving properties. - static const std::unordered_map ppt_name_to_info; + static const UnorderedMap ppt_name_to_info; private: void DumpDBMapStats(std::map* db_stats); diff --git a/db/memtable.h b/db/memtable.h index 52cfcc9d2..965404d25 100644 --- a/db/memtable.h +++ b/db/memtable.h @@ -13,7 +13,6 @@ #include #include #include -#include #include #include @@ -31,6 +30,7 @@ #include "table/multiget_context.h" #include "util/dynamic_bloom.h" #include "util/hash.h" +#include "util/hash_containers.h" namespace ROCKSDB_NAMESPACE { @@ -566,7 +566,7 @@ class MemTable { const SliceTransform* insert_with_hint_prefix_extractor_; // Insert hints for each prefix. - std::unordered_map insert_hints_; + UnorderedMapH insert_hints_; // Timestamp of oldest key std::atomic oldest_key_time_; diff --git a/db/version_set.h b/db/version_set.h index eeabb53a0..abb4046c7 100644 --- a/db/version_set.h +++ b/db/version_set.h @@ -54,6 +54,7 @@ #include "table/get_context.h" #include "table/multiget_context.h" #include "trace_replay/block_cache_tracer.h" +#include "util/hash_containers.h" namespace ROCKSDB_NAMESPACE { @@ -579,7 +580,7 @@ class VersionStorageInfo { // Map of all table files in version. Maps file number to (level, position on // level). - using FileLocations = std::unordered_map; + using FileLocations = UnorderedMap; FileLocations file_locations_; // Vector of blob files in version sorted by blob file number. @@ -1344,8 +1345,7 @@ class VersionSet { protected: using VersionBuilderMap = - std::unordered_map>; + UnorderedMap>; struct ManifestWriter; diff --git a/memory/memory_usage.h b/memory/memory_usage.h index 7f1de8c97..76b9bd130 100644 --- a/memory/memory_usage.h +++ b/memory/memory_usage.h @@ -7,6 +7,9 @@ #include #include +#ifdef USE_FOLLY +#include +#endif #include "rocksdb/rocksdb_namespace.h" @@ -25,4 +28,11 @@ size_t ApproximateMemoryUsage( umap.bucket_count() * sizeof(void*); } +#ifdef USE_FOLLY +template +size_t ApproximateMemoryUsage(const folly::F14FastMap& umap) { + return sizeof(umap) + umap.getAllocatedMemorySize(); +} +#endif + } // namespace ROCKSDB_NAMESPACE diff --git a/port/lang.h b/port/lang.h index 5d1067e1f..754f99bf2 100644 --- a/port/lang.h +++ b/port/lang.h @@ -15,6 +15,10 @@ #endif #endif +#define DECLARE_DEFAULT_MOVES(Name) \ + Name(Name&&) noexcept = default; \ + Name& operator=(Name&&) = default + // ASAN (Address sanitizer) #if defined(__clang__) diff --git a/src.mk b/src.mk index c829b9f12..9e1fa5e0f 100644 --- a/src.mk +++ b/src.mk @@ -366,12 +366,11 @@ TEST_LIB_SOURCES = \ test_util/testutil.cc \ utilities/cassandra/test_utils.cc \ -FOLLY_SOURCES = \ - third-party/folly/folly/detail/Futex.cpp \ - third-party/folly/folly/synchronization/AtomicNotification.cpp \ - third-party/folly/folly/synchronization/DistributedMutex.cpp \ - third-party/folly/folly/synchronization/ParkingLot.cpp \ - third-party/folly/folly/synchronization/WaitOptions.cpp \ +FOLLY_SOURCES = \ + $(FOLLY_DIR)/folly/container/detail/F14Table.cpp \ + $(FOLLY_DIR)/folly/lang/SafeAssert.cpp \ + $(FOLLY_DIR)/folly/lang/ToAscii.cpp \ + $(FOLLY_DIR)/folly/ScopeGuard.cpp \ TOOLS_MAIN_SOURCES = \ db_stress_tool/db_stress.cc \ diff --git a/table/block_based/block_based_table_reader.cc b/table/block_based/block_based_table_reader.cc index 51ae6aa77..99bb02001 100644 --- a/table/block_based/block_based_table_reader.cc +++ b/table/block_based/block_based_table_reader.cc @@ -2061,7 +2061,7 @@ template Status BlockBasedTable::RetrieveBlock( BlockBasedTable::PartitionedIndexIteratorState::PartitionedIndexIteratorState( const BlockBasedTable* table, - std::unordered_map>* block_map) + UnorderedMap>* block_map) : table_(table), block_map_(block_map) {} InternalIteratorBase* diff --git a/table/block_based/block_based_table_reader.h b/table/block_based/block_based_table_reader.h index 2ca1ff08e..e702de427 100644 --- a/table/block_based/block_based_table_reader.h +++ b/table/block_based/block_based_table_reader.h @@ -31,6 +31,7 @@ #include "table/table_reader.h" #include "table/two_level_iterator.h" #include "trace_replay/block_cache_tracer.h" +#include "util/hash_containers.h" namespace ROCKSDB_NAMESPACE { @@ -528,14 +529,14 @@ class BlockBasedTable::PartitionedIndexIteratorState public: PartitionedIndexIteratorState( const BlockBasedTable* table, - std::unordered_map>* block_map); + UnorderedMap>* block_map); InternalIteratorBase* NewSecondaryIterator( const BlockHandle& index_value) override; private: // Don't own table_ const BlockBasedTable* table_; - std::unordered_map>* block_map_; + UnorderedMap>* block_map_; }; // Stores all the properties associated with a BlockBasedTable. diff --git a/table/block_based/cachable_entry.h b/table/block_based/cachable_entry.h index 155097c05..eb51e98da 100644 --- a/table/block_based/cachable_entry.h +++ b/table/block_based/cachable_entry.h @@ -59,12 +59,11 @@ public: CachableEntry(const CachableEntry&) = delete; CachableEntry& operator=(const CachableEntry&) = delete; - CachableEntry(CachableEntry&& rhs) - : value_(rhs.value_) - , cache_(rhs.cache_) - , cache_handle_(rhs.cache_handle_) - , own_value_(rhs.own_value_) - { + CachableEntry(CachableEntry&& rhs) noexcept + : value_(rhs.value_), + cache_(rhs.cache_), + cache_handle_(rhs.cache_handle_), + own_value_(rhs.own_value_) { assert(value_ != nullptr || (cache_ == nullptr && cache_handle_ == nullptr && !own_value_)); assert(!!cache_ == !!cache_handle_); @@ -73,7 +72,7 @@ public: rhs.ResetFields(); } - CachableEntry& operator=(CachableEntry&& rhs) { + CachableEntry& operator=(CachableEntry&& rhs) noexcept { if (UNLIKELY(this == &rhs)) { return *this; } @@ -195,21 +194,21 @@ public: } private: - void ReleaseResource() { - if (LIKELY(cache_handle_ != nullptr)) { - assert(cache_ != nullptr); - cache_->Release(cache_handle_); - } else if (own_value_) { - delete value_; - } - } + void ReleaseResource() noexcept { + if (LIKELY(cache_handle_ != nullptr)) { + assert(cache_ != nullptr); + cache_->Release(cache_handle_); + } else if (own_value_) { + delete value_; + } + } - void ResetFields() { - value_ = nullptr; - cache_ = nullptr; - cache_handle_ = nullptr; - own_value_ = false; - } + void ResetFields() noexcept { + value_ = nullptr; + cache_ = nullptr; + cache_handle_ = nullptr; + own_value_ = false; + } static void ReleaseCacheHandle(void* arg1, void* arg2) { Cache* const cache = static_cast(arg1); diff --git a/table/block_based/filter_policy.cc b/table/block_based/filter_policy.cc index 48c49242c..6ac4b9142 100644 --- a/table/block_based/filter_policy.cc +++ b/table/block_based/filter_policy.cc @@ -28,10 +28,10 @@ #include "table/block_based/block_based_table_reader.h" #include "table/block_based/filter_policy_internal.h" #include "table/block_based/full_filter_block.h" -#include "third-party/folly/folly/ConstexprMath.h" #include "util/bloom_impl.h" #include "util/coding.h" #include "util/hash.h" +#include "util/math.h" #include "util/ribbon_config.h" #include "util/ribbon_impl.h" #include "util/string_util.h" @@ -1203,7 +1203,7 @@ inline void LegacyBloomBitsBuilder::AddHash(uint32_t h, char* data, assert(num_lines > 0 && total_bits > 0); LegacyBloomImpl::AddHash(h, num_lines, num_probes_, data, - folly::constexpr_log2(CACHE_LINE_SIZE)); + ConstexprFloorLog2(CACHE_LINE_SIZE)); } class LegacyBloomBitsReader : public BuiltinFilterBitsReader { @@ -1671,7 +1671,7 @@ BuiltinFilterBitsReader* BuiltinFilterPolicy::GetBuiltinFilterBitsReader( if (num_lines * CACHE_LINE_SIZE == len) { // Common case - log2_cache_line_size = folly::constexpr_log2(CACHE_LINE_SIZE); + log2_cache_line_size = ConstexprFloorLog2(CACHE_LINE_SIZE); } else if (num_lines == 0 || len % num_lines != 0) { // Invalid (no solution to num_lines * x == len) // Treat as zero probes (always FP) for now. diff --git a/table/block_based/partitioned_filter_block.h b/table/block_based/partitioned_filter_block.h index 555e940ea..302b30f19 100644 --- a/table/block_based/partitioned_filter_block.h +++ b/table/block_based/partitioned_filter_block.h @@ -18,6 +18,7 @@ #include "table/block_based/full_filter_block.h" #include "table/block_based/index_builder.h" #include "util/autovector.h" +#include "util/hash_containers.h" namespace ROCKSDB_NAMESPACE { class InternalKeyComparator; @@ -172,8 +173,7 @@ class PartitionedFilterBlockReader : public FilterBlockReaderCommon { protected: // For partition blocks pinned in cache. Can be a subset of blocks // in case some fail insertion on attempt to pin. - std::unordered_map> - filter_map_; + UnorderedMap> filter_map_; }; } // namespace ROCKSDB_NAMESPACE diff --git a/table/block_based/partitioned_index_reader.cc b/table/block_based/partitioned_index_reader.cc index 8e690a61c..25ea1a3a4 100644 --- a/table/block_based/partitioned_index_reader.cc +++ b/table/block_based/partitioned_index_reader.cc @@ -170,7 +170,7 @@ Status PartitionIndexReader::CacheDependencies(const ReadOptions& ro, } // For saving "all or nothing" to partition_map_ - std::unordered_map> map_in_progress; + UnorderedMap> map_in_progress; // After prefetch, read the partitions one by one biter.SeekToFirst(); diff --git a/table/block_based/partitioned_index_reader.h b/table/block_based/partitioned_index_reader.h index 22a0e3d04..58a7877ab 100644 --- a/table/block_based/partitioned_index_reader.h +++ b/table/block_based/partitioned_index_reader.h @@ -8,6 +8,7 @@ // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once #include "table/block_based/index_reader_common.h" +#include "util/hash_containers.h" namespace ROCKSDB_NAMESPACE { // Index that allows binary search lookup in a two-level index structure. @@ -49,6 +50,6 @@ class PartitionIndexReader : public BlockBasedTable::IndexReaderCommon { // For partition blocks pinned in cache. This is expected to be "all or // none" so that !partition_map_.empty() can use an iterator expecting // all partitions to be saved here. - std::unordered_map> partition_map_; + UnorderedMap> partition_map_; }; } // namespace ROCKSDB_NAMESPACE diff --git a/table/plain/plain_table_bloom.h b/table/plain/plain_table_bloom.h index 5ccc6802b..460e7ec39 100644 --- a/table/plain/plain_table_bloom.h +++ b/table/plain/plain_table_bloom.h @@ -4,18 +4,15 @@ // (found in the LICENSE.Apache file in the root directory). #pragma once +#include #include #include -#include "rocksdb/slice.h" - #include "port/port.h" +#include "rocksdb/slice.h" #include "util/bloom_impl.h" #include "util/hash.h" - -#include "third-party/folly/folly/ConstexprMath.h" - -#include +#include "util/math.h" namespace ROCKSDB_NAMESPACE { class Slice; @@ -69,7 +66,7 @@ class PlainTableBloomV1 { char* data_; static constexpr int LOG2_CACHE_LINE_SIZE = - folly::constexpr_log2(CACHE_LINE_SIZE); + ConstexprFloorLog2(CACHE_LINE_SIZE); }; #if defined(_MSC_VER) diff --git a/third-party/folly/folly/CPortability.h b/third-party/folly/folly/CPortability.h deleted file mode 100644 index 56cb6b1a5..000000000 --- a/third-party/folly/folly/CPortability.h +++ /dev/null @@ -1,27 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -/** - * Macro for marking functions as having public visibility. - */ -#if defined(__GNUC__) -#define FOLLY_EXPORT __attribute__((__visibility__("default"))) -#else -#define FOLLY_EXPORT -#endif - -#if defined(__has_feature) -#define FOLLY_HAS_FEATURE(...) __has_feature(__VA_ARGS__) -#else -#define FOLLY_HAS_FEATURE(...) 0 -#endif - -#if FOLLY_HAS_FEATURE(thread_sanitizer) || __SANITIZE_THREAD__ -#ifndef FOLLY_SANITIZE_THREAD -#define FOLLY_SANITIZE_THREAD 1 -#endif -#endif diff --git a/third-party/folly/folly/ConstexprMath.h b/third-party/folly/folly/ConstexprMath.h deleted file mode 100644 index f09167e0d..000000000 --- a/third-party/folly/folly/ConstexprMath.h +++ /dev/null @@ -1,45 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -namespace folly { -template -constexpr T constexpr_max(T a) { - return a; -} -template -constexpr T constexpr_max(T a, T b, Ts... ts) { - return b < a ? constexpr_max(a, ts...) : constexpr_max(b, ts...); -} - -namespace detail { -template -constexpr T constexpr_log2_(T a, T e) { - return e == T(1) ? a : constexpr_log2_(a + T(1), e / T(2)); -} - -template -constexpr T constexpr_log2_ceil_(T l2, T t) { - return l2 + T(T(1) << l2 < t ? 1 : 0); -} - -template -constexpr T constexpr_square_(T t) { - return t * t; -} -} // namespace detail - -template -constexpr T constexpr_log2(T t) { - return detail::constexpr_log2_(T(0), t); -} - -template -constexpr T constexpr_log2_ceil(T t) { - return detail::constexpr_log2_ceil_(constexpr_log2(t), t); -} - -} // namespace folly diff --git a/third-party/folly/folly/Indestructible.h b/third-party/folly/folly/Indestructible.h deleted file mode 100644 index 68249d865..000000000 --- a/third-party/folly/folly/Indestructible.h +++ /dev/null @@ -1,166 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include -#include -#include - -#include - -namespace folly { - -/*** - * Indestructible - * - * When you need a Meyers singleton that will not get destructed, even at - * shutdown, and you also want the object stored inline. - * - * Use like: - * - * void doSomethingWithExpensiveData(); - * - * void doSomethingWithExpensiveData() { - * static const Indestructible> data{ - * map{{"key1", 17}, {"key2", 19}, {"key3", 23}}, - * }; - * callSomethingTakingAMapByRef(*data); - * } - * - * This should be used only for Meyers singletons, and, even then, only when - * the instance does not need to be destructed ever. - * - * This should not be used more generally, e.g., as member fields, etc. - * - * This is designed as an alternative, but with one fewer allocation at - * construction time and one fewer pointer dereference at access time, to the - * Meyers singleton pattern of: - * - * void doSomethingWithExpensiveData() { - * static const auto data = // never `delete`d - * new map{{"key1", 17}, {"key2", 19}, {"key3", 23}}; - * callSomethingTakingAMapByRef(*data); - * } - */ - -template -class Indestructible final { - public: - template - constexpr Indestructible() noexcept(noexcept(T())) {} - - /** - * Constructor accepting a single argument by forwarding reference, this - * allows using list initialzation without the overhead of things like - * in_place, etc and also works with std::initializer_list constructors - * which can't be deduced, the default parameter helps there. - * - * auto i = folly::Indestructible>{{{1, 2}}}; - * - * This provides convenience - * - * There are two versions of this constructor - one for when the element is - * implicitly constructible from the given argument and one for when the - * type is explicitly but not implicitly constructible from the given - * argument. - */ - template < - typename U = T, - _t::value>>* = nullptr, - _t, remove_cvref_t>::value>>* = - nullptr, - _t::value>>* = nullptr> - explicit constexpr Indestructible(U&& u) noexcept( - noexcept(T(std::declval()))) - : storage_(std::forward(u)) {} - template < - typename U = T, - _t::value>>* = nullptr, - _t, remove_cvref_t>::value>>* = - nullptr, - _t::value>>* = nullptr> - /* implicit */ constexpr Indestructible(U&& u) noexcept( - noexcept(T(std::declval()))) - : storage_(std::forward(u)) {} - - template ()...))> - explicit constexpr Indestructible(Args&&... args) noexcept( - noexcept(T(std::declval()...))) - : storage_(std::forward(args)...) {} - template < - typename U, - typename... Args, - typename = decltype( - T(std::declval&>(), - std::declval()...))> - explicit constexpr Indestructible(std::initializer_list il, Args... args) noexcept( - noexcept( - T(std::declval&>(), - std::declval()...))) - : storage_(il, std::forward(args)...) {} - - ~Indestructible() = default; - - Indestructible(Indestructible const&) = delete; - Indestructible& operator=(Indestructible const&) = delete; - - Indestructible(Indestructible&& other) noexcept( - noexcept(T(std::declval()))) - : storage_(std::move(other.storage_.value)) { - other.erased_ = true; - } - Indestructible& operator=(Indestructible&& other) noexcept( - noexcept(T(std::declval()))) { - storage_.value = std::move(other.storage_.value); - other.erased_ = true; - } - - T* get() noexcept { - check(); - return &storage_.value; - } - T const* get() const noexcept { - check(); - return &storage_.value; - } - T& operator*() noexcept { - return *get(); - } - T const& operator*() const noexcept { - return *get(); - } - T* operator->() noexcept { - return get(); - } - T const* operator->() const noexcept { - return get(); - } - - private: - void check() const noexcept { - assert(!erased_); - } - - union Storage { - T value; - - template - constexpr Storage() noexcept(noexcept(T())) : value() {} - - template ()...))> - explicit constexpr Storage(Args&&... args) noexcept( - noexcept(T(std::declval()...))) - : value(std::forward(args)...) {} - - ~Storage() {} - }; - - Storage storage_{}; - bool erased_{false}; -}; -} // namespace folly diff --git a/third-party/folly/folly/Optional.h b/third-party/folly/folly/Optional.h deleted file mode 100644 index ee12467dd..000000000 --- a/third-party/folly/folly/Optional.h +++ /dev/null @@ -1,570 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -/* - * Optional - For conditional initialization of values, like boost::optional, - * but with support for move semantics and emplacement. Reference type support - * has not been included due to limited use cases and potential confusion with - * semantics of assignment: Assigning to an optional reference could quite - * reasonably copy its value or redirect the reference. - * - * Optional can be useful when a variable might or might not be needed: - * - * Optional maybeLogger = ...; - * if (maybeLogger) { - * maybeLogger->log("hello"); - * } - * - * Optional enables a 'null' value for types which do not otherwise have - * nullability, especially useful for parameter passing: - * - * void testIterator(const unique_ptr& it, - * initializer_list idsExpected, - * Optional> ranksExpected = none) { - * for (int i = 0; it->next(); ++i) { - * EXPECT_EQ(it->doc().id(), idsExpected[i]); - * if (ranksExpected) { - * EXPECT_EQ(it->doc().rank(), (*ranksExpected)[i]); - * } - * } - * } - * - * Optional models OptionalPointee, so calling 'get_pointer(opt)' will return a - * pointer to nullptr if the 'opt' is empty, and a pointer to the value if it is - * not: - * - * Optional maybeInt = ...; - * if (int* v = get_pointer(maybeInt)) { - * cout << *v << endl; - * } - */ - -#include -#include -#include -#include -#include -#include - -#include -#include -#include - -namespace folly { - -template -class Optional; - -namespace detail { -template -struct OptionalPromiseReturn; -} // namespace detail - -struct None { - enum class _secret { _token }; - - /** - * No default constructor to support both `op = {}` and `op = none` - * as syntax for clearing an Optional, just like std::nullopt_t. - */ - constexpr explicit None(_secret) {} -}; -constexpr None none{None::_secret::_token}; - -class FOLLY_EXPORT OptionalEmptyException : public std::runtime_error { - public: - OptionalEmptyException() - : std::runtime_error("Empty Optional cannot be unwrapped") {} -}; - -template -class Optional { - public: - typedef Value value_type; - - static_assert( - !std::is_reference::value, - "Optional may not be used with reference types"); - static_assert( - !std::is_abstract::value, - "Optional may not be used with abstract types"); - - Optional() noexcept {} - - Optional(const Optional& src) noexcept( - std::is_nothrow_copy_constructible::value) { - if (src.hasValue()) { - construct(src.value()); - } - } - - Optional(Optional&& src) noexcept( - std::is_nothrow_move_constructible::value) { - if (src.hasValue()) { - construct(std::move(src.value())); - src.clear(); - } - } - - /* implicit */ Optional(const None&) noexcept {} - - /* implicit */ Optional(Value&& newValue) noexcept( - std::is_nothrow_move_constructible::value) { - construct(std::move(newValue)); - } - - /* implicit */ Optional(const Value& newValue) noexcept( - std::is_nothrow_copy_constructible::value) { - construct(newValue); - } - - template - explicit Optional(in_place_t, Args&&... args) noexcept( - std::is_nothrow_constructible::value) - : Optional{PrivateConstructor{}, std::forward(args)...} {} - - template - explicit Optional( - in_place_t, - std::initializer_list il, - Args&&... args) noexcept(std:: - is_nothrow_constructible< - Value, - std::initializer_list, - Args...>::value) - : Optional{PrivateConstructor{}, il, std::forward(args)...} {} - - // Used only when an Optional is used with coroutines on MSVC - /* implicit */ Optional(const detail::OptionalPromiseReturn& p) - : Optional{} { - p.promise_->value_ = this; - } - - void assign(const None&) { - clear(); - } - - void assign(Optional&& src) { - if (this != &src) { - if (src.hasValue()) { - assign(std::move(src.value())); - src.clear(); - } else { - clear(); - } - } - } - - void assign(const Optional& src) { - if (src.hasValue()) { - assign(src.value()); - } else { - clear(); - } - } - - void assign(Value&& newValue) { - if (hasValue()) { - storage_.value = std::move(newValue); - } else { - construct(std::move(newValue)); - } - } - - void assign(const Value& newValue) { - if (hasValue()) { - storage_.value = newValue; - } else { - construct(newValue); - } - } - - Optional& operator=(None) noexcept { - reset(); - return *this; - } - - template - Optional& operator=(Arg&& arg) { - assign(std::forward(arg)); - return *this; - } - - Optional& operator=(Optional&& other) noexcept( - std::is_nothrow_move_assignable::value) { - assign(std::move(other)); - return *this; - } - - Optional& operator=(const Optional& other) noexcept( - std::is_nothrow_copy_assignable::value) { - assign(other); - return *this; - } - - template - Value& emplace(Args&&... args) { - clear(); - construct(std::forward(args)...); - return value(); - } - - template - typename std::enable_if< - std::is_constructible&, Args&&...>::value, - Value&>::type - emplace(std::initializer_list ilist, Args&&... args) { - clear(); - construct(ilist, std::forward(args)...); - return value(); - } - - void reset() noexcept { - storage_.clear(); - } - - void clear() noexcept { - reset(); - } - - void swap(Optional& that) noexcept(IsNothrowSwappable::value) { - if (hasValue() && that.hasValue()) { - using std::swap; - swap(value(), that.value()); - } else if (hasValue()) { - that.emplace(std::move(value())); - reset(); - } else if (that.hasValue()) { - emplace(std::move(that.value())); - that.reset(); - } - } - - const Value& value() const& { - require_value(); - return storage_.value; - } - - Value& value() & { - require_value(); - return storage_.value; - } - - Value&& value() && { - require_value(); - return std::move(storage_.value); - } - - const Value&& value() const&& { - require_value(); - return std::move(storage_.value); - } - - const Value* get_pointer() const& { - return storage_.hasValue ? &storage_.value : nullptr; - } - Value* get_pointer() & { - return storage_.hasValue ? &storage_.value : nullptr; - } - Value* get_pointer() && = delete; - - bool has_value() const noexcept { - return storage_.hasValue; - } - - bool hasValue() const noexcept { - return has_value(); - } - - explicit operator bool() const noexcept { - return has_value(); - } - - const Value& operator*() const& { - return value(); - } - Value& operator*() & { - return value(); - } - const Value&& operator*() const&& { - return std::move(value()); - } - Value&& operator*() && { - return std::move(value()); - } - - const Value* operator->() const { - return &value(); - } - Value* operator->() { - return &value(); - } - - // Return a copy of the value if set, or a given default if not. - template - Value value_or(U&& dflt) const& { - if (storage_.hasValue) { - return storage_.value; - } - - return std::forward(dflt); - } - - template - Value value_or(U&& dflt) && { - if (storage_.hasValue) { - return std::move(storage_.value); - } - - return std::forward(dflt); - } - - private: - template - friend Optional<_t>> make_optional(T&&); - template - friend Optional make_optional(Args&&... args); - template - friend Optional make_optional(std::initializer_list, As&&...); - - /** - * Construct the optional in place, this is duplicated as a non-explicit - * constructor to allow returning values that are non-movable from - * make_optional using list initialization. - * - * Until C++17, at which point this will become unnecessary because of - * specified prvalue elision. - */ - struct PrivateConstructor { - explicit PrivateConstructor() = default; - }; - template - Optional(PrivateConstructor, Args&&... args) noexcept( - std::is_constructible::value) { - construct(std::forward(args)...); - } - - void require_value() const { - if (!storage_.hasValue) { - throw OptionalEmptyException{}; - } - } - - template - void construct(Args&&... args) { - const void* ptr = &storage_.value; - // For supporting const types. - new (const_cast(ptr)) Value(std::forward(args)...); - storage_.hasValue = true; - } - - struct StorageTriviallyDestructible { - union { - char emptyState; - Value value; - }; - bool hasValue; - - StorageTriviallyDestructible() - : emptyState('\0'), hasValue{false} {} - void clear() { - hasValue = false; - } - }; - - struct StorageNonTriviallyDestructible { - union { - char emptyState; - Value value; - }; - bool hasValue; - - StorageNonTriviallyDestructible() : hasValue{false} {} - ~StorageNonTriviallyDestructible() { - clear(); - } - - void clear() { - if (hasValue) { - hasValue = false; - value.~Value(); - } - } - }; - - using Storage = typename std::conditional< - std::is_trivially_destructible::value, - StorageTriviallyDestructible, - StorageNonTriviallyDestructible>::type; - - Storage storage_; -}; - -template -const T* get_pointer(const Optional& opt) { - return opt.get_pointer(); -} - -template -T* get_pointer(Optional& opt) { - return opt.get_pointer(); -} - -template -void swap(Optional& a, Optional& b) noexcept(noexcept(a.swap(b))) { - a.swap(b); -} - -template -Optional<_t>> make_optional(T&& v) { - using PrivateConstructor = - typename folly::Optional<_t>>::PrivateConstructor; - return {PrivateConstructor{}, std::forward(v)}; -} - -template -folly::Optional make_optional(Args&&... args) { - using PrivateConstructor = typename folly::Optional::PrivateConstructor; - return {PrivateConstructor{}, std::forward(args)...}; -} - -template -folly::Optional make_optional( - std::initializer_list il, - Args&&... args) { - using PrivateConstructor = typename folly::Optional::PrivateConstructor; - return {PrivateConstructor{}, il, std::forward(args)...}; -} - -/////////////////////////////////////////////////////////////////////////////// -// Comparisons. - -template -bool operator==(const Optional& a, const V& b) { - return a.hasValue() && a.value() == b; -} - -template -bool operator!=(const Optional& a, const V& b) { - return !(a == b); -} - -template -bool operator==(const U& a, const Optional& b) { - return b.hasValue() && b.value() == a; -} - -template -bool operator!=(const U& a, const Optional& b) { - return !(a == b); -} - -template -bool operator==(const Optional& a, const Optional& b) { - if (a.hasValue() != b.hasValue()) { - return false; - } - if (a.hasValue()) { - return a.value() == b.value(); - } - return true; -} - -template -bool operator!=(const Optional& a, const Optional& b) { - return !(a == b); -} - -template -bool operator<(const Optional& a, const Optional& b) { - if (a.hasValue() != b.hasValue()) { - return a.hasValue() < b.hasValue(); - } - if (a.hasValue()) { - return a.value() < b.value(); - } - return false; -} - -template -bool operator>(const Optional& a, const Optional& b) { - return b < a; -} - -template -bool operator<=(const Optional& a, const Optional& b) { - return !(b < a); -} - -template -bool operator>=(const Optional& a, const Optional& b) { - return !(a < b); -} - -// Suppress comparability of Optional with T, despite implicit conversion. -template -bool operator<(const Optional&, const V& other) = delete; -template -bool operator<=(const Optional&, const V& other) = delete; -template -bool operator>=(const Optional&, const V& other) = delete; -template -bool operator>(const Optional&, const V& other) = delete; -template -bool operator<(const V& other, const Optional&) = delete; -template -bool operator<=(const V& other, const Optional&) = delete; -template -bool operator>=(const V& other, const Optional&) = delete; -template -bool operator>(const V& other, const Optional&) = delete; - -// Comparisons with none -template -bool operator==(const Optional& a, None) noexcept { - return !a.hasValue(); -} -template -bool operator==(None, const Optional& a) noexcept { - return !a.hasValue(); -} -template -bool operator<(const Optional&, None) noexcept { - return false; -} -template -bool operator<(None, const Optional& a) noexcept { - return a.hasValue(); -} -template -bool operator>(const Optional& a, None) noexcept { - return a.hasValue(); -} -template -bool operator>(None, const Optional&) noexcept { - return false; -} -template -bool operator<=(None, const Optional&) noexcept { - return true; -} -template -bool operator<=(const Optional& a, None) noexcept { - return !a.hasValue(); -} -template -bool operator>=(const Optional&, None) noexcept { - return true; -} -template -bool operator>=(None, const Optional& a) noexcept { - return !a.hasValue(); -} - -/////////////////////////////////////////////////////////////////////////////// - -} // namespace folly diff --git a/third-party/folly/folly/Portability.h b/third-party/folly/folly/Portability.h deleted file mode 100644 index b88b3d0c8..000000000 --- a/third-party/folly/folly/Portability.h +++ /dev/null @@ -1,99 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include - -#if defined(__arm__) -#define FOLLY_ARM 1 -#else -#define FOLLY_ARM 0 -#endif - -#if defined(__x86_64__) || defined(_M_X64) -#define FOLLY_X64 1 -#else -#define FOLLY_X64 0 -#endif - -#if defined(__aarch64__) -#define FOLLY_AARCH64 1 -#else -#define FOLLY_AARCH64 0 -#endif - -#if defined(__powerpc64__) -#define FOLLY_PPC64 1 -#else -#define FOLLY_PPC64 0 -#endif - -#if defined(__s390x__) -#define FOLLY_S390X 1 -#else -#define FOLLY_S390X 0 -#endif - -#if defined(__has_builtin) -#define FOLLY_HAS_BUILTIN(...) __has_builtin(__VA_ARGS__) -#else -#define FOLLY_HAS_BUILTIN(...) 0 -#endif - -#if defined(__has_cpp_attribute) -#if __has_cpp_attribute(nodiscard) -#define FOLLY_NODISCARD [[nodiscard]] -#endif -#endif -#if !defined FOLLY_NODISCARD -#if defined(_MSC_VER) && (_MSC_VER >= 1700) -#define FOLLY_NODISCARD _Check_return_ -#elif defined(__GNUC__) -#define FOLLY_NODISCARD __attribute__((__warn_unused_result__)) -#else -#define FOLLY_NODISCARD -#endif -#endif - -namespace folly { -constexpr bool kIsArchArm = FOLLY_ARM == 1; -constexpr bool kIsArchAmd64 = FOLLY_X64 == 1; -constexpr bool kIsArchAArch64 = FOLLY_AARCH64 == 1; -constexpr bool kIsArchPPC64 = FOLLY_PPC64 == 1; -constexpr bool kIsArchS390X = FOLLY_S390X == 1; -} // namespace folly - -namespace folly { -#ifdef NDEBUG -constexpr auto kIsDebug = false; -#else -constexpr auto kIsDebug = true; -#endif -} // namespace folly - -namespace folly { -#if defined(_MSC_VER) -constexpr bool kIsMsvc = true; -#else -constexpr bool kIsMsvc = false; -#endif -} // namespace folly - -namespace folly { -#if FOLLY_SANITIZE_THREAD -constexpr bool kIsSanitizeThread = true; -#else -constexpr bool kIsSanitizeThread = false; -#endif -} // namespace folly - -namespace folly { -#if defined(__linux__) && !FOLLY_MOBILE -constexpr auto kIsLinux = true; -#else -constexpr auto kIsLinux = false; -#endif -} // namespace folly diff --git a/third-party/folly/folly/ScopeGuard.h b/third-party/folly/folly/ScopeGuard.h deleted file mode 100644 index 711344063..000000000 --- a/third-party/folly/folly/ScopeGuard.h +++ /dev/null @@ -1,54 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include - -#include -#include - -namespace folly { -namespace scope_guard_detail { -template -class ScopeGuardImpl { - public: - explicit ScopeGuardImpl(F&& f) : f_{std::forward(f)} {} - ~ScopeGuardImpl() { - f_(); - } - - private: - F f_; -}; - -enum class ScopeGuardEnum {}; -template >> -ScopeGuardImpl operator+(ScopeGuardEnum, Func&& func) { - return ScopeGuardImpl{std::forward(func)}; -} -} // namespace scope_guard_detail -} // namespace folly - -/** - * FB_ANONYMOUS_VARIABLE(str) introduces an identifier starting with - * str and ending with a number that varies with the line. - */ -#ifndef FB_ANONYMOUS_VARIABLE -#define FB_CONCATENATE_IMPL(s1, s2) s1##s2 -#define FB_CONCATENATE(s1, s2) FB_CONCATENATE_IMPL(s1, s2) -#ifdef __COUNTER__ -#define FB_ANONYMOUS_VARIABLE(str) \ - FB_CONCATENATE(FB_CONCATENATE(FB_CONCATENATE(str, __COUNTER__), _), __LINE__) -#else -#define FB_ANONYMOUS_VARIABLE(str) FB_CONCATENATE(str, __LINE__) -#endif -#endif - -#ifndef SCOPE_EXIT -#define SCOPE_EXIT \ - auto FB_ANONYMOUS_VARIABLE(SCOPE_EXIT_STATE) = \ - ::folly::scope_guard_detail::ScopeGuardEnum{} + [&]() noexcept -#endif diff --git a/third-party/folly/folly/Traits.h b/third-party/folly/folly/Traits.h deleted file mode 100644 index ea7e1eb1c..000000000 --- a/third-party/folly/folly/Traits.h +++ /dev/null @@ -1,152 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include -#include - -namespace folly { - -#if !defined(_MSC_VER) -template -struct is_trivially_copyable - : std::integral_constant {}; -#else -template -using is_trivially_copyable = std::is_trivially_copyable; -#endif - -/*** - * _t - * - * Instead of: - * - * using decayed = typename std::decay::type; - * - * With the C++14 standard trait aliases, we could use: - * - * using decayed = std::decay_t; - * - * Without them, we could use: - * - * using decayed = _t>; - * - * Also useful for any other library with template types having dependent - * member types named `type`, like the standard trait types. - */ -template -using _t = typename T::type; - -/** - * type_t - * - * A type alias for the first template type argument. `type_t` is useful for - * controlling class-template and function-template partial specialization. - * - * Example: - * - * template - * class Container { - * public: - * template - * Container( - * type_t()...))>, - * Args&&...); - * }; - * - * void_t - * - * A type alias for `void`. `void_t` is useful for controling class-template - * and function-template partial specialization. - * - * Example: - * - * // has_value_type::value is true if T has a nested type `value_type` - * template - * struct has_value_type - * : std::false_type {}; - * - * template - * struct has_value_type> - * : std::true_type {}; - */ - -/** - * There is a bug in libstdc++, libc++, and MSVC's STL that causes it to - * ignore unused template parameter arguments in template aliases and does not - * cause substitution failures. This defect has been recorded here: - * http://open-std.org/JTC1/SC22/WG21/docs/cwg_defects.html#1558. - * - * This causes the implementation of std::void_t to be buggy, as it is likely - * defined as something like the following: - * - * template - * using void_t = void; - * - * This causes the compiler to ignore all the template arguments and does not - * help when one wants to cause substitution failures. Rather declarations - * which have void_t in orthogonal specializations are treated as the same. - * For example, assuming the possible `T` types are only allowed to have - * either the alias `one` or `two` and never both or none: - * - * template ::one>* = nullptr> - * void foo(T&&) {} - * template ::two>* = nullptr> - * void foo(T&&) {} - * - * The second foo() will be a redefinition because it conflicts with the first - * one; void_t does not cause substitution failures - the template types are - * just ignored. - */ - -namespace traits_detail { -template -struct type_t_ { - using type = T; -}; -} // namespace traits_detail - -template -using type_t = typename traits_detail::type_t_::type; -template -using void_t = type_t; - -/** - * A type trait to remove all const volatile and reference qualifiers on a - * type T - */ -template -struct remove_cvref { - using type = - typename std::remove_cv::type>::type; -}; -template -using remove_cvref_t = typename remove_cvref::type; - -template -struct IsNothrowSwappable - : std::integral_constant< - bool, - std::is_nothrow_move_constructible::value&& noexcept( - std::swap(std::declval(), std::declval()))> {}; - -template -struct Conjunction : std::true_type {}; -template -struct Conjunction : T {}; -template -struct Conjunction - : std::conditional, T>::type {}; - -template -struct Negation : std::integral_constant {}; - -template -using index_constant = std::integral_constant; - -} // namespace folly diff --git a/third-party/folly/folly/Unit.h b/third-party/folly/folly/Unit.h deleted file mode 100644 index c8cb77e2c..000000000 --- a/third-party/folly/folly/Unit.h +++ /dev/null @@ -1,59 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include - -namespace folly { - -/// In functional programming, the degenerate case is often called "unit". In -/// C++, "void" is often the best analogue. However, because of the syntactic -/// special-casing required for void, it is frequently a liability for template -/// metaprogramming. So, instead of writing specializations to handle cases like -/// SomeContainer, a library author may instead rule that out and simply -/// have library users use SomeContainer. Contained values may be ignored. -/// Much easier. -/// -/// "void" is the type that admits of no values at all. It is not possible to -/// construct a value of this type. -/// "unit" is the type that admits of precisely one unique value. It is -/// possible to construct a value of this type, but it is always the same value -/// every time, so it is uninteresting. -struct Unit { - constexpr bool operator==(const Unit& /*other*/) const { - return true; - } - constexpr bool operator!=(const Unit& /*other*/) const { - return false; - } -}; - -constexpr Unit unit{}; - -template -struct lift_unit { - using type = T; -}; -template <> -struct lift_unit { - using type = Unit; -}; -template -using lift_unit_t = typename lift_unit::type; - -template -struct drop_unit { - using type = T; -}; -template <> -struct drop_unit { - using type = void; -}; -template -using drop_unit_t = typename drop_unit::type; - -} // namespace folly - diff --git a/third-party/folly/folly/Utility.h b/third-party/folly/folly/Utility.h deleted file mode 100644 index 7e43bdc2f..000000000 --- a/third-party/folly/folly/Utility.h +++ /dev/null @@ -1,141 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include -#include - -namespace folly { - -/** - * Backports from C++17 of: - * std::in_place_t - * std::in_place_type_t - * std::in_place_index_t - * std::in_place - * std::in_place_type - * std::in_place_index - */ - -struct in_place_tag {}; -template -struct in_place_type_tag {}; -template -struct in_place_index_tag {}; - -using in_place_t = in_place_tag (&)(in_place_tag); -template -using in_place_type_t = in_place_type_tag (&)(in_place_type_tag); -template -using in_place_index_t = in_place_index_tag (&)(in_place_index_tag); - -inline in_place_tag in_place(in_place_tag = {}) { - return {}; -} -template -inline in_place_type_tag in_place_type(in_place_type_tag = {}) { - return {}; -} -template -inline in_place_index_tag in_place_index(in_place_index_tag = {}) { - return {}; -} - -template -T exchange(T& obj, U&& new_value) { - T old_value = std::move(obj); - obj = std::forward(new_value); - return old_value; -} - -namespace utility_detail { -template -struct make_seq_cat; -template < - template class S, - typename T, - T... Ta, - T... Tb, - T... Tc> -struct make_seq_cat, S, S> { - using type = - S; -}; - -// Not parameterizing by `template class, typename` because -// clang precisely v4.0 fails to compile that. Note that clang v3.9 and v5.0 -// handle that code correctly. -// -// For this to work, `S0` is required to be `Sequence` and `S1` is required -// to be `Sequence`. - -template -struct make_seq { - template - using apply = typename make_seq_cat< - typename make_seq::template apply, - typename make_seq::template apply, - typename make_seq::template apply>::type; -}; -template <> -struct make_seq<1> { - template - using apply = S1; -}; -template <> -struct make_seq<0> { - template - using apply = S0; -}; -} // namespace utility_detail - -// TODO: Remove after upgrading to C++14 baseline - -template -struct integer_sequence { - using value_type = T; - - static constexpr std::size_t size() noexcept { - return sizeof...(Ints); - } -}; - -template -using index_sequence = integer_sequence; - -template -using make_integer_sequence = typename utility_detail::make_seq< - Size>::template apply, integer_sequence>; - -template -using make_index_sequence = make_integer_sequence; -template -using index_sequence_for = make_index_sequence; - -/** - * A simple helper for getting a constant reference to an object. - * - * Example: - * - * std::vector v{1,2,3}; - * // The following two lines are equivalent: - * auto a = const_cast&>(v).begin(); - * auto b = folly::as_const(v).begin(); - * - * Like C++17's std::as_const. See http://wg21.link/p0007 - */ -template -T const& as_const(T& t) noexcept { - return t; -} - -template -void as_const(T const&&) = delete; - -} // namespace folly diff --git a/third-party/folly/folly/chrono/Hardware.h b/third-party/folly/folly/chrono/Hardware.h deleted file mode 100644 index 6635b8717..000000000 --- a/third-party/folly/folly/chrono/Hardware.h +++ /dev/null @@ -1,33 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include - -#include -#include - -#if _MSC_VER && (defined(_M_IX86) || defined(_M_X64)) -extern "C" std::uint64_t __rdtsc(); -#pragma intrinsic(__rdtsc) -#endif - -namespace folly { - -inline std::uint64_t hardware_timestamp() { -#if _MSC_VER && (defined(_M_IX86) || defined(_M_X64)) - return __rdtsc(); -#elif __GNUC__ && (__i386__ || FOLLY_X64) - return __builtin_ia32_rdtsc(); -#else - // use steady_clock::now() as an approximation for the timestamp counter on - // non-x86 systems - return std::chrono::steady_clock::now().time_since_epoch().count(); -#endif -} - -} // namespace folly - diff --git a/third-party/folly/folly/container/Array.h b/third-party/folly/folly/container/Array.h deleted file mode 100644 index bb3167b97..000000000 --- a/third-party/folly/folly/container/Array.h +++ /dev/null @@ -1,74 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include -#include -#include - -#include -#include - -namespace folly { - -namespace array_detail { -template -struct is_ref_wrapper : std::false_type {}; -template -struct is_ref_wrapper> : std::true_type {}; - -template -using not_ref_wrapper = - folly::Negation::type>>; - -template -struct return_type_helper { - using type = D; -}; -template -struct return_type_helper { - static_assert( - folly::Conjunction...>::value, - "TList cannot contain reference_wrappers when D is void"); - using type = typename std::common_type::type; -}; - -template -using return_type = std:: - array::type, sizeof...(TList)>; -} // namespace array_detail - -template -constexpr array_detail::return_type make_array(TList&&... t) { - using value_type = - typename array_detail::return_type_helper::type; - return {{static_cast(std::forward(t))...}}; -} - -namespace array_detail { -template -inline constexpr auto make_array_with( - MakeItem const& make, - folly::index_sequence) - -> std::array { - return std::array{{make(Index)...}}; -} -} // namespace array_detail - -// make_array_with -// -// Constructs a std::array<..., Size> with elements m(i) for i in [0, Size). -template -constexpr auto make_array_with(MakeItem const& make) - -> decltype(array_detail::make_array_with( - make, - folly::make_index_sequence{})) { - return array_detail::make_array_with( - make, - folly::make_index_sequence{}); -} - -} // namespace folly diff --git a/third-party/folly/folly/detail/Futex-inl.h b/third-party/folly/folly/detail/Futex-inl.h deleted file mode 100644 index 3b2a412bf..000000000 --- a/third-party/folly/folly/detail/Futex-inl.h +++ /dev/null @@ -1,117 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include -#include - -namespace folly { -namespace detail { - -/** Optimal when TargetClock is the same type as Clock. - * - * Otherwise, both Clock::now() and TargetClock::now() must be invoked. */ -template -typename TargetClock::time_point time_point_conv( - std::chrono::time_point const& time) { - using std::chrono::duration_cast; - using TimePoint = std::chrono::time_point; - using TargetDuration = typename TargetClock::duration; - using TargetTimePoint = typename TargetClock::time_point; - if (time == TimePoint::max()) { - return TargetTimePoint::max(); - } else if (std::is_same::value) { - // in place of time_point_cast, which cannot compile without if-constexpr - auto const delta = time.time_since_epoch(); - return TargetTimePoint(duration_cast(delta)); - } else { - // different clocks with different epochs, so non-optimal case - auto const delta = time - Clock::now(); - return TargetClock::now() + duration_cast(delta); - } -} - -/** - * Available overloads, with definitions elsewhere - * - * These functions are treated as ADL-extension points, the templates above - * call these functions without them having being pre-declared. This works - * because ADL lookup finds the definitions of these functions when you pass - * the relevant arguments - */ -int futexWakeImpl( - const Futex* futex, - int count, - uint32_t wakeMask); -FutexResult futexWaitImpl( - const Futex* futex, - uint32_t expected, - std::chrono::system_clock::time_point const* absSystemTime, - std::chrono::steady_clock::time_point const* absSteadyTime, - uint32_t waitMask); - -int futexWakeImpl( - const Futex* futex, - int count, - uint32_t wakeMask); -FutexResult futexWaitImpl( - const Futex* futex, - uint32_t expected, - std::chrono::system_clock::time_point const* absSystemTime, - std::chrono::steady_clock::time_point const* absSteadyTime, - uint32_t waitMask); - -template -typename std::enable_if::type -futexWaitImpl( - Futex* futex, - uint32_t expected, - Deadline const& deadline, - uint32_t waitMask) { - return futexWaitImpl(futex, expected, nullptr, &deadline, waitMask); -} - -template -typename std::enable_if::type -futexWaitImpl( - Futex* futex, - uint32_t expected, - Deadline const& deadline, - uint32_t waitMask) { - return futexWaitImpl(futex, expected, &deadline, nullptr, waitMask); -} - -template -FutexResult -futexWait(const Futex* futex, uint32_t expected, uint32_t waitMask) { - auto rv = futexWaitImpl(futex, expected, nullptr, nullptr, waitMask); - assert(rv != FutexResult::TIMEDOUT); - return rv; -} - -template -int futexWake(const Futex* futex, int count, uint32_t wakeMask) { - return futexWakeImpl(futex, count, wakeMask); -} - -template -FutexResult futexWaitUntil( - const Futex* futex, - uint32_t expected, - std::chrono::time_point const& deadline, - uint32_t waitMask) { - using Target = typename std::conditional< - Clock::is_steady, - std::chrono::steady_clock, - std::chrono::system_clock>::type; - auto const converted = time_point_conv(deadline); - return converted == Target::time_point::max() - ? futexWaitImpl(futex, expected, nullptr, nullptr, waitMask) - : futexWaitImpl(futex, expected, converted, waitMask); -} - -} // namespace detail -} // namespace folly diff --git a/third-party/folly/folly/detail/Futex.cpp b/third-party/folly/folly/detail/Futex.cpp deleted file mode 100644 index 012d1a36f..000000000 --- a/third-party/folly/folly/detail/Futex.cpp +++ /dev/null @@ -1,259 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#include -#include -#include -#include -#include -#include - -#include - -#ifdef __linux__ -#include -#endif - -#ifndef _WIN32 -#include -#endif - -namespace folly { -namespace detail { - -namespace { - -//////////////////////////////////////////////////// -// native implementation using the futex() syscall - -#ifdef __linux__ - -/// Certain toolchains (like Android's) don't include the full futex API in -/// their headers even though they support it. Make sure we have our constants -/// even if the headers don't have them. -#ifndef FUTEX_WAIT_BITSET -#define FUTEX_WAIT_BITSET 9 -#endif -#ifndef FUTEX_WAKE_BITSET -#define FUTEX_WAKE_BITSET 10 -#endif -#ifndef FUTEX_PRIVATE_FLAG -#define FUTEX_PRIVATE_FLAG 128 -#endif -#ifndef FUTEX_CLOCK_REALTIME -#define FUTEX_CLOCK_REALTIME 256 -#endif - -int nativeFutexWake(const void* addr, int count, uint32_t wakeMask) { - long rv = syscall( - __NR_futex, - addr, /* addr1 */ - FUTEX_WAKE_BITSET | FUTEX_PRIVATE_FLAG, /* op */ - count, /* val */ - nullptr, /* timeout */ - nullptr, /* addr2 */ - wakeMask); /* val3 */ - - /* NOTE: we ignore errors on wake for the case of a futex - guarding its own destruction, similar to this - glibc bug with sem_post/sem_wait: - https://sourceware.org/bugzilla/show_bug.cgi?id=12674 */ - if (rv < 0) { - return 0; - } - return static_cast(rv); -} - -template -struct timespec timeSpecFromTimePoint(std::chrono::time_point absTime) { - auto epoch = absTime.time_since_epoch(); - if (epoch.count() < 0) { - // kernel timespec_valid requires non-negative seconds and nanos in [0,1G) - epoch = Clock::duration::zero(); - } - - // timespec-safe seconds and nanoseconds; - // chrono::{nano,}seconds are `long long int` - // whereas timespec uses smaller types - using time_t_seconds = - std::chrono::duration; - using long_nanos = - std::chrono::duration; - - auto secs = std::chrono::duration_cast(epoch); - auto nanos = std::chrono::duration_cast(epoch - secs); - struct timespec result = {secs.count(), nanos.count()}; - return result; -} - -FutexResult nativeFutexWaitImpl( - const void* addr, uint32_t expected, - std::chrono::system_clock::time_point const* absSystemTime, - std::chrono::steady_clock::time_point const* absSteadyTime, - uint32_t waitMask) { - assert(absSystemTime == nullptr || absSteadyTime == nullptr); - - int op = FUTEX_WAIT_BITSET | FUTEX_PRIVATE_FLAG; - struct timespec ts; - struct timespec* timeout = nullptr; - - if (absSystemTime != nullptr) { - op |= FUTEX_CLOCK_REALTIME; - ts = timeSpecFromTimePoint(*absSystemTime); - timeout = &ts; - } else if (absSteadyTime != nullptr) { - ts = timeSpecFromTimePoint(*absSteadyTime); - timeout = &ts; - } - - // Unlike FUTEX_WAIT, FUTEX_WAIT_BITSET requires an absolute timeout - // value - http://locklessinc.com/articles/futex_cheat_sheet/ - long rv = syscall( - __NR_futex, - addr, /* addr1 */ - op, /* op */ - expected, /* val */ - timeout, /* timeout */ - nullptr, /* addr2 */ - waitMask); /* val3 */ - - if (rv == 0) { - return FutexResult::AWOKEN; - } else { - switch (errno) { - case ETIMEDOUT: - assert(timeout != nullptr); - return FutexResult::TIMEDOUT; - case EINTR: - return FutexResult::INTERRUPTED; - case EWOULDBLOCK: - return FutexResult::VALUE_CHANGED; - default: - assert(false); - // EINVAL, EACCESS, or EFAULT. EINVAL means there was an invalid - // op (should be impossible) or an invalid timeout (should have - // been sanitized by timeSpecFromTimePoint). EACCESS or EFAULT - // means *addr points to invalid memory, which is unlikely because - // the caller should have segfaulted already. We can either - // crash, or return a value that lets the process continue for - // a bit. We choose the latter. VALUE_CHANGED probably turns the - // caller into a spin lock. - return FutexResult::VALUE_CHANGED; - } - } -} - -#endif // __linux__ - -/////////////////////////////////////////////////////// -// compatibility implementation using standard C++ API - -using Lot = ParkingLot; -Lot parkingLot; - -int emulatedFutexWake(const void* addr, int count, uint32_t waitMask) { - int woken = 0; - parkingLot.unpark(addr, [&](const uint32_t& mask) { - if ((mask & waitMask) == 0) { - return UnparkControl::RetainContinue; - } - assert(count > 0); - count--; - woken++; - return count > 0 ? UnparkControl::RemoveContinue - : UnparkControl::RemoveBreak; - }); - return woken; -} - -template -FutexResult emulatedFutexWaitImpl( - F* futex, uint32_t expected, - std::chrono::system_clock::time_point const* absSystemTime, - std::chrono::steady_clock::time_point const* absSteadyTime, - uint32_t waitMask) { - static_assert( - std::is_same>::value || - std::is_same>::value, - "Type F must be either Futex or Futex"); - ParkResult res; - if (absSystemTime) { - res = parkingLot.park_until( - futex, - waitMask, - [&] { return *futex == expected; }, - [] {}, - *absSystemTime); - } else if (absSteadyTime) { - res = parkingLot.park_until( - futex, - waitMask, - [&] { return *futex == expected; }, - [] {}, - *absSteadyTime); - } else { - res = parkingLot.park( - futex, waitMask, [&] { return *futex == expected; }, [] {}); - } - switch (res) { - case ParkResult::Skip: - return FutexResult::VALUE_CHANGED; - case ParkResult::Unpark: - return FutexResult::AWOKEN; - case ParkResult::Timeout: - return FutexResult::TIMEDOUT; - } - - return FutexResult::INTERRUPTED; -} - -} // namespace - -///////////////////////////////// -// Futex<> overloads - -int futexWakeImpl( - const Futex* futex, - int count, - uint32_t wakeMask) { -#ifdef __linux__ - return nativeFutexWake(futex, count, wakeMask); -#else - return emulatedFutexWake(futex, count, wakeMask); -#endif -} - -int futexWakeImpl( - const Futex* futex, - int count, - uint32_t wakeMask) { - return emulatedFutexWake(futex, count, wakeMask); -} - -FutexResult futexWaitImpl( - const Futex* futex, uint32_t expected, - std::chrono::system_clock::time_point const* absSystemTime, - std::chrono::steady_clock::time_point const* absSteadyTime, - uint32_t waitMask) { -#ifdef __linux__ - return nativeFutexWaitImpl( - futex, expected, absSystemTime, absSteadyTime, waitMask); -#else - return emulatedFutexWaitImpl( - futex, expected, absSystemTime, absSteadyTime, waitMask); -#endif -} - -FutexResult futexWaitImpl( - const Futex* futex, uint32_t expected, - std::chrono::system_clock::time_point const* absSystemTime, - std::chrono::steady_clock::time_point const* absSteadyTime, - uint32_t waitMask) { - return emulatedFutexWaitImpl( - futex, expected, absSystemTime, absSteadyTime, waitMask); -} - -} // namespace detail -} // namespace folly diff --git a/third-party/folly/folly/detail/Futex.h b/third-party/folly/folly/detail/Futex.h deleted file mode 100644 index 987a1b895..000000000 --- a/third-party/folly/folly/detail/Futex.h +++ /dev/null @@ -1,96 +0,0 @@ -// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. -// This source code is licensed under both the GPLv2 (found in the -// COPYING file in the root directory) and Apache 2.0 License -// (found in the LICENSE.Apache file in the root directory). - -#pragma once - -#include -#include -#include -#include -#include -#include - -namespace folly { -namespace detail { - -enum class FutexResult { - VALUE_CHANGED, /* futex value didn't match expected */ - AWOKEN, /* wakeup by matching futex wake, or spurious wakeup */ - INTERRUPTED, /* wakeup by interrupting signal */ - TIMEDOUT, /* wakeup by expiring deadline */ -}; - -/** - * Futex is an atomic 32 bit unsigned integer that provides access to the - * futex() syscall on that value. It is templated in such a way that it - * can interact properly with DeterministicSchedule testing. - * - * If you don't know how to use futex(), you probably shouldn't be using - * this class. Even if you do know how, you should have a good reason - * (and benchmarks to back you up). - * - * Because of the semantics of the futex syscall, the futex family of - * functions are available as free functions rather than member functions - */ -template