Extend admission policies, statistics, add multi thread eviction and promotion #2

igchor · 2022-08-09T14:43:47Z

This change is

It's implementation is mostly based on PosixShmSegment. Also, extend ShmManager and ShmSegmentOpts to support this new segment type.

After introducing file segment type, nameToKey_ does not provide enough information to recover/remove segments on restart. This commit fixes that by replacing nameToKey_ with nameToOpts_. Previously, the Key from nameToKey_ map was only used in a single DCHECK().

* New class MemoryTierCacheConfig allows to configure a memory tier. Setting tier size and location of a file for file-backed memory are supported in this initial implementation; * New member, vector of memory tiers, is added to class CacheAllocatorConfig. * New test suite, chelib/allocator/tests/MemoryTiersTest.cpp, demonstrates the usage of and tests extended config API.

to allow using new configureMemoryTiers() API with legacy behavior. Move validation code for memory tiers to validate() method and convert ratios to sizes lazily (on get)..

It wrongly assumed that the only possible segment type is PosixSysV segment.

… handling when NVM cache state is not enabled

Now it's size is 8 bytes intead of 4. Original CompressedPtr stored only some offset with a memory Allocator. For multi-tier implementation, this is not enough. We must also store tierId and when uncompressing, select a proper allocator. An alternative could be to just resign from CompressedPtr but they are leveraged to allow the cache to be mapped to different addresses on shared memory. Changing CompressedPtr impacted CacheItem size - it increased from 32 to 44 bytes.

Without this fix removeCb called even in case when Item is moved between tiers.

It fails because CentOS is EOL. We might want to consider using CentOS Streams but for now, just remove it. Right now, we rely on build-cachelib-centos workflow anyway.

Disabled test suite allocator-test-AllocatorTypeTest to skip sporadically failing tests.

…m#43) Compensation results in ratios being different than originially specified.

The main purpose of this patch is to better simulate workloads in cachebench. Setting touchValue to true allows to see performance impact of using different mediums for memory cache.

Extend cachbench with touch value

* pmem#75: Use actual tier sizes (rounded down to slab size and decreased by header size) when creating new memory pools * Added getPoolSize method to calculate combined pool size for all tiers; added pool size validation to tests * Explicitly specified type for totalCacheSize to avoid overflow * Minor test change * Reworked tests * Minor change * Deleted redundant tests * Deleted unused constant * First set of changes to cache configuration API to enable multi-tier caches (facebook#138) Summary: These changes introduce per-tier cache configuration required to implement features discussed here: facebook#102. These specific changes enable single DRAM tier configs only which are compatible with the current version of cachelib. Configuration API will be expanded as multi-tier changes in other parts of the library are introduced. Pull Request resolved: facebook#138 Reviewed By: therealgymmy Differential Revision: D36189766 Pulled By: jiayuebao fbshipit-source-id: 947aa0cd800ea6accffc1b7b6b0c9693aa7fc0a5 Co-authored-by: Victoria McGrath <[email protected]>

Add memory usage statistics for slabs and allocation classes

Add option to print memory stats in bytes only

Added per tier pool class rolling average latency

Hot queue iterator for 2Q. Will start at Hot queue and move to Warm queue if hot queue is exhausted. Useful for promotion semantics if using 2Q replacement. rebased on to develop and added some tests.

vinser52 · 2022-08-10T13:15:03Z

cachelib/allocator/BackgroundPromoter.h

+};
+
+template <typename CacheT>
+class BackgroundPromoter : public PeriodicWorker {


BackgroundPromoter and BackgroundEvictor classes look the same. Do we really need two classes?

vinser52 · 2022-08-10T13:17:51Z

cachelib/allocator/CacheAllocator-inl.h

@@ -272,6 +266,14 @@ void CacheAllocator<CacheTrait>::initCommon(bool dramCacheAttached) {
      nvmAdmissionPolicy_->initMinTTL(config_.nvmAdmissionMinTTL);
    }
  }
+  if (numTiers_ > 1 && !config_.moveCb) {
+    XLOG(WARN, "No moveCb set, using memcpy for moving items between tiers.");
+    config_.moveCb = [](Item& oldItem, Item& newItem, Item* parentItem){


Some legacy logic depends on whether moveCb is set or not. Are we sure that we are not breaking legacy behavior?

vinser52 · 2022-08-10T13:21:07Z

cachelib/allocator/CacheAllocator-inl.h

@@ -372,7 +403,8 @@ CacheAllocator<CacheTrait>::allocateInternalTier(TierId tid,
                                             typename Item::Key key,
                                             uint32_t size,
                                             uint32_t creationTime,
-                                             uint32_t expiryTime) {
+                                             uint32_t expiryTime,
+                                             bool fromEvictorThread) {


Does fromEvictorThread also set to true when it is called from promoter thread? If so, let´s rename variable to have more generic name, e.g. fromBgThread?

and add additional parameters to control allocation and eviction of items. Co-authored-by: Daniel Byrne <[email protected]>

Summary: AdRanker ASAN canary flagged a possible UBSan violation. ## Error Failed Run: https://fburl.com/servicelab/apytosry ``` #0 0x562e3adb59bc in facebook::cachelib::objcache2::ObjectCacheSizeController<facebook::cachelib::CacheAllocator<facebook::cachelib::LruCacheTrait> >::work() buck-out/v2/gen/fbcode/47d914adeee3d982/cachelib/experimental/objcache2/__object-cache__/headers/cachelib/experimental/objcache2/ObjectCacheSizeController-inl.h #1 0x562de7610f78 in facebook::cachelib::PeriodicWorker::loop() fbcode/cachelib/common/PeriodicWorker.cpp:55 #2 0x7f7632c524e4 in execute_native_thread_routine /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:82:18 #3 0x7f7632f6ec0e in start_thread /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/nptl/pthread_create.c:434:8 #4 0x7f76330011db in clone3 /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 UndefinedBehaviorSanitizer: integer-divide-by-zero buck-out/v2/gen/fbcode/47d914adeee3d982/cachelib/experimental/objcache2/__object-cache__/headers/cachelib/experimental/objcache2/ObjectCacheSizeController-inl.h:33:40 in ``` Reviewed By: jiayuebao Differential Revision: D39024188 fbshipit-source-id: 64ad644c360565e638fa3ca74616a315038382ab

Config api integration

igchor and others added 30 commits February 10, 2022 14:55

Run centos and debian workflows on push and PR

36eae66

Introduce FileShmSegment for file-backed shared memory

790c09f

It's implementation is mostly based on PosixShmSegment. Also, extend ShmManager and ShmSegmentOpts to support this new segment type.

Adjust and enable tests for ShmFileSegment

aed38aa

Integrate Memory Tier config API with CacheAllocator.

3186b94

Add MemoryTierCacheConfig::fromShm()

38515ac

to allow using new configureMemoryTiers() API with legacy behavior. Move validation code for memory tiers to validate() method and convert ratios to sizes lazily (on get)..

Fix test_shm_manager.cpp test

48b68a3

It wrongly assumed that the only possible segment type is PosixSysV segment.

Run tests on CI

6fe4971

Run long tests (navy/bench) every day on CI

dff2296

Moved common segment code for posix and file shm segments into ShmCommon

3af7643

Enabled memory tier config API for cachebench.

c93b440

Enabled shared memory tier in cachebench.

ab752e8

Converted nvmCacheState_ to std::optional to simplify NVM cache state…

2cacc99

… handling when NVM cache state is not enabled

Run CI on prebuild docker image

9ace595

Run only centos build on CI

9812286

Initial multi-tier support implementation

e111395

Implemented async Item movement between tiers

9ae9b2e

Adding example for multitiered cache

3b68053

Enable workarounds in tests

bef878e

Add basic multi-tier test

0e8af04

Set correct size for each memory tier

4477fec

Extend cachbench with value validation

53ca174

Aadding new configs to hit_ratio/graph_cache_leader_fobj

8b83aab

Move validateValue call to make sure it is measured by latency tracker

7b36c51

Fix eviction flow and removeCb calls

d817018

Without this fix removeCb called even in case when Item is moved between tiers.

Remove failing build-cachelib workflow (pmem#42)

385128d

It fails because CentOS is EOL. We might want to consider using CentOS Streams but for now, just remove it. Right now, we rely on build-cachelib-centos workflow anyway.

Disabled test suite allocator-test-AllocatorTypeTest (pmem#41)

d13568e

Disabled test suite allocator-test-AllocatorTypeTest to skip sporadically failing tests.

Do not compensate for rounding error when calculating tier sizes (pme…

02a3bfb

…m#43) Compensation results in ratios being different than originially specified.

igchor and others added 11 commits June 15, 2022 05:15

Extend cachbench with touch value

98a2fde

The main purpose of this patch is to better simulate workloads in cachebench. Setting touchValue to true allows to see performance impact of using different mediums for memory cache.

Enable touchValue by default

21c2e31

Merge pull request pmem#85 from igchor/touch_value_develop

1d16e1a

Extend cachbench with touch value

Add memory usage statistics for slabs and allocation classes

407806a

Merge pull request pmem#91 from igchor/more_stats

57a3d1c

Add memory usage statistics for slabs and allocation classes

Add option to print memory stats in bytes only

34f9f8e

Merge pull request pmem#93 from igchor/stats_no_gb

cab87d4

Add option to print memory stats in bytes only

added per tier pool class rolling average latency

2434693

Merge pull request pmem#96 from guptask/rolling_stats

11fadaa

Added per tier pool class rolling average latency

MM2Q promotion iterators (#1)

acdfa0b

Hot queue iterator for 2Q. Will start at Hot queue and move to Warm queue if hot queue is exhausted. Useful for promotion semantics if using 2Q replacement. rebased on to develop and added some tests.

igchor force-pushed the in_progress_promotion branch from b7300e2 to a3e29dd Compare August 9, 2022 14:52

vinser52 reviewed Aug 10, 2022

View reviewed changes

Implement background promotion and eviction

a2721d1

and add additional parameters to control allocation and eviction of items. Co-authored-by: Daniel Byrne <[email protected]>

igchor force-pushed the in_progress_promotion branch from a3e29dd to a2721d1 Compare August 11, 2022 21:23

vinser52 force-pushed the develop branch from 9f5dc7c to 6122ab9 Compare September 12, 2022 15:35

byrnedj mentioned this pull request Sep 19, 2022

Background data movement #20

Merged

byrnedj force-pushed the develop branch from 4f8a592 to 9bb4db8 Compare November 16, 2022 13:17

byrnedj force-pushed the develop branch from 8a1b164 to 08bf0b4 Compare March 3, 2023 21:45

byrnedj pushed a commit that referenced this pull request Jul 23, 2023

Merge pull request #2 from igchor/config_api_integration

35cb682

Config api integration

byrnedj force-pushed the develop branch 2 times, most recently from f6ad2ae to 09d7bab Compare July 23, 2023 23:44

vinser52 force-pushed the develop branch from ff44c3c to 139e2e9 Compare March 1, 2024 20:13

byrnedj force-pushed the develop branch 2 times, most recently from 09d7bab to ebfca17 Compare May 21, 2024 13:24

byrnedj force-pushed the develop branch 4 times, most recently from 9ba4e79 to c1ff397 Compare June 26, 2024 18:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extend admission policies, statistics, add multi thread eviction and promotion #2

Extend admission policies, statistics, add multi thread eviction and promotion #2

igchor commented Aug 9, 2022 •

edited by rdower

Loading

vinser52 Aug 10, 2022

vinser52 Aug 10, 2022

vinser52 Aug 10, 2022

Extend admission policies, statistics, add multi thread eviction and promotion #2

Are you sure you want to change the base?

Extend admission policies, statistics, add multi thread eviction and promotion #2

Conversation

igchor commented Aug 9, 2022 • edited by rdower Loading

vinser52 Aug 10, 2022

Choose a reason for hiding this comment

vinser52 Aug 10, 2022

Choose a reason for hiding this comment

vinser52 Aug 10, 2022

Choose a reason for hiding this comment

igchor commented Aug 9, 2022 •

edited by rdower

Loading