Add readdir policies

README.md

@@ -256,6 +256,9 @@ These options are the same regardless of whether you use them with the
   concatenated together with the longest common prefix removed.
 * **func.FUNC=POLICY**: Sets the specific FUSE function's policy. See
   below for the list of value types. Example: **func.getattr=newest**
+* **func.readdir=seq|cosr|cor|cosr:INT|cor:INT**: Sets `readdir`
+  policy. INT value sets the number of threads to use for
+  concurrency. (default: seq)
 * **category.action=POLICY**: Sets policy of all FUSE functions in the
   action category. (default: epall)
 * **category.create=POLICY**: Sets policy of all FUSE functions in the
@@ -682,9 +685,8 @@ rather than file paths, which were created by `open` or `create`. That
 said many times the current FUSE kernel driver will not always provide
 the file handle when a client calls `fgetattr`, `fchown`, `fchmod`,
 `futimens`, `ftruncate`, etc. This means it will call the regular,
-path based, versions. `readdir` has no real need for a policy given
-the purpose is merely to return a list of entries in a
-directory. `statfs`'s behavior can be modified via other options.
+path based, versions. `statfs`'s behavior can be modified via other
+options.
 
 When using policies which are based on a branch's available space the
 base path provided is used. Not the full path to the file in
@@ -715,8 +717,7 @@ In cases where something may be searched for (such as a path to clone)
 ### Policies
 
 A policy is the algorithm used to choose a branch or branches for a
-function to work on. Think of them as ways to filter and sort
-branches.
+function to work on or generally how the function behaves.
 
 Any function in the `create` category will clone the relative path if
 needed. Some other functions (`rename`,`link`,`ioctl`) have special
@@ -725,12 +726,11 @@ requirements or behaviors which you can read more about below.
 
 #### Filtering
 
-Policies basically search branches and create a list of files / paths
+Most policies basically search branches and create a list of files / paths
 for functions to work on. The policy is responsible for filtering and
 sorting the branches. Filters include **minfreespace**, whether or not
 a branch is mounted read-only, and the branch tagging
-(RO,NC,RW). These filters are applied across all policies unless
-otherwise noted.
+(RO,NC,RW). These filters are applied across most policies.
 
 * No **search** function policies filter.
 * All **action** function policies filter out branches which are
@@ -823,6 +823,26 @@ policies is not appropriate.
 | search   | ff     |
 
 
+#### func.readdir
+
+examples: `fuse.readdir=seq`, `fuse.readdir=cor:4`
+
+`readdir` has policies to control how it manages reading directory
+content.
+
+| Policy | Description |
+|--------|-------------|
+| seq    | "sequential" : Iterate over branches in the order defined. This is the default and traditional behavior found prior to the readdir policy introduction. |
+| cosr   | "concurrent open, sequential read" : Concurrently open branch directories using a thread pool and process them in order of definition. This keeps memory and CPU usage low while also reducing the time spent waiting on branches to respond. Number of threads defaults to the number of logical cores. Can be overwritten via the syntax `fuse.readdir=cosr:N` where `N` is the number of threads. |
+| cor    | "concurrent open and read" : Concurrently open branch directories and immediately start reading their contents using a thread pool. This will result in slightly higher memory and CPU usage but reduced latency. Particularly when using higher latency / slower speed network filesystem branches. Unlike `seq` and `cosr` the order of files could change due the async nature of the thread pool. Number of threads defaults to the number of logical cores. Can be overwritten via the syntax `fuse.readdir=cor:N` where `N` is the number of threads.
+
+Keep in mind that `readdir` mostly just provides a list of file names
+in a directory and possibly some basic metadata about said files. To
+know details about the files, as one would see from commands like
+`find` or `ls`, it is required to call `stat` on the file which is
+controlled by `fuse.getattr`.
+
+
 #### ioctl
 
 When `ioctl` is used with an open file then it will use the file
@@ -891,19 +911,6 @@ returned but it will still be possible.
 **link** uses the same strategy but without the removals.
 
 
-#### readdir ####
-
-[readdir](http://linux.die.net/man/3/readdir) is different from all
-other filesystem functions. While it could have its own set of
-policies to tweak its behavior at this time it provides a simple union
-of files and directories found. Remember that any action or
-information queried about these files and directories come from the
-respective function. For instance: an **ls** is a **readdir** and for
-each file/directory returned **getattr** is called. Meaning the policy
-of **getattr** is responsible for choosing the file/directory which is
-the source of the metadata you see in an **ls**.
-
-
 #### statfs / statvfs ####
 
 [statvfs](http://linux.die.net/man/2/statvfs) normalizes the source

src/config.cpp

@@ -112,7 +112,7 @@ Config::Config()
     pid(::getpid()),
     posix_acl(false),
     readahead(0),
-    readdir(ReadDir::ENUM::POSIX),
+    readdir("seq"),
     readdirplus(false),
     rename_exdev(RenameEXDEV::ENUM::PASSTHROUGH),
     scheduling_priority(-10),
@@ -162,6 +162,7 @@ Config::Config()
   _map["func.mkdir"]             = &func.mkdir;
   _map["func.mknod"]             = &func.mknod;
   _map["func.open"]              = &func.open;
+  _map["func.readdir"]           = &readdir;
   _map["func.readlink"]          = &func.readlink;
   _map["func.removexattr"]       = &func.removexattr;
   _map["func.rename"]            = &func.rename;
@@ -189,7 +190,6 @@ Config::Config()
   _map["pin-threads"]            = &fuse_pin_threads;
   _map["posix_acl"]              = &posix_acl;
   _map["readahead"]              = &readahead;
-  //  _map["readdir"]            = &readdir;
   _map["readdirplus"]            = &readdirplus;
   _map["rename-exdev"]           = &rename_exdev;
   _map["scheduling-priority"]    = &scheduling_priority;

src/config.hpp

@@ -25,15 +25,15 @@
 #include "config_log_metrics.hpp"
 #include "config_moveonenospc.hpp"
 #include "config_nfsopenhack.hpp"
-#include "config_readdir.hpp"
 #include "config_rename_exdev.hpp"
+#include "config_set.hpp"
 #include "config_statfs.hpp"
 #include "config_statfsignore.hpp"
 #include "config_xattr.hpp"
-#include "config_set.hpp"
 #include "enum.hpp"
 #include "errno.hpp"
 #include "funcs.hpp"
+#include "fuse_readdir.hpp"
 #include "policy.hpp"
 #include "rwlock.hpp"
 #include "tofrom_wrapper.hpp"
@@ -135,7 +135,7 @@ class Config
   ConfigUINT64   pid;
   ConfigBOOL     posix_acl;
   ConfigUINT64   readahead;
-  ReadDir        readdir;
+  FUSE::ReadDir  readdir;
   ConfigBOOL     readdirplus;
   RenameEXDEV    rename_exdev;
   ConfigINT      scheduling_priority;

src/fs_devid.hpp

@@ -19,6 +19,7 @@
 #pragma once
 
 #include "fs_fstat.hpp"
+#include "fs_dirfd.hpp"
 
 
 namespace fs
@@ -37,4 +38,16 @@ namespace fs
 
     return st.st_dev;
   }
+
+  static
+  inline
+  dev_t
+  devid(DIR *dh_)
+  {
+    int dirfd;
+
+    dirfd = fs::dirfd(dh_);
+
+    return fs::devid(dirfd);
+  }
 }

src/fs_dirfd.hpp

@@ -27,8 +27,8 @@ namespace fs
   static
   inline
   int
-  dirfd(DIR *dirp_)
+  dirfd(DIR *dh_)
   {
-    return ::dirfd(dirp_);
+    return ::dirfd(dh_);
   }
 }

src/fs_inode.cpp

@@ -208,6 +208,19 @@ namespace fs
       return g_func(fusepath_,fusepath_len_,mode_,dev_,ino_);
     }
 
+    uint64_t
+    calc(std::string const &fusepath_,
+         const mode_t       mode_,
+         const dev_t        dev_,
+         const ino_t        ino_)
+    {
+      return calc(fusepath_.c_str(),
+                  fusepath_.size(),
+                  mode_,
+                  dev_,
+                  ino_);
+    }
+
     void
     calc(const char     *fusepath_,
          const uint64_t  fusepath_len_,

src/fs_inode.hpp

@@ -37,7 +37,11 @@ namespace fs
                   const uint64_t  fusepath_len,
                   const mode_t    mode,
                   const dev_t     dev,
-                  const ino_t     ion);
+                  const ino_t     ino);
+    uint64_t calc(std::string const &fusepath,
+                  mode_t const       mode,
+                  dev_t const        dev,
+                  ino_t              ino);
     void calc(const char     *fusepath,
               const uint64_t  fusepath_len,
               struct stat    *st);

src/fuse_readdir.cpp

@@ -16,35 +16,64 @@
   OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 */
 
-#include "fuse_readdir_posix.hpp"
-#include "fuse_readdir_linux.hpp"
+#include "fuse_readdir.hpp"
+#include "fuse_readdir_factory.hpp"
 
 #include "config.hpp"
-#include "dirinfo.hpp"
-#include "rwlock.hpp"
-#include "ugid.hpp"
 
-#include "fuse.h"
 
+int
+FUSE::readdir(const fuse_file_info_t *ffi_,
+              fuse_dirents_t         *buf_)
+{
+  Config::Write cfg;
+
+  return cfg->readdir(ffi_,buf_);
+}
+
+FUSE::ReadDir::ReadDir(std::string const s_)
+{
+  from_string(s_);
+  assert(_readdir);
+}
 
-namespace FUSE
+std::string
+FUSE::ReadDir::to_string() const
 {
-  int
-  readdir(const fuse_file_info_t *ffi_,
-          fuse_dirents_t         *buf_)
+  std::lock_guard<std::mutex> lg(_mutex);
+
+  return _type;
+}
+
+int
+FUSE::ReadDir::from_string(std::string const &str_)
+{
+  std::shared_ptr<FUSE::ReadDirBase> tmp;
+
+  tmp = FUSE::ReadDirFactory::make(str_);
+  if(!tmp)
+    return -EINVAL;
+
+  {
+    std::lock_guard<std::mutex> lg(_mutex);
+
+    _type    = str_;
+    _readdir = tmp;
+  }
+
+  return 0;
+}
+
+int
+FUSE::ReadDir::operator()(fuse_file_info_t const *ffi_,
+                          fuse_dirents_t         *buf_)
+{
+  std::shared_ptr<FUSE::ReadDirBase> readdir;
+
   {
-    Config::Read cfg;
-    DirInfo            *di = reinterpret_cast<DirInfo*>(ffi_->fh);
-    const fuse_context *fc = fuse_get_context();
-    const ugid::Set     ugid(fc->uid,fc->gid);
-
-    switch(cfg->readdir)
-      {
-      case ReadDir::ENUM::LINUX:
-        return FUSE::readdir_linux(cfg->branches,di->fusepath.c_str(),buf_);
-      default:
-      case ReadDir::ENUM::POSIX:
-        return FUSE::readdir_posix(cfg->branches,di->fusepath.c_str(),buf_);
-      }
+    std::lock_guard<std::mutex> lg(_mutex);
+    readdir = _readdir;
   }
+
+  return (*readdir)(ffi_,buf_);
 }

src/fuse_readdir.hpp

@@ -17,11 +17,38 @@
 #pragma once
 
 #include "fuse.h"
+#include "tofrom_string.hpp"
+#include "fuse_readdir_base.hpp"
+
+#include <memory>
+#include <mutex>
+
+#include <assert.h>
 
 
 namespace FUSE
 {
-  int
-  readdir(const fuse_file_info_t *ffi,
-          fuse_dirents_t         *buf);
+  int readdir(fuse_file_info_t const *ffi,
+              fuse_dirents_t         *buf);
+
+  class ReadDir : public ToFromString
+  {
+  public:
+    ReadDir(std::string const s_);
+
+  public:
+    std::string to_string() const;
+    int from_string(const std::string &);
+
+  public:
+    int operator()(fuse_file_info_t const *ffi,
+                   fuse_dirents_t         *buf);
+
+  private:
+    mutable std::mutex _mutex;
+
+  private:
+    std::string _type;
+    std::shared_ptr<FUSE::ReadDirBase> _readdir;
+  };
 }

src/fuse_readdir_base.hpp

@@ -0,0 +1,34 @@
+/*
+  Copyright (c) 2023, Antonio SJ Musumeci <[email protected]>
+
+  Permission to use, copy, modify, and/or distribute this software for any
+  purpose with or without fee is hereby granted, provided that the above
+  copyright notice and this permission notice appear in all copies.
+
+  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+*/
+
+#pragma once
+
+#include "fuse.h"
+
+
+namespace FUSE
+{
+  class ReadDirBase
+  {
+  public:
+    ReadDirBase() {};
+    virtual ~ReadDirBase() {};
+
+  public:
+    virtual int operator()(fuse_file_info_t const *ffi,
+                           fuse_dirents_t         *buf) = 0;
+  };
+}