diff --git a/advanced/option-database-tuning.rst b/advanced/option-database-tuning.rst deleted file mode 100644 index c4496da1a..000000000 --- a/advanced/option-database-tuning.rst +++ /dev/null @@ -1,29 +0,0 @@ -.. _option-databaseTuning: - -databaseTuning -============== - -.. versionadded:: 1.3.0 - -The ``databaseTuning`` option controls how Syncthing uses the backend -key-value database that stores the index data and other persistent data -Syncthing needs. In most cases this database is fairly small (hundred -megabytes or less) and the default tuning is optimized for this. However in -cases with large folders or many devices the database may grow, and updates -may be more frequent. In these cases it's better to use larger buffers, -allow more memory to be used for cache, allow a larger amount of overhead on -disk in order to improve compaction performance, and so on. - -The ``databaseTuning`` option can have one of three values: - -- ``small``: This is the old set of tuning parameters, recommended for small - databases. -- ``large``: This is a new set of tuning parameters, recommended for large - databases. - -- ``auto``: Syncthing will automatically use either the large or small - tuning parameters depending on the database size on disk. A database that is - 200 MiB or more in size is considered "large" for the purposes of this - option. - -The default value is ``auto``. diff --git a/dev/building.rst b/dev/building.rst index d514d1bff..879521174 100644 --- a/dev/building.rst +++ b/dev/building.rst @@ -32,7 +32,10 @@ Prerequisites - The latest stable version of Go. The previous stable version should also work; older versions will likely not work. This largely follows Go's `Release Policy `__. -- Git +- Usually, a C compiler. Syncthing uses SQLite for storing data and the + most common implementation is written in C. +- Git, unless you're building from a downloaded source tarball that includes + a ``RELEASE`` marker with the version information. - If you want to build Debian packages FPM is required. See FPM's `installation information `__. - To build Windows executables, installing `goversioninfo @@ -44,11 +47,6 @@ If you're not already a Go developer, the easiest way to get going is to download the latest version of Go as instructed in https://go.dev/doc/install. -.. note:: - Because Syncthing uses Go modules you do not need to set or care about "GOPATH". - However, the GOPATH still defaults to ``~/go`` and you'd be best to *not* - put your Syncthing source in there, for now. - Building (Unix) --------------- @@ -109,7 +107,13 @@ The following ``build.go`` subcommands and options exist. ``go run build.go build`` Builds just the named target, or ``syncthing`` by default, to the current directory. Use this when cross compiling, with parameters for what to cross - compile to: ``go run build.go --goos linux --goarch 386 build``. + compile to: ``go run build.go --goos linux --goarch 386 build``. When + cross compiling you need to have a C compiler for the target platform, and + you need to let Go know how to invoke it. The ``--cc`` flag to build.go + controls this, together with the environment variable ``CGO_ENABLED=1``. + For example, to cross build from Linux to Windows using Zig as the C + compiler, ``go run build.go -goos windows -goarch amd64 -cc "zig cc + -target x86_64-windows" build``. ``go run build.go test`` Runs the tests. @@ -120,11 +124,13 @@ The following ``build.go`` subcommands and options exist. ``go run build.go tar`` Creates a Syncthing tar.gz dist file in the current directory. Assumes a - Unixy build. + Unixy build. Otherwise same considerations and parameters as for the + ``build`` subcommand. ``go run build.go zip`` Creates a Syncthing zip dist file in the current directory. Assumes a - Windows build. + Windows build. Otherwise same considerations and parameters as for the + ``build`` subcommand. The options ``--no-upgrade``, ``--goos`` and ``--goarch`` can be given to influence ``build``, ``tar`` and ``zip``. Examples: diff --git a/includes/env-vars.rst b/includes/env-vars.rst index e0fa06aeb..c28fcece3 100644 --- a/includes/env-vars.rst +++ b/includes/env-vars.rst @@ -8,8 +8,6 @@ STTRACE Main and operational facilities: config Configuration loading and saving. - db - The database layer. main Main package. model @@ -51,6 +49,8 @@ STTRACE REST API. sha256 SHA256 hashing package (this facility currently unused). + sqlite + The SQLite database stats Persistent device and folder statistics. sync @@ -63,65 +63,14 @@ STTRACE all All of the above. -STBLOCKPROFILE - Write block profiles to ``block-$pid-$timestamp.pprof`` every 20 seconds. - -STCPUPROFILE - Write a CPU profile to ``cpu-$pid.pprof`` on exit. - -STDEADLOCKTIMEOUT - Used for debugging internal deadlocks; sets debug sensitivity. Use only - under direction of a developer. - STLOCKTHRESHOLD Used for debugging internal deadlocks; sets debug sensitivity. Use only under direction of a developer. -STGUIADDRESS - Override GUI listen address. Equivalent to passing :option:`--gui-address`. - -STGUIAPIKEY - Override the API key needed to access the GUI / REST API. Equivalent to - passing :option:`--gui-apikey`. - -STGUIASSETS - Directory to load GUI assets from. Overrides compiled in assets. Useful for - developing webgui, commonly use ``STGUIASSETS=gui bin/syncthing``. - -STHEAPPROFILE - Write heap profiles to ``heap-$pid-$timestamp.pprof`` each time heap usage - increases. - -STNODEFAULTFOLDER - Don't create a default folder when starting for the first time. This - variable will be ignored anytime after the first run. Equivalent to the - :option:`--no-default-folder` flag. - -STNORESTART - Equivalent to the :option:`--no-restart` flag. - -STNOUPGRADE - Disable automatic upgrades. Equivalent to the :option:`--no-upgrade` flag. - -STPROFILER - Set to a listen address such as "127.0.0.1:9090" to start the profiler with - HTTP access, which then can be reached at - http://localhost:9090/debug/pprof. See ``go tool pprof`` for more - information. - -STPERFSTATS - Write running performance statistics to ``perf-$pid.csv``. Not supported on - Windows. - -STRECHECKDBEVERY - Time before folder statistics (file, dir, ... counts) are recalculated from - scratch. The given duration must be parseable by Go's ``time.ParseDuration``. If - missing or not parseable, the default value of 1 month is used. To force - recalculation on every startup, set it to ``1s``. - -STGCINDIRECTEVERY - Sets the time interval in between database garbage collection runs. The - given duration must be parseable by Go's ``time.ParseDuration``. +STVERSIONEXTRA + Add extra information to the version string in logs and the version line + in the GUI. Can be set to the name of a wrapper or tool controlling + syncthing to communicate this to the end user. GOMAXPROCS Set the maximum number of CPU cores to use. Defaults to all available CPU diff --git a/rest/system-config-get.rst b/rest/system-config-get.rst index 02723f9c7..17a3891b1 100644 --- a/rest/system-config-get.rst +++ b/rest/system-config-get.rst @@ -50,9 +50,7 @@ Returns the current configuration. "pullerPauseS": 0, "maxConflicts": -1, "disableSparseFiles": false, - "disableTempIndexes": false, "paused": false, - "weakHashThresholdPct": 25, "markerName": ".stfolder", "copyOwnershipFromParent": false, "modTimeWindowS": 0, @@ -171,7 +169,6 @@ Returns the current configuration. "stunServers": [ "default" ], - "databaseTuning": "auto", "maxConcurrentIncomingRequestKiB": 0, "announceLANAddresses": true, "sendFullIndexOnUpgrade": false, @@ -226,9 +223,7 @@ Returns the current configuration. "pullerPauseS": 0, "maxConflicts": 10, "disableSparseFiles": false, - "disableTempIndexes": false, "paused": false, - "weakHashThresholdPct": 25, "markerName": ".stfolder", "copyOwnershipFromParent": false, "modTimeWindowS": 0, diff --git a/rest/system-paths-get.rst b/rest/system-paths-get.rst index f27f3f321..edb858c23 100644 --- a/rest/system-paths-get.rst +++ b/rest/system-paths-get.rst @@ -16,7 +16,7 @@ and others. "certFile": "/home/user/.config/syncthing/cert.pem", "config": "/home/user/.config/syncthing/config.xml", "csrfTokens": "/home/user/.config/syncthing/csrftokens.txt", - "database": "/home/user/.local/share/syncthing/index-v0.14.0.db", + "database": "/home/user/.local/share/syncthing/index-v2", "defFolder": "/home/user/Sync", "guiAssets": "/home/user/src/syncthing/gui", "httpsCertFile": "/home/user/.config/syncthing/https-cert.pem", diff --git a/specs/bep-v1.rst b/specs/bep-v1.rst index a384769ec..1818b47a5 100644 --- a/specs/bep-v1.rst +++ b/specs/bep-v1.rst @@ -235,13 +235,11 @@ Protocol Buffer Schema } message Folder { - string id = 1; - string label = 2; - bool read_only = 3; - bool ignore_permissions = 4; - bool ignore_delete = 5; - bool disable_temp_indexes = 6; - bool paused = 7; + string id = 1; + string label = 2; + FolderType type = 3; + FolderStopReason stop_reason = 7; + reserved 4 to 6; repeated Device devices = 16; } @@ -265,6 +263,18 @@ Protocol Buffer Schema ALWAYS = 2; } + enum FolderType { + FOLDER_TYPE_SEND_RECEIVE = 0; + FOLDER_TYPE_SEND_ONLY = 1; + FOLDER_TYPE_RECEIVE_ONLY = 2; + FOLDER_TYPE_RECEIVE_ENCRYPTED = 3; + } + + enum FolderStopReason { + FOLDER_STOP_REASON_RUNNING = 0; + FOLDER_STOP_REASON_PAUSED = 1; + } + Fields (Cluster Config Message) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -284,20 +294,12 @@ the folder. The **label** field contains the folder label, the human readable name of the folder. -The **read_only** field is set for folders that the device will accept no -updates from the network for. - -The **ignore_permissions** field is set for folders that the device will not -accept or announce file permissions for. - -The **ignore_delete** field is set for folders that the device will ignore -deletes for. - -The **disable_temp_indexes** field is set for folders that will not dispatch -and do not wish to receive progress updates about partially downloaded files -via Download Progress messages. +The **folder_type** field contains the type of the folder as defined by the +FolderType enumeration. -The **paused** field is set for folders that are currently paused. +The **stop_reason** field is set for folders that are currently stopped for +any reason. The zero means that the folder is not stopped; further reasons +are defined by the FolderStopReason enumeration. The **devices** field is a list of devices participating in sharing this folder. @@ -401,10 +403,10 @@ Protocol Buffer Schema } message BlockInfo { - int64 offset = 1; - int32 size = 2; - bytes hash = 3; - uint32 weak_hash = 4; + int64 offset = 1; + int32 size = 2; + bytes hash = 3; + reserved 4; } message Vector { @@ -511,6 +513,8 @@ Protocol Buffer Schema int32 size = 5; bytes hash = 6; bool from_temporary = 7; + int32 block_no = 9; + reserved 8; } Fields diff --git a/users/config.rst b/users/config.rst index 823f5869b..7553c1495 100644 --- a/users/config.rst +++ b/users/config.rst @@ -63,9 +63,9 @@ corresponding environment variables (``$STDATADIR`` or ``STHOMEDIR``). The database directory contains the following files, among others: -:file:`index-{*}.db` - A directory holding the database with metadata and hashes of the files - currently on disk and available from peers. +:file:`index-{*}` + The database with metadata and hashes of the files currently on disk and + available from peers. :file:`syncthing.log` Log output, on some systems. @@ -112,9 +112,7 @@ The following shows an example of a default configuration file (IDs will differ) 0 -1 false - false false - 25 .stfolder false 0 @@ -188,7 +186,6 @@ The following shows an example of a default configuration file (IDs will differ) 180 20 default - auto 0 true false @@ -217,9 +214,7 @@ The following shows an example of a default configuration file (IDs will differ) 0 10 false - false false - 25 .stfolder false 0 @@ -305,9 +300,7 @@ Folder Element 0 -1 false - false false - 25 .stfolder false 0 @@ -512,23 +505,10 @@ The following child elements may exist: to be sparse on filesystems that support this feature. When set to ``true``, sparse files will not be created. -.. option:: folder.disableTempIndexes - - By default, devices exchange information about blocks available in - transfers that are still in progress, which allows other devices to - download parts of files that are not yet fully downloaded on your own - device, essentially making transfers more torrent like. When set to - ``true``, such information is not exchanged for this folder. - .. option:: folder.paused True if this folder is (temporarily) suspended. -.. option:: folder.weakHashThresholdPct - - Use weak hash if more than the given percentage of the file has changed. Set - to ``-1`` to always use weak hash. Default is ``25``. - .. option:: folder.markerName Name of a directory or file in the folder root to be used as a marker - see @@ -1049,7 +1029,6 @@ Options Element 180 20 default - auto 0 true false @@ -1301,13 +1280,6 @@ The ``options`` element contains all other global configuration options. feature. Set ``false`` to keep Syncthing from sending panic logs on serious troubles. Defaults to ``true``, to help the developers troubleshoot. -.. option:: options.databaseTuning - - Controls how Syncthing uses the backend key-value database that stores the - index data and other persistent data it needs. The available options and - implications are explained in a :doc:`separate chapter - `. - .. option:: options.maxConcurrentIncomingRequestKiB This limits how many bytes we have "in the air" in the form of response data @@ -1391,9 +1363,7 @@ Defaults Element 0 10 false - false false - 25 .stfolder false 0 diff --git a/users/syncing.rst b/users/syncing.rst index 9d6034556..9c38fda4d 100644 --- a/users/syncing.rst +++ b/users/syncing.rst @@ -69,8 +69,8 @@ version of the file. This version is called the *global* version and is the one that each device strives to be up to date with. This information is kept in the *index database*, which is stored in the -configuration directory and called ``index-vx.y.z.db`` (for some version -x.y.z which may not be exactly the version of Syncthing you're running). +configuration or data directory and called ``index-*``, with some version +number in place of the asterisk. When new index data is received from other devices Syncthing recalculates which version for each file should be the global version, and compares this @@ -91,8 +91,8 @@ The file with the older modification time will be marked as the conflicting file and thus be renamed. If the modification times are equal, the file originating from the device which has the larger value of the first 63 bits for its device ID will be marked as the conflicting file. If the conflict is between a -modification and a deletion of the file, the modified file always wins and is -resurrected without renaming on the device where it was deleted. +modification and a deletion of the file, and the deletion wins the conflict +resolution, the file is renamed to a conflict copy as above. Beware that the ``.sync-conflict--