-
Notifications
You must be signed in to change notification settings - Fork 131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Clean up path processing #319
base: develop
Are you sure you want to change the base?
Commits on Oct 14, 2018
-
Use explicit types for some integer constants
The following constant expression: 1024 * 1024 should be evaluated by the compiler at runtime as though it were: (int)1024 * (int)1024 and one would expect the result to be: (int)1048576 However, strictly speaking, an `int' is only required to be able to represent at least the number 32767. However, a `long int' must be able to represent at least `2147483647'. Thus, strictly speaking, proper C code should specify that at least one of the integer constants is to be represented as a `long int': 1024L * 1024 Consider, also, the following program: #include <limits.h> // UINT_MAX #include <stdio.h> // printf #define PRINT_SIZE_OF(type) \ printf("sizeof(" #type ") = %u\n", (unsigned int)sizeof(type)) #define PRINT_UNSIGNED_LONG_LONG_INT(expr) \ printf(#expr " = %llu\n", (unsigned long long int)(expr)) int main() { PRINT_SIZE_OF(int); PRINT_SIZE_OF(long long int); PRINT_UNSIGNED_LONG_LONG_INT(UINT_MAX + 1); PRINT_UNSIGNED_LONG_LONG_INT(UINT_MAX + 1ULL); PRINT_UNSIGNED_LONG_LONG_INT(65536U * 65536); PRINT_UNSIGNED_LONG_LONG_INT(65536ULL * 65536); } That program might produce the following output: sizeof(int) = 4 sizeof(long long int) = 8 UINT_MAX + 1 = 0 UINT_MAX + 1ULL = 4294967296 65536U * 65536 = 0 65536ULL * 65536 = 4294967296 As you can see, specifying the type of an integer constant can be important. Take, for example, one of the above calculations: 65536U * 65536 Without the "U" suffix to specify that the integer constant has an unsigned type, the compiler would compute: (int)65536 * (int)65536 On the same machine, this would result in overflow of a signed integer type, which invokes undefined behavior; a good compiler would probably warn about it, or even produce an error.
Configuration menu - View commit details
-
Copy full SHA for ff9a53d - Browse repository at this point
Copy the full SHA ff9a53dView commit details
Commits on Oct 15, 2018
-
lib/cmdline.c: Fix error message
The main problem was that the error message used `Negativ' rather than `Negative'. However, the message was also a little too playful; this commit changes the entire message to something a little more dour.
Configuration menu - View commit details
-
Copy full SHA for d085572 - Browse repository at this point
Copy the full SHA d085572View commit details -
Configuration menu - View commit details
-
Copy full SHA for 63bc4be - Browse repository at this point
Copy the full SHA 63bc4beView commit details
Commits on Jan 22, 2019
-
lib/cfg.*: Clean up `rm_cfg_free_paths()'
This simply introduces `const' discipline. Also, `g_assert(cfg);' has been inserted.
Configuration menu - View commit details
-
Copy full SHA for a3e9d5c - Browse repository at this point
Copy the full SHA a3e9d5cView commit details -
lib/cfg.*:
rm_cfg_add_path()' now returns
bool'It was already returning values as though it did.
Configuration menu - View commit details
-
Copy full SHA for 93541fc - Browse repository at this point
Copy the full SHA 93541fcView commit details -
lib/cmdline.c: return the status of `rm_cfg_add_path()'
Before, `rm_cmd_parse_replay()' simply ignored whether there was a failure; now, it both returns a status and sets an error message.
Configuration menu - View commit details
-
Copy full SHA for 9c51e9d - Browse repository at this point
Copy the full SHA 9c51e9dView commit details -
lib/cfg.*: Refactor `rm_cfg_add_path()'
This commit introduces a number of finer-grained functions related to resolving and verifying paths, and then uses those functions not only to refactor `rm_cfg_add_path()', but also to introduce a new function, `rm_cfg_prepend_json()'. The finer-grained functions are defined in a new source file: lib/path-funcs.h Similarly, `RmPath' has been moved to its own header: lib/path.h Additionally, some of the `#include' directives have been cleaned up as part of this refactoring; if source file `X' uses an identifier whose declaration comes from source file `Y', then `X' *always* includes `Y' explicitly (even if `Y' is included indirectly through some other source file), and each `#include' directive is associated with a comment that lists each such identifier that it provides.
Configuration menu - View commit details
-
Copy full SHA for f8629f1 - Browse repository at this point
Copy the full SHA f8629f1View commit details
Commits on Jan 23, 2019
-
lib/cfg.*: Refactor `rm_cfg_add_path()'
This commit introduces a number of finer-grained functions related to resolving and verifying paths, and then uses those functions not only to refactor `rm_cfg_add_path()', but also to introduce a new function, `rm_cfg_prepend_json()'. The finer-grained functions are defined in a new source file: lib/path-funcs.h Similarly, `RmPath' has been moved to its own header: lib/path.h Additionally, some of the `#include' directives have been cleaned up as part of this refactoring; if source file `X' uses an identifier whose declaration comes from source file `Y', then `X' *always* includes `Y' explicitly (even if `Y' is included indirectly through some other source file), and each `#include' directive is associated with a comment that lists each such identifier that it provides.
Configuration menu - View commit details
-
Copy full SHA for 3539636 - Browse repository at this point
Copy the full SHA 3539636View commit details -
lib/cmdline.c: Move `rm_cmd_read_paths_from_stdin()'
Move it closer to `rm_cmd_set_paths()'.
Configuration menu - View commit details
-
Copy full SHA for fb1ba2d - Browse repository at this point
Copy the full SHA fb1ba2dView commit details -
lib/cmdline.c: Simplify size in call to `malloc()'
By definition, `sizeof(char)' evaluates to 1.
Configuration menu - View commit details
-
Copy full SHA for ee9b9a0 - Browse repository at this point
Copy the full SHA ee9b9a0View commit details -
lib/cmdline.c: Replace
RmSession' with
RmCfg' in `rm_cmd_set_paths*()'A pointer to an `RmSession' object was being passed around, but only a pointer to its associated `RmCfg' object is needed.
Configuration menu - View commit details
-
Copy full SHA for f8cea9f - Browse repository at this point
Copy the full SHA f8cea9fView commit details -
lib/cmdline.c: Abstract out struct `rm_cmd_set_paths_vars'
Variables that need to be shared across `rm_cmd_set_paths*()' may be placed in an instance of this struct.
Configuration menu - View commit details
-
Copy full SHA for 25db8f6 - Browse repository at this point
Copy the full SHA 25db8f6View commit details -
lib/cmdline.c: Add error handling to `rm_cmd_set_paths_from_stdin()'
Before, it was just ignoring failures in `malloc()' and `getdelim()'. Also, one of the error messages in `lib/treemerge.c' has been set to match the error message introduced here.
Configuration menu - View commit details
-
Copy full SHA for 1e81e41 - Browse repository at this point
Copy the full SHA 1e81e41View commit details -
lib/cfg.h: Remove member `read_stdin'
It's only used locally in one function, so it can just be a local variable in that one function.
Configuration menu - View commit details
-
Copy full SHA for fb31ce0 - Browse repository at this point
Copy the full SHA fb31ce0View commit details -
Configuration menu - View commit details
-
Copy full SHA for d4f7805 - Browse repository at this point
Copy the full SHA d4f7805View commit details -
lib/cmdline.c: Move constant loop condition (`paths') out of the loop
The compiler probably already does this, but I find it clearer and more satisfying to do this explicitly, especially since it allows avoiding a call to `g_strfreev()', rather than relying on that function to check `paths' for us again. For the sake of clarity in the diff, the existing code has not yet been indented to reflect the fact that it's part of the `if' statement's consequent.
Configuration menu - View commit details
-
Copy full SHA for 13304f7 - Browse repository at this point
Copy the full SHA 13304f7View commit details -
lib/cmdline.c: Indent code to reflect the fact that it's in a block
You can check that only white space changed: $ git diff HEAD^ --ignore-space-change && echo Nothing\ changed Nothing changed
Configuration menu - View commit details
-
Copy full SHA for a9f4de0 - Browse repository at this point
Copy the full SHA a9f4de0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 05191ee - Browse repository at this point
Copy the full SHA 05191eeView commit details -
lib/cmdline.c: Free each command-line path as it is processed.
On each iteration of the loop, a path string is freed when processing is finished, and then the whole list of strings is simply freed without having to iterate through the list again to free each string.
Configuration menu - View commit details
-
Copy full SHA for 4f14e8a - Browse repository at this point
Copy the full SHA 4f14e8aView commit details -
lib/cfg-funcs.h: Move
cfg->replay' and
cfg->path_count++'These variables were handled directly in `rm_cfg_prepend_path()'; now, they're handled directly in `rm_cmd_set_paths*()'. This has allowed for optimizing the call to `rm_cfg_prepend_path()' that is used for adding the current working directory when no other path has been supplied by the user; there's no reason to check whether the current working directory is a JSON file.
Configuration menu - View commit details
-
Copy full SHA for 5dafe3a - Browse repository at this point
Copy the full SHA 5dafe3aView commit details -
lib/path.h: s/idx/index/ in `RmPath'
The field `idx' of struct `RmPath' is now `index'.
Configuration menu - View commit details
-
Copy full SHA for a5698fd - Browse repository at this point
Copy the full SHA a5698fdView commit details -
lib/cmdline.c: Move
cfg->replay' and
cfg->path_count++' out of loopsThough an optimizing compiler might do this already, one might as well just do it explicitly, so as not to depend on the whims of any particular compiler.
Configuration menu - View commit details
-
Copy full SHA for 322d7cc - Browse repository at this point
Copy the full SHA 322d7ccView commit details -
lib/path.h: s/treat_as_single_vol/single_volume/ in `RmPath'
The field `treat_as_single_vol' of struct `RmPath' is now `single_volume'.
Configuration menu - View commit details
-
Copy full SHA for 67e0738 - Browse repository at this point
Copy the full SHA 67e0738View commit details -
lib/cfg.h: Use `RmCfg::path_count' more purposefully
Now, member `path_count' of a struct `RmCfg' has a more guaranteed and expected purpose: During and after the user-input processing of non-option path arguments, it represents the number of items in the associated list `paths'. This has allowed for the removal of a call to `g_slist_length()'.
Configuration menu - View commit details
-
Copy full SHA for e39fa29 - Browse repository at this point
Copy the full SHA e39fa29View commit details -
lib/cfg.c: git mv lib/cfg.c lib/cfg-funcs.h
This breaks the project, and is thus meant to be merged into a working commit.
Configuration menu - View commit details
-
Copy full SHA for 2f72e34 - Browse repository at this point
Copy the full SHA 2f72e34View commit details -
lib/cfg.*: Rename
rm_cfg_add_path' to
rm_cfg_prepend_path()'This makes it consistent with `rm_cfg_prepend_json()', and it also implies that the list of files is being constructed such that it will be reversed with respect to the order in which paths are provided by the user.
Configuration menu - View commit details
-
Copy full SHA for eb09ea9 - Browse repository at this point
Copy the full SHA eb09ea9View commit details -
lib/cmdline.c: Abstract out `rm_cmd_set_paths_from_cmdline()'
Code from `rm_cmd_set_paths()' has been moved to its own function: rm_cmd_set_paths_from_cmdline()
Configuration menu - View commit details
-
Copy full SHA for bc825f3 - Browse repository at this point
Copy the full SHA bc825f3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 74daf3b - Browse repository at this point
Copy the full SHA 74daf3bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6dcbfde - Browse repository at this point
Copy the full SHA 6dcbfdeView commit details -
lib/cmdline.c: Replace `strcmp()' with direct character comparisons
The way `rmlint' currently works is that it produces a shell script that sometimes performs its work by running `rmlint' many times with varying arguments; `rmlint' can be run a large number of times, and each invocation involves argument processing whereby each path argument undergoes multiple char-by-char comparisons in search of a match for a special, well-known value (e.g., "-" or "//"). Therefore, this commit optimizes the process by using tailored, idiomatic C-string calculations in lieu of invoking a generic, open-ended, arbitrary comparison function; this commit replaces the Standard C Library's `strcmp()' with boolean operators and character comparisons. The calculations are idiomatic in that they are a familiar way of processing strings, but are somewhat particular to C code. To aid reading this code, comments have been embedded to clarify the purpose, as in the following: if(/* path is "-" */ path[0] == '-' && path[1] == 0) It's true that an optimizing compiler might perform this substitution automatically, or it might even make a better, platform-specific optimization. However, this seems unlikely based on the following rudimentary benchmarks; first define some Bash functions: # Copy this script with the following command line, where "$THIS" # expands to the revision of this commit log message: # # git log -1 --format=%b "$THIS" | # sed -n '/^ #-/,/^ #-/p' | cut -c3- >/tmp/script.sh # #---------------------------------------------------------------- make_test() { # Note that the body of this function is # explicitly indented with tab characters # for the benefit of the "<<-" operator. local name="$1" expression="$2" cat >"$name".c <<-EOF #include <stdbool.h> // bool #include <string.h> // strcmp #define ITERATIONS 10000000 // 10 million int main() { int result = 0; char string[3] = {0}; const char *const volatile pointer = string; bool toggle = 0; for (long i = 0; i < ITERATIONS; ++i) { string[0] = string[1] = '/' + toggle; toggle = !toggle; const char *const s = pointer; result += $expression; } return result; } EOF } build() { local name="$1"; shift gcc -std=c99 -pedantic -Wall -static -fno-plt -fno-pie "$@" \ "$name".c -o "$name" } build_all() { build strcmp "$@" && build equals "$@"; } run() { time -p ./"$1"; } benchmark() { echo =========== "$@" run strcmp echo ----------- run equals } build_and_benchmark() { build_all "$@" || return 1 benchmark "$@"; return 0 } #---------------------------------------------------------------- Then, create and run some benchmarks: make_test strcmp 'strcmp(s,"//")' make_test equals "(s[0] == '/' && s[1] == '/' && s[2] == 0)" gcc --version | head -n 1 build_and_benchmark -O0 build_and_benchmark -Og build_and_benchmark -O1 build_and_benchmark -O2 All together, this produces the following output on my [very old and slow] system; at each optimization level, timing information is provided first for `strcmp()' and then for the manual calculation. Perversely, optimization seems to make `strcmp()' slower, not faster; in contrast, optimization drastically improves the performance of manual calculation, which is also always faster than using `strcmp()'. gcc (GCC) 7.3.1 20180312 =========== -O0 real 0.35 user 0.35 sys 0.00 ----------- real 0.24 user 0.24 sys 0.00 =========== -Og real 0.58 user 0.57 sys 0.00 ----------- real 0.15 user 0.15 sys 0.00 =========== -O1 real 0.60 user 0.59 sys 0.00 ----------- real 0.16 user 0.16 sys 0.00 =========== -O2 real 0.58 user 0.57 sys 0.00 ----------- real 0.07 user 0.07 sys 0.00 Assembler code for the unoptimized case can be produced with the following: build_all -S -O0 cat strcmp # review cat equals Here is the meat of the comparison: strcmp equals ------------------------- | ------------------------- .LC0: | movl -20(%ebp), %eax .string "//" | movzbl (%eax), %eax ... | cmpb $47, %al // s[0] == '/' pushl $.LC0 | jne .L3 pushl -20(%ebp) | movl -20(%ebp), %eax call *strcmp@GOT | addl $1, %eax | movzbl (%eax), %eax | cmpb $47, %al // s[1] == '/' | jne .L3 | movl -20(%ebp), %eax | addl $2, %eax | movzbl (%eax), %eax | testb %al, %al // s[2] == 0 | jne .L3 | movl $1, %eax | jmp .L4 | .L3: | movl $0, %eax As you can see, one is calling a function, while the other is performing the manual calculations. Now, consider the optimized version: build_all -S -O2 cat strcmp cat equals This yields: strcmp equals ------------------------- | ------------------------- .LC0: | movl -20(%ebp), %ebx .string "//" | cmpb $47, (%ebx) ... | jne .L2 movl -36(%ebp), %esi | cmpb $47, 1(%ebx) movl $.LC0, %edi | jne .L2 ... | cmpb $1, 2(%ebx) movl $3, %ecx | repz cmpsb | seta %cl | setb %bl | subl %ebx, %ecx | movsbl %cl, %ecx | There is no longer a call to `strcmp()', but the compiler has substituted what is essentially an exact replica of `strcmp()' in terms of x86 assember code; the `repz cmpsb' iterates through 2 strings in memory, comparing the bytes, stopping when there is a mismatch. Strangely, one would think that a native x86 instruction sequence might be inherently faster, but the benchmarks reveal that the naive integer comparisons are much faster. I suspect the reason for this is 2-fold: * Perhaps, `repz cmpsb' is performing a lot more memory access; in contrast, each naive integer comparison instruction can encode a comparand directly (e.g., the number 47, which is ASCII for '/'). * Maybe, `repz cmpsb' cannot benefit from instruction pipelining as readily as separate integer comparisons. Perhaps the GNU GCC team should re-consider how `strcmp()' is optimized. Anyway, the results are clear: It's better to do these small calculations by hand, rather than rely on the compiler to do something magical.
Configuration menu - View commit details
-
Copy full SHA for 4182c59 - Browse repository at this point
Copy the full SHA 4182c59View commit details -
lib/cfg.c -> lib/cfg-funcs.h: Make functions inlinable
Each of these functions is pretty much single-purpose, and should therefore be inlined at the few call sites.
Configuration menu - View commit details
-
Copy full SHA for 605b5a9 - Browse repository at this point
Copy the full SHA 605b5a9View commit details -
lib/cmdline.c: Tell compiler to optimize for the value of `replay'
When `cfg->replay' is false, there's no reason to check whether a path is a `json' file.
Configuration menu - View commit details
-
Copy full SHA for a4e4b88 - Browse repository at this point
Copy the full SHA a4e4b88View commit details -
Configuration menu - View commit details
-
Copy full SHA for 30e7156 - Browse repository at this point
Copy the full SHA 30e7156View commit details -
lib/cmdline.c: INLINE `rm_cmd_set_paths()'
It's a static function that is used in only place.
Configuration menu - View commit details
-
Copy full SHA for 2c24f77 - Browse repository at this point
Copy the full SHA 2c24f77View commit details -
lib/cfg.c -> lib/cfg-funcs.h: Refactor functions
The functions in question are now finer-grained and inlinable, which is worthwhile because these functions are only used in basically one part of the program, and thus exist mainly as a matter of organization (abstraction, documentation, etc.).
Configuration menu - View commit details
-
Copy full SHA for 3b1098c - Browse repository at this point
Copy the full SHA 3b1098cView commit details -
lib/cmdline.c: Refactor `rm_cmd_set_paths*()'
The code has been organized into finer-grained functions/macros, and has thereby been easily optimized for certain cases.
Configuration menu - View commit details
-
Copy full SHA for c0ba1b5 - Browse repository at this point
Copy the full SHA c0ba1b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0d332c9 - Browse repository at this point
Copy the full SHA 0d332c9View commit details