2020-06-29.md

< 2020-06-29 >

2,258,919 events, 1,108,140 push events, 1,832,913 commit messages, 139,253,567 characters

LuanHalaiko/LH-Kernel-wt88047@4ac427c201...

Monday 2020-06-29 03:36:16 by googyanas

fs/sync: Make sync() satisfy many requests with one invocation

Dave Jones reported RCU stalls, overly long hrtimer interrupts, and amazingly long NMI handlers from a trinity-induced workload involving lots of concurrent sync() calls (https://lkml.org/lkml/2013/7/23/369). There are any number of things that one might do to make sync() behave better under high levels of contention, but it is also the case that multiple concurrent sync() system calls can be satisfied by a single sys_sync() invocation.

Given that this situation is reminiscent of rcu_barrier(), this commit applies the rcu_barrier() approach to sys_sync(). This approach uses a global mutex and a sequence counter. The mutex is held across the sync() operation, which eliminates contention between concurrent sync() operations. The counter is incremented at the beginning and end of each sync() operation, so that it is odd while a sync() operation is in progress and even otherwise, just like sequence locks.

The code that used to be in sys_sync() is now in do_sync(), and sys_sync() now handles the concurrency. The sys_sync() function first takes a snapshot of the counter, then acquires the mutex, and then takes another snapshot of the counter. If the values of the two snapshots indicate that a full do_sync() executed during the mutex acquisition, the sys_sync() function releases the mutex and returns ("Our work is done!"). Otherwise, sys_sync() increments the counter, invokes do_sync(), and increments the counter again.

This approach allows a single call to do_sync() to satisfy an arbitrarily large number of sync() system calls, which should eliminate issues due to large numbers of concurrent invocations of the sync() system call.

Changes since v1 (https://lkml.org/lkml/2013/7/24/683):

o Add a pair of memory barriers to keep the increments from bleeding into the do_sync() code. (The failure probability is insanely low, but when you have several hundred million devices running Linux, you can expect several hundred instances of one-in-a-million failures.)

o Actually CC some people who have experience in this area.

Reported-by: Dave Jones [email protected] Signed-off-by: Paul E. McKenney [email protected] Cc: Alexander Viro [email protected] Cc: Christoph Hellwig [email protected] Cc: Jan Kara [email protected] Cc: Curt Wohlgemuth [email protected] Cc: Jens Axboe [email protected] Cc: [email protected]

Signed-off-by: Paul Reioux [email protected]

---

Stephen-M-Anderson/Olaf-the-Unhuggable@7d46b3b6f7...

Monday 2020-06-29 04:57:16 by Chrismorales116

Started fucking with configurable joints

This could work but I'm sick of looking at code for the day. Gonna draw some dope fuckin diagrams to describe how I wanna code this shit in the morning, its gonna be fuckin wild.

---

smxi/inxi@0645c3a7a6...

Monday 2020-06-29 05:22:43 by Harald Hope

New version, new man, huge update, bug fixes, cleanups, updates!!

What started as a relatively minor issue report ended up with a refactor of big chunks of some of the oldest code and logic in inxi.

So many bugs and fixes, updates, and enhancements, that I will probably miss some when I try to list them.

Bugs:

1. In the process of fixing an issue about sudo use triggering server admin emails on failure, when --sudo/--no-sudo and their respective configuration items were added, sudo was inadvertently disabled because the test ran before the options were processed, which meant the condition to set sudo data was always false, so sudo for internal use was never set. The solution was to set a flag in the option handler and set sudo after options or configs run.

2. Issue #219 reported gentoo and one other repo type would fail to show enabled repos, and would show an error as well, this was due to forgetting to make the match test case insensitive. If only all bugs were this easy to fix!!

3. I'd seen this bug before, and couldn't figure out why it existed. It turned out that the partition blacklist filters were running fine in the main partition data tool, but I had forgotten to add in corresponding lsblk partition data filters, lol, so when the logic went back and double checked for missing partitions [this feature had been if i remember right to be able to show hidden partitions, which the standard method didn't see, but lsblk did, anyway, when the double check and add missing partitions logic ran, inxi was putting back in the blacklisted partitions every time, despite the original blacklists working well and as intended. This was fixed by adding in all the required fs type blacklists, then adding in comments above each black list reminding coders that if they add or remove from one blacklist, they have to do the same on the other.

4. Found while testing something unrelated on older vm, the fallback case for cpu bugs, which was supposed to show the basic /proc/cpuinfo cpu bugs, was failing inexplicably because the data was simply being put into the wrong variable name, sigh.

Fixes:

1. While not technically an inxi bug, it would certainly appear that way to anyone who triggered it. We'd gotten issue reports before on this, but they were never complete, so couldn't figure it out. Basically, if someone puts inxi into a simple script that is $PATH [this was the missing fact needed to actually trigger this bug in order to fix it], the script [not inxi], will then enter into an endless loop as inxi queries it for its version number using

<script name> --version. This issue didn't happen if the script calling inxi was not in PATH, which is why I'd never been able to figure it out before. Only simple scripts with no argument handlers could trigger this scenario, and only if they were in PATH. Fixing this required refactoring the entire start get_shell_data logic, which ended up with a full refactor of the program_version logic as well. The fix was to expand the list of shells known by inxi so it would be able to recognize when it was in a shell running a script running inxi. This resulted in several real improvements, for instance, inxi will now almost always be able to determine the actual shell running inxi, even when started by something else. It will also never use --version attempts on programs it does not know about in a whitelist. So we lose slightly the abilty to get version data on unknown shells, but we gain inxi never being able to trigger such an infinite loop situation. 2. As part of the program_version refactor, a long standing failure to get ksh, lksh, loksh, pdksh, and the related posh shells, all of which ID their version numbers only if they are running the command in themselves. The mistake had been having the default shell run that command. These all now correctly identify themselves. 3. As part of the wm upgrades, many small failures to ID version numbers, or even wm's, in some cases, were discovered when testing, and corrected. Some I had not tested, like qtile, and the lisp variants, were not being detected correctly by the tests due to the way python or lisp items are listed in ps aux. 4. As part of the wm update and program_version refactor, updated and simplified many desktop and wm detections and logic blocks. Ideally this makes them more preditable and easy to work on for the future. 5. As some last tunings for the new -y1 key: value pair per line output option, fixed some small glitches in -b indentation. Also improved RAID indenting, and Weather, and made it all very clean and predictable in terms of indentations. 6. Something I'd slightly noticed but never done anything about, while testing desktop fixes, I realized that for Desktop: item, dm: is a secondary data type, but if it's Console:, then DM: is a primary data type, not a secondary one. So now if Console: it becomes DM: whic makes sense, previously it implied a dm: was used to start the console, which was silly. Also, since often the reason it's Console: with no dm in the first place is that it's a server with no dm. So now if console, and no dm detected, rather than showing DM: N/A it just doesn't show dm at all. 7. As part of the overall core refactor, the print_data logic was also refactored and simplified, by making -y1 a first class citizen, it led to significantly different way of being able to present inxi data on your screen, and now print_data logic is cleaner and reflects these changes more natively, all the initial hacks to get this working were removed, and the logic was made to be core, not tacked on. 8. A small thing also revealed in issue #219, battery data was not being trimmed, not sure how I missed that, but in some cases, space padding was in the values and was not removed, which leads to silly looking inxi output. 9. Several massive internal optimizations, which were tested heavily, led to in one case, 8-900x faster execution the second time a data structure is used, previously in program_values the entire list was loaded each time program_values was called, now it's loaded into a variable on first load and the variable is used for the tests after that. This was also done for the vendor_version for disk vendors, which also features a very long data structure which can be loaded > 1 times for instances where a system has > 1 disk. I also tested while I was at it, to see if loading these tyeps of data structures, arrays of arrays, or hashes of arrays, by reference, or by dereferencing their arrays, was faster, and it proved that it's about 20% faster to not dereference them, but to use them directly. So I've switched a number of the fixed data structures internally do use that method. Another tiny optimization was hard resetting the print_data iterator hash, while this would never matter in the real world, it showed that resetting the iterator hash manually was slightly more efficient than resetting it with a for loop. 10. While not seen inside inxi, I updated and improved a number of the vm's used to test inxi and various software detections, so now I have a good selection, going back to 2008 or so, up to current. This is helpful because things like shells and window managers and desktops come and go, so it is hard to test old detections on new stuff when you can't install those anymore. You'll see these fixes in many of the less well known window managers, and in a few of the better known ones, where in some cases the detections were damaged. 11. As part of the program_version refactor, updated and fixed file based version detections, those, ideally, will almost never be used. Hopefully programmers of things like window managers, shells, and desktops, can learn how to handle --version requests, even though I realize that's a lot harder than copying someone's code and then rebranding as your own project, or whatever excuse people have for not including a --version item in their softaware. Enhancements: 1. As a result of the shell, start shell, shell parent refactors, inxi was able to correctly in most cases deetermine also the user default shell and its verison, so that was added as an -Ixxx option: Shell: ksh v: A_2020.0.0 default: Bash v: 5.0.16 2. As part of the program_version refactor, a more robust version number cleaner was made, which now allows for much more manipulation of the version number string, which sometimes contains, without spaces, non version number ' info right before the actual version. 3. Many more wm IDs were created and tested, and some old virtual machines that were used years ago were used again to test old window managers and their IDs, as well as new vms created to test newer ones. Many version IDs and WM ids were fixed in this process as well. All kinds of new ones added, though the list is basiclaly endless so ideally inxi would only use its internal data tables for window managers that have actual users, or did. 4. First wayland datatype, now it may show Display ID: with -Ga, so far that's the only wayland screen/display data I can get reliably. 5. As part of the shell parent/started in: updates and fixes, added every shell I could find, and installed and tested as many of them as possible to verify that either they have no version method, or that their version method works. This shell logic also is used to determine start parent. Obviously using whitelists of things that can change over time isn't ideal, but there was no way to actually do it otherwise. The best part of the fixes is that it's now remarkably difficult to trick inxi into reporting the wrong shell, and it generally will also get the default shell right, though I found cases in testing where a shell when started replaces the value in $SHELL with itself. 6. I found a much faster and reasonably reliable way to determine toolkits used by gtk desktops, like cinnamon, gnome, and a few others. Test is to get version from gtk-launcher, which is MUCH faster than doing a package version query on the random libgtk toolkit that might be tested, and actually was tested for pacman, apt, and rpm in the old days, but that was removed because it was a silly hack. It's possible that now and then gtk desktops will be 0.0.1 versions off, but in most cases, the version matched, so I decided to restore the tk: item for a selection of gtk or gnome based desktops. So now gtk desktops, except mate, which of course will be using gtk 2 for a while longer, toolkit version should be working again, and the new method works on everything, unlike the old nasty hack that was used, which required package queries and guessing at which gtk lib was actually running the desktop, it was such a slow nasty hack that it was dumped a while ago, but this new method works reliably in most cases and solves most of the issues. 7. As part of the overall program_versions refactor, the package version tester tool was extended to support pacman, dpkg, and rpm, which in practical terms covers most gnu/linux users and systems. Since this feature is literally only used for ASH and DASH shell version detections, it was really just added as a proof of concept, and because it fit in well with the new Package counts feature of -I/-r. 8. Updated for version info a few other programs, added compositors as well. 9. Last but not least!! More disk vendor IDs, more disk vendors!! And found another source to double check vendor IDs, that's good. New Features: 1. For -Ix/-rx, -Ixx/-rxx, -Ia/-ra, now inxi shows package counts for most package managers plus snap, flatpak, and appimage. I didn't test appimage so I'm not 100% sure that works, but the others are all tested and work. If -r, Packages shows in the Repos item as first row, which makes sense, packages, repos, fits. Note that in some systems getting full package counts takes some time so it's an -x option not default. If -rx, -rxx, -ra, package info moved to -r section, and if -Ix, -Ixx, or -Ia, the following data shows: * -Ix or -rx: show total package counts: Packages: 2429 * -Ixx or -rxx: shows Packages then counts by package manager located. If there was only one package manager with packages, the total moves from right after Packages: to the package manager, like: Packages: apt: 3241 but if there were for example 2 or more found, it would show the total then: Packages 3245 apt:3241 snap: 4 * -Ia or -ra: adds package managers with 0 packages managed, those are not show with -xx, and also shows how many of those packages per package manager is a library type lib file. Sample: inxi -Iay1 Info: Processes: 470 Uptime: 8d 10h 42m Memory: 31.38 GiB used: 14.43 GiB (46.0%) Init: systemd v: 245 runlevel: 5 Compilers: gcc: 9.3.0 alt: 5/6/7/8/9 Packages: apt: 3685 lib: 2098 rpm: 0 Shell: Elvish v: 0.13.1+ds1-1 default: Bash v: 5.0.16 running in: kate pinxi: 3.1.04-1 --- ## [Spoopsta/GameJam](https://github.com/Spoopsta/GameJam)@[6c39665262...](https://github.com/Spoopsta/GameJam/commit/6c396652626afc46a05a1c68803f8ef35984aad4) #### Monday 2020-06-29 07:10:40 by BunBoi baking baking and more baking my god why does it take so long i want to die seriously what the hell i just want to make my level look pretty --- ## [lgritz/oiio](https://github.com/lgritz/oiio)@[d788a9e641...](https://github.com/lgritz/oiio/commit/d788a9e641048e736a320e88751a18c18f573382) #### Monday 2020-06-29 07:17:20 by Larry Gritz Terminal I/O of images This is a little kooky maybe, but it was something I was playing a couple years ago, finally inspired by Chris to clean it up and PR it. This patch adds a "term" format which, instead of writing files, outputs a preview image to a true color text terminal window. Is any of this useful? I have no idea. But it's kinda fun. The quick demo of this is: oiiotool myfile.exr -o out.term It will only do something useful if on a color terminal emulator with 24 bit color capability and Unicode. (I've tested it on the gnome-terminal on Linux, and iTerm2 on Mac.) The best quality results are on Mac if you are using iTerm2, it can additionally do an actual pixel output (iTerm2 has as a nonstandard feature the ability to send inline images to the terminal). This is the "iterm2" method, but it's only expected to work on Mac with iTerm2. It will automatically use this method as the default if you are running iTerm2. Next best is (thanks, Chris) is to display two pixels (vertically) in each character cell, by outputting the Unicode "upper half block" glyph (\u2508) with the foreground color set to the top pixel's color and the background color set to the bottom pixel's color. This is the default if you're not on a Mac running iTerm2. To demo (if you're on have iTerm2 but want to see this method): oiiotool myfile.exr -attrib term:method 24bit -o out.term But some terminals and fonts don't support unicode, or don't display this character, or display it with an ugly border. So you can also use an alternate method: oiiotool myfile.exr -attrib term:method 24bit-space -o out.term where doesn't use the unicode trick but instead outputs a space (' ') for each 2 vertical pixels, simply setting the background color to the average of those two pixels. There's also a "256color" method that just uses the 6x6x6 color space in the 256 color palette -- which looks horrible -- and an experimental "dither" which does a half-assed Floyd-Steinberg dithering, horizontally only, and frankly is not an improvement unless you squint really hard. In all cases, the image will automatically be resized to fit in the terminal, as well as converted to sRGB so it looks kinda ok. Along the way, I added a new ImageOutput::supports("procedural") query. If the existing input supports("procedural") query says whether the input image is being generated procedurally and doesn't correspond to an existing file on disk being ead, then the output supports("procedural") is the opposite -- is the output doing something procedural that does not correspond to a file being written. (The specific use is to use this to avoid oiiotool attempting to rename the file to move it into its final position.) Signed-off-by: Larry Gritz --- ## [CaptainJellyBS/BuildingPlayfulWorlds](https://github.com/CaptainJellyBS/BuildingPlayfulWorlds)@[32026e0914...](https://github.com/CaptainJellyBS/BuildingPlayfulWorlds/commit/32026e09143432f9ff252a3c97abc2dd91bdc777) #### Monday 2020-06-29 15:33:44 by Mana TR-0010 -HOLY SHIT ROTATION WORKS? -But now position is fucked, because of positions, which should be fixable --- ## [arnfred/Kind](https://github.com/arnfred/Kind)@[d650bee779...](https://github.com/arnfred/Kind/commit/d650bee77981933dfcd41fcc61c470094933b847) #### Monday 2020-06-29 17:16:50 by Jonas Arnfred implementing typegen using ast:traverse This change reimplements type generation using the generalised framework for traversing ASTs. It's been a trying exersice, partially because of the Erlang compiler's reluctance to include stack traces for run-time map errors. One problem has been stumping me for a while, but I think I might have come up with a pleasing resolution: If a type has both free variables and guards, how do I invoke a subtype? For example, take the following recursive type: ``` Blip a b | A, B -> Blop {k1: A, k2: B} | B, t -> Blop {k1: t, k2: B} ``` Say I want to construct a `Blop` element in my code. How do I invoke it? -- Before pattern matching, I would record which variables where used in the type following the `Blop` tag, and since I didn't have patterns where it's possible to generate new variables, they would always be a subset of the variables for `Blip`. This made it easy to create a type function `Blop` with a subset of `Blip`'s arguments. However with pattern matching we can generate new free variables that have little to do with the variables given to `Blip`. This means the type expression of `Blop` might cover over many more variables than there are arguments to `Blip` and that it's hard to map the relationship between these arguments with the variables of `Blop`. When considering this problem, I saw three solutions: 1. Make it so we can't call `Blop` on it's own if it contains variables. New `Blop`s can only be created indirectly through creating `Blip`s. 2. Make it so that when we construct `Blop`, we copy the pattern matching from `Blip` to create the new free variables. Then we need some logic to map the variables of `Blop` to the arguments of `Blip` so we can tell which of `Blip`s arguments `Blop` should be called with. 3. Make blop a type function of the free variables present in the type that `Blop` covers over. Then use the `scanner` to figure out the constraints on these variables and check that when `Blop` is invoked, these constraints are upheld, so we don't end up creating malformed `Blop`s. I've chosen solution 3 because of a few different reasons: * I think it's important to be able to create `Blop`s * I think it's important that the arguments to the `Blop` function are easy to deduce * I think trying to map Blop's free variables to `Blip`s arguments is gnarly as fuck, and I don't want to complicate my life needlessly It's possible that checking that we don't create malformed `Blop`s isn't so very straightforward. Scanning type statements isn't something the scanner currently supports, but extending that capability comes with a host of advantages, predominantly around the fact that other domain errors in types might be caught. Next steps: * Fix parsing of type clauses. I've discovered that I can't declare a type underneath a type function with clauses currently * Add support for match statements in types (should be pretty straightforward) * Implement type scanner (possibly using ast:traverse as a use-case for whether it's possible to convert the regular scanner) * Extend type generation to general code generation and reuse work to generate code --- ## [sciencehistory/scihist_digicoll](https://github.com/sciencehistory/scihist_digicoll)@[91b391c772...](https://github.com/sciencehistory/scihist_digicoll/commit/91b391c772beb4ca84b66176fc98ccc45a011acb) #### Monday 2020-06-29 17:43:02 by Jonathan Rochkind Freeze blacklight in Gemfile to 7.7.x While BL minor version releases are theoretically intended to be backwards compatible, our experience is that upgrading blacklight even to a minor version frequently causes problems. By locking to 7.7.x, we can run `bundle update` or `bundle update some_arg` to update other dependencies, and bundler will never update blacklight beyond 7.7.x, becuase we've told it not to here. This will let us use bundler for dependency management in a more convenient way, to keep other dependecies up to date, without worrying it'll update blacklight. The downside is that blacklight really will be fixed to 7.7.x unless we take manual action to change the Gemfile to try a new version. So new 7.x's will be out with bugfixes or improvements, but we'll just be stuck on the old one. We will need to periodically remember to try new ones and see if they work, to avoid getting stuck on an old version. Or eventually something will force our hand, we'll need to update to a new version of Blacklight becuase 7.7.x is incompatible with something else we need to update, etc. But for now, we think this is more manageable, we want to be able to periodically run `bundle update` to update ALL our dependencies to latest non-breaking versions, and can't afford to deal with the individual attention that Blacklight minor versions often require to avoid problems or copious deprecation warnings. (Right now the reaosn we are sticking to 7.7.x instead of 7.8.0 is to avoid some copious deprecation warnings that would fill up our logs, which are non-trivial to deal with, in 7.8.0). --- ## [Exerra/exerrawebsite](https://github.com/Exerra/exerrawebsite)@[587c9f66d7...](https://github.com/Exerra/exerrawebsite/commit/587c9f66d76cbc644adaa331fa78a0827ccd1b14) #### Monday 2020-06-29 18:42:34 by SCP-049 fuck this shit ass fucking meta fucking cunt ass shit --- ## [postgres/postgres](https://github.com/postgres/postgres)@[c410af098c...](https://github.com/postgres/postgres/commit/c410af098c46949e36607eb13689e697fa2def97) #### Monday 2020-06-29 21:12:56 by Tom Lane Mop up some no-longer-necessary hacks around printf %.*s format. Commit 54cd4f045 added some kluges to work around an old glibc bug, namely that %.*s could misbehave if glibc thought any characters in the supplied string were incorrectly encoded. Now that we use our own snprintf.c implementation, we need not worry about that bug (even if it still exists in the wild). Revert a couple of particularly ugly hacks, and remove or improve assorted comments. Note that there can still be encoding-related hazards here: blindly clipping at a fixed length risks producing wrongly-encoded output if the clip splits a multibyte character. However, code that's doing correct multibyte-aware clipping doesn't really need a comment about that, while code that isn't needs an explanation why not, rather than a red-herring comment about an obsolete bug. Discussion: https://postgr.es/m/[email protected] --- ## [Kayoshi-dev/Portfolio](https://github.com/Kayoshi-dev/Portfolio)@[6817e2ed46...](https://github.com/Kayoshi-dev/Portfolio/commit/6817e2ed461898e8d6bca87523035c7b7809f65b) #### Monday 2020-06-29 22:33:37 by Maxime add a working fallback for browser that doesn't support web format i hate my life. i said "working fallback" but I'm not sure --- ## [rsahlstrom/family-meal-schedule](https://github.com/rsahlstrom/family-meal-schedule)@[c57bdb025b...](https://github.com/rsahlstrom/family-meal-schedule/commit/c57bdb025b8a1c3345f2e29f2905e0327cb48bca) #### Monday 2020-06-29 23:15:09 by Russell Ahlstrom Adds Support for Breakfast, Lunch, Dinner, Notes, and Slow Cookers Instead of specifying a meal, you specify a breakfast, lunch, and dinner on the day. If your day has more than one meal, it outputs the type of the meal. Notes can be displayed on a day instead of a meal and shows in italics. A slow cooker alert can be added so you know you don't cook at your normal time. Adds meal plan for 06282020. --- # [<](2020-06-28.md) 2020-06-29 [>](2020-06-30.md)