TODO

Ongoing:
- Animated traces:
  * Anim scores
  * For multiple players, perhaps color gold differently depending on how many players have collected / remain to collect each piece.
  * Precompute bounding boxes for all frames, and use this to estimate how long the encoding might take, then decide if it's worth it to update the Discord message with progress information, such as frames rendered or percentage done.
  * Remove level name from msg, so that it may not interfere with other parameters of the command.
  * Add level name to screenshot background.
  * Ensure each trace is always drawn over the previous one, by adding a condition that will not paint a pixel if it's already been painted by a previous trace.
  * Interpolate between frames, so that we can trace at slower speeds at 50fps, or basically at any speed.
  * Other ideas: anti-aliasing, bezier curves, frame counter / timer, controls for which segment of the run gets animated (instead of the whole thing), more than 4 players, changing line colors (perhaps to reflect speed or direction changes), ...
- Display names and implicit player parsing do not work for userlevels now. Merge userlevel_players with players:
  * Add metanet_id column to userlevel_scores
  * Merge userlevel_players (not already in players) with players
  * Fill in proper player_id in userlevel_scores by looking at the metanet_id
  * Ditto for userlevel_histories
  * Remove metanet_id field from both tables

- Persist task counters in db (using task_ prefix). Don't do it constantly, perhaps only sum them when shutting down.
- Migrate highscoreable_type to integers from 0 to 2.
- Add a special command to run arbitrary Ruby code, sent from Discord, and print the return inside a text block. Ensure to execute inside guards for exceptions. Perhaps even code a simple Debugger that can be started in a separate Thread, which waits for commands (e.g. using a Queue).
- Add Interaction module to handle them, now the code is too unorganized.
- Add robust test suite that automatically tests as many outte features as possible. At least that they execute without throwing, even better if we can assert equality in some cases.
- Add lock to prevent restarting outte when a score submission is in progress
- For when we enforce hashes: Have secondary db table to temporarily store rejected runs due to invalid hash, just in case they're legit and need to be restored
- Add indices to many tables, apparently add_column does not have an :index option (as opposed to create/change_table, which does), so many of the migrations we used the last years failed to create an index at all. Go table by table, seeing what's useful to add, and benchmark the differences.
- Add a live control panel (a special command that puts a collection of buttons and perhaps select menus and modals on Discord, that allows to execute managerial functions, like batch update the completions (as well as cancelling them)).
- Revamp help / commands, with navigation buttons, select menus, etc. Use classes for each command, which will contain all the info necessary to print the corresponding window of info. Then, before each command in messages.rb we can also create the corresponding help object, to keep things organized (or just in a new file). The help (categories, command lists, help texts, examples...) should be autogenerated. Then remove all that from the README file.
- Simplify README, with a brief description but without commands, update stuff like contributors and dates
- Distribution of scores: top20 episode distribution = first show some stats regarding top20s per episode (most top20s in an ep, least, average, etc), then a file with all episodes sorted by # of top20s). Parameters: type, rank (top20, 0th, etc), distribution type (by episodes, by rows, by tabs).
- Restrict display names to ASCII only
- For Archive scores function: Make sure to sort ties in descending order by using the replay id.
- Improve the migration of video challenges to the database: Parse the spreadsheet directly, otherwise it takes longer to update it and it doesn't work unless you have the csv files, which are not in the github repo.
- Suggestion: Change Steam ID when we get the Bad Gateway error, since I think it depends on the player and thus we might benefit from a new ID, although we don't want to lose the other one since we know it's active, so maybe we could keep a cache of recently active Steam IDs that lasts, say, 1 hour. Another idea is to have a column in the database "last seen active" that we could refer to for this purpose.
- Implement "index" (maybe "tally"): Outputs file with a list (like "list") of something, like all 0ths. With a parameter we specify what, for instance, 0th scores or 0th frames. With another parameter we specify the order, like the natural one, or by score in descending order...
- Compare local scores (nprofile) vs server scores to find differences, a la NReality Score Comparer.
- Add total <rank> score, and show total score and total 19th score gained daily and weekly.
- Compile usage stats (number of pings, most used commands, data uploaded / downloaded, etc).
- Screenscores for userlevels.
- Add "maxed" and "maxable" to "list" and "how many".
- Add "ownage" and others to "table".
- Perhaps add rank to how many function, i.e. "username has n/N top20 scores (ranked kth)".
- Report: Add # of 0ths, i.e., "• There were 23 new scores by 9 players in 18 levels, making the boards 15.083 seconds harder including X new 0ths increasing the total 0th score by 1.050 seconds."
- Save how many times each streamer has streamed the game, and perhaps indicate it somehow if it's among the top ones (dope gamer alert!)
- Command to update certain userlevels (e.g. within a range, from a author, etc., with some hard limit e.g. 50, perhaps a cooldown).
- Add other versions? (v1.4, v2.0, etc)
- "nubs" command to show ignored hackers and cheaters
- Allow to add "rankings" to any userlevel query (browse) to instead perform rankings on the result set.
- Add regex to functions (userlevel search, etc)
- aliases: Make sure they cannot be repeated. Add edit/remove alias functions only for botmaster.
- In spread function, if rank == 19, assume it means "last", so that it works in boards with hacked scores. Perhaps do that for 18 (2nd from last), etc. At some point the pattern needs to break, though. Just use clamp here.
- Personal demo analysis function (for when a run is not in the global leaderboard)
- Browse uncleared userlevels (levels with no completions)
- Unmaxed missing (ignore maxes in missing)
- Add "graphic" / "image" parameter to many commands which output level lists, so that instead of the list, we get images with thumbnails of 5/10 levels, with arrow button navigation to move, a-la userlevel browse.
- Add access to all functions using slash commands
- Multiple matches:
  * Add for player names
  * kk's suggestion: nav bar for matches, navigate screenies etc
- Userlevel tabs not working correctly? (dupped levels in top weekly, what about featured, etc)
- Handle cheaters within database (rather than via hardcoding in source code), have botmaster function to add cheaters, etc
- Histograms of userlevels played by ID (perhaps also chart anything, e.g. 0ths, top10s, parse_range)
- Analysis: Implement for episodes and stories and userlevels and mappacks
- "percent" for table
- maybe modify function 'nav' from 'HighScore' module to allow for arbitrary offsets, and have some special sign to denote 'first' and 'last'
- add "full" to everything that outputs lists (missing, etc), maybe nav too
- add "suggestions" command (random selection of scores the player doesn't have)
- allow "missing" parameter in table, just like in "how many" or "list"
- use Score.filter with a Player for some of the Player methods, like range_ns, table...
- add Score::filter to rank_exclude
- CKC rankings weighed by the number of cool kids in each board
- Have all people tied in rankings have the same rank, with hyphens. Show slightly over 20 if tied at the bottom, up to another limit, like 25.
- In order to sort ties, rather than alphabetical, use chronological. For simplicity, use date of most recent 0th, rather than the whole history.
- Migrate the scores table to using integer scores (in frames), like the others.
- Dual 0ths (both hs and sr 0ths), and dual ownages
- Ability to subscribe to mappacks or certain things (e.g. "subscribe to cla sr top20" will warn when the user loses a sr top20 in mappack CLA)
- Package outte as a gem
- Across-mappacks rankings and stuff
- TASes in outte, perhaps add to CLE too (scores around mine?)
- String distance for player/author names (and also partial matching), and maybe even outte commands
- Generated nprofiles have extra challenges unlocked?
- Add generic nprofile functionality
- Can we track Youtube N++ streams like with Twitch? 
- Ownages with ties (add "with ties"), also add to mappacks
- Minimum scores to appear in mappack average rank rankings are 0, fix.
- Create a class QueryResult for when we parse a highscoreable, that contains the array of results, the header of text, a function to format the result, etc. Perhaps do the same for parsing players and other things.
- Add function to update stars in scores table, since due to errors we could sporadicaly lose some of them, but we can rebuild them by looking at the archives table.
- Use average rank for tie-breaking rankings
- For mappack ranking navigation, add button for HS/SR, and perhaps select menu for mappacks
- Add other most/least gold functions apart from rankings: how many, missing, etc
- Player comparison for mappacks (add speedrun, G++/G--, ...)
- Instead of calling the compiled version of STB's SHA1, do it natively (gem with native extensions?)

\\ ------------------------ SQL -----------------------

General tips:
- https://dba.stackexchange.com/questions/311593/mysql-indexes-on-all-column-combinations

Indices:
- https://mysql.rjweb.org/doc.php/index_cookbook_mysql

Partitioning:
- https://mysql.rjweb.org/doc.php/partitionmaint

Memory tables (for userlevels, read-only): 
- https://dev.mysql.com/doc/refman/8.0/en/memory-storage-engine.html
- https://stackoverflow.com/questions/10692398/how-do-i-make-a-mysql-database-run-completely-in-memory

Find a way to forcefully clear dead connections. Perhaps MySQL server needs to be restarted in extreme cases.

Monitor SQL connections. Add warnings when the connection count gets close to the limit, deal with by:
	Increasing the limit.
	Restarting outte, like we do with the memory limit. Or even MySQL.

Save number of concurrent connections each time we monitor, so that we have a registry and can e.g. plot them later, to see the average num of connections, peaks, etc.

Perhaps we should look into using acquire_connection (and release_connection) in _thread, and make sure we use that wrapper everywhere.

\\ ------------------------ mappacks -----------------------

- Add hs scores around mine in 2nd leaderboard.
- Add outte command to choose what to put in the 2nd/3rd leaderboard (toggle between hs, sr, g--, etc, with different rank ranges too)

\\ ------------------------ ntrace ------------------------

- Option to change color of plot line with time (let's say, loop through rainbow as many times as specified, or once), to aid readability for long runs with many crosses.
- Look into matplotlib blitting, or implement it myself
- Try using Python itself, rather than Ruby's wrapper, to see if it's quicker
- Try optimizing it in C altogether

\\ ------------------------ nprofile ------------------------

- Create an nprofile module / class to handle different functionality
- Upload zipped nprofile and perform stats (e.g. graphs with svg-graph)
- Allow multiple nprofiles to perform cumulative stats (e.g. attempts or times)

\\ ------------------------ HC ------------------------


\\ ------------------------ Art ------------------------

- Use different objects for different colors.
- Create a custom palette that best fits the image.
- Dithering.

\\ ------------------------ Userlevels ------------------------

- We need to update all maps regularly, for the ++'s, etc.
- Userlevel of the day (filter by levels with at least n scores and at least a certain delta of scores to prevent impossible and maxed levels)
- Add Metanet levels to the db.

Less priority:
- Short/compact search mode (apt for mobile; show only ID, name and author; shorter truncations)
- Missing
- NUMA-esque features: We could have a command (e.g. "post") that prints the userlevel info (name, author, screenshot, etc) and adds buttons to cast a vote (we keep track of the users so that a single vote can be casted per Discord user), as well as a text input to put a comment (or perhaps, save a conversation regarding a userlevel as the "comment section" of that map, somehow), ec
- Demo analysis.
- Userlevel rankings by page (featured, best, hardest, etc.). Also, the special hardest rankings with point ponderation.
- Imagesearch: Show pictures of results (miniature) instead of text.
- Userlevel stats: How many unique authors, maps, averages, etc. Break down by mode and total.
- When an ID is not in the database, if it's close (or always), download newest, but without printing it.
- Store a daily list of all top 50 userlevels (just ID's for instance), so we can see how the hardest levels evolve.
- Map Steam ID's to player names by looking it up in the database, so that they can be used when searching the server / database.
- Browse user dependant categories
- Browse other (made up but interesting) categories
- Download maps (plural), like, all maps by an author or something.
- Upload maps to accounts (need to figure out request by using proxy first)
- Map editor with slots for each user
- Comparison function (like for Metanet scores). Be able to filter maps by author, name, etc.

Others:
- Add the possibility of making secondary orders (to break ties).

\\ ------------------------ Scores ------------------------

- Ping players who lost 0ths, using the previous function.

\\ ------------------------ Screenshots ------------------------

- Optimizations:
  * Draw all tiles using polygon (in OilyPNG) rather than compose (and bench it)
  * Initialize the canvas in C

- Implement vector silhouettes (would need one more layer per object, and perhaps new colors in the master image, make them optional with a parameter to the screenshot function)
- Allow to compose images out of bounds by cropping them appropriately. Make a commit to oily_png, like the original author intended. 
- Show glitch tiles with some soft shade, e.g. red or the inverse of the background, perhaps with a pattern too, e.g. lines or dotted.
- Properly rotate laser turrets when they're against a tile
- Draw lasers, even if in basic fashion (straight lines till tile)
- Optionally show editor info in the screenshot (door-switch lines, etc)
- Minis are missing the eye (remember the second color needs to be composed in a special way to obtain the final color)

- Implement custom palettes, and commands for users to submit their palettes, instead of editing the spreadsheet. Commands to list palettes, showcase examples, etc.
- Go through palette channel and add custom palettes that haven't been added (list zip files or something).
- Generate a gif with one palette per frame

\\ ------------------------ Total Scores ------------------------
- Create 2 tables: Levels and Episodes. One row per player, one column per level/episode. The idea is to keep the bot running 24/7, cycling through Steam ID's until an active one is found, and then latch onto that player and start downloading his scores. First, download those which have not yet been downloaded, and after that's done, keep downloading them, but not willy nilly, instead, also save the date at which each score was downloaded, and always update the scores in chronological order of last updated. Eventually, the goal is to have all scores by all players (in the list), and have them as updated as possible (even if they haven't completed the level).

\\ ------------------------ Replays ------------------------

- Generate attract files from Metanet level & userlevel replays, as a way to keep demos.
- Implement demo analysis to attract files.
- Compare inputs of two runs like github does, also maybe do this comparison by just specifying changes (*read below*)

\\ ------------------------ History ------------------------

-- Fix the doubling through code, by halving the older scores.
-- Add the possibility of limiting the scope of graphs (upper bound of y axis), using an additional parameters to the user command (e.g. "intro history 100"). In this case, if some player doesn't even appear in the graph because he's above it, maybe delete him from the legend and add new players. This adds an aditional difficulty since it's not just changing the y axis, but also changing the players to be plotted. To do so, the maximum would have to be computed first, and then decide which players are to be plotted.
-- Medium priority: Another solution for the above problem: Use logarithmic scale.
-- Manually join players that changed username (e.g. Maelstrom=drunk driver=asleep at the wheel=coronavirus).
-- Add some chronological marks in the x axis (at least years, if not months). Maybe more and better marks in the y axis too.
-- Retroactively delete (in the code, not the database) ignored players, like Mishu.
-- Medium priority: Figure out what causes the sudden drops in the graphs, and fix it.
-- Low priotity: Select the number of players to be plotted, or the minimum number of 0ths to select such players, using user input. This would have to be limited with some hardcoded constants to prevent spam or a computation overload).
-- Low priotity: Improve legend (align players and put on the side instead of top, maybe put maxes next to them).
-- Low priotity: Overall improvements to style (frame graph, and other decorations).

\\ ------------------------ Cheaters ------------------------


\\ ------------------------ Hardcore ------------------------

- Cleanliness, splits, and other functions.

\\ ------------------------ Humanlike features ------------------------

- Add guessing game: inne shows a picture of a level and people must figure out the level (by typing the name or level code). Maybe different difficulty settings, the easy one being a full picture and hard one being a section only, maybe ensuring that this section makes sense somehow (ie. it's not just empty space). Since we need to hold state, make sure you can run the game in a thread, with a limit of maybe 5 simultaneous games.
- Add some backstory, like what happens when outte is asked about inne (murder?)
- Add some more easter eggs, like a command that shows (or links) eddysipw, or a very small chance that a drone in a screenshot is dronexd or the ninja has eyes (see gofish meme).

\\ ------------------------ General ------------------------

- Maybe change "total score" to default to TLS instead of TLS + TES, which doesn't make much sense.
- SAVEFILE PARSER, ability to send inne your savefile and use it to elaborate completion leaderboards and whatnot (ask CCS if he ever finished it).
- 0th/Top20 changes (new ones, and per player).
- Comparison between players.
- Charts with colors.

- Possibility to return lists in order of level and not rank.
- For "splits", also download episode 0th replay and parse it to get the actual 0th splits, so that we can compare the 0th splits with the ideal splits we are currently producing by adding up the level 0ths. This isn't actually possible unless we assume G++, which we can check by adding up.
- Implement "missing" in the "list" function, so that we could get "bottom 5 list for jirka that I don't already have".
- Add relaxed episode ownages (without episode), and ownages with ties.

\\ ------------------------ NPC ------------------------

- Add 2 color boxes, to change every instance of color 1 in the palette by color 2.


\\ ------------------------ Notes ------------------------

*below*

"holds jump for 1 less frame, switches right for 2 frames then left until end", or in oter words, "as long as the difference between two inputs is the same as in the previous frame, discard it, as you said"