State of Disgo (v0.10.0)

[switchupcb]

State of Disgo (v0.10.0)

As a DiscordGo contributor, Disgo was created in response to a number of issues with DiscordGo that may be considered unsolvable at this point in time. For an in-depth overview on the problems a DiscordGo user can solve by switching to Disgo, view this issue: #23. This State of Disgo will focus on the current features of Disgo in relation to the roadmap.

Developer Operations

Most important to Disgo is its developer operations. It's been 6 years since the Discord API was released, yet that same API continues to sustain numerous changes in a somewhat volatile manner. Sufficient Developer Operations means that Disgo can adapt quickly and with a low amount of effort, while also verifying those changes across its ecosystem.

For the unaware, Discord does NOT actually provide documentation for its API. Instead, a community is used to determine the current state of the API. While the employees of Discord may release the occasional changelog feature in a Discord Server, the majority of features remain undocumented or outright incorrect. Features are historically only considered with the company in mind, which results in controversy when a feature has poor execution.

The most popular example of this "poor execution" is the release of Slash Commands. The controversy surrounding this issue is only a Google Search away and is a direct result of the decisions above. In this case, slash commands - which is a good idea - was executed without any practical answer on how for API Consumer's could use it with their current requirements. This poor execution led to one of Discord's largest API Wrappers shutting down.

Another example of this "poor self-centered execution" strategy is rate limits, which DOES have documentation provided, but in a very vague manner. As a result, a majority of API Wrappers are unable to handle every rate limit Discord provides (in an efficient manner). Upon further research, it became clear that even Discord Employees have limited knowledge on how their own rate limit mechanism works. The implementation is not standard, and this doesn't help either.

In essence, it seems that Discord employees are rewarded for "finishing" new features at the expense of its API Consumers. There is also no Machine Readable API. For this reason among others, Disgo must be able to ensure that it maintains practices that enable high velocity (after initial creation). We do this through the following features.

Static Code Analysis

Static Code Analysis is used to maintain code that is free from known security issues, fix formatting, and increase correctness of the program. While DiscordGo does feature GolangCI Lint, its settings disable every linter except for the golint preset which is deprecated. In contrast, Disgo enables all linters, then disables stylistic ones which it cannot adhere to.

Tests

Both libraries contain unit and integration tests. However, DiscordGo only tests fundamental areas of ITS code, rather than containing an extensive integration test. At this time, Disgo's near-100% coverage of the Discord API test is still in development.

@t-rog is "relaxing before school starts" and as a result, I will have to pick up this unexpected workload at some point in the near future. Otherwise, one can complete that issue by finishing the "story" with 168/174 endpoints (some endpoints are not tested and are/should be listed in the _gen/flow file).

Data Race Detection

Disgo uses the Go data race detector in its integration tests to identify any data races, or potential deadlock scenarios. This allows it to be more reliable compared to other language wrappers with regards to data races. There are languages that "mock" multi-threaded environments (JavaScript) at the cost of performance. Disgo provides reliable data race protection without sacrificing the ability to run code concurrently.

Developer Ergonomics

Disgo places a high priority on simplicity for the end developer. One issue we found with other API Wrappers was the knowledge gap required to use the API. To be specific, the Discord API provides a number of types, requests, events, etc. However, API Wrappers which implement the Discord API seem to have their own types and functions. This results in the developer requiring knowledge of two APIs instead of one.

Disgo solves this problem by sticking as close to the data as possible. Abstractions are based on the actual concepts the Discord API implements. In addition, almost everything in the Discord API is modeled via struct, which maintains properties the Discord API allows. As a result, the developer may use the Disgo API without ever looking at its auto-generated documentation.

In the main example, a request struct is created (with necessary parameters) and then sent (by the bot). This is in stark contrast to other API Wrappers which will have the developer create a bot object, then use a specific custom function (that you must learn) to send a specific request; and handle in a custom manner. Since Disgo is a 100% one-to-one implementation, it saves time for developers by removing the need to search for renamed functionality.

Full Coverage of the Discord API

Disgo aims to implement every feature provided by the Discord API. Disgo pursued a machine-generated yet human readable approach in order to eventually support every feature available. Other Wrappers in the Go space have gotten close, but they never fully reach this point due to issues described below.

Dasgo

DiscordGo was originally created by Java developers. This led to an implementation that is somewhat unidiomatic and less efficient, but also results in issues due to its type definition. The Go language uses JSON tags to unmarshal and marshal fields into its structs. However, this means that distinguishing between "empty" and "null" is important. DiscordGo does this retroactively, resulting in some features of the Discord API being unachievable without a fork.

In order to solve this issue, we created a Go API Types Library for the Discord API called Dasgo. This library is built with a specification and is not implementation specific to Disgo; meaning that other Go API Wrappers can use it for their types. If Dasgo is used by many API Wrappers, we can unite the entire Discord Go community while still allowing multiple API Wrappers to exist.

One benefit of Dasgo when this occurs is that the load on maintainers will decrease drastically. Whenever a new change to Discord comes out, Dasgo simply needs to be updated and versioned. Go provides hashed versioning which means that libraries can specify a specific version or branch to use Dasgo. This allows us to easily maintain a types library over the multiple versioned endpoints the Discord API provides. As a result, we would also be able to easily maintain every API version a Discord Bot may use.

Exclusive Features

Disgo v.0.10.0 was released in order to place it in the market for external contributors. It technically fits all the requirements required by a Discord API to be introduced to the community resources section. However, it IS missing a few features that could be important for a production-ready bot: These are explained in the next section.

OAuth2

In contrast, there are features Disgo v.0.10.0 contains that are not present in other API Wrappers. OAuth2 is very important for users who want to host dashboards for their Discord Bots, but most Go Wrappers leave you on your own to implement those flows. While it's true that certain steps must be completed by the end developer, Disgo makes implementing OAuth2 easy by providing the necessary endpoints and detailed steps.

Automatic Intent Handling

Gateway Intents are annoying to deal with; especially in current API Wrappers. Every time you want to handle a new event, you must make sure it doesn't require an intent. When it does, you must add that intent to your bot at some point in time the API Wrapper specifies. That is a lot of work. Instead, Disgo - in pursuit of its no-reflection event handling - is able to calculate the intents the developer need automatically (when an event handler is created). This allows the developer to focus more on their bot's functionality, and less on technical details.

All Requests and Events (-5)

Every request and event is logically supported at the current time with the exception of 5 requests involving the usage of multipart/form files. Support for these requests will be added as soon as possible.

Rate Limits

As explained above, most of the details regarding Discord Rate Limits were discovered by... me. I am not exaggerating when I state that most API Wrappers — in every language — fail to handle every Discord Rate Limit in an optimal manner. It doesn't help that Discord themselves hold the belief that being rate limited is unavoidable. However, Discord also penalizes you for being rate limited.

The whole "get 429ed and then retry" method doesn't work that well, as it seems this is incredibly prone to getting the bot in a cascading failure mode that just spams the API server.
— discord/discord-api-docs#108

What does "optimal" mean?

Some API Wrappers patch rate limit issues through the use of custom rate limits. For example, DiscordGo avoids Emoji Rate Limits by limiting the amount to 4 requests per second. In reality, the actual limit is 4 rps PER CHANNEL. This is an example of a "Per Resource Per Route" rate limit which has no details on the actual documentation, except for this statement.

During calculation, per-route rate limits often account for top-level resources within the path using an identifier.

Great.

In other cases, rate limits aren't handled. DiscordGo does not handle Global Rate Limits, and the advice most people give in the Discord Developer server (every language) is to ask Discord for more requests per second. This is not the greatest plan to have when bots are evaluated by the number of servers they are in (rather than the amount of users that interact with them in a given time frame).

In short, Disgo will eventually maintain the most optimal rate limits for the Discord API.

Per Resource Per Route Rate Limits

In its current form, Disgo will occasionally hit 429s. The cause of this issue is due to Per Resource per Route. For more information, read this issue #22.

Utility

Disgo provides a tools package in order to increase developer ergonomics while also maintaining security.

Missing Features

UDP Voice Connection

Support for Opus is still subpar among the Go ecosystem which means UDP Voice connections has been delayed to a later release.

Sharding

Certain Go API Wrappers maintain a sharding feature. DiscordGo does not. This is because it's a feature that requires extra requirements (stated in bwmarrin/discordgo#265), such that it became ingrained into the library. On one hand, it's understandable to allow the user to shard. However, an Ideal API Wrapper will provide a shard manager for the user. The state of sharding in Disgo can be followed in #26.

Cache

In its current stage, the Disgo Cache is only an idea with space allocated for it. On one hand, both dominant libraries maintain caches. On the other hand, what is useful to cache is entirely dependent on the actual Discord Bot. Caching just to cache only results in a waste of memory.

DiscordGo maintains the "State" cache which is prone to data race issues. Disgord is entirely based upon its cache, which is not optional. For this reason, we plan to implement an optional cache, but the strategy on what and how to cache is not yet decided.

Backwards Compatibility

Backwards compatibility should be a function of a previous design's quality. Go's package manager allows versioning such that backwards compatibility should not be a problem. We will not use Backwards Compatibility as an excuse to prevent necessary improvements.

Pre-Release

Disgo v0.10.0 is in a usable yet unfinished state. Unfortunately, the other contributors are on hiatus due to a number of reasons (explained in an incoming article). As a result, a marketing push will be made now. Disgo is still in a state where contributors may be placed on the README (with a link to their websites). If you are interested in contributing to Disgo or the priority of incoming features, please read the Roadmap.