Color Code Bot

Share code snippets as beautiful syntax-highlighted images and HTML on Telegram

It's a small bit of Python glue between great projects, including:

• highlight (lua, renders HTML)
• pyTelegramBotAPI
• weasyprint (HTML -> image)
• guesslang (uses TensorFlow; saves you the step of specifying the snippet's language)
• Iosevka (the most wonderful monospaced font)

Usage

Send @colorcodebot the code you want highlighted, as a forwarded or original direct message.

As a convenience, you can get to a direct chat with it from any other chat, by typing @colorcodebot and tapping the button that pops up. A button returning you (with a shiny new image) to your original chat will be presented after you send the code.

Behold: a demo video

Development & Deployment

The bot should run anywhere with Python, fontconfig, highlight, either GraphicsMagick or ImageMagick, and the ability to install TensorFlow. Or anywhere that can run a container image.

Depending on your hardware, you may see faster syntax guessing (from guesslang) by installing cuda and cudnn packages. This is not done for the currently hosted container images, which is the result of ./mk/ctnr.sh -d prod --push run by a GitHub Action.

Outside of the core Python app, sops is used for secrets, buildah for container building, GitHub Actions for automated container image builds and other CI tasks, and wheezy.template and yamlpath are extremely handy for defining+rendering service definitions and other dev/ops maneuvers.

Most of the mk/ and start/ scripts are POSIX, but mk/svcs.zsh requires Zsh, and mk/ctnr.sh calls mk/svcs.zsh.

Please do send a message or open an issue with any questions.

Organization

An abbreviated file tree overview:

colorcodebot/
├──app/                  # core app that gets deployed
│  ├──requirements.in    # loosely versioned reqs for the bot
│  └──sops/              # encrypted deployment-specific data
├──vars.<deployment>.yml # unencrypted deployment-specific data
├──start/                # scripts that help start the bot
├──mk/                   # scripts that make things
├──templates/            # used by mk/ scripts to generate files
└──dev-requirements.in   # loosely versioned reqs for mk/ and start/ scripts

The following are generated by mk/ scripts:

colorcodebot/
├──app/
│  ├──requirements.txt   # mk/reqs.sh     - lockfile for the bot
│  ├──svcs/              # mk/svcs.zsh    - supervised process definitions for s6 [untracked]
│  └──theme_previews.yml # mk/file_ids.sh - {theme_name: image_id}                [untracked]
└──dev-requirements.txt  # mk/reqs.sh     - lockfile for mk/ and start/ scripts

When building a container image with mk/ctnr.sh, app becomes /home/colorcodebot.

If you want to use the container images already built from this repo, you'll probably write or mount over:

• /home/colorcodebot/theme_previews.yml
• /home/colorcodebot/svcs
• /home/colorcodebot/sops
• /home/colorcodebot/.sops.yaml

Getting Started

To run colorcodebot.py, the environment variable TG_API_KEY must be set, with a token from @botfather.

$ python3 -m venv app/venv
$ . ./app/venv/bin/activate
$ python -m pip install -r app/requirements.txt
$ TG_API_KEY='...' ./app/colorcodebot.py

After chatting with the bot, check the logs for your chat_id. Pass this as an additional environment variable ADMIN_CHAT_ID to get:

• permission to use the admin /previews command for Generating Theme Previews
• an updated SQLite db file sent to that chat whenever a user sets a preferred theme

Deployments, Secrets, and Scripts

Encrypted Variables

Configure Sops

Create one or more age keys to use with sops:

$ mkdir -p ~/.config/sops/age
$ printf '%s\n' '' '# --- colorcodebot ---' >>~/.config/sops/age/keys.txt
$ age-keygen >>~/.config/sops/age/keys.txt
Public key: age1r50agxl277e24h4ammj0kvpqh224ut8ds67qc2d537dq0uy74shq98dh97

And use that public key in .sops.yaml to match your desired deployments.

Write colorcodebot Variables

Overwrite app/sops/colorcodebot.<deployment>.yml with

TG_API_KEY: <put-the-real-token-here>

(and optionally ADMIN_CHAT_ID) and encrypt it with

$ sops -e -i app/sops/colorcodebot.<deployment>.yaml

Load colorcodebot Variables

$ ./start/local.sh -h
Start the bot locally, without process supervision or other svcs
Args: [-d <deployment>=dev]

You can use start/local.sh to:

• ensure Python lockfile is updated
• ensure a virtual environment exists
• ensure the venv has all Python dependencies installed
• ensure the venv is activated if one is not already
• update or create app/theme_previews.yml if file IDs are present in vars.<deployment>.yml
• load decrypted values from app/sops/colorcodebot.<deployment>.yml into environment variables
• launch the bot (unsupervised, no other services)

You can do just those last two (as seen in the script) with

$ sops exec-env "app/sops/colorcodebot.${deployment}.yml" app/colorcodebot.py

Unencrypted Variables

A deployment's unencrypted variables are defined by vars.<name>.yml.

There are two top-level keys:

theme_previews

mapping of theme names to Telegram file IDs; see Generating Theme Previews

used by: mk/file_ids.sh, mk/ctnr.sh

svcs

list of mappings that each define a long-running supervised service (the bot and optionally a log sender for Papertrail)

used by: mk/svcs.sh, mk/ctnr.sh

The deployments dev and prod are both intended to run inside a container, built by mk/ctnr.sh. Note the difference between the svc definitions of vars.dev.yml and vars.prod.yml:

--- vars.dev.yml  2021-06-28 11:13:46.347838948 -0400
+++ vars.prod.yml 2021-07-12 14:22:07.638842356 -0400
@@ -4,7 +4,7 @@
     exec: >-
       sops exec-env
-      sops/colorcodebot.dev.yml
+      sops/colorcodebot.prod.yml

       "s6-setuidgid colorcodebot ./venv/bin/python
       ./colorcodebot.py"
@@ -16,7 +16,7 @@
     exec: >-
       sops exec-file --filename log_files.yml
-      ../log_files.dev.yml
+      ../log_files.prod.yml

       "remote_syslog -D -c {}"
@@ -24,7 +24,7 @@
     sops_templates:
       - src: papertrail.log_files.yml.wz
-        dest: log_files.dev.yml
+        dest: log_files.prod.yml

• differences:

  • which encrypted variables get set in the environment of the bot process
  • which encrypted config file is created for and read by the remote logger

Now let's compare vars.dev.yml to vars.local.yml:

--- vars.dev.yml  2021-06-28 11:13:46.347838948 -0400
+++ vars.local.yml   2021-07-12 13:57:00.414719676 -0400
@@ -6,14 +6,15 @@
-      "s6-setuidgid colorcodebot ./venv/bin/python
+      "./venv/bin/python
       ./colorcodebot.py"
     folder:
       run: ../../
       log: ../../../logs/colorcodebot
+      cgroups: /sys/fs/cgroup/user.slice/user-1000.slice/[email protected]/app.slice/svcs

   - name: papertrail
-    enabled: true
+    enabled: false
@@ -22,6 +23,7 @@
     folder:
       run: log
       log: ../../../logs/papertrail
+      cgroups: /sys/fs/cgroup/user.slice/user-1000.slice/[email protected]/app.slice/svcs

• similarities:

  • which encrypted configs are used

• differences:

  • local: no user changing (no s6-setuidgid)
  • local: overrides the default cgroup path used by services with a systemd-flavored one
  • local: disables optional Papertrail remote logging service

Modify one of these to your liking, or copy to vars.<name>.yml with your own deployment name, e.g.:

$ cp vars.local.yml "vars.$(hostname).yml"

Generating Theme Previews

highlight has many themes, so we picked a subset.

For the user to choose a theme, we need to generate preview images, and save their file IDs.

Start by creating app/theme_previews.yml either manually or with ./mk/file_ids.sh

$ ./mk/file_ids.sh -h
Generate theme_previews.yml, with data from vars.<deployment>.yml
Args: [-d <deployment>=dev] [<dest>=app/theme_previews.yml]

For now the value of each entry can be garbage, what's important is that the keys are the names of the themes you wish to offer.

Send the /previews command to the bot, and the file IDs you need will show up in the log as preview images are generated and sent your way.

Enter those into vars.<deployment>.yml, then generate app/theme_previews.yml for local deployment with mk/file_ids.sh, which is automatically called by start/local.sh and mk/ctnr.sh.