Is is possible to purge automation logs after N days?

[hawkinsjon]

I have a few automations that are receiving a large number of webhook events per day. I have no reason to preserve this logs for the 1 year time period my pro plan offers. My concern now is that I'm unclear from the documentation on if the logs will be purged/pruned at some point of if I'm going to spend a lot of my storage space growing these logs I really don't need.

Questions

• Is my assumption correct that the automation log storage will continue growing?
• Is there an automation or workaround I can perform to help wipe my automation logs after let's say 7 days?

More information on the BB docs page would be helpful

If possible it would be great to include a little bit of detail here about log expiration options for paid users

[melohagan]

Hey @hawkinsjon

Currently there is no way of purging automation logs, but I have raised a feature request here: #11238

As for storage concern, the logs are just a bit of text so even over a year I can't imagine them being enormous unless you are using a very high volume of automations?

[melohagan]

This script that @shogunpurple wrote may also help you out in the meantime!

#!/bin/bash

# URL of your CouchDB instance
couchdb_url="http://localhost:5984"

# Name of your database
database="prod_app_db"

# Get the timestamp for one week ago
timestamp_one_week_ago=$(date -v-1w -u +'%Y-%m-%dT%H:%M:%SZ')

# Query to match the documents
query='{
  "selector": {
    "_id": {
      "$regex": "^log_au"
    },
    "createdAt": {
      "$lt": "'"$timestamp_one_week_ago"'"
    }
  }
}'

# Get the documents matching the query
docs=$(curl -s -X POST "$couchdb_url/$database/_find" -H "Content-Type: application/json" -d "$query")

# Extract document IDs and revision IDs
ids_and_revs=$(echo "$docs" | jq -r '.docs[] | "\(.["_id"]) \(.["_rev"])"')

echo $ids_and_revs

# Loop over each document and delete it
while read -r id rev; do
    curl -s -X DELETE "$couchdb_url/$database/$id?rev=$rev"
done <<< "$ids_and_revs"

curl -X POST "$couchdb_url/$database/_compact" -H "Content-Type: application/json"

This line in the mango query "$lt": "'"$timestamp_one_week_ago"'" will grab all of the logs more than a week ago (i.e. a timestamp less than the timestamp of a week ago). Then the script does on to loop over each and delete for your provided couchdb URL and app database.

You will also need to make sure you have jq installed to run this script: https://formulae.brew.sh/formula/jq

[hawkinsjon]

Thanks @melohagan and @shogunpurple. I really appreciate this!