Add ActiveStorage and AzureBlob

As the "azure-storage-blob" has been recently deprecated, we've decided to adopt a new community driven alternative, "azure-blob".

.env.example

@@ -41,3 +41,8 @@ GET_INTO_TEACHING_API_URL=https://get-into-teaching-api-dev.london.cloudapps.dig
 BIG_QUERY_PROJECT_ID=project-id
 DFE_BIGQUERY_API_JSON_KEY="{}"
 NODE_OPTIONS=--openssl-legacy-provider
+
+# Active Storage credentials
+AZURE_STORAGE_ACCOUNT_NAME=
+AZURE_STORAGE_ACCESS_KEY=
+AZURE_STORAGE_CONTAINER=

Gemfile

@@ -150,6 +150,9 @@ gem 'rack-mini-profiler', require: ['prepend_net_http_patch']
 # BigQuery
 gem 'dfe-analytics', github: 'DFE-Digital/dfe-analytics', tag: 'v1.12.7'
 
+# Azure Blob Storage
+gem 'azure-blob'
+
 group :development do
   gem 'listen', '>= 3.0.5', '< 3.10'
   gem 'rails-erd'

Gemfile.lock

@@ -154,6 +154,8 @@ GEM
     audited (5.4.3)
       activerecord (>= 5.0, < 7.2)
       request_store (~> 1.2)
+    azure-blob (0.4.1)
+      rexml
     base64 (0.2.0)
     bcrypt (3.1.20)
     better_html (2.0.2)
@@ -853,6 +855,7 @@ DEPENDENCIES
   ar-sequence
   archive-zip
   audited
+  azure-blob
   blazer
   brakeman
   bullet

adr/0023-add-active-storage-with-azure-storage.md

@@ -0,0 +1,18 @@
+# 23. Add Active Storage with Azure Storage
+
+**Date:** 31/05/2024
+
+## Status:
+
+Proposed
+
+## Context
+
+Our `data_exports` table currently stores csv files in binary format. This causes each row to be very large and slow to query. We want to move these files to Active Storage to improve performance and reduce the size of the table. This will also make database backups much more efficient.
+
+## Consequences:
+
+- We will have an additional 3rd party dependency in production - Azure Storage.
+- We will need to update our CI/CD pipeline to include spinning up Azure Storage instances.
+- We will need to add Azure Storage env vars to the production config.
+- We will also need to backup Azure Storage accounts/containers as part of our disaster recovery plan.