Setting schema_format to sql prevents queue migration #525

longthanhtran · 2025-02-28T01:30:27Z

When I use rails 8 and want to use solid_queue, I think one setting of config.active_record.schema_format in config/application.rb

module MyRailsApp
  class Application < Rails::Application
    # omit some existing config lines
    config.active_record.schema_format = :sql # <- this line creates issue
  end
end

causes queue schema could not be created via bin/rails db:migrate:queue.

All the steps I use was following the RoR guides

The text was updated successfully, but these errors were encountered:

rosa · 2025-03-05T11:52:48Z

Hey @longthanhtran, you don't need to use bin/rails db:migrate:queue to create the Solid Queue schema 🤔 You just need to run bin/rails db:prepare after bin/rails solid_queue:install, as explained in the README. What happens when you do that? Why are you trying to run bin/rails db:migrate:queue?

glundgrenm · 2025-03-05T17:42:53Z

@rosa i am having this issue too, it doesn't generate queue migrations, if i run db:migrate or prepare, this only clears the queue schema.

rosa · 2025-03-05T17:48:54Z

Hey @glundgrenm, migrations aren't supposed to be generated. The queue schema clearing... that sounds like a bug in Rails, though.

glundgrenm · 2025-03-05T17:50:47Z

@rosa cool, but how am i suppose to handle this?

rosa · 2025-03-05T17:52:58Z

Could you test this from scratch? Run bin/rails solid_queue:install, check db/queue.rb, then run bin/rails db:prepare and check db/queue.rb again. Could you also share the version of Rails you're running?

glundgrenm · 2025-03-05T18:00:49Z

Running on Rails 8.0.1. Here it is:

➜  hikari-api git:(main) ✗ bin/rails solid_queue:install
    conflict  config/queue.yml
Overwrite /Users/glundgren/projects/hikari/hikari-api/config/queue.yml? (enter "h" for help) [Ynaqdhm] n
        skip  config/queue.yml
   identical  config/recurring.yml
    conflict  db/queue_schema.rb
Overwrite /Users/glundgren/projects/hikari/hikari-api/db/queue_schema.rb? (enter "h" for help) [Ynaqdhm] y
       force  db/queue_schema.rb
   identical  bin/jobs
        gsub  config/environments/production.rb

the db/queue_schema.rb:

ActiveRecord::Schema[7.1].define(version: 1) do
  create_table "solid_queue_blocked_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.string "queue_name", null: false
    t.integer "priority", default: 0, null: false
    t.string "concurrency_key", null: false
    t.datetime "expires_at", null: false
    t.datetime "created_at", null: false
    t.index [ "concurrency_key", "priority", "job_id" ], name: "index_solid_queue_blocked_executions_for_release"
    t.index [ "expires_at", "concurrency_key" ], name: "index_solid_queue_blocked_executions_for_maintenance"
    t.index [ "job_id" ], name: "index_solid_queue_blocked_executions_on_job_id", unique: true
  end

  create_table "solid_queue_claimed_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.bigint "process_id"
    t.datetime "created_at", null: false
    t.index [ "job_id" ], name: "index_solid_queue_claimed_executions_on_job_id", unique: true
    t.index [ "process_id", "job_id" ], name: "index_solid_queue_claimed_executions_on_process_id_and_job_id"
  end

  create_table "solid_queue_failed_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.text "error"
    t.datetime "created_at", null: false
    t.index [ "job_id" ], name: "index_solid_queue_failed_executions_on_job_id", unique: true
  end

  create_table "solid_queue_jobs", force: :cascade do |t|
    t.string "queue_name", null: false
    t.string "class_name", null: false
    t.text "arguments"
    t.integer "priority", default: 0, null: false
    t.string "active_job_id"
    t.datetime "scheduled_at"
    t.datetime "finished_at"
    t.string "concurrency_key"
    t.datetime "created_at", null: false
    t.datetime "updated_at", null: false
    t.index [ "active_job_id" ], name: "index_solid_queue_jobs_on_active_job_id"
    t.index [ "class_name" ], name: "index_solid_queue_jobs_on_class_name"
    t.index [ "finished_at" ], name: "index_solid_queue_jobs_on_finished_at"
    t.index [ "queue_name", "finished_at" ], name: "index_solid_queue_jobs_for_filtering"
    t.index [ "scheduled_at", "finished_at" ], name: "index_solid_queue_jobs_for_alerting"
  end

  create_table "solid_queue_pauses", force: :cascade do |t|
    t.string "queue_name", null: false
    t.datetime "created_at", null: false
    t.index [ "queue_name" ], name: "index_solid_queue_pauses_on_queue_name", unique: true
  end

  create_table "solid_queue_processes", force: :cascade do |t|
    t.string "kind", null: false
    t.datetime "last_heartbeat_at", null: false
    t.bigint "supervisor_id"
    t.integer "pid", null: false
    t.string "hostname"
    t.text "metadata"
    t.datetime "created_at", null: false
    t.string "name", null: false
    t.index [ "last_heartbeat_at" ], name: "index_solid_queue_processes_on_last_heartbeat_at"
    t.index [ "name", "supervisor_id" ], name: "index_solid_queue_processes_on_name_and_supervisor_id", unique: true
    t.index [ "supervisor_id" ], name: "index_solid_queue_processes_on_supervisor_id"
  end

  create_table "solid_queue_ready_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.string "queue_name", null: false
    t.integer "priority", default: 0, null: false
    t.datetime "created_at", null: false
    t.index [ "job_id" ], name: "index_solid_queue_ready_executions_on_job_id", unique: true
    t.index [ "priority", "job_id" ], name: "index_solid_queue_poll_all"
    t.index [ "queue_name", "priority", "job_id" ], name: "index_solid_queue_poll_by_queue"
  end

  create_table "solid_queue_recurring_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.string "task_key", null: false
    t.datetime "run_at", null: false
    t.datetime "created_at", null: false
    t.index [ "job_id" ], name: "index_solid_queue_recurring_executions_on_job_id", unique: true
    t.index [ "task_key", "run_at" ], name: "index_solid_queue_recurring_executions_on_task_key_and_run_at", unique: true
  end

  create_table "solid_queue_recurring_tasks", force: :cascade do |t|
    t.string "key", null: false
    t.string "schedule", null: false
    t.string "command", limit: 2048
    t.string "class_name"
    t.text "arguments"
    t.string "queue_name"
    t.integer "priority", default: 0
    t.boolean "static", default: true, null: false
    t.text "description"
    t.datetime "created_at", null: false
    t.datetime "updated_at", null: false
    t.index [ "key" ], name: "index_solid_queue_recurring_tasks_on_key", unique: true
    t.index [ "static" ], name: "index_solid_queue_recurring_tasks_on_static"
  end

  create_table "solid_queue_scheduled_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.string "queue_name", null: false
    t.integer "priority", default: 0, null: false
    t.datetime "scheduled_at", null: false
    t.datetime "created_at", null: false
    t.index [ "job_id" ], name: "index_solid_queue_scheduled_executions_on_job_id", unique: true
    t.index [ "scheduled_at", "priority", "job_id" ], name: "index_solid_queue_dispatch_all"
  end

  create_table "solid_queue_semaphores", force: :cascade do |t|
    t.string "key", null: false
    t.integer "value", default: 1, null: false
    t.datetime "expires_at", null: false
    t.datetime "created_at", null: false
    t.datetime "updated_at", null: false
    t.index [ "expires_at" ], name: "index_solid_queue_semaphores_on_expires_at"
    t.index [ "key", "value" ], name: "index_solid_queue_semaphores_on_key_and_value"
    t.index [ "key" ], name: "index_solid_queue_semaphores_on_key", unique: true
  end

  add_foreign_key "solid_queue_blocked_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_claimed_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_failed_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_ready_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_recurring_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_scheduled_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
end

just ran bin/rails db:prepare, it ran without any mesage, but the development_queue.sqlite3 doesnt seems to have the tables from schema.

rosa · 2025-03-05T18:08:36Z

just ran bin/rails db:prepare, it ran without any mesage

But did this empty your db/queue_schema.rb file?

Could you also copy your database.yml configuration? Are you setting this in your development.rb environment file?

config.solid_queue.connects_to = { database: { writing: :queue } }

glundgrenm · 2025-03-05T18:11:19Z

No, it didnt emptied.

Actually the application is running in development (i am not using any queue, actually) and production well. I started having issues when deploying it with kamal to the staging env (that also have the config.solid_queue.connects_to = { database: { writing: :queue } } in staging environment as in production).

Whats happening only on the staging environment is that the app keeps restarting, saying:

➜  hikari-api git:(main) kamal logs -d staging                                             
  INFO Following logs on xxx...
  INFO ssh -t root@xxx -p 22 'sh -c '\''docker ps --latest --quiet --filter label=service=hikari_api --filter label=destination=staging --filter label=role=web --filter status=running --filter status=restarting --filter ancestor=$(docker image ls --filter reference=registry.digitalocean.com/hikari/hikari_api:latest-staging --format '\''\'\'''\''{{.ID}}'\''\'\'''\'') ; docker ps --latest --quiet --filter label=service=hikari_api --filter label=destination=staging --filter label=role=web --filter status=running --filter status=restarting'\'' | head -1 | xargs docker logs --timestamps --tail 10 --follow 2>&1'
2025-03-05T15:23:19.442950226Z  from /usr/local/bundle/ruby/3.2.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
2025-03-05T15:23:19.442952581Z  from ./bin/rails:4:in `<main>'
2025-03-05T15:23:19.469225897Z [13] ! reaped unknown child process pid=66 status=pid 66 exit 1
2025-03-05T15:23:21.412042583Z [13] Detected Solid Queue has gone away, stopping Puma...
2025-03-05T15:23:21.412588580Z [13] - Gracefully shutting down workers...
2025-03-05T15:23:21.617385404Z [13] === puma shutdown: 2025-03-05 15:23:21 +0000 ===
2025-03-05T15:23:21.617456305Z [13] - Goodbye!
2025-03-05T15:23:21.617462207Z Exiting
2025-03-05T15:23:21.641151992Z {"time":"2025-03-05T15:23:21.641013735Z","level":"INFO","msg":"Server stopping"}
2025-03-05T15:23:21.641371068Z {"time":"2025-03-05T15:23:21.641313947Z","level":"INFO","msg":"Server stopped"}
2025-03-05T15:23:32.661465668Z   ActiveRecord::SchemaMigration Load (4.5ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
2025-03-05T15:23:32.661524611Z   ActiveRecord::SchemaMigration Load (0.2ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
2025-03-05T15:23:32.661564385Z   ActiveRecord::SchemaMigration Load (0.2ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
2025-03-05T15:23:32.661571192Z   ActiveRecord::SchemaMigration Load (0.3ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
2025-03-05T15:23:33.145579436Z {"time":"2025-03-05T15:23:33.145266182Z","level":"INFO","msg":"Server started","http":":80"}
2025-03-05T15:23:34.441368198Z => Booting Puma
2025-03-05T15:23:34.441414733Z => Rails 8.0.1 application starting in staging 
2025-03-05T15:23:34.441421556Z => Run `bin/rails server --help` for more startup options
2025-03-05T15:23:36.242355776Z [13] Puma starting in cluster mode...
2025-03-05T15:23:36.242972123Z [13] * Puma version: 6.6.0 ("Return to Forever")
2025-03-05T15:23:36.243339250Z [13] * Ruby version: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
2025-03-05T15:23:36.243692170Z [13] *  Min threads: 3
2025-03-05T15:23:36.243991751Z [13] *  Max threads: 3
2025-03-05T15:23:36.244335180Z [13] *  Environment: staging
2025-03-05T15:23:36.244674002Z [13] *   Master PID: 13
2025-03-05T15:23:36.245065159Z [13] *      Workers: 5
2025-03-05T15:23:36.245079763Z [13] *     Restarts: (✔) hot (✖) phased
2025-03-05T15:23:36.247012788Z [13] * Preloading application
2025-03-05T15:23:36.247499716Z [13] * Listening on http://0.0.0.0:3000
2025-03-05T15:23:36.247514038Z [13] Use Ctrl-C to stop
2025-03-05T15:23:36.294683020Z [13] - Worker 0 (PID: 21) booted in 0.04s, phase: 0
2025-03-05T15:23:36.297143916Z [13] - Worker 1 (PID: 23) booted in 0.04s, phase: 0
2025-03-05T15:23:36.298084745Z [13] - Worker 2 (PID: 35) booted in 0.03s, phase: 0
2025-03-05T15:23:36.298782971Z [13] - Worker 3 (PID: 44) booted in 0.02s, phase: 0
2025-03-05T15:23:36.300137401Z [13] - Worker 4 (PID: 53) booted in 0.01s, phase: 0
2025-03-05T15:23:36.353878484Z SolidQueue-1.1.3 Error registering Supervisor (34.4ms)  pid: 66, hostname: "xxx-3256b018cfb5", name: "supervisor-7e0b20a0c1d83673ac70", error: "ActiveRecord::StatementInvalid Could not find table 'solid_queue_processes'"
2025-03-05T15:23:36.354931667Z SolidQueue-1.1.3 Started Supervisor (35.9ms)  pid: 66, hostname: "xxx-3256b018cfb5", process_id: nil, name: "supervisor-7e0b20a0c1d83673ac70"
2025-03-05T15:23:36.355419750Z /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/sqlite3_adapter.rb:512:in `table_structure': Could not find table 'solid_queue_processes' (ActiveRecord::StatementInvalid)
2025-03-05T15:23:36.355464620Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/abstract/schema_statements.rb:109:in `columns'
2025-03-05T15:23:36.355469331Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:345:in `block (2 levels) in columns'
2025-03-05T15:23:36.355475557Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:412:in `with_connection'
2025-03-05T15:23:36.355479877Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:344:in `block in columns'
2025-03-05T15:23:36.355483268Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:343:in `fetch'
2025-03-05T15:23:36.355486989Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:343:in `columns'
2025-03-05T15:23:36.355490566Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:354:in `block in columns_hash'
2025-03-05T15:23:36.355494072Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:353:in `fetch'
2025-03-05T15:23:36.355497570Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:353:in `columns_hash'
2025-03-05T15:23:36.355501000Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:54:in `columns_hash'
2025-03-05T15:23:36.355504626Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/schema_cache.rb:198:in `columns_hash'
2025-03-05T15:23:36.355508335Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/model_schema.rb:592:in `load_schema!'
2025-03-05T15:23:36.355511880Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/counter_cache.rb:187:in `load_schema!'
2025-03-05T15:23:36.355516640Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/encryption/encryptable_record.rb:127:in `load_schema!'
2025-03-05T15:23:36.355521004Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/model_schema.rb:539:in `block in load_schema'
2025-03-05T15:23:36.355535312Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/model_schema.rb:536:in `synchronize'
2025-03-05T15:23:36.355537767Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/model_schema.rb:536:in `load_schema'
2025-03-05T15:23:36.355540076Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/model_schema.rb:428:in `columns_hash'
2025-03-05T15:23:36.355542501Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/attributes.rb:244:in `block in _default_attributes'
2025-03-05T15:23:36.355544878Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:418:in `with_connection'
2025-03-05T15:23:36.355547269Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/connection_handling.rb:310:in `with_connection'
2025-03-05T15:23:36.355549624Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/attributes.rb:243:in `_default_attributes'
2025-03-05T15:23:36.355552001Z  from /usr/local/bundle/ruby/3.2.0/gems/activemodel-8.0.1/lib/active_model/attribute_registration.rb:38:in `attribute_types'
2025-03-05T15:23:36.355554850Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/attribute_methods.rb:261:in `_has_attribute?'
2025-03-05T15:23:36.355557164Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/inheritance.rb:61:in `new'
2025-03-05T15:23:36.355560534Z  from /usr/local/bundle/ruby/3.2.0/gems/activerecord-8.0.1/lib/active_record/persistence.rb:54:in `create!'
2025-03-05T15:23:36.355566680Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/app/models/solid_queue/process.rb:13:in `block in register'
2025-03-05T15:23:36.355573087Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/notifications.rb:210:in `block in instrument'
2025-03-05T15:23:36.355576709Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
2025-03-05T15:23:36.355580108Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/notifications.rb:210:in `instrument'
2025-03-05T15:23:36.355583827Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/solid_queue.rb:83:in `instrument'
2025-03-05T15:23:36.355587080Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/app/models/solid_queue/process.rb:12:in `register'
2025-03-05T15:23:36.355590649Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/solid_queue/processes/registrable.rb:22:in `register'
2025-03-05T15:23:36.355594201Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:361:in `block in make_lambda'
2025-03-05T15:23:36.355597483Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:207:in `call'
2025-03-05T15:23:36.355600658Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:562:in `block in invoke_after'
2025-03-05T15:23:36.355604037Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:562:in `each'
2025-03-05T15:23:36.355612399Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:562:in `invoke_after'
2025-03-05T15:23:36.355616020Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:110:in `run_callbacks'
2025-03-05T15:23:36.355619135Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/solid_queue/supervisor.rb:49:in `block in boot'
2025-03-05T15:23:36.355622332Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/notifications.rb:210:in `block in instrument'
2025-03-05T15:23:36.355625521Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
2025-03-05T15:23:36.355628921Z  from /usr/local/bundle/ruby/3.2.0/gems/activesupport-8.0.1/lib/active_support/notifications.rb:210:in `instrument'
2025-03-05T15:23:36.355632180Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/solid_queue.rb:83:in `instrument'
2025-03-05T15:23:36.355635371Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/solid_queue/supervisor.rb:48:in `boot'
2025-03-05T15:23:36.355638623Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/solid_queue/supervisor.rb:30:in `start'
2025-03-05T15:23:36.355641758Z  from <internal:kernel>:90:in `tap'
2025-03-05T15:23:36.355645239Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/solid_queue/supervisor.rb:14:in `start'
2025-03-05T15:23:36.355648704Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/puma/plugin/solid_queue.rb:17:in `block (2 levels) in start'
2025-03-05T15:23:36.355652800Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/puma/plugin/solid_queue.rb:15:in `fork'
2025-03-05T15:23:36.355656086Z  from /usr/local/bundle/ruby/3.2.0/gems/solid_queue-1.1.3/lib/puma/plugin/solid_queue.rb:15:in `block in start'
2025-03-05T15:23:36.355659610Z  from /usr/local/bundle/ruby/3.2.0/gems/puma-6.6.0/lib/puma/events.rb:17:in `block in fire'
2025-03-05T15:23:36.355663012Z  from /usr/local/bundle/ruby/3.2.0/gems/puma-6.6.0/lib/puma/events.rb:17:in `each'
2025-03-05T15:23:36.355666876Z  from /usr/local/bundle/ruby/3.2.0/gems/puma-6.6.0/lib/puma/events.rb:17:in `fire'
2025-03-05T15:23:36.355670520Z  from /usr/local/bundle/ruby/3.2.0/gems/puma-6.6.0/lib/puma/events.rb:46:in `fire_on_booted!'
2025-03-05T15:23:36.355674104Z  from /usr/local/bundle/ruby/3.2.0/gems/puma-6.6.0/lib/puma/cluster.rb:500:in `run'
2025-03-05T15:23:36.355677472Z  from /usr/local/bundle/ruby/3.2.0/gems/puma-6.6.0/lib/puma/launcher.rb:203:in `run'
2025-03-05T15:23:36.355680704Z  from /usr/local/bundle/ruby/3.2.0/gems/puma-6.6.0/lib/rack/handler/puma.rb:79:in `run'
2025-03-05T15:23:36.355684015Z  from /usr/local/bundle/ruby/3.2.0/gems/rackup-2.2.1/lib/rackup/server.rb:341:in `start'
2025-03-05T15:23:36.355687331Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/commands/server/server_command.rb:38:in `start'
2025-03-05T15:23:36.355712198Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/commands/server/server_command.rb:145:in `block in perform'
2025-03-05T15:23:36.355721876Z  from <internal:kernel>:90:in `tap'
2025-03-05T15:23:36.355725749Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/commands/server/server_command.rb:136:in `perform'
2025-03-05T15:23:36.355729450Z  from /usr/local/bundle/ruby/3.2.0/gems/thor-1.3.2/lib/thor/command.rb:28:in `run'
2025-03-05T15:23:36.355732907Z  from /usr/local/bundle/ruby/3.2.0/gems/thor-1.3.2/lib/thor/invocation.rb:127:in `invoke_command'
2025-03-05T15:23:36.355736347Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/command/base.rb:178:in `invoke_command'
2025-03-05T15:23:36.355740220Z  from /usr/local/bundle/ruby/3.2.0/gems/thor-1.3.2/lib/thor.rb:538:in `dispatch'
2025-03-05T15:23:36.355743937Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/command/base.rb:73:in `perform'
2025-03-05T15:23:36.355747827Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/command.rb:65:in `block in invoke'
2025-03-05T15:23:36.355751671Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/command.rb:143:in `with_argv'
2025-03-05T15:23:36.355755441Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/command.rb:63:in `invoke'
2025-03-05T15:23:36.355759944Z  from /usr/local/bundle/ruby/3.2.0/gems/railties-8.0.1/lib/rails/commands.rb:18:in `<main>'
2025-03-05T15:23:36.355763992Z  from /usr/local/bundle/ruby/3.2.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
2025-03-05T15:23:36.355767901Z  from /usr/local/bundle/ruby/3.2.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
2025-03-05T15:23:36.355771707Z  from ./bin/rails:4:in `<main>'
2025-03-05T15:23:36.393181230Z [13] ! reaped unknown child process pid=66 status=pid 66 exit 1
2025-03-05T15:23:38.296290692Z [13] Detected Solid Queue has gone away, stopping Puma...
2025-03-05T15:23:38.297198948Z [13] - Gracefully shutting down workers...
2025-03-05T15:23:38.503063906Z [13] === puma shutdown: 2025-03-05 15:23:38 +0000 ===
2025-03-05T15:23:38.503761030Z [13] - Goodbye!
2025-03-05T15:23:38.504059851Z Exiting
2025-03-05T15:23:38.530008430Z {"time":"2025-03-05T15:23:38.529577643Z","level":"INFO","msg":"Server stopping"}
2025-03-05T15:23:38.530840448Z {"time":"2025-03-05T15:23:38.529911267Z","level":"INFO","msg":"Server stopped"}
Connection to xxx closed.

Here is the database.yml, see that it kind of asks for migrations (migrations_paths: db/queue_migrate):

# SQLite. Versions 3.8.0 and up are supported.
#   gem install sqlite3
#
#   Ensure the SQLite 3 gem is defined in your Gemfile
#   gem "sqlite3"
#
default: &default
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  primary:
    adapter: postgresql
    pool: 5
    timeout: 5000
    host: localhost
    database: hikari_api_development
  cache:
    <<: *default
    database: storage/development_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/development_cable.sqlite3
    migrations_paths: db/cable_migrate

staging:
  primary:
    adapter: postgresql
    pool: 5
    timeout: 5000
    url: <%= ENV['POSTGRES_URL_STAGING'] %>
  cache:
    <<: *default
    database: storage/staging_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/staging_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/staging_cable.sqlite3
    migrations_paths: db/cable_migrate

# Store production database in the storage/ directory, which by default
# is mounted as a persistent Docker volume in config/deploy.yml.
production:
  primary:
    adapter: postgresql
    pool: 5
    timeout: 5000
    url: <%= ENV['POSTGRES_URL_PRODUCTION'] %>
  cache:
    <<: *default
    database: storage/production_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/production_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/production_cable.sqlite3
    migrations_paths: db/cable_migrate

# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
  primary:
    adapter: postgresql
    pool: 5
    timeout: 5000
    host: localhost
    database: hikari_api_test
  cache:
    <<: *default
    database: storage/test_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/test_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/test_cable.sqlite3
    migrations_paths: db/cable_migrate

If you need we can have a call.

rosa · 2025-03-05T18:19:20Z

Ahhh! You need to run db:prepare in your staging environment as well, so you create the DB and its tables there.

glundgrenm · 2025-03-05T18:25:51Z

that's the point hehe kamal is running it from a file called docker-entrypoint, that is called from the Dockerfile:

#!/bin/bash -e

# Enable jemalloc for reduced memory usage and latency.
if [ -z "${LD_PRELOAD+x}" ]; then
    LD_PRELOAD=$(find /usr/lib -name libjemalloc.so.2 -print -quit)
    export LD_PRELOAD
fi

# If running the rails server then create or migrate existing database
if [ "${@: -2:1}" == "./bin/rails" ] && [ "${@: -1:1}" == "server" ]; then
  ./bin/rails db:prepare
fi

exec "${@}"

It seems that its not created because the file was empty, i will try it with the schema filled.

Did not worked. Same error. I will try in a clean VM.

rosa · 2025-03-05T18:53:47Z

Are you running solid queue in that same container? Or just the server? You have this condition:

# If running the rails server then create or migrate existing database
if [ "${@: -2:1}" == "./bin/rails" ] && [ "${@: -1:1}" == "server" ]; then

glundgrenm · 2025-03-05T19:08:30Z

@rosa it worked on a clean VM and with the schema file "filled". Let me sum up with i think is wrong with the "solid" pack.

I believe the main bug is that the db:migrate is somehow emptying the schema for solid_cable, solid_queue and solid_cache.

maybe if the bug is addressed or even better, a helper to generate the migrations (it is more clear to see whats going on).

rosa · 2025-03-05T19:10:12Z

Ahh, great to know!

I believe the main bug is that the db:migrate is somehow emptying the schema for solid_cable, solid_queue and solid_cache.

Huh, this shouldn't be the case, and it used to be a bug in Rails that got supposedly fixed 🤔 Are you using Rails 8? I wonder if that's the problem... if you're on Rails 8, this might have been a regression.

glundgrenm · 2025-03-05T19:10:56Z

rails 8.0.1. thank you very much for the help rosa!

rosa · 2025-03-05T19:29:33Z

Thank you! This is the original bug I was thinking about: rails/rails#52829. In theory, it should be fixed in 8.0.1 😕

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Setting schema_format to sql prevents queue migration #525

Setting schema_format to sql prevents queue migration #525

longthanhtran commented Feb 28, 2025

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 •

edited

Loading

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 •

edited

Loading

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 •

edited

Loading

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 •

edited

Loading

rosa commented Mar 5, 2025

Setting schema_format to sql prevents queue migration #525

Setting schema_format to sql prevents queue migration #525

Comments

longthanhtran commented Feb 28, 2025

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 • edited Loading

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 • edited Loading

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 • edited Loading

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 • edited Loading

rosa commented Mar 5, 2025

glundgrenm commented Mar 5, 2025 •

edited

Loading

glundgrenm commented Mar 5, 2025 •

edited

Loading

glundgrenm commented Mar 5, 2025 •

edited

Loading

glundgrenm commented Mar 5, 2025 •

edited

Loading