Feature not supported: capped:true on AWS DocumentDB

[ciberpiranha]

Hello, I have a problem running rocketjob command, also when I try to enter in RocketJob Mission COntrol system gives my a beautiful 504. Always when I use AWS DOcumentDB.

If I switch to MongoDB everything works fine.
The error DocumentDb throws is Feature not supported: capped:true , and in fact they don't support that feature.

But in /lib/rocket_job/event.rb line 103 in self.create_capped_collection the gem explicitly set this capped: true as option.

Environment

• Ruby Version 3.1.3
• Rocket Job Version 6.1.1.
• Rails 7.0.4.2 with Puma 5.6.5
• AWS Document DB 4.0.0

Stack Trace

#<Thread:0x00007f456f5d3f98 /bundle/ruby/3.1.0/gems/rocketjob-6.1.1/lib/rocket_job/supervisor.rb:36 run> terminated with exception (report_on_exception is true):
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/result.rb:364:in `raise_operation_failure': [303]: Feature not supported: capped:true (on docdb-cluster-ddq-0.ceojorgnntl8.eu-west-1.docdb.amazonaws.com:27017) (Mongo::Error::OperationFailure)
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/result.rb:330:in `validate!'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:36:in `block (3 levels) in validate_result'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:108:in `add_server_diagnostics'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:35:in `block (2 levels) in validate_result'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:54:in `add_error_labels'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:34:in `block in validate_result'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:94:in `unpin_maybe'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:33:in `validate_result'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/executable.rb:81:in `block in execute'
        from <internal:kernel>:90:in `tap'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/executable.rb:80:in `execute'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/op_msg_executable.rb:49:in `execute_with_connection'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/op_msg_executable.rb:36:in `block in execute'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/server/connection_pool.rb:667:in `with_connection'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/server.rb:448:in `with_connection'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/op_msg_executable.rb:35:in `execute'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/collection.rb:397:in `block (2 levels) in create'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/collection/queryable_encryption.rb:43:in `maybe_create_qe_collections'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/collection.rb:386:in `block in create'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/client.rb:1132:in `with_session'
        from /bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/collection.rb:378:in `create'
        from /bundle/ruby/3.1.0/gems/rocketjob-6.1.1/lib/rocket_job/event.rb:103:in `create_capped_collection'
        from /bundle/ruby/3.1.0/gems/rocketjob-6.1.1/lib/rocket_job/event.rb:88:in `listener'
        from /bundle/ruby/3.1.0/gems/rocketjob-6.1.1/lib/rocket_job/supervisor.rb:36:in `block in run'
2023-02-01 13:11:58.519338 E [77:rocketjob event event.rb:93] RocketJob::Event -- #listener Event listener is terminating due to unhandled exception -- Exception: Mongo::Error::OperationFailure: [303]: Feature not supported: capped:true (on docdb-cluster-ddq-0.ceojorgnntl8.eu-west-1.docdb.amazonaws.com:27017)
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/result.rb:364:in `raise_operation_failure'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/result.rb:330:in `validate!'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:36:in `block (3 levels) in validate_result'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:108:in `add_server_diagnostics'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:35:in `block (2 levels) in validate_result'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:54:in `add_error_labels'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:34:in `block in validate_result'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:94:in `unpin_maybe'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/response_handling.rb:33:in `validate_result'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/executable.rb:81:in `block in execute'
<internal:kernel>:90:in `tap'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/executable.rb:80:in `execute'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/op_msg_executable.rb:49:in `execute_with_connection'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/op_msg_executable.rb:36:in `block in execute'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/server/connection_pool.rb:667:in `with_connection'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/server.rb:448:in `with_connection'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/operation/shared/op_msg_executable.rb:35:in `execute'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/collection.rb:397:in `block (2 levels) in create'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/collection/queryable_encryption.rb:43:in `maybe_create_qe_collections'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/collection.rb:386:in `block in create'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/client.rb:1132:in `with_session'
/bundle/ruby/3.1.0/gems/mongo-2.18.2/lib/mongo/collection.rb:378:in `create'
/bundle/ruby/3.1.0/gems/rocketjob-6.1.1/lib/rocket_job/event.rb:103:in `create_capped_collection'
/bundle/ruby/3.1.0/gems/rocketjob-6.1.1/lib/rocket_job/event.rb:88:in `listener'
/bundle/ruby/3.1.0/gems/rocketjob-6.1.1/lib/rocket_job/supervisor.rb:36:in `block in run'

Expected Behavior

• To connect && write on DocumentDB using rocketjob without any error and/or warnings
• To access to MissionControl

[reidmorrison]

That is very disappointing that AWS would choose not to implement such a basic feature of MongoDB.

We tested a much earlier version of Rocket Job against AWS DocumentDB and it worked fine. Since then we have moved to "real-time" management that uses a capped collection for sending events immediately to workers, to stop, start, or otherwise manage workers.

I updated the documentation with a note that AWS DocumentDB is not supported.