Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update configuration schema for MongoDB adaptor. #354

Merged
merged 1 commit into from
Aug 22, 2023

Conversation

bensonmiller
Copy link

Fixes #353

Summary

Fixes JSON schema verification for mongodb clusterUrl configuration

Details

In packages/mongodb/src/Adaptor.js, the connect() function constructs the URI for the MongoDB cluster as follows:

  const uri = `mongodb+srv://${encodeURIComponent(
    username
  )}:${encodeURIComponent(
    password
  )}@${clusterUrl}/test?retryWrites=true&w=majority`;

The adaptor expects the clusterUrl configuration property be provided as a hostname and not a full URI, but the current JSON schema specifies the format of this property as uri, which fails validation when providing a normal hostname. This PR changes the JSON schema to verify this property as the correct format (hostname, not uri). The PR also renames the property and fixes all references to the property in the adaptor and unit tests.

Issues

#353

Review Checklist

Before merging, the reviewer should check the following items:

  • Does the PR do what it claims to do?
  • If this is a new adaptor, has the migration tool been run and the
    migration guide followed?
  • Are there any unit tests? Should there be?
  • Is there a changeset associated with this PR? Should there be? Note that
    dev only changes don't need a changeset.

Copy link
Collaborator

@josephjclark josephjclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great @bensonmiller, thanks!

We've got a lot going on here but we'll try and get this released today.

@josephjclark
Copy link
Collaborator

Hi @bensonmiller - I've been digging a bit deeper and I think I've found a couple of issues in the mongo adaptor. Bear with us while we get this sorted out!

@josephjclark
Copy link
Collaborator

Hi @bensonmiller - we're going to merge and release this, but it looks like the mongo adaptor could use a little love :'(

I've raised a bunch of issues - would you mind taking a quick look and letting me know if any of these are going to block you? I'm also interested if you have any other feedback, suggestions or insights.

Thanks!

@bensonmiller
Copy link
Author

@josephjclark Thank you for the deep dive here on the MongoDB adaptor. None of these changes would block me. I would note that I'm not deeply experienced with MongoDB, so my current uses are relatively unsophisticated. Nevertheless, I've looked at each of the issues you opened and they all look reasonable to me. If I were to rank these issues according to my own interest/priority:

HIGH:
Support databases not hosted on mongodb atlas (356)
Allow a database to be set in the credential (357)

MED:
Update client (359) -- mainly because it's low-effort

LOW:
Expose connection options (360)
Support TLS certs (358)

@josephjclark
Copy link
Collaborator

@bensonmiller hey, thank you for going through this, that's so helpful!

Nothing looks critical for right now so we'll get this released for you.

(By the way, I've not forgotten the JSON schema stuff, I am still looking into it! There are some security concerns I'm exploring)

@josephjclark josephjclark merged commit 101df0f into OpenFn:main Aug 22, 2023
1 check passed
@bensonmiller bensonmiller deleted the mongodb_creds_schema_change branch August 22, 2023 23:33
@josephjclark
Copy link
Collaborator

Hi @bensonmiller, just to let you know that this should be live on Lightning. We did a major bump on the adaptor so you'll need to be using Mongo 2.0.0 (or latest)

josephjclark pushed a commit that referenced this pull request Jun 11, 2024
Add support to supplying a project id when pulling
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MongoDB clusterUrl should be type hostname not URI
3 participants