Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs to clone a new DANDI instance #104

Merged
merged 99 commits into from
Feb 18, 2025
Merged

Conversation

aaronkanzer
Copy link
Member

@aaronkanzer aaronkanzer commented Dec 1, 2023

Looking for review for now, no need to merge

These documents provide a step-by-step process if another user would like to launch their own Dandi-like ecosystem

please see here if you'd like to observe a live link: https://aquamarine-profiterole-e20e84.netlify.app/

or specifically:

https://lincbrain.github.io/handbook/40_initialization/

@jwodder
Copy link
Member

jwodder commented Dec 1, 2023

@aaronkanzer Why do the instructions say to create an account on PyPI? That should only be done if you're planning to release packages on PyPI, which has nothing to do with interacting with DANDI.

@kabilar
Copy link
Member

kabilar commented Dec 1, 2023

@aaronkanzer Why do the instructions say to create an account on PyPI? That should only be done if you're planning to release packages on PyPI, which has nothing to do with interacting with DANDI.

Hi @jwodder, These instructions are meant for the developers of the data archive and associated tools, especially for developing a new DANDI-like ecosystem which we are doing for the LINC project. Since the DANDI CLI and Python API are a method of interacting with the archive, Aaron added instructions here for releasing the Python package to PyPI. Hope this helps to answer your question.

@jwodder
Copy link
Member

jwodder commented Dec 1, 2023

@kabilar

Since the DANDI CLI and Python API are a method of interacting with the archive, Aaron added instructions here for releasing the Python package to PyPI.

Releasing what package? This sentence implies you'll be releasing dandi, which only four people can do, none of which are you.

@kabilar
Copy link
Member

kabilar commented Dec 1, 2023

@kabilar

Since the DANDI CLI and Python API are a method of interacting with the archive, Aaron added instructions here for releasing the Python package to PyPI.

Releasing what package? This sentence implies you'll be releasing dandi, which only four people can do, none of which are you.

Hi @jwodder, we are releasing a clone of the dandi client, as the lincbrain client for interacting with LINC datasets. (Please disregard the current semantic version as it will be deleted and released as 0.X.0.)

You'll be prompted with the following form -- see the example values populated in this screenshot -- more to come in other sections for where these values might be populated:

<br/><br/>
<img
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We (EMBER) ended up needing to change these to the following to get auth working:

rather than the existing:
image


### Creating "Sites" and "Social App" Objects

**In order to complete this step, you will need to have deployed an initial DANDI Archive API**. See [Creating the DANDI Archive API](../64_dandi_archive) for more details.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommendation: Consider re-ordering the pages for Authentication, CLI, Infrastructure, and Archive.

We (EMBER) roughly did things in the following order:

  • Infrastructure
  • Archive
  • Authentication
  • CLI

Of course there was a little back and forth but in general this order seemed to work well

Comment on lines +127 to +144
### Heroku Dyno Sizes

`dandi-infrastructure` defines "dyno" (a.k.a process) sizes for each service being run. For specific reference,
[see here in api.tf](https://github.com/dandi/dandi-infrastructure/blob/master/terraform/api.tf#L14-L18).

While your DANDI Archive clone may differ in traffic and activity, the defaults set in `dandi-infrastructure` rarely
exceed 75% usage.

Keep in mind the different [pricing structures](https://www.heroku.com/pricing) that come with choosing different Heroku dyno sizes

### Heroku Add-Ons

In addition to the Heroku 'dynos' that are added for compute, multiple Heroku 'add-ons' are included

- [CloudAMQP](https://elements.heroku.com/addons/cloudamqp) -- **Use Case**: Message Broker
- [Postgres](https://elements.heroku.com/addons/heroku-postgresql) -- **Use Case**: Database
- [Papertrail](https://elements.heroku.com/addons/papertrail) -- **Use Case**: Log Management

Copy link
Member

@kabilar kabilar Feb 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move these topics to the Understanding the DANDI Infrastructure section in this page?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe they are already in that section? Unless you had something else in mind?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think they are currently in the Domain Management section, but its not a big deal.

@kabilar kabilar changed the title Add docs framework for how to clone a new DANDI instance Add docs to clone a new DANDI instance Feb 18, 2025
Copy link
Member

@kabilar kabilar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @aaronkanzer. I am sure this took a huge effort. Great work.

There are just a few remaining suggestions above.

@aaronkanzer
Copy link
Member Author

@kabilar -- updates should be included from your review

@NEStock thanks for the recommendations -- just so I don't lose track of all these changes, I'm going to do a follow-up PR to update the docs per EMBER

kabilar and others added 2 commits February 18, 2025 16:00
Co-authored-by: Yaroslav Halchenko <[email protected]>
Co-authored-by: Yaroslav Halchenko <[email protected]>
Copy link
Member

@kabilar kabilar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @aaronkanzer.

After discussion with Aaron, we will proceed with the merge here and he will be submitting a subsequent pull request for the few unresolved suggestions.

@kabilar kabilar merged commit 663bb07 into dandi:master Feb 18, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

7 participants