Skip to content

Add API endpoint to request contents of multiple files simultaniously #34139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 41 commits into from
Apr 21, 2025

Conversation

denyskon
Copy link
Member

@denyskon denyskon commented Apr 7, 2025

Adds an API POST endpoint under /repos/{owner}/{repo}/file-contents which receives a list of paths and returns a list of the contents of these files.

This API endpoint will be helpful for applications like headless CMS (reference: sveltia/sveltia-cms#198) which need to retrieve a large number of files by reducing the amount of needed API calls.

Close #33495

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Apr 7, 2025
@github-actions github-actions bot added modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code labels Apr 7, 2025
@denyskon denyskon added this to the 1.24.0 milestone Apr 7, 2025
@denyskon denyskon added the type/feature Completely new functionality. Can only be merged if feature freeze is not active. label Apr 7, 2025
@denyskon denyskon requested a review from wxiaoguang April 7, 2025 14:27
@wxiaoguang
Copy link
Contributor

Some thoughts in my mind:

  1. Why it is a "POST" endpoint? If I understand correctly, it "GETs"
  2. Does GitHub have a simliar API? If yes, we'd better to follow, if no, let's invent
  3. It needs some tests (almost all existing APIs are covered by tests, otherwise it would easily break in the future if something changes)

@denyskon
Copy link
Member Author

denyskon commented Apr 7, 2025

  1. IIRC, the HTTP spec says or has said that GET requests should not contain a body - that's why it isn't an option. Post is the next best choice I think.

  2. GitHub doesn't have a similar API, as such functionality is now being served through their GraphQL API. This, however, is clearly out of scope for near future, so I think that such an API endpoint would be helpful :)

  3. I can add some tests...

@silverwind
Copy link
Member

  1. IIRC, the HTTP spec says or has said that GET requests should not contain a body - that's why it isn't an option. Post is the next best choice I think.

HTTP QUERY is a proposed standard to solve the GET-with-body problem. Tooling support is probably not there yet, but if it works in your case, it might be an option to use it already.

Copy link
Contributor

@wxiaoguang wxiaoguang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. IIRC, the HTTP spec says or has said that GET requests should not contain a body - that's why it isn't an option. Post is the next best choice I think.

I see, I missed the Files []string field, it would be quite large so it is POST's job.

@denyskon denyskon requested a review from wxiaoguang April 10, 2025 07:14
@denyskon
Copy link
Member Author

@wxiaoguang Added an integration test

@denyskon denyskon requested a review from a team April 11, 2025 14:13
Copy link
Member

@lunny lunny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Apr 11, 2025
@wxiaoguang
Copy link
Contributor

Found various legacy (related) problems, need to fix together. If I open a separate refactoring PR, the PRs would conflict largely. So I made the changes in this PR directly. 🙏

@denyskon
Copy link
Member Author

@wxiaoguang Looks good, I already noticed that the mechanism for getting the ref was flawed. I also missed that base64 encoding increases the size - I adapted the test to it. Could you fix the lint issues in your bugfix?

@wxiaoguang
Copy link
Contributor

wxiaoguang commented Apr 21, 2025

One more thing, it's impossible to make "POST" work with "read" permission at the moment, so I also added the "GET" method support. It is somewhat hacky but I think it is the only thing we can do at the moment.

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Apr 21, 2025
@lunny lunny added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Apr 21, 2025
@wxiaoguang wxiaoguang merged commit 9a071a5 into go-gitea:main Apr 21, 2025
26 checks passed
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Apr 21, 2025
@denyskon denyskon deleted the feat/multiple-files-api branch April 21, 2025 17:47
zjjhot added a commit to zjjhot/gitea that referenced this pull request Apr 23, 2025
* giteaofficial/main:
  [skip ci] Updated translations via Crowdin
  Display when a release attachment was uploaded (go-gitea#34261)
  Fix Set Email Preference dropdown and button placement (go-gitea#34255)
  [skip ci] Updated translations via Crowdin
  Update compare.tmpl (go-gitea#34251)
  Make public URL generation configurable (go-gitea#34250)
  Add API endpoint to request contents of multiple files simultaniously (go-gitea#34139)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-update-needed The document needs to be updated synchronously lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code type/feature Completely new functionality. Can only be merged if feature freeze is not active.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[API] Retrieve a list of metadata and content for the requested file paths.
6 participants