Merge lando-api repo #13
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change upgrades all of the python dependencies and moves lando api to python 3.7. The standard python docker image is now used to greatly simplify dependency installation and better take advantage of shared layer caching between applications. The largest change that had to be worked around was in connexion. It no longer supports having handlers themselves take care of security schemes, and forbids the use of multiple schemes on a single endpoint. This is problematic because we allow anonymous access to some endpoints that also accept an api key. Integrating our authentication with the new built-in auth wrappers would be quite a lot of work and wouldn't even be properly possible without support for multiple schemes. Because of this all security definitions have been removed from the swagger spec. While unfortunate, they are only informational when running in production.
Add DEBUG logging to conduit API calls to aid local development.
Pin the PostgreSQL container version to v9.6 in docker-compose.yml. This matches the version of PostgreSQL in GCP. The service dies with an error about incompatible init scripts if you try to start the app with the postgres:latest image, which is Postgres v11.
The default error handlers eat the PhabricatorAPIException root cause and traceback. This makes debugging API failures difficult and breaks PyCharm unittest integration. We can make the error handler optional in the same way that the Flask default error handler is optional: in production it wraps all errors, but in the test and debug environments exceptions are allowed to propagate through the stack to make debugging easier.
Flask uses the TESTING config variable to determine if test suite errors should propagate or be wrapped in generic HTTP status codes. The correct behaviour is to set TESTING=True during test suite runs. Also kill the dead 'testing' parameter to construct_app.
Revision entities returned by the API publish a new "is_secure" property. Revisions for which this property is True contain security-sensitive information and should be handled appropriately by clients.
Add an API operation that accepts sanitized commit messages for revisions. Executing the operation kicks off the sec-approval workflow, including asking for review from the proper project.
…ed for a revision (Bug 1569721)(#75) In `calculate_landable_subgraphs` we iterate over the repository values attached to the revisions in our stack and check if the repo is valid using `repo not in landable_repos`. This blocks the revision from landing and displays a sensible error message about an unsupported repo. However, in the case a repo has not been associated with a revision, the condition will still be true as the empty value is not in the set of landable repos. This causes confusion to end users regarding what to do to fix the problem. In this commit we add an extra check to assert the value of `repo` is not a False-y value. If the value is False-y, we provide a more instructive error message and block the revision from landing. Otherwise, we continue with the existing block logic.
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 0.15.2 to 0.15.3. **This update includes a security fix.** - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/master/CHANGES.rst) - [Commits](pallets/werkzeug@0.15.2...0.15.3) Signed-off-by: dependabot-preview[bot] <[email protected]>
Wrap the Phabricator API's 'transaction.search' endpoint with some useful functions.
This is a port of transplant's code for dealing with mercurial using the command server. It installs the required hg libraries and hg itself under python3 and provides code for calling out to hg. Some minimal tests and fixtures are provided in order to test hg interactions.
Bumps [ecdsa](https://github.com/warner/python-ecdsa) from 0.13.2 to 0.13.3. - [Release notes](https://github.com/warner/python-ecdsa/releases) - [Changelog](https://github.com/warner/python-ecdsa/blob/python-ecdsa-0.13.3/NEWS) - [Commits](tlsfuzzer/python-ecdsa@python-ecdsa-0.13.2...python-ecdsa-0.13.3) Signed-off-by: dependabot-preview[bot] <[email protected]>
…#83) Add a PhabricatorDouble method that takes a mock object returned from the PhabricatorDouble factory methods and returns the same object as it would appear in a Phabricator API search result. Refactor existing test code to use the new method.
Allow the creation of fake Phabricator comment objects, like you would have from adding a comment to a Revision. Having comments properly mocked makes the Transaction mocks match real Phabricator Transaction objects more closely.
Add a pytest custom assertion handler to decorate failed API responses. In addition to reporting the failed HTTP status code the response JSON, including the error message within it, is also reported in the test output.
Transaction objects can now be searched for by both PHID and by object name, such as "D123". This is necessary for situations where only the object name is locally available without a network request. It also matches the Phabricator API, which allows searches by both PHID and object name.
Exclude the sec-approval team from the list of code reviewers when displaying a commit message in the UI or building a commit message to land with the transplant service.
If a revision is part of the sec-approval workflow then publish that revision's alternate commit message and related metadata in the Lando API response.
When a user requests landing of a commit that has an alternate commit message going through the sec-approval process use the alternate commit title and commit summary as the patch title and summary.
The conduit suite uses the repository named 'test-repo' for testing. test-repo was made unlandable in a prior commit. Fix the Lando repo name registry so test-repo is landable again.
Note: this change also updates SecApprovalRequest to have an updated_at field. The `Base` class can be extended to create a model that has some common attributes and fields, namely: `updated_at` and `created_at` fields, an `id` field, a default `__tablename__` (snake-case of the class name), as well as a default `__repr__` method.
Put the override-able version.json into a directory so that it can be mounted by docker-compose. Note that a version.json isn't created by default, because we want lando to default to the "dev version", while still giving devs the ability mock out the file if necessary.
The message will include instructions on how to fix an expired token issue. Since we have different types of tokens, users can easily mix up what to do without more specific messaging.
Commit 1 of 2. This commit is retained to maintain authorship info.
…#328) Adds a new POST /try endpoint to Lando that accepts try patch data and creates a landing job to submit the patches to try. Patches are submitted as an array of base64 encoded bytes and the base commit for the try submission in mozilla-unified is passed so the patch applies as it would in the developer's local machine. The Repo object sees several changes to allow Lando to work with the try repo. A force_push option is added that controls adding the -f option during hg push. A phabricator_repo flag is also added which indicates if the repo is present on Phabricator. At the moment this flag controls pinging Phabricator to issue a repo update API call, which is irrelevant for try. The try repo is added to the repo configs, with mozilla-unified as the pull_url and try as the push_url. This means patches will be applied to mozilla-unified in Lando and pushed out to try, in the same fashion as the current try push model on a developer's local machine. A decorator is added to enforce active_scm_level_1 group permissions with appropriate error messages. Patches can be submitted in the hg export or git format-patch formats. Support for git format-patch is added via a Git patch parser similar to the HgPatchHelper.
Refactor some patch parsing code in the `HgPatchHelper` to clarify what is taking place. Use `enumerate` instead of a manually incremented looping variable. Flatten if-statement logic to reduce repeated behaviour and add a comment describing what is taking place within the loop.
Adds a `strip_depends_on_from_commit_message` function that removes `Depends on` lines from commit messages. Pass the commit message of the original revision through this function before creating the uplift revision to avoid linking uplift revisions to the original revision.
…5) (#325) - add mots to requirements - add Revision.data field that will store meta information - upon landing request, store list of reviewers who accepted revision - upon landing, store list of peers and owners who accepted revision - add tests The new Revision.data field will be used to store information about who approved a revision and of those users, who is a peer or an owner. The data will be used for analysis down the road to determine how many revisions land with peer or owner approval.
…343) When pushes fail due to timeouts waiting for the repo lock, Lando should wait and retry the push. Add a new exception for this condition that includes snippets of the expected output from Mercurial, and add the exception to the block of temporary failure exceptions when trying to push. --------- Co-authored-by: Zeid Zabaneh <[email protected]>
Remove the buffer read size argument from calls to `BytesIO.read` to instead read to EOF. Parametrize the integrated landing job with a test case that would fail on the current behaviour due to an improperly decoded string.
…) (#347) Transition from using `io.BytesIO` to `io.StringIO` for content buffers. Update uses of `NamedTemporaryFile` to open in `w+` mode with a specified encoding.
… pull (Bug 1858500) (#349) Add a `PullFailureException` with a snippet that looks for errors during `hg pull` and add it to the list of Lando exceptions that can be raised from an hglib exception. Add an except case to the `update_repo` try block that is similar to the case in the `push` try block, to allow the job to be retried when this exception is encountered.
…860067) (#351) The error snippet for `PullFailureException` is a piece of text that comes from Lando, not Mercurial. The correct piece of text to look for is `abort: HTTP Error 500`, which is a more general error that can happen when load on hg is high. Rename `PullFailureException` to `HgmoInternalServerError` and update the snippet to reflect the output from hg. Since this error is more general, we also add it to the set of exceptions to consider a temporary failure at push time.
…(#352) Add `--upgrade` to the `pip-compile` command in our `build-requirements` docker-compose recipe so packages are upgraded to later versions when possible.
…po (Bug 1843041) (#353) Add the `metrics` attachment to the Conduit API call for the target repo, which adds the most recent commit hash from that repo to the returned data. Use this commit hash as the `sourceControlBaseRevision` value when uploading the diff to Phabricator. This mirrors how the value is set in `moz-phab`.
…(#355) Make the Phabricator landing emails less Phabricator-specific. Update the landing job identifier to include the context that the job belongs to a try push with the included commit message title. Add a check of the landing job identifier to only display a link to a Lando-UI URL when the identifier is a Phab revision ID.
…69) (#356) PyYaml <= 6.0.0 has a bug that causes build failues on later versions of Python due to a Cython version bump. I noticed this when I forgot to source a virtualenv and attempted to install the Lando-API requirements into a Python 3.12 interpreter. The fix is available on PyYaml 6.0.1, so we should get ahead of this problem and upgrade requirements now while it's low risk.
…t succeed (Bug 1864912) (#359)
Add a type hint for `LandingJob.revisions`. This fixes LSP features on this field, which don't work at the moment as the LSP can't infer the type from SQLAlchemy's string-based type hints.
When empty Git commits are sent to Lando in `git format-patch` format, Lando fails to parse the commit correctly. Currently Lando will fail to find the `---` line used to separate the commit message from the diff, and assume all lines in the patch body are part of the commit message. The parsing will then fail later down the line as Lando will not find a valid diff line in the body. Add an `else` clause to the loop which looks for the `---` line separating the extended commit message and the diff, which when reached indicates that no diff is present on the commit. In this case we strip the last two lines of the gathered commit message and return an empty diff. Add a test for parsing a patch generated from an empty commit to `test_hgexports.py`.
zzzeid
force-pushed
the
zeid/bug-1867357-lando-api
branch
from
e2ef945 to
7f1eed4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.