Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: grail filter segments client #135

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

warber
Copy link
Contributor

@warber warber commented Sep 24, 2024

What this PR does / Why we need it:

This PR introduces the http client for grail filter segments to be used by monaco and terraform.

Special notes for your reviewer:

Does this PR introduce a user-facing change?

@warber warber changed the title Feat/grail filter segments client feat: grail filter segments client Sep 24, 2024
Copy link

github-actions bot commented Sep 24, 2024

Test Results

  3 files  ± 0   54 suites  +6   32s ⏱️ -1s
268 tests +16  268 ✅ +16  0 💤 ±0  0 ❌ ±0 
804 runs  +48  804 ✅ +48  0 💤 ±0  0 ❌ ±0 

Results for commit 1d2457e. ± Comparison against base commit 4a7af80.

♻️ This comment has been updated with latest results.

@jskelin jskelin force-pushed the feat/grail-filter-segments-client branch 3 times, most recently from 295719b to 429ee69 Compare October 21, 2024 07:30
@jskelin jskelin self-assigned this Oct 21, 2024
@jskelin jskelin marked this pull request as ready for review October 21, 2024 07:37
@jskelin jskelin requested a review from a team as a code owner October 21, 2024 07:37
@jskelin jskelin force-pushed the feat/grail-filter-segments-client branch 5 times, most recently from 87361f2 to e902702 Compare October 21, 2024 09:13
api/clients/grailfiltersegements/filtersegments_test.go Outdated Show resolved Hide resolved
api/clients/grailfiltersegements/filtersegments_test.go Outdated Show resolved Hide resolved
clients/grailfiltersegments/filtersegments.go Show resolved Hide resolved
clients/grailfiltersegments/export_test.go Show resolved Hide resolved
api/clients/grailfiltersegements/filtersegments.go Outdated Show resolved Hide resolved
clients/grailfiltersegments/filtersegment_test.go Outdated Show resolved Hide resolved
clients/grailfiltersegments/filtersegments.go Outdated Show resolved Hide resolved
clients/grailfiltersegments/filtersegments.go Outdated Show resolved Hide resolved
clients/grailfiltersegments/filtersegments.go Outdated Show resolved Hide resolved
var getRequestOptions = rest.RequestOptions{
CustomShouldRetryFunc: rest.RetryIfTooManyRequests,
QueryParams: map[string][]string{
"add-fields": {"INCLUDES", "VARIABLES", "RESOURCECONTEXT"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These differ from the other fields, why is that? Is it not necessary?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, not all parts of the configuration are included. If a user doesn't specify what exactly it needs, we assume that it requests everything.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question is why the ExteralId is missing here, as the list queries all by default.

@Laubi
Copy link
Contributor

Laubi commented Oct 24, 2024

To the points mentioned above, please also add documentation to all public methods, with all the assumptions and special cases we're taking

@jskelin jskelin force-pushed the feat/grail-filter-segments-client branch 4 times, most recently from f734a74 to e15bb00 Compare November 11, 2024 13:47
@jskelin jskelin force-pushed the feat/grail-filter-segments-client branch 3 times, most recently from c006cc9 to 8042673 Compare November 13, 2024 08:41
@jskelin jskelin force-pushed the feat/grail-filter-segments-client branch from 8042673 to 22ba664 Compare November 14, 2024 10:03
Copy link
Contributor

@Laubi Laubi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major points that are still required:

  1. 0% test coverage of the filtersegments. I gave an example in my comments.
    a) This is because of the mocks. We should do fakes instead of mocks and use a local server and respond the HTTP results directly. Then, the inner code would be tested.
    b) Because of that, a real bug was not detected that should have been easily detectable, as it's not even in the method doc.

  2. Inconsistent error return values. Sometimes you return a plain err, sometimes you wrap it. Please consistently wrap it.

  3. The inner test names are not consistent in clients/grailfiltersegments/filtersegment_test.go

}

// set default behavior to pick all information by default
if ro.QueryParams == nil && ro.QueryParams.Has("add-fields") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if ro.QueryParams == nil && ro.QueryParams.Has("add-fields") {
if ro.QueryParams == nil && !ro.QueryParams.Has("add-fields") {

When I saw this, I noticed that there are no unit tests at all present. Please add some.
For this small check, I came up with this test:

func TestGet_AllFieldsArePresentIfNotSpecified(t *testing.T) {
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		t.Log(r.URL.String())

		fields := r.URL.Query()["add-fields"]
		require.ElementsMatch(t, []string{"INCLUDES", "VARIABLES", "EXTERNALID", "RESOURCECONTEXT"}, fields)
	}))
	defer server.Close()
	u, err := url.Parse(server.URL)
	require.NoError(t, err)

	c := NewClient(rest.NewClient(u, server.Client()))

	resp, err := c.Get(context.TODO(), "id", rest.RequestOptions{})
	require.NoError(t, err)

	require.Equal(t, http.StatusOK, resp.StatusCode)
}

Please add some basic tests for all endpoints, e.g. input validation


func (c Client) List(ctx context.Context) (*http.Response, error) {
path := endpointPath + ":lean" // minimal set of information is enough
return c.client.GET(ctx, path, rest.RequestOptions{CustomShouldRetryFunc: rest.RetryIfTooManyRequests})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing I noticed: Sometimes (like here) you are directly returning from the client, while in other cases (like Create) you are checking err and add context to it. I'd suggest doing this everywhere, so here and in Update adding fetching all items failed or updating failed below

clients/grailfiltersegments/export_test.go Show resolved Hide resolved
var getRequestOptions = rest.RequestOptions{
CustomShouldRetryFunc: rest.RetryIfTooManyRequests,
QueryParams: map[string][]string{
"add-fields": {"INCLUDES", "VARIABLES", "RESOURCECONTEXT"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question is why the ExteralId is missing here, as the list queries all by default.

func (c Client) GetAll(ctx context.Context) ([]Response, error) {
listResp, err := c.List(ctx)
if err != nil {
return nil, err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here you're doing something similar to what I commented before: Sometimes you do add information to the errors, sometimes you don't (e.g. L124 you add information). I suggest adding everywhere the context what's going wrong. Please have a look where you only return the err without extra information and add some.


func TestUpsert(t *testing.T) {

t.Run("Create - Create OK", func(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the other file, the prefix is not needed. Here it's even wrong

Suggested change
t.Run("Create - Create OK", func(t *testing.T) {
t.Run("Creates if config does not exist", func(t *testing.T) {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file has 0% test coverage

assert.ErrorAs(t, err, &api.APIError{})

var apiErr api.APIError
errors.As(err, &apiErr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
errors.As(err, &apiErr)
`assert.ErrorAs(t, err, &apiErr)`

Nit: This will check that the error is really of that type. Otherwise, you are currently silently swallowing the error. In case the As would fail, the Equals after will fail, but we would not know why/what error type it really was

To update an OpenPipeline configuration, the `updateToken` must be either empty, or set to the latest version. To be consistent with existing code, we set the `updateToken`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants