Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: import from 2.1 #689

Merged
merged 1 commit into from
Feb 17, 2025
Merged

fix: import from 2.1 #689

merged 1 commit into from
Feb 17, 2025

Conversation

gfyrag
Copy link
Contributor

@gfyrag gfyrag commented Feb 17, 2025

No description provided.

@gfyrag gfyrag requested a review from a team as a code owner February 17, 2025 15:42
Copy link

coderabbitai bot commented Feb 17, 2025

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

This pull request implements significant updates across the repository. The Ledger API documentation is revised to transition from version 1 to version 2, with detailed endpoint descriptions and examples. Internally, ID fields for logs and transactions are converted from value types to pointer types, and new helper methods (WithID) are added. The router initialization is modified to include an environment string (e.g. "develop"), and tests are updated accordingly. SQL migration scripts and legacy storage handling are also adjusted to match these changes.

Changes

File(s) Change Summary
docs/api/README.md, pkg/client/README.md, pkg/client/USAGE.md, pkg/client/docs/sdks/ledger/README.md, pkg/client/docs/sdks/v2/README.md Updated Ledger API documentation to reflect the transition to v2, added new endpoint details and usage examples, and removed deprecated sections.
internal/README.md, internal/doc.go Enhanced internal documentation with updated links, function/type references, and clearer declarations for exported entities.
internal/api/router.go, internal/api/v1/*, internal/api/v2/* (including tests) Modified router initialization to include an environment/version parameter, renamed functions (e.g. getInfo to GetInfo), and updated endpoint routing for v2 API usage.
internal/log.go, internal/transaction.go, internal/transaction_test.go Changed ID fields from int to pointer types, added WithID methods for Log and Transaction, and adjusted related functions to use proper pointer dereferencing.
internal/controller/ledger/* (and corresponding test files) Updated logging and error messages to dereference pointer IDs; modified mocks and expectations in tests to set IDs using pointer values.
internal/storage/bucket/migrations/* Revised SQL migration scripts with new functions and triggers, updated column defaults and encoding settings, and streamlined trigger management.
internal/storage/ledger/legacy/*, internal/storage/ledger/paginator_column.go, internal/storage/ledger/resource.go Introduced new paginated resource adapter types/methods and updated tests to correctly dereference IDs; added minor debugging aids.
pkg/client/formance.go, pkg/client/ledger.go, pkg/client/v2.go, pkg/testserver/* Updated SDK method calls to use versioned endpoints (e.g. Ledger.V2.GetInfo), adjusted version strings, and removed deprecated methods.
test/e2e/*, test/performance/benchmark_test.go Modified test cases to invoke updated API endpoints with the new environment parameter and to use pointer-dereferenced ID values; added new test scenarios for data import.

Sequence Diagram(s)

sequenceDiagram
    participant C as Client
    participant R as Router (v2)
    participant Ctrl as Controller
    participant Store as Ledger Store

    C->>R: Sends API request (e.g. CreateTransaction)
    R->>Ctrl: Forwards request with environment parameter ("develop")
    Ctrl->>Store: Executes operations (e.g. InsertTransaction using dereferenced IDs)
    Store-->>Ctrl: Returns transaction/log result with pointer-based IDs
    Ctrl-->>R: Sends back processed data (using WithID methods)
    R-->>C: Responds with updated v2 API response
Loading

Possibly related PRs

Suggested reviewers

  • flemzord
  • paul-nicolas

Poem

Hopping through lines of code with glee,
I find new pointers dancing merrily.
Docs and tests all hop in tune,
As routers sing a versioned tune.
With every change I twirl around,
A bunny’s joy in bugs unbound!
🥕🐰 Happy coding in every bound!


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@gfyrag gfyrag changed the base branch from main to release/v2.2 February 17, 2025 15:43
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments.

Actionable comments posted: 5

🧹 Nitpick comments (11)
internal/storage/ledger/legacy/adapters.go (3)

55-68: Optional: enforce maximum page size for pagination.
If user inputs are unbounded, large page sizes could impact performance. Consider bounding or validating query.PageSize to avoid potential performance issues.


136-149: Consider bounding the transactions Paginate query size.
Like with accounts, you may want to validate or limit query.PageSize for performance reasons.


157-181: Clarify usage of big.Int when constructing AggregatedVolumes.
Using new(big.Int).Add(new(big.Int), balance) effectively copies the balance value, but it's slightly verbose. For clarity, consider a direct set:

ret.Aggregated[asset] = ledger.Volumes{
-    Input:  new(big.Int).Add(new(big.Int), balance),
+    Input:  new(big.Int).Set(balance),
     Output: new(big.Int),
}
pkg/client/v2.go (2)

32-219: Consider explicitly handling 4xx responses & reducing code duplication.

  1. Error Handling for 4xx: The function currently branches for status code 200, the 5xx range, and a default case. Explicitly handling 4xx (e.g., 400, 401, 403, 404) can clarify error flow and prevent unexpected lumps of client errors under the “default” branch.

  2. Repeated Request/Retry Logic: The block for hooking, retrying, security population, etc. reappears in multiple functions (e.g., GetMetrics). You could extract it as a helper to improve readability and maintainability.


223-397: Refactor repeated patterns and consider large-body retries.

  1. Code Duplication: The request creation, retry setup, and hooking logic closely mirrors the approach in GetInfo. Consider centralizing these patterns to avoid repeated code blocks.

  2. Potential Large Payload Overhead: Copying the request body with GetBody() each time for retries can be expensive if future placements of metrics or other endpoints involve large payloads. You could handle small GET requests differently from large requests, or allow a one-time read/caching strategy for large bodies.

internal/api/v2/controllers_transactions_read_test.go (1)

31-31: Consider using dynamic transaction ID for more robust testing.

The change from using tx.ID to a hardcoded value 0 makes the test less dynamic and might not effectively test real-world scenarios.

Apply this diff to restore dynamic ID testing:

-    Builder: query.Match("id", 0),
+    Builder: query.Match("id", tx.ID),
internal/controller/ledger/log_process_test.go (1)

72-75: LGTM! Mock behavior updated for pointer-based IDs.

The mock now correctly simulates setting the ID field using a pointer, which aligns with the new ID field type changes.

Consider extracting the ID assignment to a helper function for reuse across tests:

-		DoAndReturn(func(ctx context.Context, log *ledger.Log) error {
-			log.ID = pointer.For(0)
-			return nil
-		})
+		DoAndReturn(func(ctx context.Context, log *ledger.Log) error {
+			return assignLogID(log, 0)
+		})

+func assignLogID(log *ledger.Log, id int) error {
+	log.ID = pointer.For(id)
+	return nil
+}
internal/api/v2/controllers_transactions_revert_test.go (1)

84-84: LGTM! Consider using a constant for the environment string.

The addition of the environment parameter aligns with the compatibility fix. However, consider defining the environment string as a constant to maintain consistency across test files.

+const testEnvironment = "develop"
+
 func TestTransactionsRevert(t *testing.T) {
     // ...
-    router := NewRouter(systemController, auth.NewNoAuth(), "develop", os.Getenv("DEBUG") == "true")
+    router := NewRouter(systemController, auth.NewNoAuth(), testEnvironment, os.Getenv("DEBUG") == "true")
internal/api/v1/routes.go (1)

32-32: LGTM! Consider adding documentation for the exported function.

The change to export GetInfo aligns with the fix for the GET /_info endpoint. Since this is now a public API, consider adding documentation to describe its purpose and usage.

+// GetInfo handles the GET /_info endpoint request.
+// It returns information about the system controller and version.
+// Parameters:
+//   - systemController: The system controller instance
+//   - version: The API version string
 router.Get("/_info", GetInfo(systemController, version))
internal/storage/bucket/migrations/11-make-stateless/up.sql (1)

32-45: Schema Alterations and Column Renaming in Moves Table
The migration alters the moves table by adding the transactions_id column, dropping NOT NULL constraints for certain volume columns, and renaming account_address to accounts_address (and similarly for the address array). These changes improve consistency with updated naming conventions. Verify that all application queries and indices are adjusted accordingly.
[verified]

pkg/client/docs/sdks/v2/README.md (1)

85-134: GetMetrics Documentation Section Added
The GetMetrics section follows a similar template as GetInfo and is clear and complete. It outlines:

  • A concise introduction ("Read in memory metrics")—consider hyphenating this to "Read in-memory metrics" for consistency.
  • A full example usage demonstrating the client construction and endpoint call.
  • Detailed tables for parameters and responses.

Ensure that the linked response type ([*operations.GetMetricsResponse]) accurately reflects the API’s output and that the anchor link (#getmetrics) is generated correctly by your markdown renderer.

🛑 Comments failed to post (5)
internal/storage/ledger/legacy/adapters.go (5)

183-185: 🛠️ Refactor suggestion

Stop using panic for unimplemented methods.
Similar to the logs adapter, return an error or remove the method if truly never needed.


208-214: 🛠️ Refactor suggestion

Avoid panics in GetOne and Count methods.
Prevent unexpected runtime failures with a safer stub or error return.


22-39: 🛠️ Refactor suggestion

Handle potential type mismatch in GetOne to avoid panic.
This code performs value.(string) without verifying that value is actually a string. If the underlying type differs, it will panic.

Consider returning an error instead of panicking:

 func (p accountsPaginatedResourceAdapter) GetOne(ctx context.Context, query ledgercontroller.ResourceQuery[any]) (*ledger.Account, error) {
   var address string
   err := query.Builder.Walk(func(_ string, _ string, value any) error {
-    address = value.(string)
-    return nil
+    s, ok := value.(string)
+    if !ok {
+      return fmt.Errorf("expected string but got %T", value)
+    }
+    address = s
+    return nil
   })
   if err != nil {
     return nil, err
   }
   return p.store.GetAccountWithVolumes(ctx, GetAccountQuery{
     ...
   })
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

func (p accountsPaginatedResourceAdapter) GetOne(ctx context.Context, query ledgercontroller.ResourceQuery[any]) (*ledger.Account, error) {
	var address string
	err := query.Builder.Walk(func(_ string, _ string, value any) error {
		s, ok := value.(string)
		if !ok {
			return fmt.Errorf("expected string but got %T", value)
		}
		address = s
		return nil
	})
	if err != nil {
		return nil, err
	}
	return p.store.GetAccountWithVolumes(ctx, GetAccountQuery{
		PITFilterWithVolumes: PITFilterWithVolumes{
			PITFilter: PITFilter{
				PIT: query.PIT,
				OOT: query.OOT,
			},
			ExpandVolumes:          slices.Contains(query.Expand, "volumes"),
			ExpandEffectiveVolumes: slices.Contains(query.Expand, "effectiveVolumes"),
		},
		Addr: address,
	})
}

72-82: 🛠️ Refactor suggestion

Avoid implementing placeholder methods that panic.
Relying on panic("never used") can lead to unexpected runtime failures if the method is ever called. Consider returning an error or leaving them unimplemented with a clear message.

 func (p logsPaginatedResourceAdapter) GetOne(_ context.Context, _ ledgercontroller.ResourceQuery[any]) (*ledger.Log, error) {
-  panic("never used")
+  return nil, errors.New("GetOne for logsPaginatedResourceAdapter is not implemented")
 }
 
 func (p logsPaginatedResourceAdapter) Count(_ context.Context, _ ledgercontroller.ResourceQuery[any]) (int, error) {
-  panic("never used")
+  return 0, errors.New("Count for logsPaginatedResourceAdapter is not implemented")
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

type logsPaginatedResourceAdapter struct {
	store *Store
}

func (p logsPaginatedResourceAdapter) GetOne(_ context.Context, _ ledgercontroller.ResourceQuery[any]) (*ledger.Log, error) {
	return nil, errors.New("GetOne for logsPaginatedResourceAdapter is not implemented")
}

func (p logsPaginatedResourceAdapter) Count(_ context.Context, _ ledgercontroller.ResourceQuery[any]) (int, error) {
	return 0, errors.New("Count for logsPaginatedResourceAdapter is not implemented")
}

103-120: 🛠️ Refactor suggestion

Handle integer type assertion safely in GetOne.
Similar to the accounts adapter, the code asserts value.(int). This will panic if value is not an int.

 func (p transactionsPaginatedResourceAdapter) GetOne(ctx context.Context, query ledgercontroller.ResourceQuery[any]) (*ledger.Transaction, error) {
   var txID int
   err := query.Builder.Walk(func(_ string, _ string, value any) error {
-    txID = value.(int)
-    return nil
+    i, ok := value.(int)
+    if !ok {
+      return fmt.Errorf("expected int but got %T", value)
+    }
+    txID = i
+    return nil
   })
   if err != nil {
     return nil, err
   }
   return p.store.GetTransactionWithVolumes(ctx, GetTransactionQuery{
     ...
   })
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

func (p transactionsPaginatedResourceAdapter) GetOne(ctx context.Context, query ledgercontroller.ResourceQuery[any]) (*ledger.Transaction, error) {
	var txID int
	err := query.Builder.Walk(func(_ string, _ string, value any) error {
		i, ok := value.(int)
		if !ok {
			return fmt.Errorf("expected int but got %T", value)
		}
		txID = i
		return nil
	})
	if err != nil {
		return nil, err
	}
	return p.store.GetTransactionWithVolumes(ctx, GetTransactionQuery{
		PITFilterWithVolumes: PITFilterWithVolumes{
			PITFilter: PITFilter{
				PIT: query.PIT,
				OOT: query.OOT,
			},
			ExpandVolumes:          slices.Contains(query.Expand, "volumes"),
			ExpandEffectiveVolumes: slices.Contains(query.Expand, "effectiveVolumes"),
		},
		ID: txID,
	})
}

@gfyrag gfyrag force-pushed the hotfix/import-from-2.1 branch 3 times, most recently from 283e5eb to 20edf69 Compare February 17, 2025 16:21
@gfyrag gfyrag force-pushed the hotfix/import-from-2.1 branch from 20edf69 to e9ba16a Compare February 17, 2025 16:38
@gfyrag gfyrag merged commit aa20c11 into release/v2.2 Feb 17, 2025
8 checks passed
@gfyrag gfyrag deleted the hotfix/import-from-2.1 branch February 17, 2025 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants