Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pluggy.ai bank sync #530

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

Conversation

lelemm
Copy link
Contributor

@lelemm lelemm commented Dec 27, 2024

Pluggy.ai bank sync for Brazilian Banks
Needs this actualbudget/actual#4049

@actual-github-bot actual-github-bot bot changed the title Pluggy.ai bank sync [WIP] Pluggy.ai bank sync Dec 27, 2024
@lelemm
Copy link
Contributor Author

lelemm commented Dec 27, 2024

@coderabbitai review

Copy link
Contributor

coderabbitai bot commented Dec 27, 2024

Walkthrough

The pull request introduces a new module for integrating the PluggyAI service into the application. The changes include creating a new Express.js application (app-pluggyai.js) that provides API endpoints for checking service status, retrieving accounts, and fetching transactions. A corresponding service module (pluggyai-service.js) was developed to handle interactions with the Pluggy.ai API, including authentication and data retrieval. The main application (app.js) was updated to include the new PluggyAI route, and the secrets service was extended to support new secret names related to the PluggyAI service configuration. These modifications enable the application to interact with the Pluggy.ai API, retrieve financial account and transaction information, and manage the necessary authentication credentials.

Suggested labels

sparkles:Merged, feature:integrations, service:pluggyai

✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

👋 Hi! It looks like this PR has not had any changes for a week now. Would you like someone to review this PR? If so - please remove the "[WIP]" prefix from the PR title. That will let the community know that this PR is open for a review.

@github-actions github-actions bot added the stale label Jan 22, 2025
@lelemm lelemm changed the title [WIP] Pluggy.ai bank sync Pluggy.ai bank sync Jan 23, 2025
@matt-fidd
Copy link
Contributor

@coderabbitai full review

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (8)
src/app-pluggyai/app-pluggyai.js (4)

7-10: Consider more descriptive naming or a dedicated router for pluggyAI.
Using a simple app with export { app as handlers } works, but a separate router or more descriptive naming can improve clarity and maintainability when scaling.


12-25: Review the use of POST method for '/status' endpoint.
Endpoints that only retrieve status are often exposed as GET for semantic clarity. Consider switching to GET unless you have a specific reason for POST.


66-88: Use safer pagination approach to avoid potential large response accumulation.
This while loop fetches all transactions, which might be extremely large for certain users. Consider a more controlled or streaming approach, or limiting the maximum pages fetched to prevent memory exhaustion.


221-223: Expand or rename getDate for time-sensitive use cases.
Currently, getDate strips time to YYYY-MM-DD. If time zones or partial dates become relevant, consider returning a full ISO string or storing time detail in a separate field.

src/app-pluggyai/pluggyai-service.js (4)

7-19: Consider concurrency and caching strategy for pluggyApiKey.
pluggyApiKey is stored as a global variable; concurrent requests could result in race conditions if the token expires mid-request. Consider storing the token on a per-request basis or leveraging a caching mechanism with proper locking.


25-79: Handle JSON parsing and success checks robustly when generating API key.
While you handle 403 and 401 statuses, other status codes or network anomalies could still cause unexpected results. Consider uniform status code handling and error messaging to simplify debugging.


97-105: Move token persistence behind a stable interface.
Directly calling secretsService.set from within the token logic can create coupling between this layer and the secrets layer. Consider an abstraction that can handle all token-related operations consistently.


185-228: Examine large date range queries in getTransactionsByAccountId.
Constructing a giant query combining multiple dates and pages may lead to potential timeouts or performance bottlenecks. Offer a robust approach, e.g., date-chunking or cursors, if users often retrieve large date ranges.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5ada00c and bf0a0bc.

⛔ Files ignored due to path filters (1)
  • upcoming-release-notes/530.md is excluded by !**/*.md
📒 Files selected for processing (4)
  • src/app-pluggyai/app-pluggyai.js (1 hunks)
  • src/app-pluggyai/pluggyai-service.js (1 hunks)
  • src/app.js (2 hunks)
  • src/services/secrets-service.js (1 hunks)
🔇 Additional comments (5)
src/app-pluggyai/app-pluggyai.js (2)

185-189: Double-check sign logic for amounts with credit accounts.
For 'BANK' vs. 'CREDIT' accounts, the code reuses the 'BANK' condition for the sign. Validate that you’ve accurately handled all relevant account types if there are more possibilities (e.g., 'INVESTMENT').


27-59: Validate existence of item IDs before processing.
The code splits and maps the item IDs, but there is no check to ensure the secret is defined. A missing or empty pluggyai_itemIds secret could cause runtime errors.

+ if (!itemIds || itemIds.length === 0) {
+   return res.status(400).send({
+     status: 'error',
+     message: 'No itemIds found in configuration.'
+   });
+ }
✅ Verification successful

Validate existence and validity of item IDs before processing
The validation is necessary as other routes in the codebase explicitly check for null/forbidden secrets. Update the code to match existing error handling patterns:

+ const itemIdsSecret = secretsService.get(SecretName.pluggyai_itemIds);
+ if (!itemIdsSecret || itemIdsSecret === 'Forbidden') {
+   return res.status(400).send({
+     status: 'error',
+     reason: 'missing-item-ids-configuration'
+   });
+ }

- const itemIds = secretsService
+ const itemIds = itemIdsSecret
    .split(',')
    .map((item) => item.trim());

+ if (itemIds.length === 0) {
+   return res.status(400).send({
+     status: 'error',
+     reason: 'empty-item-ids-configuration'
+   });
+ }
🏁 Scripts executed

The following scripts were executed for the analysis:

Error: No shell scripts found


Script:

#!/bin/bash
# Find secretsService implementation
ast-grep --pattern 'class $_ {
  $$$
  get($name) {
    $$$
  }
  $$$
}'

# Look for similar patterns of secret usage
rg "secretsService\.get\(" -A 2 -B 2

# Check error handling patterns in routes
rg "res\.status\(.*\)\.send\(" -A 2

Length of output: 11963

src/services/secrets-service.js (1)

14-17: Ensure secret usage follows security best practices.
Adding these secrets expands the surface area of sensitive data. Make sure you are encrypting secrets at rest, limiting read/write access, and rotating credentials as needed.

src/app.js (2)

16-16: LGTM! Import follows established patterns.

The import statement follows the consistent pattern used for other bank sync modules, maintaining code organization standards.


53-53: LGTM! Route registration is secure and consistent.

The /pluggyai route registration:

  • Follows the established pattern used by other bank sync routes
  • Inherits security measures:
    • Rate limiting (500 requests/minute)
    • CORS protection
    • Request size limits
    • Proxy trust configuration

Let's verify the security implementation in the Pluggy.ai handlers:

Comment on lines +136 to +139
if (transactionDate < startDate) {
continue;
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Ensure request body date fields are reliably parsed before comparison.
Currently, you compare transactionDate < startDate directly, but startDate might be a string from req.body. Confirm it’s converted to a Date object to avoid unexpected behaviors.

- if (transactionDate < startDate) {
+ const parsedStart = new Date(startDate);
+ if (transactionDate < parsedStart) {
    continue;
  }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (transactionDate < startDate) {
continue;
}
const parsedStart = new Date(startDate);
if (transactionDate < parsedStart) {
continue;
}

@matt-fidd
Copy link
Contributor

Did you see that there is an official node sdk for pluggy? https://www.npmjs.com/package/pluggy-sdk

@lelemm
Copy link
Contributor Author

lelemm commented Jan 28, 2025

Did you see that there is an official node sdk for pluggy? https://www.npmjs.com/package/pluggy-sdk

I didn't. I found the API so simple that I implemented myself. Do you prefer that I use the official sdk?

@matt-fidd
Copy link
Contributor

Did you see that there is an official node sdk for pluggy? https://www.npmjs.com/package/pluggy-sdk

I didn't. I found the API so simple that I implemented myself. Do you prefer that I use the official sdk?

Another opinion would be good but IMO using the official SDK gives us a level of security with sanitisation etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants