feat(utils): implement nested field extraction #91
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Simplify field extraction in `typesenseDocumentFromSnapshot` by utilizing lodash.get for safe property access. - Remove redundant filtering of entries and streamline the mapping process for improved clarity and efficiency.
- Expand test coverage for `typesenseDocumentFromSnapshot` function to include scenarios with nested fields, dot notation, missing fields, and array indexing. - Ensure proper extraction of complex data structures from Firestore documents.
jasonbosco
requested changes
Oct 4, 2024
- Replace external 'flat' library with a custom flattenDocument function for improved control over nested object and array handling in Typesense document preparation.
- Implement custom nested value handling functions to replace lodash.get. - Add support for array fields and improve object flattening.
- Refactor test case for nested arrays inside fields, to more properly match real world cases
- Rename spec files that include test cases with flattening to reflect that
jasonbosco
approved these changes
Oct 14, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is this?
This pull request enhances our Firestore to Typesense synchronization by adding support for nested field extraction. It allows users to specify nested fields using dot notation, making it easier to sync complex data structures from Firestore to Typesense.
Rationale
This change resolves issue #90, which requests dot notation support for the "Firestore Collection Fields" parameter. By implementing this feature, we enable users to access any level property on fields they want to explicitly include in the trigger.
Changes
Added Features:
functions/package.json:lodash.get(version 4.4.2) for safe nested property access.Code Changes:
functions/src/utils.js:lodash.getfor nested field extraction.typesenseDocumentFromSnapshotfunction to uselodash.getfor accessing nested fields.Test Updates:
test/utils.spec.js:Context
This change improves the flexibility of our Firestore to Typesense synchronization extension. Users can now easily map nested Firestore fields to Typesense, allowing for more complex data structures to be indexed and searched efficiently. This feature is particularly useful for projects with deeply nested document structures, enabling fine-grained control over which specific nested fields are synchronized to Typesense.
PR Checklist