Improve perfs for spatial intersection for sql views (refs : #3600)

[bruhnild]

refs : #3601 (contributed by @amandine-sahl)

Description

Related Issue

Checklist

• I have followed the guidelines in our Contributing document
• My code respects the Definition of done available in the Development section of the documentation
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes
• I added an entry in the changelog file
• My commits are all using prefix convention (emoji + tag name) and references associated issues
• I added a label to the PR corresponding to the perimeter of my contribution
• The title of my PR mentionned the issue associated

[cypress]

Passing run #7850 ↗︎

0

24

0

0

0

Details:

Merge 663e4ed into 326b23e...
Project: Geotrek-admin	Commit: 05f375ebfc ℹ️
Status: Passed	Duration: 02:13 💡
Started: Jan 24, 2024 5:23 PM	Ended: Jan 24, 2024 5:25 PM

Review all test suite changes for PR #3888 ↗︎

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (326b23e) 98.35% compared to head (663e4ed) 98.35%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3888   +/-   ##
=======================================
  Coverage   98.35%   98.35%           
=======================================
  Files         293      293           
  Lines       22174    22174           
=======================================
  Hits        21810    21810           
  Misses        364      364           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[babastienne]

Otherwise except from changelog that need to be updated, lgtm

[submarcos]

@CodiumAI-Agent /review

[CodiumAI-Agent]

PR Analysis

(review updated until commit 16fcf59)

• 🎯 Main theme: Performance improvement for spatial intersection in SQL views
• 📝 PR summary: This PR aims to improve the performance of spatial intersection for SQL views. It replaces the previous JOIN operations with LATERAL JOINs to optimize the spatial intersection operations in several SQL views. The changes are applied across multiple files, including those related to trekking, tourism, signage, infrastructure, maintenance, core, outdoor, feedback, and sensitivity.
• 📌 Type of PR: Enhancement
• 🧪 Relevant tests added: No
• ⏱️ Estimated effort to review [1-5]: 4, because the PR involves changes in SQL queries which require a good understanding of SQL performance optimization and spatial intersection operations.
• 🔒 Security concerns: No security concerns found

PR Feedback

💡 General suggestions: The PR seems to be well-structured and the changes are consistent across all the files. However, it would be beneficial to include some performance metrics to quantify the improvement. This could be in the form of execution times before and after the changes, or some other relevant measure. Additionally, it would be useful to add tests to ensure that the changes do not affect the functionality of the code.

🤖 Code feedback:

---

relevant file	geotrek/trekking/templates/trekking/sql/post_30_views.sql
suggestion	Consider adding indexes on the geom columns of zoning_city and zoning_district tables if not already present. This could potentially improve the performance of the ST_Intersects function. [important]
relevant line	'+ WHERE st_intersects(a.geom, b_1.geom)'

---

relevant file	geotrek/tourism/templates/tourism/sql/post_20_views.sql
suggestion	It's recommended to avoid using functions like array_to_string and array_agg in the SELECT clause of a query as they can slow down the query performance. If possible, try to handle this data manipulation in the application layer. [medium]
relevant line	'+ SELECT array_to_string(array_agg(b_1.name ORDER BY b_1.name), ', '::text, '_'::text) AS zoning_city'

---

relevant file	geotrek/infrastructure/templates/infrastructure/sql/post_10_views.sql
suggestion	It's good practice to avoid using TRUE in the ON clause of a JOIN operation. Instead, specify the actual condition that should be met for the join. [medium]
relevant line	'+ ) f ON TRUE'

---

relevant file	geotrek/maintenance/templates/maintenance/sql/post_20_views.sql
suggestion	It's recommended to avoid using functions like ST_Intersects in the WHERE clause of a query as they can slow down the query performance. If possible, try to handle this spatial intersection in a different way. [medium]
relevant line	'+ WHERE st_intersects(a.geom_3d, b_1.geom)'

---

---

✨ Usage guide:

---

Overview:
The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

Utilizing extra instructions The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize. Examples for extra instructions: [pr_reviewer] # /review # extra_instructions=""" In the code feedback section, emphasize the following: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """ Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

How to enable\disable automation When you first install PR-Agent app, the default mode for the review tool is: pr_commands = ["/review", ...] meaning the review tool will run automatically on every PR, with the default configuration. Edit this field to enable/disable the tool, or to change the used configurations

About the 'Code feedback' section The review tool provides several type of feedbacks, one of them is code suggestions. If you are interested only in the code suggestions, it is recommended to use the improve feature instead, since it dedicated only to code suggestions, and usually gives better results. Use the review tool if you want to get a more comprehensive feedback, which includes code suggestions as well.

Auto-labels The review tool can auto-generate two specific types of labels for a PR: a possible security issue label, that detects possible security issues (enable_review_labels_security flag) a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)

Extra sub-tools The review tool provides a collection of possible feedbacks about a PR. It is recommended to review the possible options, and choose the ones relevant for your use case. Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example: require_score_review, require_soc2_review, enable_review_labels_effort, and more.

More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.

[CodiumAI-Agent]

Persistent review updated to latest commit 16fcf59

[submarcos]

@CodiumAI-Agent /improve

[bruhnild]

We should keep st_intersects functions as it uses any available spatial index : https://www.cockroachlabs.com/docs/stable/st_intersects. See also : https://mentin.medium.com/which-predicate-cb608b470471