docs(views): add documentation of freeform views #21
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Trivy scanning results. |
|
Code Coverage SummaryDiff against mainResults for commit: 2660222 Minimum allowed coverage is ♻️ This comment has been updated with latest results |
| # Concept: Freeform Views | ||
|
|
||
| Freeform views are a type of [view](views.md) that provides a way for developers using db-ally to define what they need from the LLM without requiring a fixed response structure. This flexibility is beneficial when the data structure is unknown beforehand or when potential queries are too diverse to be covered by a structured view. Though freeform views offer more flexibility than structured views, they are less predictable, efficient, and secure, and may be more challenging to integrate with other systems. For these reasons, we recommend using [structured views](./structured_views.md) when possible. | ||
|
|
There was a problem hiding this comment.
Maybe add a sentence or two about strategy of:
- Starting with FreeFormView
- Collecting statistics about most common questions and failure cases
- Incorporating StructuredViews to overcome them
|
|
||
| Freeform views are a type of [view](views.md) that provides a way for developers using db-ally to define what they need from the LLM without requiring a fixed response structure. This flexibility is beneficial when the data structure is unknown beforehand or when potential queries are too diverse to be covered by a structured view. Though freeform views offer more flexibility than structured views, they are less predictable, efficient, and secure, and may be more challenging to integrate with other systems. For these reasons, we recommend using [structured views](./structured_views.md) when possible. | ||
|
|
||
| Unlike structured views, which define a response format and a set of operations the LLM may use in response to natural language queries, freeform views only have one task - to respond directly to natural language queries with data from the datasource. They accomplish this by implementing the [`ask`][dbally.views.base.BaseView] method. This method takes a natural language query as input and returns a response. The method also has access to the LLM model (via the `llm_client` attribute), which is typically used to retrieve the correct data from the source (for example, by generating a source-specific query string). To learn more about implementing freeform views, refer to the [How to: Custom Freeform Views](../how-to/custom_freeform_views.md) guide. |
There was a problem hiding this comment.
freeform views only have one task -> Isn't the task of StructuredView the same?
I mean we can also run freeform in the dry mode and obtain directly the sql or api call.
The main difference is in the IQL generation by the StructuredView so I'd emphasise it
|
|
||
| For instance, an LLM might generate an IQL query like this when asked "Find me French candidates suitable for a senior data scientist position": | ||
|
|
||
| ``` | ||
| from_country('France') AND senior_data_scientist_position() | ||
| ``` | ||
|
|
||
| The capabilities made available to the AI model via IQL differ between projects. Developers control these by defining [Views](views.md). db-ally automatically exposes special methods defined in views, known as "filters", via IQL. For instance, the expression above suggests that the specific project contains a view that includes the `from_country` and `senior_data_scientist_position` methods (and possibly others that the LLM did not choose to use for this particular question). Additionally, the LLM can use Boolean operators (`and`,`or`, `not`) to combine individual filters into more complex expressions. | ||
| The capabilities made available to the AI model via IQL differ between projects. Developers control these by defining special [Views](structured_views.md). db-ally automatically exposes special methods defined in structured views, known as "filters", via IQL. For instance, the expression above suggests that the specific project contains a view that includes the `from_country` and `senior_data_scientist_position` methods (and possibly others that the LLM did not choose to use for this particular question). Additionally, the LLM can use Boolean operators (`and`,`or`, `not`) to combine individual filters into more complex expressions. |
There was a problem hiding this comment.
Word special somehow implied "hard" when I was reading it, maybe just use the structured to familiarize users with this term
There was a problem hiding this comment.
I know it will create a repetition, but sometimes they are not that bad
| @@ -0,0 +1,39 @@ | |||
| # Concept: Structured Views | |||
|
|
|||
| Structured views are a type of [view](../concepts/views.md), which provide a way for developers using db-ally to define what they need from the LLM, including: | |||
There was a problem hiding this comment.
maybe change "are a type of a view" into something more like -> Implements [View] interface, providing more controlled approach of using db-ally, including ?
| Structured views are a type of [view](../concepts/views.md), which provide a way for developers using db-ally to define what they need from the LLM, including: | ||
|
|
||
| * The desired data structure, such as the specific fields to include from the data source. | ||
| * A set of operations the LLM may employ in response to natural language queries (currently only “filters” are supported, with more to come) |
There was a problem hiding this comment.
I'd love to see alink to filters but we do not have any page, so maybe we should create one? Maybe a header in the IQL concept page?
|
|
||
| Given different natural language queries, a db-ally view will produce different responses while maintaining a consistent data structure. This consistency offers a reliable interface for integration - the code consuming responses from a particular structured view knows what data structure to expect and can utilize this knowledge when displaying or processing the data. This feature of db-ally makes it stand out in terms of reliability and stability compared to standard text-to-SQL approaches. | ||
|
|
||
| Each structured view can contain one or more “filters”, which the LLM may decide to choose and apply to the extracted data so that it meets the criteria specified in the natural language query. Given such a query, LLM chooses which filters to use, provides arguments to the filters, and connects the filters with Boolean operators. The LLM expresses these filter combinations using a special language called [IQL](iql.md), in which the defined view filters provide a layer of abstraction between the LLM and the raw syntax used to query the data source (e.g., SQL). |
There was a problem hiding this comment.
Here filters are clearly explained but I think there should be separate header in other section explaining just filters
| return Candidate.country == country | ||
| ``` | ||
|
|
||
| In addition to structured views, db-ally also provides [freeform views](freeform_views.md), which are more flexible and can be used to create views that do not require a fixed data structure. Freeform views come in handy when the data structure is not predefined or when the scope of potential queries is too vast to be addressed by a structured view. Conversely, structured views are more predictable, efficient, secure, and easier to integrate with other systems. Therefore, we recommend using structured views where possible. To read about the advantages and disadvantages of both kinds of views, refer to [Concept: Views](views.md). |
There was a problem hiding this comment.
Maybe put this Freeform vs Structured View into separate section or page so that one can link to it?
| @@ -1,37 +1,19 @@ | |||
| # Concept: Views | |||
There was a problem hiding this comment.
Maybe structured vs unstructured views use-cases and scenarios section shoould go here
ds-sebastianchwilczynski
approved these changes
Apr 25, 2024
mhordynski
approved these changes
Apr 25, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.