diff --git a/docs/ai/fastapi_gelai_searchbot.rst b/docs/ai/fastapi_gelai_searchbot.rst new file mode 100644 index 00000000000..673648a04a7 --- /dev/null +++ b/docs/ai/fastapi_gelai_searchbot.rst @@ -0,0 +1,1716 @@ +.. _ref_guide_fastapi_gelai_searchbot: + +=================== +FastAPI (Searchbot) +=================== + +:edb-alt-title: Building a search bot with memory using FastAPI and Gel AI + +In this tutorial we're going to walk you through building a chat bot with search +capabilities using Gel and `FastAPI `_. + +FastAPI is a framework designed to help you build web apps *fast*. Gel is a +data layer designed to help you figure out storage in your application - also +*fast*. By the end of this tutorial, you will have tried out different aspects +of using those two together. + +We will start by creating an app with FastAPI, adding web search capabilities, +and then putting search results through a language model to get a +human-friendly answer. After that, we'll use Gel to implement chat history so +that the bot remembers previous interactions with the user. We'll finish it off +with semantic search-based cross-chat memory. + +The end result is going to look something like this: + +.. image:: + /docs/tutorials/placeholder.png + :alt: Placeholder + :width: 100% + +1. Initialize the project +========================= + +.. edb:split-section:: + + We're going to start by installing `uv `_ - a Python + package manager that's going to simplify environment management for us. You can + follow their `installation instructions + `_ or simply run: + + .. code-block:: bash + + $ curl -LsSf https://astral.sh/uv/install.sh | sh + +.. edb:split-section:: + + Once that is done, we can use uv to create scaffolding for our project following + the `documentation `_: + + .. code-block:: bash + + $ uv init searchbot \ + && cd searchbot + +.. edb:split-section:: + + For now, we know we're going to need Gel and FastAPI, so let's add those + following uv's instructions on `managing dependencies + `_, + as well as FastAPI's `installation docs + `_. Running ``uv sync`` after + that will create our virtual environment in a ``.venv`` directory and ensure + it's ready. As the last step, we'll activate the environment and get started. + + .. note:: + + Every time you open a new terminal session, you should source the + environment before running ``python``, ``gel`` or ``fastapi`` commands. + + .. code-block:: bash + + $ uv add "fastapi[standard]" \ + && uv add gel \ + && uv sync \ + && source .venv/bin/activate + + +2. Get started with FastAPI +=========================== + +.. edb:split-section:: + + At this stage we need to follow FastAPI's `tutorial + `_ to create the foundation of our app. + + We're going to make a minimal web API with one endpoint that takes in a user + query as an input and echoes it as an output. First, let's make a directory + called ``app`` in our project root, and put an empty ``__init__.py`` there. + + .. code-block:: bash + + $ mkdir app && touch app/__init__.py + +.. edb:split-section:: + + Now let's create a file called ``main.py`` inside the ``app`` directory and put + the "Hello World" example in it: + + .. code-block:: python + :caption: app/main.py + + from fastapi import FastAPI + + app = FastAPI() + + + @app.get("/") + async def root(): + return {"message": "Hello World"} + + +.. edb:split-section:: + + To start the server, we'll run: + + .. code-block:: bash + + $ fastapi dev app/main.py + + +.. edb:split-section:: + + Once the server gets up and running, we can make sure it works using FastAPI's + built-in UI at _, or manually with ``curl``: + + .. code-block:: bash + + $ curl -X 'GET' \ + 'http://127.0.0.1:8000/' \ + -H 'accept: application/json' + + {"message":"Hello World"} + + +.. edb:split-section:: + + Now, to create the search endpoint we mentioned earlier, we need to pass our + query as a parameter to it. We'd prefer to have it in the request's body + since user messages can be long. + + In FastAPI land, this is done by creating a Pydantic schema and making it the + type of the input parameter. `Pydantic `_ is + a data validation library for Python. It has many features, but we don't + actually need to know about them for now. All we need to know is that FastAPI + uses Pydantic types to automatically figure out schemas for `input + `_, as well as `output + `_. + + Let's add the following to our ``main.py``: + + .. code-block:: python + :caption: app/main.py + + from pydantic import BaseModel + + + class SearchTerms(BaseModel): + query: str + + class SearchResult(BaseModel): + response: str | None = None + + +.. edb:split-section:: + + Now, we can define our endpoint. We'll set the two classes we just created as + the new endpoint's argument and return type. + + .. code-block:: python + :caption: app/main.py + + @app.post("/search") + async def search(search_terms: SearchTerms) -> SearchResult: + return SearchResult(response=search_terms.query) + + +.. edb:split-section:: + + Same as before, we can test the endpoint using the UI, or by sending a request + with ``curl``: + + .. code-block:: bash + + $ curl -X 'POST' \ + 'http://127.0.0.1:8000/search' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ "query": "string" }' + + { + "response": "string", + } + +3. Implement web search +======================= + +Now that we have our web app infrastructure in place, let's add some substance +to it by implementing web search capabilities. + +.. edb:split-section:: + + There're many powerful feature-rich products for LLM-driven web search. But + in this tutorial we're going to use a much more reliable source of real-world + information that is comment threads on `Hacker News + `_. Their `web API + `_ is free of charge and doesn't require an + account. Below is a simple function that requests a full-text search for a + string query and extracts a nice sampling of comment threads from each of the + stories that came up in the result. + + We are not going to cover this code sample in too much depth. Feel free to grab + it save it to ``app/web.py``, or make your own. + + Notice that we've created another Pydantic type called ``WebSource`` to store + our web search results. There's no framework-related reason for that, it's just + nicer than passing dictionaries around. + + .. code-block:: python + :caption: app/web.py + :class: collapsible + + import requests + from pydantic import BaseModel + from datetime import datetime + import html + + + class WebSource(BaseModel): + """Type that stores search results.""" + + url: str | None = None + title: str | None = None + text: str | None = None + + + def extract_comment_thread( + comment: dict, + max_depth: int = 3, + current_depth: int = 0, + max_children=3, + ) -> list[str]: + """ + Recursively extract comments from a thread up to max_depth. + Returns a list of formatted comment strings. + """ + if not comment or current_depth > max_depth: + return [] + + results = [] + + # Get timestamp, author and the body of the comment, + # then pad it with spaces so that it's offset appropriately for its depth + + if comment["text"]: + timestamp = datetime.fromisoformat(comment["created_at"].replace("Z", "+00:00")) + author = comment["author"] + text = html.unescape(comment["text"]) + formatted_comment = f"[{timestamp.strftime('%Y-%m-%d %H:%M')}] {author}: {text}" + results.append((" " * current_depth) + formatted_comment) + + # If there're children comments, we are going to extract them too, + # and add them to the list. + + if comment.get("children"): + for child in comment["children"][:max_children]: + child_comments = extract_comment_thread(child, max_depth, current_depth + 1) + results.extend(child_comments) + + return results + + + def fetch_web_sources(query: str, limit: int = 5) -> list[WebSource]: + """ + For a given query perform a full-text search for stories on Hacker News. + From each of the matched stories extract the comment thread and format it into a single string. + For each story return its title, url and comment thread. + """ + search_url = "http://hn.algolia.com/api/v1/search_by_date?numericFilters=num_comments>0" + + # Search for stories + response = requests.get( + search_url, + params={ + "query": query, + "tags": "story", + "hitsPerPage": limit, + "page": 0, + }, + ) + + response.raise_for_status() + search_result = response.json() + + # For each search hit fetch and process the story + web_sources = [] + for hit in search_result.get("hits", []): + item_url = f"https://hn.algolia.com/api/v1/items/{hit['story_id']}" + response = requests.get(item_url) + response.raise_for_status() + item_result = response.json() + + site_url = f"https://news.ycombinator.com/item?id={hit['story_id']}" + title = hit["title"] + comments = extract_comment_thread(item_result) + text = "\n".join(comments) if len(comments) > 0 else None + web_sources.append( + WebSource(url=site_url, title=title, text=text) + ) + + return web_sources + + + if __name__ == "__main__": + web_sources = fetch_web_sources("edgedb", limit=5) + + for source in web_sources: + print(source.url) + print(source.title) + print(source.text) + + +.. edb:split-section:: + + One more note: this snippet comes with an extra dependency called ``requests``, + which is a library for making HTTP requests. Let's add it by running: + + .. code-block:: bash + + $ uv add requests + + +.. edb:split-section:: + + Now, we can test our web search on its own by running it like this: + + .. code-block:: bash + + $ python3 app/web.py + + +.. edb:split-section:: + + It's time to reflect the new capabilities in our web app. + + .. code-block:: python + :caption: app/main.py + + from .web import fetch_web_sources, WebSource + + async def search_web(query: str) -> list[WebSource]: + raw_sources = fetch_web_sources(query, limit=5) + return [s for s in raw_sources if s.text is not None] + + +.. edb:split-section:: + + Now we can update the ``/search`` endpoint as follows: + + .. code-block:: python-diff + :caption: app/main.py + + class SearchResult(BaseModel): + response: str | None = None + + sources: list[WebSource] | None = None + + + @app.post("/search") + async def search(search_terms: SearchTerms) -> SearchResult: + + web_sources = await search_web(search_terms.query) + - return SearchResult(response=search_terms.query) + + return SearchResult( + + response=search_terms.query, sources=web_sources + + ) + + +4. Connect to the LLM +===================== + +Now that we're capable of scraping text from search results, we can forward +those results to the LLM to get a nice-looking summary. + +.. edb:split-section:: + + There's a million different LLMs accessible via a web API (`one + `_, `two + `_, `three + `_, `four `_ to name + a few), feel free to choose whichever you prefer. In this tutorial we will + roll with OpenAI, primarily for how ubiquitous it is. To keep things somewhat + provider-agnostic, we're going to get completions via raw HTTP requests. + Let's grab API descriptions from OpenAI's `API documentation + `_, and set up + LLM generation like this: + + .. code-block:: python + :caption: app/main.py + + import requests + from dotenv import load_dotenv + + _ = load_dotenv() + + + def get_llm_completion(system_prompt: str, messages: list[dict[str, str]]) -> str: + api_key = os.getenv("OPENAI_API_KEY") + url = "https://api.openai.com/v1/chat/completions" + headers = {"Content-Type": "application/json", "Authorization": f"Bearer {api_key}"} + + response = requests.post( + url, + headers=headers, + json={ + "model": "gpt-4o-mini", + "messages": [ + {"role": "developer", "content": system_prompt}, + *messages, + ], + }, + ) + response.raise_for_status() + result = response.json() + return result["choices"][0]["message"]["content"] + + +.. edb:split-section:: + + Note that this cloud LLM API (and many others) requires a secret key to be + set as an environment variable. A common way to manage those is to use the + ``python-dotenv`` library in combinations with a ``.env`` file. Feel free to + browse `the readme + `_, + to learn more. Create a file called ``.env`` in the root directory and put + your api key in there: + + .. code-block:: .env + :caption: .env + + OPENAI_API_KEY="sk-..." + + +.. edb:split-section:: + + Don't forget to add the new dependency to the environment: + + .. code-block:: bash + + uv add python-dotenv + + +.. edb:split-section:: + + And now we can integrate this LLM-related code with the rest of the app. First, + let's set up a function that prepares LLM inputs: + + + .. code-block:: python + :caption: app/main.py + + async def generate_answer( + query: str, + web_sources: list[WebSource], + ) -> SearchResult: + system_prompt = ( + "You are a helpful assistant that answers user's questions" + + " by finding relevant information in Hacker News threads." + + " When answering the question, describe conversations that people have around the subject," + + " provided to you as a context, or say i don't know if they are completely irrelevant." + ) + + prompt = f"User search query: {query}\n\nWeb search results:\n" + + for i, source in enumerate(web_sources): + prompt += f"Result {i} (URL: {source.url}):\n" + prompt += f"{source.text}\n\n" + + messages = [{"role": "user", "content": prompt}] + + llm_response = get_llm_completion( + system_prompt=system_prompt, + messages=messages, + ) + + search_result = SearchResult( + response=llm_response, + sources=web_sources, + ) + + return search_result + + +.. edb:split-section:: + + Then we can plug that function into the ``/search`` endpoint: + + .. code-block:: python-diff + :caption: app/main.py + + @app.post("/search") + async def search(search_terms: SearchTerms) -> SearchResult: + web_sources = await search_web(search_terms.query) + + search_result = await generate_answer(search_terms.query, web_sources) + + return search_result + - return SearchResult( + - response=search_terms.query, sources=web_sources + - ) + + +.. edb:split-section:: + + And now we can test the result as usual. + + .. code-block:: bash + + $ curl -X 'POST' \ + 'http://127.0.0.1:8000/search' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ "query": "gel" }' + + +5. Use Gel to implement chat history +==================================== + +So far we've built an application that can take in a query, fetch some Hacker +News threads for it, sift through them using an LLM, and generate a nice +summary. + +However, right now it's hardly user-friendly since you have to speak in +keywords and basically start over every time you want to refine the query. To +enable a more organic multi-turn interaction, we need to add chat history and +infer the query from the context of the entire conversation. + +Now's a good time to introduce Gel. + +.. edb:split-section:: + + In case you need installation instructions, take a look at the :ref:`Quickstart + `. Once Gel CLI is present in your system, initialize the + project like this: + + .. code-block:: bash + + $ gel project init --non-interactive + + +This command is going to put some project scaffolding inside our app, spin up a +local instace of Gel, and then link the two together. From now on, all +Gel-related things that happen inside our project directory are going to be +automatically run on the correct database instance, no need to worry about +connection incantations. + + +Defining the schema +------------------- + +The database :ref:`schema ` in Gel is defined +declaratively. The :ref:`gel project init ` +command has created a file called ``dbchema/default.esdl``, which we're going to +use to define our types. + +.. edb:split-section:: + + We obviously want to keep track of the messages, so we need to represent + those in the schema. By convention established in the LLM space, each message + is going to have a role in addition to the message content itself. We can + also get Gel to automatically keep track of message's creation time by adding + a property callled ``timestamp`` and setting its :ref:`default value + ` to the output of the :ref:`datetime_current() + ` function. Finally, LLM messages in our search bot have + source URLs associated with them. Let's keep track of those too, by adding a + :ref:`multi-property `. + + .. code-block:: sdl + :caption: dbschema/default.esdl + + type Message { + role: str; + body: str; + timestamp: datetime { + default := datetime_current(); + } + multi sources: str; + } + + +.. edb:split-section:: + + Messages are grouped together into a chat, so let's add that entity to our + schema too. + + .. code-block:: sdl + :caption: dbschema/default.esdl + + type Chat { + multi messages: Message; + } + + +.. edb:split-section:: + + And chats all belong to a certain user, making up their chat history. One other + thing we'd like to keep track of about our users is their username, and it would + make sense for us to make sure that it's unique by using an ``excusive`` + :ref:`constraint `. + + .. code-block:: sdl + :caption: dbschema/default.esdl + + type User { + name: str { + constraint exclusive; + } + multi chats: Chat; + } + + +.. edb:split-section:: + + We're going to keep our schema super simple. One cool thing about Gel is that + it will enable us to easily implement advanced features such as authentication + or AI down the road, but we're gonna come back to that later. + + For now, this is the entire schema we came up with: + + .. code-block:: sdl + :caption: dbschema/default.esdl + + module default { + type Message { + role: str; + body: str; + timestamp: datetime { + default := datetime_current(); + } + multi sources: str; + } + + type Chat { + multi messages: Message; + } + + type User { + name: str { + constraint exclusive; + } + multi chats: Chat; + } + } + + +.. edb:split-section:: + + Let's use the :ref:`gel migration create ` CLI + command, followed by :ref:`gel migrate ` in order to + migrate to our new schema and proceed to writing some queries. + + .. code-block:: bash + + $ gel migration create + $ gel migrate + + +.. edb:split-section:: + + Now that our schema is applied, let's quickly populate the database with some + fake data in order to be able to test the queries. We're going to explore + writing queries in a bit, but for now you can just run the following command in + the shell: + + .. code-block:: bash + :class: collapsible + + $ mkdir app/sample_data && cat << 'EOF' > app/sample_data/inserts.edgeql + # Create users first + insert User { + name := 'alice', + }; + insert User { + name := 'bob', + }; + # Insert chat histories for Alice + update User + filter .name = 'alice' + set { + chats := { + (insert Chat { + messages := { + (insert Message { + role := 'user', + body := 'What are the main differences between GPT-3 and GPT-4?', + timestamp := '2024-01-07T10:00:00Z', + sources := {'arxiv:2303.08774', 'openai.com/research/gpt-4'} + }), + (insert Message { + role := 'assistant', + body := 'The key differences include improved reasoning capabilities, better context understanding, and enhanced safety features...', + timestamp := '2024-01-07T10:00:05Z', + sources := {'openai.com/blog/gpt-4-details', 'arxiv:2303.08774'} + }) + } + }), + (insert Chat { + messages := { + (insert Message { + role := 'user', + body := 'Can you explain what policy gradient methods are in RL?', + timestamp := '2024-01-08T14:30:00Z', + sources := {'Sutton-Barto-RL-Book-Ch13', 'arxiv:1904.12901'} + }), + (insert Message { + role := 'assistant', + body := 'Policy gradient methods are a class of reinforcement learning algorithms that directly optimize the policy...', + timestamp := '2024-01-08T14:30:10Z', + sources := {'Sutton-Barto-RL-Book-Ch13', 'spinning-up.openai.com'} + }) + } + }) + } + }; + # Insert chat histories for Bob + update User + filter .name = 'bob' + set { + chats := { + (insert Chat { + messages := { + (insert Message { + role := 'user', + body := 'What are the pros and cons of different sharding strategies?', + timestamp := '2024-01-05T16:15:00Z', + sources := {'martin-kleppmann-ddia-ch6', 'aws.amazon.com/sharding-patterns'} + }), + (insert Message { + role := 'assistant', + body := 'The main sharding strategies include range-based, hash-based, and directory-based sharding...', + timestamp := '2024-01-05T16:15:08Z', + sources := {'martin-kleppmann-ddia-ch6', 'mongodb.com/docs/sharding'} + }), + (insert Message { + role := 'user', + body := 'Could you elaborate on hash-based sharding?', + timestamp := '2024-01-05T16:16:00Z', + sources := {'mongodb.com/docs/sharding'} + }) + } + }) + } + }; + EOF + + +.. edb:split-section:: + + This created the ``app/sample_data/inserts.edgeql`` file, which we can now execute + using the CLI like this: + + .. code-block:: bash + + $ gel query -f app/sample_data/inserts.edgeql + + {"id": "862de904-de39-11ef-9713-4fab09220c4a"} + {"id": "862e400c-de39-11ef-9713-2f81f2b67013"} + {"id": "862de904-de39-11ef-9713-4fab09220c4a"} + {"id": "862e400c-de39-11ef-9713-2f81f2b67013"} + + +.. edb:split-section:: + + The :ref:`gel query ` command is one of many ways we can + execute a query in Gel. Now that we've done it, there's stuff in the database. + Let's verify it by running: + + .. code-block:: bash + + $ gel query "select User { name };" + + {"name": "alice"} + {"name": "bob"} + + +Writing queries +--------------- + +With schema in place, it's time to focus on getting the data in and out of the +database. + +In this tutorial we're going to write queries using :ref:`EdgeQL +` and then use :ref:`codegen ` to +generate typesafe function that we can plug directly into out Python code. If +you are completely unfamiliar with EdgeQL, now is a good time to check out the +basics before proceeding. + + +.. edb:split-section:: + + Let's move on. First, we'll create a directory inside ``app`` called + ``queries``. This is where we're going to put all of the EdgeQL-related stuff. + + We're going to start by writing a query that fetches all of the users. In + ``queries`` create a file named ``get_users.edgeql`` and put the following query + in there: + + .. code-block:: edgeql + :caption: app/queries/get_users.edgeql + + select User { name }; + + +.. edb:split-section:: + + Now run the code generator from the shell: + + .. code-block:: bash + + $ gel-py + + +.. edb:split-section:: + + It's going to automatically locate the ``.edgeql`` file and generate types for + it. We can inspect generated code in ``app.queries/get_users_async_edgeql.py``. + Once that is done, let's use those types to create the endpoint in ``main.py``: + + .. code-block:: python + :caption: app/main.py + + from edgedb import create_async_client + from .queries.get_users_async_edgeql import get_users as get_users_query, GetUsersResult + + + gel_client = create_async_client() + + @app.get("/users") + async def get_users() -> list[GetUsersResult]: + return await get_users_query(gel_client) + + +.. edb:split-section:: + + Let's verify it that works as expected: + + .. code-block:: bash + + $ curl -X 'GET' \ + 'http://127.0.0.1:8000/users' \ + -H 'accept: application/json' + + [ + { + "id": "862de904-de39-11ef-9713-4fab09220c4a", + "name": "alice" + }, + { + "id": "862e400c-de39-11ef-9713-2f81f2b67013", + "name": "bob" + } + ] + + +.. edb:split-section:: + + While we're at it, let's also implement the option to fetch a user by their + username. In order to do that, we need to write a new query in a separate file + ``app/queries/get_user_by_name.edgeql``: + + .. code-block:: edgeql + :caption: app/queries/get_user_by_name.edgeql + + select User { name } + filter .name = $name; + + +.. edb:split-section:: + + After that, we will run the code generator again by calling ``gel-py``. In the + app, we are going to reuse the same endpoint that fetches the list of all users. + From now on, if the user calls it without any arguments (e.g. + ``http://127.0.0.1/users``), they are going to receive the list of all users, + same as before. But if they pass a username as a query argument like this: + ``http://127.0.0.1/users?username=bob``, the system will attempt to fetch a user + named ``bob``. + + In order to achieve this, we're going to need to add a ``Query``-type argument + to our endpoint function. You can learn more about how to configure this type of + arguments in `FastAPI's docs + `_. It's default value is + going to be ``None``, which will enable us to implement our conditional logic: + + .. code-block:: python + :caption: app/main.py + + from fastapi import Query, HTTPException + from http import HTTPStatus + from .queries.get_user_by_name_async_edgeql import ( + get_user_by_name as get_user_by_name_query, + GetUserByNameResult, + ) + + + @app.get("/users") + async def get_users( + username: str = Query(None), + ) -> list[GetUsersResult] | GetUserByNameResult: + """List all users or get a user by their username""" + if username: + user = await get_user_by_name_query(gel_client, name=username) + if not user: + raise HTTPException( + HTTPStatus.NOT_FOUND, + detail={"error": f"Error: user {username} does not exist."}, + ) + return user + else: + return await get_users_query(gel_client) + + +.. edb:split-section:: + + And once again, let's verify that everything works: + + .. code-block:: bash + + $ curl -X 'GET' \ + 'http://127.0.0.1:8000/users?username=alice' \ + -H 'accept: application/json' + + { + "id": "862de904-de39-11ef-9713-4fab09220c4a", + "name": "alice" + } + + +.. edb:split-section:: + + Finally, let's also implement the option to add a new user. For this, just as + before, we'll create a new file ``app/queries/create_user.edgeql``, add a query + to it and run code generation. + + Note that in this query we've wrapped the ``insert`` in a ``select`` statement. + This is a common pattern in EdgeQL, that can be used whenever you would like to + get something other than object ID when you just inserted it. + + .. code-block:: edgeql + :caption: app/queries/create_user.edgeql + + select( + insert User { + name := $username + } + ) { + name + } + + + +.. edb:split-section:: + + In order to integrate this query into our app, we're going to add a new + endpoint. Note that this one has the same name ``/users``, but is for the POST + HTTP method. + + .. code-block:: python + :caption: app/main.py + + from gel import ConstraintViolationError + from .queries.create_user_async_edgeql import ( + create_user as create_user_query, + CreateUserResult, + ) + + @app.post("/users", status_code=HTTPStatus.CREATED) + async def post_user(username: str = Query()) -> CreateUserResult: + try: + return await create_user_query(gel_client, username=username) + except ConstraintViolationError: + raise HTTPException( + status_code=HTTPStatus.BAD_REQUEST, + detail={"error": f"Username '{username}' already exists."}, + ) + + +.. edb:split-section:: + + Once more, let's verify that the new endpoint works as expected: + + .. code-block:: bash + + $ curl -X 'POST' \ + 'http://127.0.0.1:8000/users?username=charlie' \ + -H 'accept: application/json' \ + -d '' + + { + "id": "20372a1a-ded5-11ef-9a08-b329b578c45c", + "name": "charlie" + } + + +.. edb:split-section:: + + This wraps things up for our user-related functionality. Of course, we now need + to deal with Chats and Messages, too. We're not going to go in depth for those, + since the process would be quite similar to what we've just done. Instead, feel + free to implement those endpoints yourself as an exercise, or copy the code + below if you are in rush. + + .. code-block:: bash + :class: collapsible + + $ echo 'select Chat { + messages: { role, body, sources }, + user := .$username;' > app/queries/get_chats.edgeql && echo 'select Chat { + messages: { role, body, sources }, + user := .$username and .id = $chat_id;' > app/queries/get_chat_by_id.edgeql && echo 'with new_chat := (insert Chat) + select ( + update User filter .name = $username + set { + chats := assert_distinct(.chats union new_chat) + } + ) { + new_chat_id := new_chat.id + }' > app/queries/create_chat.edgeql && echo 'with + user := (select User filter .name = $username), + chat := ( + select Chat filter .$chat_id + ) + select Message { + role, + body, + sources, + chat := . app/queries/get_messages.edgeql && echo 'with + user := (select User filter .name = $username), + update Chat + filter .id = $chat_id and .$message_role, + body := $message_body, + sources := array_unpack(>$sources) + } + )) + }' > app/queries/add_message.edgeql + + +.. edb:split-section:: + + And these are the endpoint definitions, provided in bulk. + + .. code-block:: python + :caption: app/main.py + :class: collapsible + + from .queries.get_chats_async_edgeql import get_chats as get_chats_query, GetChatsResult + from .queries.get_chat_by_id_async_edgeql import ( + get_chat_by_id as get_chat_by_id_query, + GetChatByIdResult, + ) + from .queries.get_messages_async_edgeql import ( + get_messages as get_messages_query, + GetMessagesResult, + ) + from .queries.create_chat_async_edgeql import ( + create_chat as create_chat_query, + CreateChatResult, + ) + from .queries.add_message_async_edgeql import ( + add_message as add_message_query, + ) + + + @app.get("/chats") + async def get_chats( + username: str = Query(), chat_id: str = Query(None) + ) -> list[GetChatsResult] | GetChatByIdResult: + """List user's chats or get a chat by username and id""" + if chat_id: + chat = await get_chat_by_id_query( + gel_client, username=username, chat_id=chat_id + ) + if not chat: + raise HTTPException( + HTTPStatus.NOT_FOUND, + detail={"error": f"Chat {chat_id} for user {username} does not exist."}, + ) + return chat + else: + return await get_chats_query(gel_client, username=username) + + + @app.post("/chats", status_code=HTTPStatus.CREATED) + async def post_chat(username: str) -> CreateChatResult: + return await create_chat_query(gel_client, username=username) + + + @app.get("/messages") + async def get_messages( + username: str = Query(), chat_id: str = Query() + ) -> list[GetMessagesResult]: + """Fetch all messages from a chat""" + return await get_messages_query(gel_client, username=username, chat_id=chat_id) + + +.. edb:split-section:: + + For the ``post_messages`` function we're going to do something a little bit + different though. Since this is now the primary way for the user to add their + queries to the system, it functionally superceeds the ``/search`` endpoint we + made before. To this end, this function is where we're going to handle saving + messages, retrieving chat history, invoking web search and generating the + answer. + + .. code-block:: python-diff + :caption: app/main.py + + - @app.post("/search") + - async def search(search_terms: SearchTerms) -> SearchResult: + - web_sources = await search_web(search_terms.query) + - search_result = await generate_answer(search_terms.query, web_sources) + - return search_result + + + @app.post("/messages", status_code=HTTPStatus.CREATED) + + async def post_messages( + + search_terms: SearchTerms, + + username: str = Query(), + + chat_id: str = Query(), + + ) -> SearchResult: + + chat_history = await get_messages_query( + + gel_client, username=username, chat_id=chat_id + + ) + + + _ = await add_message_query( + + gel_client, + + username=username, + + message_role="user", + + message_body=search_terms.query, + + sources=[], + + chat_id=chat_id, + + ) + + + search_query = search_terms.query + + web_sources = await search_web(search_query) + + + search_result = await generate_answer( + + search_terms.query, chat_history, web_sources + + ) + + + _ = await add_message_query( + + gel_client, + + username=username, + + message_role="assistant", + + message_body=search_result.response, + + sources=search_result.sources, + + chat_id=chat_id, + + ) + + + return search_result + + +.. edb:split-section:: + + Let's not forget to modify the ``generate_answer`` function, so it can also be + history-aware. + + .. code-block:: python-diff + :caption: app/main.py + + async def generate_answer( + query: str, + + chat_history: list[GetMessagesResult], + web_sources: list[WebSource], + ) -> SearchResult: + system_prompt = ( + "You are a helpful assistant that answers user's questions" + + " by finding relevant information in HackerNews threads." + + " When answering the question, describe conversations that people have around the subject," + + " provided to you as a context, or say i don't know if they are completely irrelevant." + ) + + prompt = f"User search query: {query}\n\nWeb search results:\n" + + for i, source in enumerate(web_sources): + prompt += f"Result {i} (URL: {source.url}):\n" + prompt += f"{source.text}\n\n" + + - messages = [{"role": "user", "content": prompt}] + + messages = [ + + {"role": message.role, "content": message.body} for message in chat_history + + ] + + messages.append({"role": "user", "content": prompt}) + + llm_response = get_llm_completion( + system_prompt=system_prompt, + messages=messages, + ) + + search_result = SearchResult( + response=llm_response, + sources=web_sources, + ) + + return search_result + + +.. edb:split-section:: + + Ok, this should be it for setting up the chat history. Let's test it. First, we + are going to start a new chat for our user: + + .. code-block:: bash + + $ curl -X 'POST' \ + 'http://127.0.0.1:8000/chats?username=charlie' \ + -H 'accept: application/json' \ + -d '' + + { + "id": "20372a1a-ded5-11ef-9a08-b329b578c45c", + "new_chat_id": "544ef3f2-ded8-11ef-ba16-f7f254b95e36" + } + + +.. edb:split-section:: + + Next, let's add a couple messages and wait for the bot to respond: + + .. code-block:: bash + + $ curl -X 'POST' \ + 'http://127.0.0.1:8000/messages?username=charlie&chat_id=544ef3f2-ded8-11ef-ba16-f7f254b95e36' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "query": "best database in existence" + }' + + $ curl -X 'POST' \ + 'http://127.0.0.1:8000/messages?username=charlie&chat_id=544ef3f2-ded8-11ef-ba16-f7f254b95e36' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "query": "gel" + }' + + +.. edb:split-section:: + + Finally, let's check that the messages we saw are in fact stored in the chat + history: + + .. code-block:: bash + + $ curl -X 'GET' \ + 'http://127.0.0.1:8000/messages?username=charlie&chat_id=544ef3f2-ded8-11ef-ba16-f7f254b95e36' \ + -H 'accept: application/json' + + +In reality this workflow would've been handled by the frontend, providing the +user with a nice inteface to interact with. But even without one our chatbot is +almost functional by now. + +Generating a Google search query +-------------------------------- + +Congratulations! We just got done implementing multi-turn conversations for our +search bot. + +However, there's still one crucial piece missing. Right now we're simply +forwarding the users message straight to the full-text search. But what happens +if their message is a followup that cannot be used as a standalone search +query? + +Ideally what we should do is we should infer the search query from the entire +conversation, and use that to perform the search. + +Let's implement an extra step in which the LLM is going to produce a query for +us based on the entire chat history. That way we can be sure we're progressively +working on our query rather than rewriting it from scratch every time. + + +.. edb:split-section:: + + This is what we need to do: every time the user submits a message, we need to + fetch the chat history, extract a search query from it using the LLM, and the + other steps are going to the the same as before. Let's make the follwing + modifications to the ``main.py``: first we need to create a function that + prepares LLM inputs for the search query inference. + + + .. code-block:: python + :caption: app/main.py + + async def generate_search_query( + query: str, message_history: list[GetMessagesResult] + ) -> str: + system_prompt = ( + "You are a helpful assistant." + + " Your job is to extract a keyword search query" + + " from a chat between an AI and a human." + + " Make sure it's a single most relevant keyword to maximize matching." + + " Only provide the query itself as your response." + ) + + formatted_history = "\n---\n".join( + [ + f"{message.role}: {message.body} (sources: {message.sources})" + for message in message_history + ] + ) + prompt = f"Chat history: {formatted_history}\n\nUser message: {query} \n\n" + + llm_response = get_llm_completion( + system_prompt=system_prompt, messages=[{"role": "user", "content": prompt}] + ) + + return llm_response + + +.. edb:split-section:: + + And now we can use this function in ``post_messages`` in order to get our + search query: + + + .. code-block:: python-diff + :caption: app/main.py + + class SearchResult(BaseModel): + response: str | None = None + + search_query: str | None = None + sources: list[WebSource] | None = None + + + @app.post("/messages", status_code=HTTPStatus.CREATED) + async def post_messages( + search_terms: SearchTerms, + username: str = Query(), + chat_id: str = Query(), + ) -> SearchResult: + # 1. Fetch chat history + chat_history = await get_messages_query( + gel_client, username=username, chat_id=chat_id + ) + + # 2. Add incoming message to Gel + _ = await add_message_query( + gel_client, + username=username, + message_role="user", + message_body=search_terms.query, + sources=[], + chat_id=chat_id, + ) + + # 3. Generate a query and perform googling + - search_query = search_terms.query + + search_query = await generate_search_query(search_terms.query, chat_history) + + web_sources = await search_web(search_query) + + + # 5. Generate answer + search_result = await generate_answer( + search_terms.query, + chat_history, + web_sources, + ) + + search_result.search_query = search_query # add search query to the output + + # to see what the bot is searching for + # 6. Add LLM response to Gel + _ = await add_message_query( + gel_client, + username=username, + message_role="assistant", + message_body=search_result.response, + sources=[s.url for s in search_result.sources], + chat_id=chat_id, + ) + + # 7. Send result back to the client + return search_result + + +.. edb:split-section:: + + Done! We've now fully integrated the chat history into out app and enabled + natural language conversations. As before, let's quickly test out the + improvements before moving on: + + + .. code-block:: bash + + $ curl -X 'POST' \ + 'http://localhost:8000/messages?username=alice&chat_id=d4eed420-e903-11ef-b8a7-8718abdafbe1' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "query": "what are people saying about gel" + }' + + $ curl -X 'POST' \ + 'http://localhost:8000/messages?username=alice&chat_id=d4eed420-e903-11ef-b8a7-8718abdafbe1' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "query": "do they like it or not" + }' + + +6. Use Gel's advanced features to create a RAG +============================================== + +At this point we have a decent search bot that can refine a search query over +multiple turns of a conversation. + +It's time to add the final touch: we can make the bot remember previous similar +interactions with the user using retrieval-augmented generation (RAG). + +To achieve this we need to implement similarity search across message history: +we're going to create a vector embedding for every message in the database using +a neural network. Every time we generate a Google search query, we're also going +to use it to search for similar messages in user's message history, and inject +the corresponding chat into the prompt. That way the search bot will be able to +quickly "remember" similar interactions with the user and use them to understand +what they are looking for. + +Gel enables us to implement such a system with only minor modifications to the +schema. + + +.. edb:split-section:: + + We begin by enabling the ``ai`` extension by adding the following like on top of + the ``dbschema/default.esdl``: + + .. code-block:: sdl-diff + :caption: dbschema/default.esdl + + + using extension ai; + + +.. edb:split-section:: + + ... and do the migration: + + + .. code-block:: bash + + $ gel migration create + $ gel migrate + + +.. edb:split-section:: + + Next, we need to configure the API key in Gel for whatever embedding provider + we're going to be using. As per documentation, let's open up the CLI by typing + ``gel`` and run the following command (assuming we're using OpenAI): + + .. code-block:: edgeql-repl + + searchbot:main> configure current database + insert ext::ai::OpenAIProviderConfig { + secret := 'sk-....', + }; + + OK: CONFIGURE DATABASE + + +.. edb:split-section:: + + In order to get Gel to automatically keep track of creating and updating + message embeddings, all we need to do is create a deferred index like this. + Don't forget to run a migration one more time! + + .. code-block:: sdl-diff + + type Message { + role: str; + body: str; + timestamp: datetime { + default := datetime_current(); + } + multi sources: str; + + + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') + + on (.body); + } + + +.. edb:split-section:: + + And we're done! Gel is going to cook in the background for a while and generate + embedding vectors for our queries. To make sure nothing broke we can follow + Gel's AI documentation and take a look at instance logs: + + .. code-block:: bash + + $ gel instance logs -I searchbot | grep api.openai.com + + INFO 50121 searchbot 2025-01-30T14:39:53.364 httpx: HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" + + +.. edb:split-section:: + + It's time to create the second half of the similarity search - the search query. + The query needs to fetch ``k`` chats in which there're messages that are most + similar to our current message. This can be a little difficult to visualize in + your head, so here's the query itself: + + .. code-block:: edgeql + :caption: app/queries/search_chats.edgeql + + with + user := (select User filter .name = $username), + chats := ( + select Chat + filter .$current_chat_id + ) + + select chats { + distance := min( + ext::ai::search( + .messages, + >$embedding, + ).distance, + ), + messages: { + role, body, sources + } + } + + order by .distance + limit $limit; + + +.. edb:split-section:: + + .. note:: + + Before we can integrate this query into our Python app, we also need to add a + new dependency for the Python binding: ``httpx-sse``. It's enables streaming + outputs, which we're not going to use right now, but we won't be able to + create the AI client without it. + + Let's place in in ``app/queries/search_chats.edgeql``, run the codegen and modify + our ``post_messages`` endpoint to keep track of those similar chats. + + .. code-block:: python-diff + :caption: app.main.py + + + from edgedb.ai import create_async_ai, AsyncEdgeDBAI + + from .queries.search_chats_async_edgeql import ( + + search_chats as search_chats_query, + + ) + + class SearchResult(BaseModel): + response: str | None = None + search_query: str | None = None + sources: list[WebSource] | None = None + + similar_chats: list[str] | None = None + + + @app.post("/messages", status_code=HTTPStatus.CREATED) + async def post_messages( + search_terms: SearchTerms, + username: str = Query(), + chat_id: str = Query(), + ) -> SearchResult: + # 1. Fetch chat history + chat_history = await get_messages_query( + gel_client, username=username, chat_id=chat_id + ) + + # 2. Add incoming message to Gel + _ = await add_message_query( + gel_client, + username=username, + message_role="user", + message_body=search_terms.query, + sources=[], + chat_id=chat_id, + ) + + # 3. Generate a query and perform googling + search_query = await generate_search_query(search_terms.query, chat_history) + web_sources = await search_web(search_query) + + + # 4. Fetch similar chats + + db_ai: AsyncEdgeDBAI = await create_async_ai(gel_client, model="gpt-4o-mini") + + embedding = await db_ai.generate_embeddings( + + search_query, model="text-embedding-3-small" + + ) + + similar_chats = await search_chats_query( + + gel_client, + + username=username, + + current_chat_id=chat_id, + + embedding=embedding, + + limit=1, + + ) + + # 5. Generate answer + search_result = await generate_answer( + search_terms.query, + chat_history, + web_sources, + + similar_chats, + ) + search_result.search_query = search_query # add search query to the output + # to see what the bot is searching for + # 6. Add LLM response to Gel + _ = await add_message_query( + gel_client, + username=username, + message_role="assistant", + message_body=search_result.response, + sources=[s.url for s in search_result.sources], + chat_id=chat_id, + ) + + # 7. Send result back to the client + return search_result + + +.. edb:split-section:: + + Finally, the answer generator needs to get updated one more time, since we need + to inject the additional messages into the prompt. + + .. code-block:: python-diff + :caption: app/main.py + + async def generate_answer( + query: str, + chat_history: list[GetMessagesResult], + web_sources: list[WebSource], + + similar_chats: list[list[GetMessagesResult]], + ) -> SearchResult: + system_prompt = ( + "You are a helpful assistant that answers user's questions" + + " by finding relevant information in HackerNews threads." + + " When answering the question, describe conversations that people have around the subject, provided to you as a context, or say i don't know if they are completely irrelevant." + + + " You can reference previous conversation with the user that" + + + " are provided to you, if they are relevant, by explicitly referring" + + + " to them by saying as we discussed in the past." + ) + + prompt = f"User search query: {query}\n\nWeb search results:\n" + + for i, source in enumerate(web_sources): + prompt += f"Result {i} (URL: {source.url}):\n" + prompt += f"{source.text}\n\n" + + + prompt += "Similar chats with the same user:\n" + + + formatted_chats = [] + + for i, chat in enumerate(similar_chats): + + formatted_chat = f"Chat {i}: \n" + + for message in chat.messages: + + formatted_chat += f"{message.role}: {message.body}\n" + + formatted_chats.append(formatted_chat) + + + prompt += "\n".join(formatted_chats) + + messages = [ + {"role": message.role, "content": message.body} for message in chat_history + ] + messages.append({"role": "user", "content": prompt}) + + llm_response = get_llm_completion( + system_prompt=system_prompt, + messages=messages, + ) + + search_result = SearchResult( + response=llm_response, + sources=web_sources, + + similar_chats=formatted_chats, + ) + + return search_result + + +.. edb:split-section:: + + And one last time, let's check to make sure everything works: + + .. code-block:: bash + + $ curl -X 'POST' \ + 'http://localhost:8000/messages?username=alice&chat_id=d4eed420-e903-11ef-b8a7-8718abdafbe1' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "query": "remember that cool db i was talking to you about?" + }' + + +Keep going! +=========== + +This tutorial is over, but this app surely could use way more features! + +Basic functionality like deleting messages, a user interface or real web +search, sure. But also authentication or access policies -- Gel will let you +set those up in minutes. + +Thanks! + + + + + + + diff --git a/docs/ai/guide_edgeql.rst b/docs/ai/guide_edgeql.rst new file mode 100644 index 00000000000..f21f57157c1 --- /dev/null +++ b/docs/ai/guide_edgeql.rst @@ -0,0 +1,286 @@ +.. _ref_ai_guide_edgeql: + +========================= +Guide to Gel AI in EdgeQL +========================= + +:edb-alt-title: How to set up Gel AI in EdgeQL + + +|Gel| AI brings vector search capabilities and retrieval-augmented generation +directly into the database. + + +Enable and configure the extension +================================== + +.. edb:split-section:: + + AI is a |Gel| extension. To enable it, we will need to add the extension + to the app’s schema: + + .. code-block:: sdl + + using extension ai; + + +.. edb:split-section:: + + |Gel| AI uses external APIs in order to get vectors and LLM completions. For it + to work, we need to configure an API provider and specify their API key. Let's + open EdgeQL REPL and run the following query: + + .. code-block:: edgeql + + configure current database + insert ext::ai::OpenAIProviderConfig { + secret := 'sk-....', + }; + + +Now our |Gel| application can take advantage of OpenAI's API to implement AI +capabilities. + + +.. note:: + + |Gel| AI comes with its own :ref:`UI ` that can + be used to configure providers, set up prompts and test them in a sandbox. + + +.. note:: + + Most API providers require you to set up and account and charge money for + model use. + + +Add vectors and perform similarity search +========================================= + +.. edb:split-section:: + + Before we start introducing AI capabilities, let's set up our database with a + schema and populate it with some data (we're going to be helping Komi-san keep + track of her friends). + + .. code-block:: sdl + + module default { + type Friend { + required name: str { + constraint exclusive; + }; + + summary: str; # A brief description of personality and role + relationship_to_komi: str; # Relationship with Komi + defining_trait: str; # Primary character trait or quirk + } + } + +.. edb:split-section:: + + Here's a shell command you can paste and run that will populate the + database with some sample data. + + .. code-block:: bash + :class: collapsible + + $ cat << 'EOF' > populate_db.edgeql + insert Friend { + name := 'Tadano Hitohito', + summary := 'An extremely average high school boy with a remarkable ability to read the atmosphere and understand others\' feelings, especially Komi\'s.', + relationship_to_komi := 'First friend and love interest', + defining_trait := 'Perceptiveness', + }; + + insert Friend { + name := 'Osana Najimi', + summary := 'An extremely outgoing person who claims to have been everyone\'s childhood friend. Gender: Najimi.', + relationship_to_komi := 'Second friend and social catalyst', + defining_trait := 'Universal childhood friend', + }; + + insert Friend { + name := 'Yamai Ren', + summary := 'An intense and sometimes obsessive classmate who is completely infatuated with Komi.', + relationship_to_komi := 'Self-proclaimed guardian and admirer', + defining_trait := 'Obsessive devotion', + }; + + insert Friend { + name := 'Katai Makoto', + summary := 'A intimidating-looking but shy student who shares many communication problems with Komi.', + relationship_to_komi := 'Fellow communication-challenged friend', + defining_trait := 'Scary appearance but gentle nature', + }; + + insert Friend { + name := 'Nakanaka Omoharu', + summary := 'A self-proclaimed wielder of dark powers who acts like an anime character and is actually just a regular gaming enthusiast.', + relationship_to_komi := 'Gaming buddy and chuunibyou friend', + defining_trait := 'Chuunibyou tendencies', + }; + EOF + $ gel query -f populate_db.edgeql + + +.. edb:split-section:: + + In order to get |Gel| to produce embedding vectors, we need to create a special + ``deferred index`` on the type we would like to perform similarity search on. + More specifically, we need to specify an EdgeQL expression that produces a + string that we're going to create an embedding vector for. This is how we would + set up an index if we wanted to perform similarity search on + ``Friend.summary``: + + .. code-block:: sdl-diff + + module default { + type Friend { + required name: str { + constraint exclusive; + }; + + summary: str; # A brief description of personality and role + relationship_to_komi: str; # Relationship with Komi + defining_trait: str; # Primary character trait or quirk + + + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') + + on (.summary); + } + } + + +.. edb:split-section:: + + But actually, in our case it would be better if we could similarity search + across all properties at the same time. We can define the index on a more + complex expression - like a concatenation of string properties - like this: + + + .. code-block:: sdl-diff + + module default { + type Friend { + required name: str { + constraint exclusive; + }; + + summary: str; # A brief description of personality and role + relationship_to_komi: str; # Relationship with Komi + defining_trait: str; # Primary character trait or quirk + + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') + - on (.summary); + + on ( + + .name ++ ' ' ++ .summary ++ ' ' + + ++ .relationship_to_komi ++ ' ' + + ++ .defining_trait + + ); + } + } + + +.. edb:split-section:: + + Once we're done with schema modification, we need to apply them by going + through a migration: + + .. code-block:: bash + + $ gel migration create + $ gel migrate + + +.. edb:split-section:: + + That's it! |Gel| will make necessary API requests in the background and create an + index that will enable us to perform efficient similarity search like this: + + .. code-block:: edgeql + + select ext::ai::search(Friend, query_vector); + + +.. edb:split-section:: + + Note that this function accepts an embedding vector as the second argument, not + a text string. This means that in order to similarity search for a string, we + need to create a vector embedding for it using the same model as we used to + create the index. |Gel| offers an HTTP endpoint ``/ai/embeddings`` that can + handle it for us. All we need to do is to pass the vector it produces into the + search query: + + .. note:: + + Note that we're passing our login and password in order to autheticate the + request. We can find those using the CLI: ``gel instance credentials + --json``. Learn about all the other ways you can authenticate a request + :ref:`here `. + + .. code-block:: bash + + $ curl --user user:password \ + --json '{"input": "Who helps Komi make friends?", "model": "text-embedding-3-small"}' \ + http://localhost:/branch/main/ai/embeddings \ + | jq -r '.data[0].embedding' \ # extract the embedding out of the JSON + | tr -d '\n' \ # remove newlines + | sed 's/^\[//;s/\]$//' \ # remove square brackets + | awk '{print "select ext::ai::search(Friend, >[" $0 "]);"}' \ # assemble the query + | gel query --file - # pass the query into Gel CLI + + + +Use the built-in RAG +==================== + +One more feature |Gel| AI offers is built-in retrieval-augmented generation, also +known as RAG. + +.. edb:split-section:: + + |Gel| comes preconfigured to be able to process our text query, perform + similarity search across the index we just created, pass the results to an LLM + and return a response. We can access the built-in RAG using the ``/ai/rag`` + HTTP endpoint: + + + .. code-block:: bash + + $ curl --user user:password --json '{ + "query": "Who helps Komi make friends?", + "model": "gpt-4-turbo-preview", + "context": {"query":"select Friend"} + }' http://localhost:/branch/main/ai/rag + + +.. edb:split-section:: + + We can also stream the response like this: + + + .. code-block:: bash-diff + + $ curl --user user:password --json '{ + "query": "Who helps Komi make friends?", + "model": "gpt-4-turbo-preview", + "context": {"query":"select Friend"}, + + "stream": true, + }' http://localhost:/branch/main/ai/rag + + +Keep going! +=========== + +You are now sufficiently equipped to use |Gel| AI in your applications. + +If you'd like to build something on your own, make sure to check out the +:ref:`Reference manual ` in order to learn the details +about using different APIs and models, configuring prompts or using the UI. +Make sure to also check out the |Gel| AI bindings in Python and JavaScript if +those languages are relevant to you. + +And if you would like more guidance for how |Gel| AI can be fit into an +application, take a look at the FastAPI Gel AI Tutorial, where we're building a +search bot using features you learned about above. + diff --git a/docs/ai/guide_python.rst b/docs/ai/guide_python.rst new file mode 100644 index 00000000000..aed9004bd66 --- /dev/null +++ b/docs/ai/guide_python.rst @@ -0,0 +1,369 @@ +.. _ref_ai_guide_python: + +========================= +Guide to Gel AI in Python +========================= + +:edb-alt-title: How to set up Gel AI in Python + +.. edb:split-section:: + + |Gel| AI brings vector search capabilities and retrieval-augmented + generation directly into the database. It's integrated into the |Gel| + Python binding via the ``gel.ai`` module. + + .. code-block:: bash + + $ pip install 'gel[ai]' + + +Enable and configure the extension +================================== + +.. edb:split-section:: + + AI is an |Gel| extension. To enable it, we will need to add the extension + to the app’s schema: + + .. code-block:: sdl + + using extension ai; + + +.. edb:split-section:: + + |Gel| AI uses external APIs in order to get vectors and LLM completions. + For it to work, we need to configure an API provider and specify their API + key. Let's open EdgeQL REPL and run the following query: + + .. code-block:: edgeql + + configure current database + insert ext::ai::OpenAIProviderConfig { + secret := 'sk-....', + }; + + +Now our |Gel| application can take advantage of OpenAI's API to implement AI +capabilities. + + +.. note:: + + |Gel| AI comes with its own :ref:`UI ` that can + be used to configure providers, set up prompts and test them in a sandbox. + + +.. note:: + + Most API providers require you to set up and account and charge money for + model use. + + +Add vectors +=========== + +.. edb:split-section:: + + Before we start introducing AI capabilities, let's set up our database with a + schema and populate it with some data (we're going to be helping Komi-san keep + track of her friends). + + .. code-block:: sdl + + module default { + type Friend { + required name: str { + constraint exclusive; + }; + + summary: str; # A brief description of personality and role + relationship_to_komi: str; # Relationship with Komi + defining_trait: str; # Primary character trait or quirk + } + } + +.. edb:split-section:: + + Here's a shell command you can paste and run that will populate the + database with some sample data. + + .. code-block:: bash + :class: collapsible + + $ cat << 'EOF' > populate_db.edgeql + insert Friend { + name := 'Tadano Hitohito', + summary := 'An extremely average high school boy with a remarkable ability to read the atmosphere and understand others\' feelings, especially Komi\'s.', + relationship_to_komi := 'First friend and love interest', + defining_trait := 'Perceptiveness', + }; + + insert Friend { + name := 'Osana Najimi', + summary := 'An extremely outgoing person who claims to have been everyone\'s childhood friend. Gender: Najimi.', + relationship_to_komi := 'Second friend and social catalyst', + defining_trait := 'Universal childhood friend', + }; + + insert Friend { + name := 'Yamai Ren', + summary := 'An intense and sometimes obsessive classmate who is completely infatuated with Komi.', + relationship_to_komi := 'Self-proclaimed guardian and admirer', + defining_trait := 'Obsessive devotion', + }; + + insert Friend { + name := 'Katai Makoto', + summary := 'A intimidating-looking but shy student who shares many communication problems with Komi.', + relationship_to_komi := 'Fellow communication-challenged friend', + defining_trait := 'Scary appearance but gentle nature', + }; + + insert Friend { + name := 'Nakanaka Omoharu', + summary := 'A self-proclaimed wielder of dark powers who acts like an anime character and is actually just a regular gaming enthusiast.', + relationship_to_komi := 'Gaming buddy and chuunibyou friend', + defining_trait := 'Chuunibyou tendencies', + }; + EOF + $ gel query -f populate_db.edgeql + + +.. edb:split-section:: + + In order to get |Gel| to produce embedding vectors, we need to create a + special ``deferred index`` on the type we would like to perform similarity + search on. More specifically, we need to specify an EdgeQL expression that + produces a string that we're going to create an embedding vector for. This + is how we would set up an index if we wanted to perform similarity search + on ``Friend.summary``: + + .. code-block:: sdl-diff + + module default { + type Friend { + required name: str { + constraint exclusive; + }; + + summary: str; # A brief description of personality and role + relationship_to_komi: str; # Relationship with Komi + defining_trait: str; # Primary character trait or quirk + + + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') + + on (.summary); + } + } + + +.. edb:split-section:: + + But actually, in our case it would be better if we could similarity search + across all properties at the same time. We can define the index on a more + complex expression - like a concatenation of string properties - like this: + + + .. code-block:: sdl-diff + + module default { + type Friend { + required name: str { + constraint exclusive; + }; + + summary: str; # A brief description of personality and role + relationship_to_komi: str; # Relationship with Komi + defining_trait: str; # Primary character trait or quirk + + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') + - on (.summary); + + on ( + + .name ++ ' ' ++ .summary ++ ' ' + + ++ .relationship_to_komi ++ ' ' + + ++ .defining_trait + + ); + } + } + + +.. edb:split-section:: + + Once we're done with schema modification, we need to apply them by going + through a migration: + + .. code-block:: bash + + $ gel migration create + $ gel migrate + + +That's it! |Gel| will make necessary API requests in the background and create an +index that will enable us to perform efficient similarity search. + + +Perform similarity search in Python +=================================== + +.. edb:split-section:: + + In order to run queries against the index we just created, we need to create a + |Gel| client and pass it to a |Gel| AI instance. + + .. code-block:: python + + import gel + import gel.ai + + gel_client = gel.create_client() + gel_ai = gel.ai.create_rag_client(client) + + text = "Who helps Komi make friends?" + vector = gel_ai.generate_embeddings( + text, + "text-embedding-3-small", + ) + + gel_client.query( + "select ext::ai::search(Friend, >$embedding_vector", + embedding_vector=vector, + ) + + +.. edb:split-section:: + + We are going to execute a query that calls a single function: + ``ext::ai::search(, )``. That function accepts an + embedding vector as the second argument, not a text string. This means that in + order to similarity search for a string, we need to create a vector embedding + for it using the same model as we used to create the index. The |Gel| AI binding + in Python comes with a ``generate_embeddings`` function that does exactly that: + + + .. code-block:: python-diff + + import gel + import gel.ai + + gel_client = gel.create_client() + gel_ai = gel.ai.create_rag_client(client) + + + text = "Who helps Komi make friends?" + + vector = gel_ai.generate_embeddings( + + text, + + "text-embedding-3-small", + + ) + + +.. edb:split-section:: + + Now we can plug that vector directly into our query to get similarity search + results: + + + .. code-block:: python-diff + + import gel + import gel.ai + + gel_client = gel.create_client() + gel_ai = gel.ai.create_rag_client(client) + + text = "Who helps Komi make friends?" + vector = gel_ai.generate_embeddings( + text, + "text-embedding-3-small", + ) + + + gel_client.query( + + "select ext::ai::search(Friend, >$embedding_vector", + + embedding_vector=vector, + + ) + + +Use the built-in RAG +==================== + +One more feature |Gel| AI offers is built-in retrieval-augmented generation, +also known as RAG. + +.. edb:split-section:: + + |Gel| comes preconfigured to be able to process our text query, perform + similarity search across the index we just created, pass the results to an + LLM and return a response. In order to access the built-in RAG, we need to + start by selecting an LLM and passing its name to the |Gel| AI instance + constructor: + + + .. code-block:: python-diff + + import gel + import gel.ai + + gel_client = gel.create_client() + gel_ai = gel.ai.create_rag_client( + client, + + model="gpt-4-turbo-preview" + ) + + +.. edb:split-section:: + + Now we can access the RAG using the ``query_rag`` function like this: + + + .. code-block:: python-diff + + import gel + import gel.ai + + gel_client = gel.create_client() + gel_ai = gel.ai.create_rag_client( + client, + model="gpt-4-turbo-preview" + ) + + + gel_ai.query_rag( + + "Who helps Komi make friends?", + + context="Friend", + + ) + + +.. edb:split-section:: + + We can also stream the response like this: + + + .. code-block:: python-diff + + import gel + import gel.ai + + gel_client = gel.create_client() + gel_ai = gel.ai.create_rag_client( + client, + model="gpt-4-turbo-preview" + ) + + - gel_ai.query_rag( + + gel_ai.stream_rag( + "Who helps Komi make friends?", + context="Friend", + ) + + +Keep going! +=========== + +You are now sufficiently equipped to use |Gel| AI in your applications. + +If you'd like to build something on your own, make sure to check out the +:ref:`Reference manual ` in order to learn the details +about using different APIs and models, configuring prompts or using the UI. + +And if you would like more guidance for how |Gel| AI can be fit into an +application, take a look at the FastAPI Gel AI Tutorial, where we're building a +search bot using features you learned about above. + + diff --git a/docs/ai/images/ui_playground.png b/docs/ai/images/ui_playground.png new file mode 100644 index 00000000000..5fa19839aee Binary files /dev/null and b/docs/ai/images/ui_playground.png differ diff --git a/docs/ai/images/ui_prompts.png b/docs/ai/images/ui_prompts.png new file mode 100644 index 00000000000..2a5b2d50784 Binary files /dev/null and b/docs/ai/images/ui_prompts.png differ diff --git a/docs/ai/images/ui_providers.png b/docs/ai/images/ui_providers.png new file mode 100644 index 00000000000..cf1adfb6c8e Binary files /dev/null and b/docs/ai/images/ui_providers.png differ diff --git a/docs/ai/index.rst b/docs/ai/index.rst index 88134092fdf..a23c768bd26 100644 --- a/docs/ai/index.rst +++ b/docs/ai/index.rst @@ -1,267 +1,39 @@ .. _ref_ai_overview: -== -AI -== +====== +Gel AI +====== .. toctree:: :hidden: :maxdepth: 3 + quickstart_fastapi_ai + reference_extai + reference_http + reference_python javascript - python - reference + guide_edgeql + guide_python :edb-alt-title: Using Gel AI -|Gel| AI allows you to ship AI-enabled apps with practically no effort. It -automatically generates embeddings for your data. Works with OpenAI, Mistral -AI, Anthropic, and any other provider with a compatible API. +|Gel| AI is a set of tools designed to enable you to ship AI-enabled apps with +practically no effort. This is what comes in the box: +1. ``ext::ai``: this Gel extension automatically generates embeddings for your + data. Works with OpenAI, Mistral AI, Anthropic, and any other provider with a + compatible API. -Enable extension in your schema -=============================== +2. ``ext::vectorstore``: this extension is designed to replicate workflows that + might be familiar to you from vectorstore-style databases. Powered by + ``pgvector``, it allows you to store and search for embedding vectors, and + integrates with popular AI frameworks. -AI is a |Gel| extension. To enable it, you will need to add the extension -to your app's schema: +3. Python library: ``gel.ai``. Access all Gel AI features straight from your + Python application. -.. code-block:: sdl +4. JavaScript library: ``@gel/ai``. - using extension ai; -Extension configuration -======================= - -The AI extension may be configured via our UI or via EdgeQL. To use the -built-in UI, access it by running :gelcmd:`ui`. If you have the extension -enabled in your schema as shown above and have migrated that schema change, you -will see the "AI Admin" icon in the left-hand toolbar. - -.. image:: images/ui-ai.png - :alt: The Gel local development server UI highlighting the AI admin - icon in the left-hand toolbar. The icon is two stars, one larger and - one smaller, the smaller being a light pink color and the larger - being a light blue when selected. - :width: 100% - -The default tab "Playground" allows you to test queries against your data after -you first configure the model, prompt, and context query in the right sidebar. - -The "Prompts" tab allows you to configure prompts for use in the playground. -The "Providers" tab must be configured for the API you want to use for -embedding generation and querying. We currently support OpenAI, Mistral AI, and -Anthropic. - - -Configuring a provider ----------------------- - -To configure a provider, you will first need to obtain an API key for your -chosen provider, which you may do from their respective sites: - -* `OpenAI API keys `__ -* `Mistral API keys `__ -* `Anthropic API keys `__ - -With your API key, you may now configure in the UI by clickin the "Add -Provider" button, selecting the appropriate API, and pasting your key in the -"Secret" field. - -.. image:: images/ui-ai-add-provider.png - :alt: The "Add Provider" form of the Gel local development server UI. - On the left, the sidebar navigation for the view showing Playground, - Prompts, and Providers options, with Provider selected (indicated - with a purple border on the left). The main content area shows a - heading Providers with a form under it. The form contains a dropdown - to select the API. (Anthropic is currently selected.) The form - contains two fields: an optional Client ID and a Secret. The Secret - field is filled with your-api-key-here. Under the fields to the - right, the form has a gray button to cancel and a purple Add Provider - button. - :width: 100% - -You may alternatively configure a provider via EdgeQL: - -.. code-block:: edgeql - - configure current branch - insert ext::ai::OpenAIProviderConfig { - secret := 'sk-....', - }; - -This object has other properties as well, including ``client_id`` and -``api_url``, which can be set as strings to override the defaults for the -chosen provider. - -We have provider config types for each of the three supported APIs: - -* ``OpenAIProviderConfig`` -* ``MistralProviderConfig`` -* ``AnthropicProviderConfig`` - - -Usage -===== - -Using |Gel| AI requires some changes to your schema. - - -Add an index ------------- - -To start using |Gel| AI on a type, create an index: - -.. code-block:: sdl-diff - - module default { - type Astronomy { - content: str; - + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') - + on (.content); - } - }; - -In this example, we have added an AI index on the ``Astronomy`` type's -``content`` property using the ``text-embedding-3-small`` model. Once you have -the index in your schema, :ref:`create ` and -:ref:`apply ` your migration, and you're ready -to start running queries! - -.. note:: - - The particular embedding model we've chosen here - (``text-embedding-3-small``) is an OpenAI model, so it will require an - OpenAI provider to be configured as described above. - - You may use any of :ref:`our pre-configured embedding generation models - `. - -You may want to include multiple properties in your AI index. Fortunately, you -can define an AI index on an expression: - -.. code-block:: sdl - - module default { - type Astronomy { - climate: str; - atmosphere: str; - deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') - on (.climate ++ ' ' ++ .atmosphere); - } - }; - -.. note:: When AI indexes aren't working… - - If you find your queries are not returning the expected results, try - inspecting your instance logs. On a |Gel| Cloud instance, use the "Logs" - tab in your instance dashboard. On local or :ref:`CLI-linked remote - instances `, use :gelcmd:`instance logs -I - `. You may find the problem there. - - Providers impose rate limits on their APIs which can often be the source of - AI index problems. If index creation hits a rate limit, |Gel| will wait - the ``indexer_naptime`` (see the docs on :ref:`ext::ai configuration - `) and resume index creation. - - If your indexed property contains values that exceed the token limit for a - single request, you may consider truncating the property value in your - index expression. You can do this with a string by slicing it: - - .. code-block:: sdl - - module default { - type Astronomy { - content: str; - deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') - on (.content[0:10000]); - } - }; - - This example will slice the first 10,000 characters of the ``content`` - property for indexing. - - Tokens are not equivalent to characters. For OpenAI embedding generation, - you may test values via `OpenAI's web-based tokenizer - `__. You may alternatively download - the library OpenAI uses for tokenization from that same page if you prefer. - By testing, you can get an idea how much of your content can be sent for - indexing. - - -Run a semantic similarity query -------------------------------- - -Once your index has been migrated, running a query against the embeddings is -super simple: - -.. code-block:: edgeql - - select ext::ai::search(Astronomy, query) - -Simple, but you'll still need to generate embeddings from your query or pass in -existing embeddings. If your ultimate goal is retrieval-augmented generation -(i.e., RAG), we've got you covered. - -.. _ref_ai_overview_rag: - -Use RAG via HTTP ----------------- - -By making an HTTP request to -``https://:/branch//ai/rag``, you can generate -text via the generative AI API of your choice within the context of a type with -a deferred embedding index. - -.. note:: - - Making HTTP requests to |Gel| requires :ref:`authentication - `. - -.. code-block:: bash - - $ curl --json '{ - "query": "What color is the sky on Mars?", - "model": "gpt-4-turbo-preview", - "context": {"query":"select Astronomy"} - }' https://:/branch//ai/rag - {"response": "The sky on Mars is red."} - -Since LLMs are often slow, it may be useful to stream the response. To do this, -add ``"stream": true`` to your request JSON. - -.. note:: - - The particular text generation model we've chosen here - (``gpt-4-turbo-preview``) is an OpenAI model, so it will require an OpenAI - provider to be configured as described above. - - You may use any of our supported :ref:`text generation models - `. - - -Use RAG via JavaScript ----------------------- - -``@gel/ai`` offers a convenient wrapper around ``ext::ai``. Install it with -``npm install @gel/ai`` (or via your package manager of choice) and -implement it like this example: - -.. code-block:: typescript - - import { createClient } from "gel"; - import { createAI } from "@gel/ai"; - - const client = createClient(); - - const gpt4AI = createAI(client, { - model: "gpt-4-turbo-preview", - }); - - const blogAI = gpt4AI.withContext({ - query: "select Astronomy" - }); - - console.log(await blogAI.queryRag( - "What color is the sky on Mars?" - )); diff --git a/docs/ai/javascript.rst b/docs/ai/javascript.rst index e1822023cef..fc5e7f6305f 100644 --- a/docs/ai/javascript.rst +++ b/docs/ai/javascript.rst @@ -41,7 +41,7 @@ model): }); You may use any of the supported :ref:`text generation models -`. Add your query as context: +`. Add your query as context: .. code-block:: typescript diff --git a/docs/ai/quickstart_fastapi_ai.rst b/docs/ai/quickstart_fastapi_ai.rst new file mode 100644 index 00000000000..9708e1fc9ae --- /dev/null +++ b/docs/ai/quickstart_fastapi_ai.rst @@ -0,0 +1,346 @@ +.. _ref_quickstart_ai: + +====================== +Using the built-in RAG +====================== + +.. edb:split-section:: + + In this section we'll use |Gel|'s built-in vector search and + retrieval-augmented generation capabilities to decorate our flashcard app + with a couple AI features. We're going to create a ``/fetch_similar`` + endpoint that's going to look up flashcards similar to a text search query, + as well as a ``/fetch_rag`` endpoint that's going to enable us to talk to + an LLM about the content of our flashcard deck. + + We're going to start with the same schema we left off with in the primary + quickstart. + + + .. code-block:: sdl + :caption: dbschema/default.gel + + module default { + abstract type Timestamped { + required created_at: datetime { + default := datetime_of_statement(); + }; + required updated_at: datetime { + default := datetime_of_statement(); + }; + } + + type Deck extending Timestamped { + required name: str; + description: str; + cards := ( + select . configure current database + insert ext::ai::OpenAIProviderConfig { + secret := 'sk-....', + }; + + +.. edb:split-section:: + + Once last thing before we move on. Let's add some sample data to give the + embedding model something to work with. You can copy and run this command + in the terminal, or come up with your own sample data. + + + .. code-block:: edgeql + :class: collapsible + + $ cat << 'EOF' | gel query --file - + with deck := ( + insert Deck { + name := 'Smelly Cheeses', + description := 'To impress everyone with stinky cheese trivia.' + } + ) + for card_data in {( + 1, + 'Époisses de Bourgogne', + 'Known as the "king of cheeses", this French cheese is so pungent it\'s banned on public transport in France. Washed in brandy, it becomes increasingly funky as it ages. Orange-red rind, creamy interior.' + ), ( + 2, + 'Vieux-Boulogne', + 'Officially the smelliest cheese in the world according to scientific studies. This northern French cheese has a reddish-orange rind from being washed in beer. Smooth, creamy texture with a powerful aroma.' + ), ( + 3, + 'Durian Cheese', + 'This Malaysian creation combines durian fruit with cheese, creating what some consider the ultimate "challenging" dairy product. Combines the pungency of blue cheese with durian\'s notorious aroma.' + ), ( + 4, + 'Limburger', + 'German cheese famous for its intense smell, often compared to foot odor due to the same bacteria. Despite its reputation, has a surprisingly mild taste with notes of mushroom and grass.' + ), ( + 5, + 'Roquefort', + 'The "king of blue cheeses", aged in limestone caves in southern France. Contains Penicillium roqueforti mold. Strong, tangy, and salty with a crumbly texture. Legend says it was discovered when a shepherd left his lunch in a cave.' + ), ( + 6, + 'What makes washed-rind cheeses so smelly?', + 'The process of washing cheese rinds in brine, alcohol, or other solutions promotes the growth of Brevibacterium linens, the same bacteria responsible for human body odor. This bacteria contributes to both the orange color and distinctive aroma.' + ), ( + 7, + 'Stinking Bishop', + 'Named after the Stinking Bishop pear (not a religious figure). This English cheese is washed in perry made from these pears. Known for its powerful aroma and sticky, pink-orange rind. Gained fame after being featured in Wallace & Gromit.' + )} + union ( + insert Card { + deck := deck, + order := card_data.0, + front := card_data.1, + back := card_data.2 + } + ); + EOF + + +.. edb:split-section:: + + Now we can finally start producing embedding vectors. Since |Gel| is fully + aware of when your data gets inserted, updated and deleted, it's perfectly + equipped to handle all the tedious work of keeping those vectors up to + date. All that's left for us is to create a special ``deferred index`` on + the data we would like to perform similarity search on. + + + .. code-block:: sdl-diff + :caption: dbschema/default.gel + + using extension ai; + + module default { + abstract type Timestamped { + required created_at: datetime { + default := datetime_of_statement(); + }; + required updated_at: datetime { + default := datetime_of_statement(); + }; + } + + type Deck extending Timestamped { + required name: str; + description: str; + cards := ( + select .>$embedding_vector)", + + embedding_vector=embedding_vector, + + ) + + + return similar_cards + + +.. edb:split-section:: + + Let's test the endpoint to see that everything works the way we expect. + + + .. code-block:: bash + + $ curl -X 'GET' \ + 'http://localhost:8000/fetch_similar?query=the%20stinkiest%20cheese' \ + -H 'accept: application/json' + + +.. edb:split-section:: + + Finally, let's create the second endpoint we mentioned, called + ``/fetch_rag``. We'll be able to use this one to, for example, ask an LLM + to quiz us on the contents of our deck. + + The RAG feature is represented in the Python binding with the ``query_rag`` + method of the ``GelRAG`` class. To use it, we're going to instantiate the + class and call the method... And that's it! + + + .. code-block:: python-diff + :caption: main.py + + import gel + import gel.ai + + from fastapi import FastAPI + + + client = gel.create_async_client() + + app = FastAPI() + + + @app.get("/fetch_similar") + async def fetch_similar_cards(query: str): + rag = await gel.ai.create_async_rag_client(client, model="gpt-4-turbo-preview") + embedding_vector = await rag.generate_embeddings( + query, model="text-embedding-3-small" + ) + + similar_cards = await client.query( + "select ext::ai::search(Card, >$embedding_vector)", + embedding_vector=embedding_vector, + ) + + return similar_cards + + + + @app.get("/fetch_rag") + + async def fetch_rag_response(query: str): + + rag = await gel.ai.create_async_rag_client(client, model="gpt-4-turbo-preview") + + response = await rag.query_rag( + + message=query, + + context=gel.ai.QueryContext(query="select Card"), + + ) + + return response + + +.. edb:split-section:: + + Let's test the endpoint to see if it works: + + + .. code-block:: bash + + $ curl -X 'GET' \ + 'http://localhost:8000/fetch_rag?query=what%20cheese%20smells%20like%20feet' \ + -H 'accept: application/json' + + +.. edb:split-section:: + + Congratulations! We've now implemented AI features in our flashcards app. + Of course, there's more to learn when it comes to using the AI extension. + Make sure to check out the Reference manual, or build an LLM-powered search + bot from the ground up with the FastAPI Gel AI tutorial. diff --git a/docs/ai/reference.rst b/docs/ai/reference.rst deleted file mode 100644 index 85557ff33fb..00000000000 --- a/docs/ai/reference.rst +++ /dev/null @@ -1,671 +0,0 @@ -.. _ref_ai_reference: - -======= -ext::ai -======= - -To activate |Gel| AI functionality, you can use the :ref:`extension -` mechanism: - -.. code-block:: sdl - - using extension ai; - - -.. _ref_ai_reference_config: - -Configuration -============= - -Use the ``configure`` command to set configuration for the AI extension. Update -the values using the ``configure session`` or the ``configure current branch`` -command depending on the scope you prefer: - -.. code-block:: edgeql-repl - - db> configure current branch - ... set ext::ai::Config::indexer_naptime := '0:00:30'; - OK: CONFIGURE DATABASE - -The only property available currently is ``indexer_naptime`` which specifies -the minimum delay between deferred ``ext::ai::index`` indexer runs on any given -branch. - -Examine the ``extensions`` link of the ``cfg::Config`` object to check the -current config values: - -.. code-block:: edgeql-repl - - db> select cfg::Config.extensions[is ext::ai::Config]{*}; - { - ext::ai::Config { - id: 1a53f942-d7ce-5610-8be2-c013fbe704db, - indexer_naptime: '0:00:30' - } - } - -You may also restore the default config value using ``configure session -reset`` if you set it on the session or ``configure current branch reset`` -if you set it on the branch: - -.. code-block:: edgeql-repl - - db> configure current branch reset ext::ai::Config::indexer_naptime; - OK: CONFIGURE DATABASE - - -Providers ---------- - -Provider configs are required for AI indexes (for embedding generation) and for -RAG (for text generation). They may be added via :ref:`ref_cli_gel_ui` or by -via EdgeQL: - -.. code-block:: edgeql - - configure current branch - insert ext::ai::OpenAIProviderConfig { - secret := 'sk-....', - }; - -The extension makes available types for each provider and for a custom provider -compatible with one of the supported API styles. - -* ``ext::ai::OpenAIProviderConfig`` -* ``ext::ai::MistralProviderConfig`` -* ``ext::ai::AnthropicProviderConfig`` -* ``ext::ai::CustomProviderConfig`` - -All provider types require the ``secret`` property be set with a string -containing the secret provided by the AI vendor. Other properties may -optionally be set: - -* ``name``- A unique provider name -* ``display_name``- A human-friendly provider name -* ``api_url``- The provider's API URL -* ``client_id``- ID for the client provided by model API vendor - -In addition to the required ``secret`` property, -``ext::ai::CustomProviderConfig requires an ``api_style`` property be set. -Available values are ``ext::ai::ProviderAPIStyle.OpenAI`` and -``ext::ai::ProviderAPIStyle.Anthropic``. - -Prompts -------- - -You may add prompts either via :ref:`ref_cli_gel_ui` or via EdgeQL. Here's -an example of how you might add a prompt with a single message: - -.. code-block:: edgeql - - insert ext::ai::ChatPrompt { - name := 'test-prompt', - messages := ( - insert ext::ai::ChatPromptMessage { - participant_role := ext::ai::ChatParticipantRole.System, - content := "Your message content" - } - ) - }; - -``participant_role`` may be any of these values: - -* ``ext::ai::ChatParticipantRole.System`` -* ``ext::ai::ChatParticipantRole.User`` -* ``ext::ai::ChatParticipantRole.Assistant`` -* ``ext::ai::ChatParticipantRole.Tool`` - -``ext::ai::ChatPromptMessage`` also has a ``participant_name`` property which -is an optional ``str``. - - -.. _ref_guide_ai_reference_index: - -Index -===== - -The ``ext::ai::index`` creates a deferred semantic similarity index of an -expression on a type. - -.. code-block:: sdl-diff - - module default { - type Astronomy { - content: str; - + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') - + on (.content); - } - }; - -It can accept several named arguments: - -* ``embedding_model``- The name of the model to use for embedding generation as - a string. - - .. _ref_ai_reference_embedding_models: - - You may use any of these pre-configured embedding generation models: - - **OpenAI** - - * ``text-embedding-3-small`` - * ``text-embedding-3-large`` - * ``text-embedding-ada-002`` - - `Learn more about the OpenAI embedding models `__ - - **Mistral** - - * ``mistral-embed`` - - `Learn more about the Mistral embedding model `__ -* ``distance_function``- The function to use for determining semantic - similarity. Default: ``ext::ai::DistanceFunction.Cosine`` - - The distance function may be any of these: - - * ``ext::ai::DistanceFunction.Cosine`` - * ``ext::ai::DistanceFunction.InnerProduct`` - * ``ext::ai::DistanceFunction.L2`` -* ``index_type``- The type of index to create. Currently the only option is the - default: ``ext::ai::IndexType.HNSW``. -* ``index_parameters``- A named tuple of additional index parameters: - - * ``m``- The maximum number of edges of each node in the graph. Increasing - can increase the accuracy of searches at the cost of index size. Default: - ``32`` - * ``ef_construction``- Dictates the depth and width of the search when - building the index. Higher values can lead to better connections and more - accurate results at the cost of time and resource usage when building the - index. Default: ``100`` - - -When indexes aren't working… ----------------------------- - -If you find your queries are not returning the expected results, try -inspecting your instance logs. On a |Gel| Cloud instance, use the "Logs" -tab in your instance dashboard. On local or :ref:`CLI-linked remote -instances `, use :gelcmd:`instance logs -I -`. You may find the problem there. - -Providers impose rate limits on their APIs which can often be the source of -AI index problems. If index creation hits a rate limit, Gel will wait -the ``indexer_naptime`` (see the docs on :ref:`ext::ai configuration -`) and resume index creation. - -If your indexed property contains values that exceed the token limit for a -single request, you may consider truncating the property value in your -index expression. You can do this with a string by slicing it: - -.. code-block:: sdl - - module default { - type Astronomy { - content: str; - deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') - on (.content[0:10000]); - } - }; - -This example will slice the first 10,000 characters of the ``content`` -property for indexing. - -Tokens are not equivalent to characters. For OpenAI embedding generation, -you may test values via `OpenAI's web-based tokenizer -`__. You may alternatively download -the library OpenAI uses for tokenization from that same page if you prefer. -By testing, you can get an idea how much of your content can be sent for -indexing. - - -Functions -========= - -.. list-table:: - :class: funcoptable - - * - :eql:func:`ext::ai::to_context` - - :eql:func-desc:`ext::ai::to_context` - - * - :eql:func:`ext::ai::search` - - :eql:func-desc:`ext::ai::search` - - ------------- - - -.. eql:function:: ext::ai::to_context(object: anyobject) -> str - - Evaluates the expression of an :ref:`ai::index - ` on the passed object and returns it. - - This can be useful for confirming the basis of embedding generation for a - particular object or type. - - Given this schema: - - .. code-block:: sdl - - module default { - type Astronomy { - topic: str; - content: str; - deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') - on (.topic ++ ' ' ++ .content); - } - }; - - and with these inserts: - - .. code-block:: edgeql-repl - - db> insert Astronomy { - ... topic := 'Mars', - ... content := 'Skies on Mars are red.' - ... } - db> insert Astronomy { - ... topic := 'Earth', - ... content := 'Skies on Earth are blue.' - ... } - - ``to_context`` returns these results: - - .. code-block:: edgeql-repl - - db> select ext::ai::to_context(Astronomy); - {'Mars Skies on Mars are red.', 'Earth Skies on Earth are blue.'} - db> select ext::ai::to_context((select Astronomy limit 1)); - {'Mars Skies on Mars are red.'} - - ------------- - - -.. eql:function:: ext::ai::search( \ - object: anyobject, \ - query: array \ - ) -> optional tuple - - Search an object using its :ref:`ai::index ` - index. - - Returns objects that match the specified semantic query and the - similarity score. - - .. note:: - - The ``query`` argument should *not* be a textual query but the - embeddings generated *from* a textual query. To have |Gel| generate - the query for you along with a text response, try :ref:`our built-in - RAG `. - - .. code-block:: edgeql-repl - - db> with query := >$query - ... select ext::ai::search(Knowledge, query); - { - ( - object := default::Knowledge {id: 9af0d0e8-0880-11ef-9b6b-4335855251c4}, - distance := 0.20410746335983276 - ), - ( - object := default::Knowledge {id: eeacf638-07f6-11ef-b9e9-57078acfce39}, - distance := 0.7843298847773637 - ), - ( - object := default::Knowledge {id: f70863c6-07f6-11ef-b9e9-3708318e69ee}, - distance := 0.8560434728860855 - ), - } - - -HTTP endpoints -============== - -Use the AI extension's HTTP endpoints to perform retrieval-augmented generation -using your AI indexes or to generate embeddings against a model of your choice. - -.. note:: - - All |Gel| server HTTP endpoints require :ref:`authentication - `. By default, you may use `HTTP Basic Authentication - `_ - with your Gel username and password. - - -RAG ---- - -``POST``: ``https://:/branch//ai/rag`` - -Responds with text generated by the specified text generation model in response -to the provided query. - - -Request -^^^^^^^ - -Make a ``POST`` request to the endpoint with a JSON body. The body may have -these properties: - -* ``model`` (string, required): The name of the text generation model to use. - - .. _ref_ai_reference_text_generation_models: - - You may use any of these text generation models: - - **OpenAI** - - * ``gpt-3.5-turbo`` - * ``gpt-4-turbo-preview`` - - `Learn more about the OpenAI text generation models `__ - - **Mistral** - - * ``mistral-small-latest`` - * ``mistral-medium-latest`` - * ``mistral-large-latest`` - - `Learn more about the Mistral text generation models `__ - - **Anthropic** - - * ``claude-3-haiku-20240307`` - * ``claude-3-sonnet-20240229`` - * ``claude-3-opus-20240229`` - - `Learn more about the Athropic text generation models `__ - -* ``query`` (string, required): The query string use as the basis for text - generation. - -* ``context`` (object, required): Settings that define the context of the - query. - - * ``query`` (string, required): Specifies an expression to determine the - relevant objects and index to serve as context for text generation. You may - set this to any expression that produces a set of objects, even if it is - not a standalone query. - - * ``variables`` (object, optional): A dictionary of variables for use in the - context query. - - * ``globals`` (object, optional): A dictionary of globals for use in the - context query. - - * ``max_object_count`` (int, optional): Maximum number of objects to return; - default is 5. - -* ``stream`` (boolean, optional): Specifies whether the response should be - streamed. Defaults to false. - -* ``prompt`` (object, optional): Settings that define a prompt. Omit to use the - default prompt. - - You may specify an existing prompt by its ``name`` or ``id``, you may define - a custom prompt inline by sending an array of objects, or you may do both to - augment an existing prompt with additional custom messages. - - * ``name`` (string, optional) or ``id`` (string, optional): The ``name`` or - ``id`` of an existing custom prompt to use. Provide only one of these if - you want to use or start from an existing prompt. - - * ``custom`` (array of objects, optional): Custom prompt messages, each - containing a ``role`` and ``content``. If no ``name`` or ``id`` was - provided, the custom messages provided here become the prompt. If one of - those was provided, these messages will be added to that existing prompt. - -**Example request** - -.. code-block:: - - curl --user : --json '{ - "query": "What color is the sky on Mars?", - "model": "gpt-4-turbo-preview", - "context": {"query":"Knowledge"} - }' http://:/branch/main/ai/rag - - -Response -^^^^^^^^ - -**Example successful response** - -* **HTTP status**: 200 OK -* **Content-Type**: application/json -* **Body**: - - .. code-block:: json - - {"response": "The sky on Mars is red."} - -**Example error response** - -* **HTTP status**: 400 Bad Request -* **Content-Type**: application/json -* **Body**: - - .. code-block:: json - - { - "message": "missing required 'query' in request 'context' object", - "type": "BadRequestError" - } - - -Streaming response (SSE) -^^^^^^^^^^^^^^^^^^^^^^^^ - -When the ``stream`` parameter is set to ``true``, the server uses `Server-Sent -Events -`__ -(SSE) to stream responses. Here is a detailed breakdown of the typical -sequence and structure of events in a streaming response: - -* **HTTP Status**: 200 OK -* **Content-Type**: text/event-stream -* **Cache-Control**: no-cache - -The stream consists of a sequence of five events, each encapsulating part of -the response in a structured format: - -1. **Message start** - - * Event type: ``message_start`` - - * Data: Starts a message, specifying identifiers and roles. - - .. code-block:: json - - { - "type": "message_start", - "message": { - "id": "", - "role": "assistant", - "model": "" - } - } - -2. **Content block start** - - * Event type: ``content_block_start`` - - * Data: Marks the beginning of a new content block. - - .. code-block:: json - - { - "type": "content_block_start", - "index": 0, - "content_block": { - "type": "text", - "text": "" - } - } - -3. **Content block delta** - - * Event type: ``content_block_delta`` - - * Data: Incrementally updates the content, appending more text to the - message. - - .. code-block:: json - - { - "type": "content_block_delta", - "index": 0, - "delta": { - "type": "text_delta", - "text": "The" - } - } - - Subsequent ``content_block_delta`` events add more text to the message. - -4. **Content block stop** - - * Event type: ``content_block_stop`` - - * Data: Marks the end of a content block. - - .. code-block:: json - - { - "type": "content_block_stop", - "index": 0 - } - -5. **Message stop** - - * Event type: ``message_stop`` - - * Data: Marks the end of the message. - - .. code-block:: json - - {"type": "message_stop"} - -Each event is sent as a separate SSE message, formatted as shown above. The -connection is closed after all events are sent, signaling the end of the -stream. - -**Example SSE response** - -.. code-block:: - - event: message_start - data: {"type": "message_start", "message": {"id": "chatcmpl-9MzuQiF0SxUjFLRjIdT3mTVaMWwiv", "role": "assistant", "model": "gpt-4-0125-preview"}} - - event: content_block_start - data: {"type": "content_block_start","index":0,"content_block":{"type":"text","text":""}} - - event: content_block_delta - data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": "The"}} - - event: content_block_delta - data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " skies"}} - - event: content_block_delta - data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " on"}} - - event: content_block_delta - data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " Mars"}} - - event: content_block_delta - data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " are"}} - - event: content_block_delta - data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " red"}} - - event: content_block_delta - data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": "."}} - - event: content_block_stop - data: {"type": "content_block_stop","index":0} - - event: message_delta - data: {"type": "message_delta", "delta": {"stop_reason": "stop"}} - - event: message_stop - data: {"type": "message_stop"} - - -Embeddings ----------- - -``POST``: ``https://:/branch//ai/embeddings`` - -Responds with embeddings generated by the specified embeddings model in -response to the provided input. - -Request -^^^^^^^ - -Make a ``POST`` request to the endpoint with a JSON body. The body may have -these properties: - -* ``input`` (array of strings or a single string, required): The text to use as - the basis for embeddings generation. - -* ``model`` (string, required): The name of the embedding model to use. You may - use any of the supported :ref:`embedding models - `. - -**Example request** - -.. code-block:: - - curl --user : --json '{ - "input": "What color is the sky on Mars?", - "model": "text-embedding-3-small" - }' http://localhost:10931/branch/main/ai/embeddings - - -Response -^^^^^^^^ - -**Example successful response** - -* **HTTP status**: 200 OK -* **Content-Type**: application/json -* **Body**: - - -.. code-block:: json - - { - "object": "list", - "data": [ - { - "object": "embedding", - "index": 0, - "embedding": [-0.009434271, 0.009137661] - } - ], - "model": "text-embedding-3-small", - "usage": { - "prompt_tokens": 8, - "total_tokens": 8 - } - } - -.. note:: - - The ``embedding`` property is shown here with only two values for brevity, - but an actual response would contain many more values. - -**Example error response** - -* **HTTP status**: 400 Bad Request -* **Content-Type**: application/json -* **Body**: - - .. code-block:: json - - { - "message": "missing or empty required \"model\" value in request", - "type": "BadRequestError" - } diff --git a/docs/ai/reference_extai.rst b/docs/ai/reference_extai.rst new file mode 100644 index 00000000000..69c0ab34453 --- /dev/null +++ b/docs/ai/reference_extai.rst @@ -0,0 +1,491 @@ +.. _ref_ai_extai_reference: + +============ +AI Extension +============ + +This reference documents the |Gel| AI extension's components, configuration +options, and APIs. + + +Enabling the Extension +====================== + +The AI extension can be enabled using the :ref:`extension ` mechanism: + +.. code-block:: sdl + + using extension ai; + +Configuration +============= + +The AI extension can be configured using ``configure session`` or ``configure current branch``: + +.. code-block:: edgeql + + configure current branch + set ext::ai::Config::indexer_naptime := 'PT30S'; + +Configuration Properties +------------------------ + +* ``indexer_naptime``: Duration + Specifies minimum delay between deferred ``ext::ai::index`` indexer runs. + +View current configuration: + +.. code-block:: edgeql + + select cfg::Config.extensions[is ext::ai::Config]{*}; + +Reset configuration: + +.. code-block:: edgeql + + configure current branch reset ext::ai::Config::indexer_naptime; + + +.. _ref_ai_extai_reference_ui: + +UI +== + +The AI section of the UI can be accessed via the sidebar after the extension +has been enabled in the schema. It provides ways to manage provider +configurations and RAG prompts, as well as try out different settings in the +playground. + +Playground tab +-------------- + +Provides an interactive environment for testing and configuring the built-in +RAG. + +.. image:: images/ui_playground.png + :alt: Screenshot of the Playground tab of the UI depicting an empty message window and three input fields set with default values. + :width: 100% + +Components: + +* Message window: Displays conversation history between the user and the LLM. +* Model: Dropdown menu for selecting the text generation model. +* Prompt: Dropdown menu for selecting the RAG prompt template. +* Context Query: Input field for entering an EdgeQL expression returning a set of objects with AI indexes. + + +Prompts tab +----------- + +Provides ways to manage system prompts used in the built-in RAG. + +.. image:: images/ui_prompts.png + :alt: Screenshot of the Prompts tab of the UI depicting an expanded prompt configuration menu. + :width: 100% + +Providers tab +------------- + +Enables management of API configurations for AI API providers. + +.. image:: images/ui_providers.png + :alt: Screenshot of the Providers tab of the UI depicting an expanded provider configuration menu. + :width: 100% + + +.. _ref_ai_extai_reference_index: + +Index +===== + +The ``ext::ai::index`` creates a deferred semantic similarity index of an +expression on a type. + +.. code-block:: sdl-diff + + module default { + type Astronomy { + content: str; + + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') + + on (.content); + } + }; + + +Parameters: + +* ``embedding_model``- The name of the model to use for embedding generation as + a string. +* ``distance_function``- The function to use for determining semantic + similarity. Default: ``ext::ai::DistanceFunction.Cosine`` +* ``index_type``- The type of index to create. Currently the only option is the + default: ``ext::ai::IndexType.HNSW``. +* ``index_parameters``- A named tuple of additional index parameters: + + * ``m``- The maximum number of edges of each node in the graph. Increasing + can increase the accuracy of searches at the cost of index size. Default: + ``32`` + * ``ef_construction``- Dictates the depth and width of the search when + building the index. Higher values can lead to better connections and more + accurate results at the cost of time and resource usage when building the + index. Default: ``100`` + +* ``dimensions``: int64 (Optional) - Embedding dimensions +* ``truncate_to_max``: bool (Default: False) + + +Built-in resources +================== + +.. _ref_ai_extai_reference_embedding_models: + +Embedding models +---------------- + +OpenAI (`documentation `__) + +* ``text-embedding-3-small`` +* ``text-embedding-3-large`` +* ``text-embedding-ada-002`` + +Mistral (`documentation `__) + +* ``mistral-embed`` + + +.. _ref_ai_extai_reference_text_generation_models: + +Text generation models +---------------------- + +OpenAI (`documentation `__) + +* ``gpt-3.5-turbo`` +* ``gpt-4-turbo-preview`` + +Mistral (`documentation `__) + +* ``mistral-small-latest`` +* ``mistral-medium-latest`` +* ``mistral-large-latest`` + +Anthropic (`documentation `__) + +* ``claude-3-haiku-20240307`` +* ``claude-3-sonnet-20240229`` +* ``claude-3-opus-20240229`` + + +Functions +========= + +.. list-table:: + :class: funcoptable + + * - :eql:func:`ext::ai::to_context` + - :eql:func-desc:`ext::ai::to_context` + + * - :eql:func:`ext::ai::search` + - :eql:func-desc:`ext::ai::search` + + +------------ + + +.. eql:function:: ext::ai::to_context(object: anyobject) -> str + + Returns the indexed expression value for an object with an ``ext::ai::index``. + + **Example**: + + Schema: + + .. code-block:: sdl + + module default { + type Astronomy { + topic: str; + content: str; + deferred index ext::ai::index(embedding_model := 'text-embedding-3-small') + on (.topic ++ ' ' ++ .content); + } + }; + + Data: + + .. code-block:: edgeql-repl + + db> insert Astronomy { + ... topic := 'Mars', + ... content := 'Skies on Mars are red.' + ... } + db> insert Astronomy { + ... topic := 'Earth', + ... content := 'Skies on Earth are blue.' + ... } + + Results of calling ``to_context``: + + .. code-block:: edgeql-repl + + db> select ext::ai::to_context(Astronomy); + + {'Mars Skies on Mars are red.', 'Earth Skies on Earth are blue.'} + + +------------ + + +.. eql:function:: ext::ai::search( \ + object: anyobject, \ + query: array \ + ) -> optional tuple + + Searches objects using their :ref:`ai::index + `. + + Returns tuples of (object, distance). + + .. note:: + + The ``query`` argument should *not* be a textual query but the + embeddings generated *from* a textual query. + + .. code-block:: edgeql-repl + + db> with query := >$query + ... select ext::ai::search(Knowledge, query); + + { + ( + object := default::Knowledge {id: 9af0d0e8-0880-11ef-9b6b-4335855251c4}, + distance := 0.20410746335983276 + ), + ( + object := default::Knowledge {id: eeacf638-07f6-11ef-b9e9-57078acfce39}, + distance := 0.7843298847773637 + ), + ( + object := default::Knowledge {id: f70863c6-07f6-11ef-b9e9-3708318e69ee}, + distance := 0.8560434728860855 + ), + } + + +Types +===== + +Provider Configuration Types +---------------------------- + +.. list-table:: + :class: funcoptable + + * - :eql:type:`ext::ai::ProviderAPIStyle` + - Enum defining supported API styles + + * - :eql:type:`ext::ai::ProviderConfig` + - Abstract base configuration for AI providers. + + +Provider configurations are required for AI indexes and RAG functionality. + +Example provider configuration: + +.. code-block:: edgeql + + configure current database + insert ext::ai::OpenAIProviderConfig { + secret := 'sk-....', + }; + +.. note:: + + All provider types require the ``secret`` property be set with a string + containing the secret provided by the AI vendor. + + +.. note:: + + ``ext::ai::CustomProviderConfig requires an ``api_style`` property be set. + + +--------- + + +.. eql:type:: ext::ai::ProviderAPIStyle + + Enum defining supported API styles: + + * ``OpenAI`` + * ``Anthropic`` + + +--------- + + +.. eql:type:: ext::ai::ProviderConfig + + Abstract base configuration for AI providers. + + Properties: + + * ``name``: str (Required) - Unique provider identifier + * ``display_name``: str (Required) - Human-readable name + * ``api_url``: str (Required) - Provider API endpoint + * ``client_id``: str (Optional) - Provider-supplied client ID + * ``secret``: str (Required) - Provider API secret + * ``api_style``: ProviderAPIStyle (Required) - Provider's API style + + Provider-specific types: + + * ``ext::ai::OpenAIProviderConfig`` + * ``ext::ai::MistralProviderConfig`` + * ``ext::ai::AnthropicProviderConfig`` + * ``ext::ai::CustomProviderConfig`` + + Each inherits from :eql:type:`ext::ai::ProviderConfig` with provider-specific defaults. + + +Model Types +----------- + +.. list-table:: + :class: funcoptable + + * - :eql:type:`ext::ai::Model` + - Abstract base type for AI models. + + * - :eql:type:`ext::ai::EmbeddingModel` + - Abstract type for embedding models. + + * - :eql:type:`ext::ai::TextGenerationModel` + - Abstract type for text generation models. + +--------- + +.. eql:type:: ext::ai::Model + + Abstract base type for AI models. + + Annotations: + * ``model_name`` - Model identifier + * ``model_provider`` - Provider identifier + +--------- + +.. eql:type:: ext::ai::EmbeddingModel + + Abstract type for embedding models. + + Annotations: + * ``embedding_model_max_input_tokens`` - Maximum tokens per input + * ``embedding_model_max_batch_tokens`` - Maximum tokens per batch + * ``embedding_model_max_output_dimensions`` - Maximum embedding dimensions + * ``embedding_model_supports_shortening`` - Input shortening support flag + + +--------- + +.. eql:type:: ext::ai::TextGenerationModel + + Abstract type for text generation models. + + Annotations: + * ``text_gen_model_context_window`` - Model's context window size + + +Indexing Types +-------------- + +.. list-table:: + :class: funcoptable + + * - :eql:type:`ext::ai::DistanceFunction` + - Enum for similarity metrics. + + * - :eql:type:`ext::ai::IndexType` + - Enum for index implementations. + +--------- + +.. eql:type:: ext::ai::DistanceFunction + + Enum for similarity metrics. + + * ``Cosine`` + * ``InnerProduct`` + * ``L2`` + +--------- + +.. eql:type:: ext::ai::IndexType + + Enum for index implementations. + + * ``HNSW`` + + + +Prompt Types +------------ + +.. list-table:: + :class: funcoptable + + * - :eql:type:`ext::ai::ChatParticipantRole` + - Enum for chat roles. + + * - :eql:type:`ext::ai::ChatPromptMessage` + - Type for chat prompt messages. + + * - :eql:type:`ext::ai::ChatPrompt` + - Type for chat prompt configuration. + +Example custom prompt configuration: + +.. code-block:: edgeql + + insert ext::ai::ChatPrompt { + name := 'test-prompt', + messages := ( + insert ext::ai::ChatPromptMessage { + participant_role := ext::ai::ChatParticipantRole.System, + content := "Your message content" + } + ) + }; + + +--------- + +.. eql:type:: ext::ai::ChatParticipantRole + + Enum for chat roles. + + * ``System`` + * ``User`` + * ``Assistant`` + * ``Tool`` + +--------- + +.. eql:type:: ext::ai::ChatPromptMessage + + Type for chat prompt messages. + + Properties: + * ``participant_role``: ChatParticipantRole (Required) + * ``participant_name``: str (Optional) + * ``content``: str (Required) + +--------- + +.. eql:type:: ext::ai::ChatPrompt + + Type for chat prompt configuration. + + Properties: + * ``name``: str (Required) + * ``messages``: set of ChatPromptMessage (Required) + diff --git a/docs/ai/reference_http.rst b/docs/ai/reference_http.rst new file mode 100644 index 00000000000..44d86a95bd4 --- /dev/null +++ b/docs/ai/reference_http.rst @@ -0,0 +1,382 @@ +.. _ref_ai_http_reference: + +===================== +AI HTTP API Reference +===================== + +:edb-alt-title: AI Extension HTTP API + +.. note:: + + All |Gel| server HTTP endpoints require :ref:`authentication + `, such as `HTTP Basic Authentication + `_ + with Gel username and password. + + +Embeddings +========== + +``POST``: ``https://:/branch//ai/embeddings`` + +Generates text embeddings using the specified embeddings model. + + +Request headers +--------------- + +* ``Content-Type: application/json`` (required) + + +Request body +------------ + +.. code-block:: json + + { + "model": string, // Required: Name of the embedding model + "inputs": string[], // Required: Array of texts to embed + "dimensions": number, // Optional: Number of dimensions to truncate to + "user": string // Optional: User identifier + } + +* ``input`` (array of strings or a single string, required): The text to use as + the basis for embeddings generation. + +* ``model`` (string, required): The name of the embedding model to use. You may + use any of the supported :ref:`embedding models + `. + + +Example request +--------------- + +.. code-block:: bash + + curl --user : --json '{ + "input": "What color is the sky on Mars?", + "model": "text-embedding-3-small" + }' http://localhost:10931/branch/main/ai/embeddings + + +Response +-------- + +* **HTTP status**: 200 OK +* **Content-Type**: application/json +* **Body**: + + +.. code-block:: json + + { + "object": "list", + "data": [ + { + "object": "embedding", + "index": 0, + "embedding": [-0.009434271, 0.009137661] + } + ], + "model": "text-embedding-3-small", + "usage": { + "prompt_tokens": 8, + "total_tokens": 8 + } + } + +.. note:: + + The ``embedding`` property is shown here with only two values for brevity, + but an actual response would contain many more values. + + +Error response +-------------- + +* **HTTP status**: 400 Bad Request +* **Content-Type**: application/json +* **Body**: + + .. code-block:: json + + { + "message": "missing or empty required \"model\" value in request", + "type": "BadRequestError" + } + +RAG +=== + +``POST``: ``https://:/branch//ai/rag`` + +Performs retrieval-augmented text generation using the specified model based on +the provided text query and the database content selected using similarity +search. + + +Request headers +--------------- + +* ``Content-Type: application/json`` (required) + + +Request body +------------ + +.. code-block:: json + + { + "context": { + "query": string, // Required: EdgeQL query for context retrieval + "variables": object, // Optional: Query variables + "globals": object, // Optional: Query globals + "max_object_count": number // Optional: Max objects to retrieve (default: 5) + }, + "model": string, // Required: Name of the generation model + "query": string, // Required: User query + "stream": boolean, // Optional: Enable streaming (default: false) + "prompt": { + "name": string, // Optional: Name of predefined prompt + "id": string, // Optional: ID of predefined prompt + "custom": [ // Optional: Custom prompt messages + { + "role": string, // "system"|"user"|"assistant"|"tool" + "content": string|object, + "tool_call_id": string, + "tool_calls": array + } + ] + }, + "temperature": number, // Optional: Sampling temperature + "top_p": number, // Optional: Nucleus sampling parameter + "max_tokens": number, // Optional: Maximum tokens to generate + "seed": number, // Optional: Random seed + "safe_prompt": boolean, // Optional: Enable safety features + "top_k": number, // Optional: Top-k sampling parameter + "logit_bias": object, // Optional: Token biasing + "logprobs": number, // Optional: Return token log probabilities + "user": string // Optional: User identifier + } + + +* ``model`` (string, required): The name of the text generation model to use. + + +* ``query`` (string, required): The query string use as the basis for text + generation. + +* ``context`` (object, required): Settings that define the context of the + query. + + * ``query`` (string, required): Specifies an expression to determine the + relevant objects and index to serve as context for text generation. You may + set this to any expression that produces a set of objects, even if it is + not a standalone query. + + * ``variables`` (object, optional): A dictionary of variables for use in the + context query. + + * ``globals`` (object, optional): A dictionary of globals for use in the + context query. + + * ``max_object_count`` (int, optional): Maximum number of objects to return; + default is 5. + +* ``stream`` (boolean, optional): Specifies whether the response should be + streamed. Defaults to false. + +* ``prompt`` (object, optional): Settings that define a prompt. Omit to use the + default prompt. + + You may specify an existing prompt by its ``name`` or ``id``, you may define + a custom prompt inline by sending an array of objects, or you may do both to + augment an existing prompt with additional custom messages. + + * ``name`` (string, optional) or ``id`` (string, optional): The ``name`` or + ``id`` of an existing custom prompt to use. Provide only one of these if + you want to use or start from an existing prompt. + + * ``custom`` (array of objects, optional): Custom prompt messages, each + containing a ``role`` and ``content``. If no ``name`` or ``id`` was + provided, the custom messages provided here become the prompt. If one of + those was provided, these messages will be added to that existing prompt. + + +Example request +--------------- + +.. code-block:: + + curl --user : --json '{ + "query": "What color is the sky on Mars?", + "model": "gpt-4-turbo-preview", + "context": {"query":"Knowledge"} + }' http://:/branch/main/ai/rag + + +Response +-------- + +* **HTTP status**: 200 OK +* **Content-Type**: application/json +* **Body**: + + .. code-block:: json + + {"response": "The sky on Mars is red."} + +Error response +-------------- + +* **HTTP status**: 400 Bad Request +* **Content-Type**: application/json +* **Body**: + + .. code-block:: json + + { + "message": "missing required 'query' in request 'context' object", + "type": "BadRequestError" + } + + +Streaming response (SSE) +------------------------ + +When the ``stream`` parameter is set to ``true``, the server uses `Server-Sent +Events +`__ +(SSE) to stream responses. Here is a detailed breakdown of the typical +sequence and structure of events in a streaming response: + +* **HTTP Status**: 200 OK +* **Content-Type**: text/event-stream +* **Cache-Control**: no-cache + +The stream consists of a sequence of five events, each encapsulating part of +the response in a structured format: + +1. **Message start** + + * Event type: ``message_start`` + + * Data: Starts a message, specifying identifiers and roles. + + .. code-block:: json + + { + "type": "message_start", + "message": { + "id": "", + "role": "assistant", + "model": "" + } + } + +2. **Content block start** + + * Event type: ``content_block_start`` + + * Data: Marks the beginning of a new content block. + + .. code-block:: json + + { + "type": "content_block_start", + "index": 0, + "content_block": { + "type": "text", + "text": "" + } + } + +3. **Content block delta** + + * Event type: ``content_block_delta`` + + * Data: Incrementally updates the content, appending more text to the + message. + + .. code-block:: json + + { + "type": "content_block_delta", + "index": 0, + "delta": { + "type": "text_delta", + "text": "The" + } + } + + Subsequent ``content_block_delta`` events add more text to the message. + +4. **Content block stop** + + * Event type: ``content_block_stop`` + + * Data: Marks the end of a content block. + + .. code-block:: json + + { + "type": "content_block_stop", + "index": 0 + } + +5. **Message stop** + + * Event type: ``message_stop`` + + * Data: Marks the end of the message. + + .. code-block:: json + + {"type": "message_stop"} + +Each event is sent as a separate SSE message, formatted as shown above. The +connection is closed after all events are sent, signaling the end of the +stream. + +**Example SSE response** + +.. code-block:: + :class: collapsible + + event: message_start + data: {"type": "message_start", "message": {"id": "chatcmpl-9MzuQiF0SxUjFLRjIdT3mTVaMWwiv", "role": "assistant", "model": "gpt-4-0125-preview"}} + + event: content_block_start + data: {"type": "content_block_start","index":0,"content_block":{"type":"text","text":""}} + + event: content_block_delta + data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": "The"}} + + event: content_block_delta + data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " skies"}} + + event: content_block_delta + data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " on"}} + + event: content_block_delta + data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " Mars"}} + + event: content_block_delta + data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " are"}} + + event: content_block_delta + data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": " red"}} + + event: content_block_delta + data: {"type": "content_block_delta","index":0,"delta":{"type": "text_delta", "text": "."}} + + event: content_block_stop + data: {"type": "content_block_stop","index":0} + + event: message_delta + data: {"type": "message_delta", "delta": {"stop_reason": "stop"}} + + event: message_stop + data: {"type": "message_stop"} + + diff --git a/docs/ai/python.rst b/docs/ai/reference_python.rst similarity index 84% rename from docs/ai/python.rst rename to docs/ai/reference_python.rst index 19d0af9a791..fc0b1cecb87 100644 --- a/docs/ai/python.rst +++ b/docs/ai/reference_python.rst @@ -1,87 +1,61 @@ -.. _ref_ai_python: +.. _ref_ai_python_reference: -====== -Python -====== +============= +AI Python API +============= -:edb-alt-title: Gel AI's Python package +:edb-alt-title: AI Extension Python API The ``gel.ai`` package is an optional binding of the AI extension in |Gel|. -To use the AI binding, you need to install ``gel`` Python package with the -``ai`` extra dependencies: .. code-block:: bash $ pip install 'gel[ai]' -Usage -===== +Blocking and async API +====================== -Start by importing ``gel`` and ``gel.ai``: +The AI binding is built on top of the regular |Gel| client objects, providing +both blocking and asynchronous versions of its API. + +**Blocking client example**: .. code-block:: python import gel import gel.ai - -Blocking --------- - -The AI binding is built on top of the regular |Gel| client objects, providing -both blocking and asynchronous versions of its API. For example, a blocking AI -client is initialized like this: - -.. code-block:: python - client = gel.create_client() - gpt4ai = gel.ai.create_ai( + gpt4ai = gel.ai.create_rag_client( client, model="gpt-4-turbo-preview" ) -Add your query as context: - -.. code-block:: python - astronomy_ai = gpt4ai.with_context( query="Astronomy" ) -The default text generation prompt will ask your selected provider to limit -answer to information provided in the context and will pass the queried -objects' AI index as context along with that prompt. - -Call your AI client's ``query_rag`` method, passing in a text query. - -.. code-block:: python - print( astronomy_ai.query_rag("What color is the sky on Mars?") ); -or stream back the results by using ``stream_rag`` instead: - -.. code-block:: python - for data in astronomy_ai.stream_rag("What color is the sky on Mars?"): print(data) -Async ------ - -To use an async client instead, do this: +**Async client example**: .. code-block:: python - import asyncio # alongside the Gel imports + import gel + import gel.ai + import asyncio client = gel.create_async_client() async def main(): - gpt4ai = await gel.ai.create_async_ai( + gpt4ai = await gel.ai.create_async_rag_client( client, model="gpt-4-turbo-preview" ) @@ -100,12 +74,12 @@ To use an async client instead, do this: asyncio.run(main()) -API reference -============= +Factory functions +================= -.. py:function:: create_ai(client, **kwargs) -> GelAI +.. py:function:: create_rag_client(client, **kwargs) -> RAGClient - Creates an instance of ``GelAI`` with the specified client and options. + Creates an instance of ``RAGClient`` with the specified client and options. This function ensures that the client is connected before initializing the AI with the specified options. @@ -114,7 +88,7 @@ API reference A |Gel| client instance. :param kwargs: - Keyword arguments that are passed to the ``AIOptions`` data class to + Keyword arguments that are passed to the ``RAGOptions`` data class to configure AI-specific options. These options are: * ``model``: The name of the model to be used. (required) @@ -122,9 +96,9 @@ API reference ``None`` will result in the client using the default prompt. (default: ``None``) -.. py:function:: create_async_ai(client, **kwargs) -> AsyncGelAI +.. py:function:: create_async_rag_client(client, **kwargs) -> AsyncRAGClient - Creates an instance of ``AsyncGelAI`` w/ the specified client & options. + Creates an instance of ``AsyncRAGClient`` w/ the specified client & options. This function ensures that the client is connected asynchronously before initializing the AI with the specified options. @@ -133,21 +107,17 @@ API reference An asynchronous |Gel| client instance. :param kwargs: - Keyword arguments that are passed to the ``AIOptions`` data class to + Keyword arguments that are passed to the ``RAGOptions`` data class to configure AI-specific options. These options are: * ``model``: The name of the model to be used. (required) * ``prompt``: An optional prompt to guide the model's behavior. (default: None) -AI client classes ------------------ - +Core classes +============ -BaseGelAI -^^^^^^^^^ - -.. py:class:: BaseGelAI +.. py:class:: BaseRAGClient The base class for |Gel| AI clients. @@ -158,7 +128,7 @@ BaseGelAI these methods are available on an AI client of either type. :ivar options: - An instance of :py:class:`AIOptions`, storing the AI options. + An instance of :py:class:`RAGOptions`, storing the RAG options. :ivar context: An instance of :py:class:`QueryContext`, storing the context for AI @@ -210,10 +180,7 @@ BaseGelAI objects returned by the query. -GelAI -^^^^^ - -.. py:class:: GelAI +.. py:class:: RAGClient A synchronous class for creating |Gel| AI clients. @@ -253,11 +220,17 @@ GelAI the query. If not provided, uses the default context of this AI client instance. +.. py:method:: generate_embeddings(*inputs: str, model: str) -> list[float] + + Generates embeddings for input texts. + + :param inputs: + Input texts. + :param model: + The embedding model to use -AsyncGelAI -^^^^^^^^^^ -.. py:class:: AsyncGelAI +.. py:class:: AsyncRAGClient An asynchronous class for creating |Gel| AI clients. @@ -301,9 +274,19 @@ AsyncGelAI the query. If not provided, uses the default context of this AI client instance. +.. py:method:: generate_embeddings(*inputs: str, model: str) -> list[float] + :noindex: -Other classes -------------- + Generates embeddings for input texts. + + :param inputs: + Input texts. + :param model: + The embedding model to use + + +Configuration classes +===================== .. py:class:: ChatParticipantRole @@ -343,9 +326,9 @@ Other classes role-specific content within the prompt. -.. py:class:: AIOptions +.. py:class:: RAGOptions - A data class for AI options, specifying model and prompt settings. + A data class for RAG options, specifying model and prompt settings. :ivar model: The name of the AI model. @@ -354,8 +337,8 @@ Other classes the model. :method derive(kwargs): - Creates a new instance of :py:class:`AIOptions` by merging existing options - with provided keyword arguments. Returns a new :py:class:`AIOptions` + Creates a new instance of :py:class:`RAGOptions` by merging existing options + with provided keyword arguments. Returns a new :py:class:`RAGOptions` instance with updated attributes. :param kwargs: @@ -414,3 +397,4 @@ Other classes :method to_httpx_request(): Converts the RAGRequest into a dictionary suitable for making an HTTP request using the httpx library. + diff --git a/docs/datamodel/access_policies.rst b/docs/datamodel/access_policies.rst index 184994804ad..e94e14013ea 100644 --- a/docs/datamodel/access_policies.rst +++ b/docs/datamodel/access_policies.rst @@ -7,12 +7,23 @@ Access Policies .. index:: access policy, object-level security, row-level security, RLS, allow, deny, using -Object types can contain security policies that restrict the set of objects -that can be selected, inserted, updated, or deleted by a particular query. -This is known as *object-level security* and it is similar in function to SQL's -row-level security. +Object types in |Gel| can contain security policies that restrict the set of +objects that can be selected, inserted, updated, or deleted by a particular +query. This is known as *object-level security* and is similar in function +to SQL's row-level security. -Let's start with a simple schema for a blog without any access policies. +When no access policies are defined, object-level security is not activated: +any properly authenticated client can carry out any operation on any object +in the database. Access policies allow you to ensure that the database itself +handles access control logic rather than having to implement it in every +application or service that connects to your database. + +Access policies can greatly simplify your backend code, centralizing access +control logic in a single place. They can also be extremely useful for +implementing AI agentic flows, where you want to have guardrails around +your data that agents can't break. + +We'll illustrate access policies in this document with this simple schema: .. code-block:: sdl @@ -25,37 +36,43 @@ Let's start with a simple schema for a blog without any access policies. required author: User; } -When no access policies are defined, object-level security is not activated. -Any properly authenticated client can carry out any operation on any object -in the database. At the moment, we would need to ensure that the app handles -the logic to restrict users from accessing other users' posts. Access -policies allow us to ensure that the database itself handles this logic, -thereby freeing us up from implementing access control in each and every -piece of software that accesses the data. .. warning:: - Once a policy is added to a particular object type, **all operations** - (``select``, ``insert``, ``delete``, ``update``, etc.) on any object of - that type are now *disallowed by default* unless specifically allowed by an - access policy! See the subsection on resolution order below for details. - -Defining a global -^^^^^^^^^^^^^^^^^ - -Global variables are the a convenient way to provide the context needed to -determine what sort of access should be allowed for a given object, as they -can be set and reset by the application as needed. - -To start, we'll add two global variables to our schema. We'll use one global -``uuid`` to represent the identity of the user executing the query, and an -enum for the other to represent the type of country that the user is currently -in. The enum represents three types of countries: those where the service has -not been rolled out, those with read-only access, and those with full access. -A global makes sense in this case because a user's current country is -context-specific: the same user who can access certain content in one country -might not be able to in another country due to different legal frameworks -(such as copyright length). + Once a policy is added to a particular object type, **all operations** + (``select``, ``insert``, ``delete``, ``update``, etc.) on any object of + that type are now *disallowed by default* unless specifically allowed by + an access policy! See :ref:`resolution order ` + below for details. + +Global variables +================ + +Global variables are a convenient way to set up the context for your access +policies. Gel's global variables are tightly integrated with the Gel's +data model, client APIs, EdgeQL and SQL, and the tooling around them. + +Global variables in Gel are not pre-defined. Users are free to define +as many globals in their schema as they want to represent the business +logic of their application. + +A common scenario is storing a ``current_user`` global representing +the user executing queries. We'd like to have a slightly more complex example +showing that you can use more than one global variable. Let's do that: + +* We'll use one *global* ``uuid`` to represent the identity of the user + executing the query. +* We'll have the ``Country`` *enum* to represent the type of country + that the user is currently in. The enum represents three types of + countries: those where the service has not been rolled out, those with + read-only access, and those with full access. +* We'll use the ``current_country`` *global* to represent the user's + current country. In our *example schema*, we want *country* to be + context-specific: the same user who can access certain content in one + country might not be able to in another country (let's imagine that's + due to different country-specific legal frameworks). + +Here is an illustration: .. code-block:: sdl-diff @@ -74,8 +91,7 @@ might not be able to in another country due to different legal frameworks required author: User; } -The value of these global variables is attached to the *client* you use to -execute queries. The exact API depends on which client library you're using: +You can set and reset these globals in Gel client libraries, for example: .. tabs:: @@ -83,11 +99,17 @@ execute queries. The exact API depends on which client library you're using: import createClient from 'gel'; - const client = createClient().withGlobals({ + const client = createClient(); + + // 'authedClient' will share the network connection with 'client', + // but will have the 'current_user' global set. + const authedClient = client.withGlobals({ current_user: '2141a5b4-5634-4ccc-b835-437863534c51', }); - await client.query(`select global current_user;`); + const result = await authedClient.query( + `select global current_user;`); + console.log(result); .. code-tab:: python @@ -100,6 +122,7 @@ execute queries. The exact API depends on which client library you're using: result = client.query(""" select global current_user; """) + print(result) .. code-tab:: go @@ -166,66 +189,53 @@ execute queries. The exact API depends on which client library you're using: .expect("Returning value"); -Defining a policy -^^^^^^^^^^^^^^^^^ +Defining policies +================= -Let's add two policies to our sample schema. +A policy example for our simple blog schema might look like: .. code-block:: sdl-diff - global current_user: uuid; - required global current_country: Country { - default := Country.None - } - scalar type Country extending enum; + global current_user: uuid; + required global current_country: Country { + default := Country.None + } + scalar type Country extending enum; - type User { - required email: str { constraint exclusive; } - } + type User { + required email: str { constraint exclusive; } + } - type BlogPost { - required title: str; - required author: User; + type BlogPost { + required title: str; + required author: User; - + access policy author_has_full_access - + allow all - + using (global current_user ?= .author.id - + and global current_country ?= Country.Full) { - + errmessage := "User does not have full access"; - + } - + access policy author_has_read_access - + allow select - + using (global current_user ?= .author.id - + and global current_country ?= Country.ReadOnly); - } + + access policy author_has_full_access + + allow all + + using (global current_user ?= .author.id + + and global current_country ?= Country.Full) { + + errmessage := "User does not have full access"; + + } -Let's break down the access policy syntax piece-by-piece. These policies grant -full read-write access (``all``) to the ``author`` of each ``BlogPost``, if -the author is in a country that allows full access to the service. Otherwise, -the same author will be restricted to either read-only access or no access at -all, depending on the country. + + access policy author_has_read_access + + allow select + + using (global current_user ?= .author.id + + and global current_country ?= Country.ReadOnly); + } -.. note:: +Explanation: + +- ``access policy `` introduces a new policy in an object type. +- ``allow all`` grants ``select``, ``insert``, ``update``, and ``delete`` + access if the condition passes. We also used a separate policy to allow + only ``select`` in some cases. +- ``using ()`` is a boolean filter restricting the set of objects to + which the policy applies. (We used the coalescing operator ``?=`` to + handle empty sets gracefully.) +- ``errmessage`` is an optional custom message to display in case of a write + violation. - We're using the *coalescing equality* operator ``?=`` because it returns - ``false`` even if one of its arguments is an empty set. - -- ``access policy``: The keyword used to declare a policy inside an object - type. -- ``author_has_full_access`` and ``author_has_read_access``: The names of these - policies; could be any string. -- ``allow``: The kind of policy; could be ``allow`` or ``deny`` -- ``all``: The set of operations being allowed/denied; a comma-separated list - of any number of the following: ``all``, ``select``, ``insert``, ``delete``, - ``update``, ``update read``, and ``update write``. -- ``using ()``: A boolean expression. Think of this as a ``filter`` - expression that defines the set of objects to which the policy applies. -- ``errmessage``: Here we have added an error message that will be shown in - case the policy expression returns ``false``. We could have added other - annotations of our own inside this code block instead of, or in addition - to ``errmessage``. - -Let's do some experiments. +Let's run some experiments in the REPL: .. code-block:: edgeql-repl @@ -242,58 +252,22 @@ Let's do some experiments. ... }; {default::BlogPost {id: e76afeae-03db-11ed-b346-fbb81f537ca6}} -We've created a ``User``, set the value of ``current_user`` to its ``id``, the -country to ``Country.Full``, and created a new ``BlogPost``. When we try to -select all ``BlogPost`` objects, we'll see the post we just created. - -.. code-block:: edgeql-repl - - db> select BlogPost; - {default::BlogPost {id: e76afeae-03db-11ed-b346-fbb81f537ca6}} - db> select count(BlogPost); - {1} - -Next, let's test what happens when the same user is in two other countries: -one that allows read-only access to our app, and another where we haven't -yet been given permission to roll out our service. +Because the user is in a "full access" country and the current user ID +matches the author, the new blog post is permitted. When the same user sets +``global current_country := Country.ReadOnly;``: .. code-block:: edgeql-repl db> set global current_country := Country.ReadOnly; OK: SET GLOBAL db> select BlogPost; - {default::BlogPost {id: dd274432-94ff-11ee-953e-0752e8ad3010}} + {default::BlogPost {id: e76afeae-03db-11ed-b346-fbb81f537ca6}} db> insert BlogPost { ... title := "My second post", ... author := (select User filter .id = global current_user) ... }; gel error: AccessPolicyError: access policy violation on insert of default::BlogPost (User does not have full access) - db> set global current_country := Country.None; - OK: SET GLOBAL - db> select BlogPost; - {} - -Note that for a ``select`` operation, the access policy works as a filter -by simply returning an empty set. Meanwhile, when attempting an ``insert`` -operation, the operation may or may not work and thus we have provided a -helpful error message in the access policy to give users a heads up on what -went wrong. - -Now let's move back to a country with full access, but set the -``global current_user`` to some other id: a new user that has yet to write -any blog posts. Now the number of ``BlogPost`` objects returned via -the ``count`` function is zero: - -.. code-block:: edgeql-repl - - db> set global current_country := Country.Full; - OK: SET GLOBAL - db> set global current_user := - ... 'd1c64b84-8e3c-11ee-86f0-d7ddecf3e9bd'; - OK: SET GLOBAL - db> select count(BlogPost); - {0} Finally, let's unset ``current_user`` and see how many blog posts are returned when we count them. @@ -314,76 +288,43 @@ When ``current_user`` has no value or has a different value from the But thanks to ``Country`` being set to ``Country.Full``, this user will be able to write a new blog post. -The access policies use global variables to define a "subgraph" of data that -is visible to a particular query. +**The bottom line:** access policies use global variables to define a +"subgraph" of data that is visible to your queries. + Policy types -^^^^^^^^^^^^ +============ -.. index:: accesss policy, select, insert, delete, update, update read, +.. index:: access policy, select, insert, delete, update, update read, update write, all -For the most part, the policy types correspond to EdgeQL's *statement types*: - -- ``select``: Applies to all queries; objects without a ``select`` permission - cannot be modified either. -- ``insert``: Applies to insert queries; executed *post-insert*. If an - inserted object violates the policy, the query will fail. -- ``delete``: Applies to delete queries. -- ``update``: Applies to update queries. - -Additionally, the ``update`` operation can be broken down into two -sub-policies: ``update read`` and ``update write``. +The types of policy rules map to the statement type in EdgeQL: -- ``update read``: This policy restricts *which* objects can be updated. It - runs *pre-update*; that is, this policy is executed before the updates have - been applied. As a result, an empty set is returned on an ``update read`` - when a query lacks access to perform the operation. -- ``update write``: This policy restricts *how* you update the objects; you - can think of it as a *post-update* validity check. As a result, an error - is returned on an ``update write`` when a query lacks access to perform - the operation. Preventing a ``User`` from transferring a ``BlogPost`` to - another ``User`` is one example of an ``update write`` access policy. - -Finally, there's an umbrella policy that can be used as a shorthand for all -the others. - -- ``all``: A shorthand policy that can be used to allow or deny full read/ - write permissions. Exactly equivalent to ``select, insert, update, delete``. +- ``select``: Controls which objects are visible to any query. +- ``insert``: Post-insert check. If the inserted object violates the policy, + the operation fails. +- ``delete``: Controls which objects can be deleted. +- ``update read``: Pre-update check on which objects can be updated at all. +- ``update write``: Post-update check for how objects can be updated. +- ``all``: Shorthand for granting or denying ``select, insert, update, + delete``. Resolution order -^^^^^^^^^^^^^^^^ - -An object type can contain an arbitrary number of access policies, including -several conflicting ``allow`` and ``deny`` policies. |Gel| uses a particular -algorithm for resolving these policies. - -.. figure:: images/ols.png - - The access policy resolution algorithm, explained with Venn diagrams. +================ -1. When no policies are defined on a given object type, all objects of that - type can be read or modified by any appropriately authenticated connection. +If multiple policies apply (some are ``allow`` and some are ``deny``), the +logic is: -2. Gel then applies all ``allow`` policies. Each policy grants a - *permission* that is scoped to a particular *set of objects* as defined by - the ``using`` clause. Conceptually, these permissions are merged with - the ``union`` / ``or`` operator to determine the set of allowable actions. +1. If there are no policies, access is allowed. +2. All ``allow`` policies collectively form a *union* / *or* of allowed sets. +3. All ``deny`` policies *subtract* from that union, overriding allows! +4. The final set of objects is the intersection of the above logic for each + operation: ``select, insert, update read, update write, delete``. -3. After the ``allow`` policies are resolved, the ``deny`` policies can be - used to carve out exceptions to the ``allow`` rules. Deny rules *supersede* - allow rules! As before, the set of objects targeted by the policy is - defined by the ``using`` clause. - -4. This results in the final access level: a set of objects targetable by each - of ``select``, ``insert``, ``update read``, ``update write``, and - ``delete``. - -Currently, by default the access policies affect the values visible -in expressions of *other* access -policies. This means that they can affect each other in various ways. Because -of this, great care needs to be taken when creating access policies based on -objects other than the ones they are defined on. For example: +By default, once you define any policy on an object type, you must explicitly +allow the operations you need. This is a common **pitfall** when you are +starting out with access policies (but you will develop an intuition for this +quickly). Let's look at an example: .. code-block:: sdl @@ -410,45 +351,34 @@ objects other than the ones they are defined on. For example: using (global current_user ?= .author.id); } -In the above schema only the admin will see a non-empty ``author`` link, -because only the admin can see any user objects at all. This means that -instead of making ``BlogPost`` visible to its author, all non-admin authors -won't be able to see their own posts. The above issue can be remedied by -making the current user able to see their own ``User`` record. - -.. _ref_datamodel_access_policies_nonrecursive: -.. _nonrecursive: +In the above schema only admins will see a non-empty ``author`` link when +running ``select BlogPost { author }``. Why? Because only admins can see +``User`` objects at all: ``admin_only`` policy is the only one defined on +the ``User`` type! -.. note:: +This means that instead of making ``BlogPost`` visible to its author, all +non-admin authors won't be able to see their own posts. The above issue can be +remedied by making the current user able to see their own ``User`` record. - Starting with |EdgeDB| 3.0, access policy restrictions will **not** apply to - any access policy expression. This means that when reasoning about access - policies it is no longer necessary to take other policies into account. - Instead, all data is visible for the purpose of *defining* an access - policy. - This change is being made to simplify reasoning about access policies and - to allow certain patterns to be express efficiently. Since those who have - access to modifying the schema can remove unwanted access policies, no - additional security is provided by applying access policies to each - other's expressions. +Interaction between policies +============================ - It is possible (and recommended) to enable this :ref:`future - ` behavior in |EdgeDB| 2.6 and later by adding the - following to the schema: ``using future nonrecursive_access_policies;`` +Policy expressions themselves do not take other policies into account +(since |EdgeDB| 3). This makes it easier to reason about policies. Custom error messages -^^^^^^^^^^^^^^^^^^^^^ +===================== .. index:: access policy, errmessage, using -When you run a query that attempts a write and is restricted by an access -policy, you will get a generic error message. +When an ``insert`` or ``update write`` violates an access policy, Gel will +raise a generic ``AccessPolicyError``: .. code-block:: - gel error: AccessPolicyError: access policy violation on insert of - + gel error: AccessPolicyError: access policy violation + on insert of .. note:: @@ -458,9 +388,8 @@ policy, you will get a generic error message. simply won't get the data that is being restricted. Other operations (``insert`` and ``update write``) will return an error message. -If you have multiple access policies, it can be useful to know which policy is -restricting your query and provide a friendly error message. You can do this -by adding a custom error message to your policy. +If multiple policies are in effect, it can be helpful to define a distinct +``errmessage`` in your policy: .. code-block:: sdl-diff @@ -499,134 +428,126 @@ will receive this error: gel error: AccessPolicyError: access policy violation on insert of default::User (Only admins may query Users) + Disabling policies -^^^^^^^^^^^^^^^^^^ +================== .. index:: apply_access_policies You may disable all access policies by setting the ``apply_access_policies`` :ref:`configuration parameter ` to ``false``. -You may also toggle access policies using the "Disable Access Policies" -checkbox in the "Config" dropdown in the Gel UI (accessible by running -the CLI command :gelcmd:`ui` from inside your project). This is the most -convenient way to temporarily disable access policies since it applies only to -your UI session. +You may also temporarily disable access policies using the Gel UI configuration +checkbox (or via :gelcmd:`ui`), which only applies to your UI session. +More examples +============= -Examples -^^^^^^^^ +Here are some additional patterns: -Blog posts are publicly visible if ``published`` but only writable by the -author. +1. Publicly visible blog posts, only writable by the author: -.. code-block:: sdl-diff - - global current_user: uuid; + .. code-block:: sdl-diff - type User { - required email: str { constraint exclusive; } - } + global current_user: uuid; - type BlogPost { - required title: str; - required author: User; - + required published: bool { default := false }; + type User { + required email: str { constraint exclusive; } + } - access policy author_has_full_access - allow all - using (global current_user ?= .author.id); - + access policy visible_if_published - + allow select - + using (.published); - } + type BlogPost { + required title: str; + required author: User; + + required published: bool { default := false }; -Blog posts are visible to friends but only modifiable by the author. + access policy author_has_full_access + allow all + using (global current_user ?= .author.id); + + access policy visible_if_published + + allow select + + using (.published); + } -.. code-block:: sdl-diff +2. Visible to friends, only modifiable by the author: - global current_user: uuid; + .. code-block:: sdl-diff - type User { - required email: str { constraint exclusive; } - + multi friends: User; - } + global current_user: uuid; - type BlogPost { - required title: str; - required author: User; + type User { + required email: str { constraint exclusive; } + + multi friends: User; + } - access policy author_has_full_access - allow all - using (global current_user ?= .author.id); - + access policy friends_can_read - + allow select - + using ((global current_user in .author.friends.id) ?? false); - } + type BlogPost { + required title: str; + required author: User; -Blog posts are publicly visible except to users that have been ``blocked`` by -the author. + access policy author_has_full_access + allow all + using (global current_user ?= .author.id); + + access policy friends_can_read + + allow select + + using ((global current_user in .author.friends.id) ?? false); + } -.. code-block:: sdl-diff +3. Publicly visible except to those blocked by the author: - type User { - required email: str { constraint exclusive; } - + multi blocked: User; - } + .. code-block:: sdl-diff - type BlogPost { - required title: str; - required author: User; + type User { + required email: str { constraint exclusive; } + + multi blocked: User; + } - access policy author_has_full_access - allow all - using (global current_user ?= .author.id); - + access policy anyone_can_read - + allow select; - + access policy exclude_blocked - + deny select - + using ((global current_user in .author.blocked.id) ?? false); - } + type BlogPost { + required title: str; + required author: User; + access policy author_has_full_access + allow all + using (global current_user ?= .author.id); + + access policy anyone_can_read + + allow select; + + access policy exclude_blocked + + deny select + + using ((global current_user in .author.blocked.id) ?? false); + } -"Disappearing" posts that become invisible after 24 hours. +4. "Disappearing" posts that become invisible after 24 hours: -.. code-block:: sdl-diff + .. code-block:: sdl-diff - type User { - required email: str { constraint exclusive; } - } + type User { + required email: str { constraint exclusive; } + } - type BlogPost { - required title: str; - required author: User; - + required created_at: datetime { - + default := datetime_of_statement() # non-volatile - + } + type BlogPost { + required title: str; + required author: User; + + required created_at: datetime { + + default := datetime_of_statement() # non-volatile + + } - access policy author_has_full_access - allow all - using (global current_user ?= .author.id); - + access policy hide_after_24hrs - + allow select - + using (datetime_of_statement() - .created_at < '24 hours'); - } + access policy author_has_full_access + allow all + using (global current_user ?= .author.id); + + access policy hide_after_24hrs + + allow select + + using ( + + datetime_of_statement() - .created_at < '24 hours' + + ); + } Super constraints -***************** - -Access policies support arbitrary EdgeQL and can be used to define "super -constraints". Policies on ``insert`` and ``update write`` can -be thought of as post-write "validity checks"; if the check fails, the write -will be rolled back. - -.. note:: +================= - Due to an underlying Postgres limitation, :ref:`constraints on object types - ` can only reference properties, not - links. +Access policies can act like "super constraints." For instance, a policy on +``insert`` or ``update write`` can do a post-write validity check, rejecting +the operation if a certain condition is not met. -Here's a policy that limits the number of blog posts a ``User`` can post. +E.g. here's a policy that limits the number of blog posts a +``User`` can post: .. code-block:: sdl-diff @@ -647,9 +568,236 @@ Here's a policy that limits the number of blog posts a ``User`` can post. + using (count(.author.posts) > 500); } -.. list-table:: - :class: seealso +.. _ref_eql_sdl_access_policies: +.. _ref_eql_sdl_access_policies_syntax: + +Declaring access policies +========================= + +This section describes the syntax to declare access policies in your schema. + +Syntax +------ + +.. sdl:synopsis:: + + access policy + [ when () ] + { allow | deny } [, ... ] + [ using () ] + [ "{" + [ errmessage := value ; ] + [ ] + "}" ] ; + + # where is one of + all + select + insert + delete + update [{ read | write }] + +Where: + +:eql:synopsis:`` + The name of the access policy. + +:eql:synopsis:`when ()` + Specifies which objects this policy applies to. The + :eql:synopsis:`` has to be a :eql:type:`bool` expression. + + When omitted, it is assumed that this policy applies to all objects of a + given type. + +:eql:synopsis:`allow` + Indicates that qualifying objects should allow access under this policy. + +:eql:synopsis:`deny` + Indicates that qualifying objects should *not* allow access under this + policy. This flavor supersedes any :eql:synopsis:`allow` policy and can + be used to selectively deny access to a subset of objects that otherwise + explicitly allows accessing them. + +:eql:synopsis:`all` + Apply the policy to all actions. It is exactly equivalent to listing + :eql:synopsis:`select`, :eql:synopsis:`insert`, :eql:synopsis:`delete`, + :eql:synopsis:`update` actions explicitly. + +:eql:synopsis:`select` + Apply the policy to all selection queries. Note that any object that + cannot be selected, cannot be modified either. This makes + :eql:synopsis:`select` the most basic "visibility" policy. + +:eql:synopsis:`insert` + Apply the policy to all inserted objects. If a newly inserted object would + violate this policy, an error is produced instead. + +:eql:synopsis:`delete` + Apply the policy to all objects about to be deleted. If an object does not + allow access under this kind of policy, it is not going to be considered + by any :eql:stmt:`delete` command. + + Note that any object that cannot be selected, cannot be modified either. + +:eql:synopsis:`update read` + Apply the policy to all objects selected for an update. If an object does + not allow access under this kind of policy, it is not visible cannot be + updated. + + Note that any object that cannot be selected, cannot be modified either. + +:eql:synopsis:`update write` + Apply the policy to all objects at the end of an update. If an updated + object violates this policy, an error is produced instead. + + Note that any object that cannot be selected, cannot be modified either. + +:eql:synopsis:`update` + This is just a shorthand for :eql:synopsis:`update read` and + :eql:synopsis:`update write`. + + Note that any object that cannot be selected, cannot be modified either. + +:eql:synopsis:`using ` + Specifies what the policy is with respect to a given eligible (based on + :eql:synopsis:`when` clause) object. The :eql:synopsis:`` has to be + a :eql:type:`bool` expression. The specific meaning of this value also + depends on whether this policy flavor is :eql:synopsis:`allow` or + :eql:synopsis:`deny`. + + The expression must be :ref:`Stable `. + + When omitted, it is assumed that this policy applies to all eligible + objects of a given type. + +:eql:synopsis:`set errmessage := ` + Set a custom error message of :eql:synopsis:`` that is displayed + when this access policy prevents a write action. + +:sdl:synopsis:`` + Set access policy :ref:`annotation ` + to a given *value*. + +Any sub-type extending a type inherits all of its access policies. +You can define additional access policies on sub-types. + + +.. _ref_eql_ddl_access_policies: + +DDL commands +============ + +This section describes the low-level DDL commands for creating, altering, and +dropping access policies. You typically don't need to use these commands +directly, but knowing about them is useful for reviewing migrations. + +Create access policy +-------------------- + +:eql-statement: + +Define a new object access policy on a type: + +.. eql:synopsis:: + + [ with [, ...] ] + { create | alter } type "{" + [ ... ] + create access policy + [ when () ; ] + { allow | deny } action [, action ... ; ] + [ using () ; ] + [ "{" + [ set errmessage := value ; ] + [ create annotation := value ; ] + "}" ] + "}" + + # where is one of + all + select + insert + delete + update [{ read | write }] + +See the meaning of each parameter in the `Declaring access policies`_ section. + +The following subcommands are allowed in the ``create access policy`` block: + +:eql:synopsis:`set errmessage := ` + Set a custom error message of :eql:synopsis:`` that is displayed + when this access policy prevents a write action. + +:eql:synopsis:`create annotation := ` + Set access policy annotation :eql:synopsis:`` to + :eql:synopsis:``. + + See :eql:stmt:`create annotation` for details. + + +Alter access policy +------------------- + +:eql-statement: + +Modify an existing access policy: + +.. eql:synopsis:: + + [ with [, ...] ] + alter type "{" + [ ... ] + alter access policy "{" + [ when () ; ] + [ reset when ; ] + { allow | deny } [, ... ; ] + [ using () ; ] + [ set errmessage := value ; ] + [ reset expression ; ] + [ create annotation := ; ] + [ alter annotation := ; ] + [ drop annotation ; ] + "}" + "}" + +You can change the policy's condition, actions, or error message, or add/drop +annotations. + +The parameters describing the action policy are identical to the parameters +used by ``create action policy``. There are a handful of additional +subcommands that are allowed in the ``alter access policy`` block: + +:eql:synopsis:`reset when` + Clear the :eql:synopsis:`when ()` so that the policy applies to + all objects of a given type. This is equivalent to ``when (true)``. + +:eql:synopsis:`reset expression` + Clear the :eql:synopsis:`using ()` so that the policy always + passes. This is equivalent to ``using (true)``. + +:eql:synopsis:`alter annotation ;` + Alter access policy annotation :eql:synopsis:``. + See :eql:stmt:`alter annotation` for details. + +:eql:synopsis:`drop annotation ;` + Remove access policy annotation :eql:synopsis:``. + See :eql:stmt:`drop annotation` for details. + + +All the subcommands allowed in the ``create access policy`` block are also +valid subcommands for ``alter access policy`` block. + +Drop access policy +------------------ + +:eql-statement: + +Remove an existing policy: + +.. eql:synopsis:: - * - **See also** - * - :ref:`SDL > Access policies ` - * - :ref:`DDL > Access policies ` + [ with [, ...] ] + alter type "{" + [ ... ] + drop access policy ; + "}" diff --git a/docs/datamodel/aliases.rst b/docs/datamodel/aliases.rst index fc56a1dd911..587c9eb4c3f 100644 --- a/docs/datamodel/aliases.rst +++ b/docs/datamodel/aliases.rst @@ -6,85 +6,99 @@ Aliases .. index:: alias, virtual type -.. important:: +You can think of *aliases* as a way to give schema names to arbitrary EdgeQL +expressions. You can later refer to aliases in queries and in other aliases. - This section assumes a basic understanding of EdgeQL. If you aren't familiar - with it, feel free to skip this page for now. +Aliases are functionally equivalent to expression aliases defined in EdgeQL +statements in :ref:`with block `, but are available +to all queries using the schema and can be introspected. +Like computed properties, the aliased expression is evaluated on the fly +whenever the alias is referenced. -An **alias** is a *pointer* to a set of values. This set is defined with an -arbitrary EdgeQL expression. -Like computed properties, this expression is evaluated on the fly whenever the -alias is referenced in a query. Unlike computed properties, aliases are -defined independent of an object type; they are standalone expressions. -As such, aliases are fairly open ended. Some examples are: - -**Scalar alias** +Scalar alias +============ .. code-block:: sdl + # in your schema: alias digits := {0,1,2,3,4,5,6,7,8,9}; -**Object type alias** +Later, in some query: + +.. code-block:: edgeql + + select count(digits); + + +Object type alias +================= The name of a given object type (e.g. ``User``) is itself a pointer to the *set of all User objects*. After declaring the alias below, you can use ``User`` and -``UserAlias`` interchangably. +``UserAlias`` interchangeably: .. code-block:: sdl alias UserAlias := User; -**Object type alias with computeds** +Object type alias with computeds +================================ -Object type aliases can include a *shape* that declare additional computed -properties or links. +Object type aliases can include a *shape* that declares additional computed +properties or links: .. code-block:: sdl - type Post { - required title: str; - } + type Post { + required title: str; + } + + alias PostWithTrimmedTitle := Post { + trimmed_title := str_trim(.title) + } - alias PostAlias := Post { - trimmed_title := str_trim(.title) - } +Later, in some query: -In effect, this creates a *virtual subtype* of the base type, which can be -referenced in queries just like any other type. +.. code-block:: edgeql -**Other arbitrary expressions** + select PostWithTrimmedTitle { + trimmed_title + }; + +Arbitrary expressions +===================== Aliases can correspond to any arbitrary EdgeQL expression, including entire queries. .. code-block:: sdl - # Tuple alias - alias Color := ("Purple", 128, 0, 128); - - # Named tuple alias - alias GameInfo := ( - name := "Li Europan Lingues", - country := "Iceland", - date_published := 2023, - creators := ( - (name := "Bob Bobson", age := 20), - (name := "Trina Trinadóttir", age := 25), - ), - ); - - type BlogPost { - required title: str; - required is_published: bool; - } - - # Query alias - alias PublishedPosts := ( - select BlogPost - filter .is_published = true - ); + # Tuple alias + alias Color := ("Purple", 128, 0, 128); + + # Named tuple alias + alias GameInfo := ( + name := "Li Europan Lingues", + country := "Iceland", + date_published := 2023, + creators := ( + (name := "Bob Bobson", age := 20), + (name := "Trina Trinadóttir", age := 25), + ), + ); + + type BlogPost { + required title: str; + required is_published: bool; + } + + # Query alias + alias PublishedPosts := ( + select BlogPost + filter .is_published = true + ); .. note:: @@ -92,11 +106,127 @@ queries. `. +.. _ref_eql_sdl_aliases: +.. _ref_eql_sdl_aliases_syntax: + +Defining aliases +================ + +Syntax +------ + +Define a new alias corresponding to the :ref:`more explicit DDL +commands `. + +.. sdl:synopsis:: + + alias := ; + + alias "{" + using ; + [ ] + "}" ; + +Where: + +:eql:synopsis:`` + The name (optionally module-qualified) of an alias to be created. + +:eql:synopsis:`` + The aliased expression. Must be a :ref:`Stable ` + EdgeQL expression. + +The valid SDL sub-declarations are listed below: + +:sdl:synopsis:`` + Set alias :ref:`annotation ` + to a given *value*. + + +.. _ref_eql_ddl_aliases: + +DDL commands +============ + +This section describes the low-level DDL commands for creating and +dropping aliases. You typically don't need to use these commands +directly, but knowing about them is useful for reviewing migrations. + +Create alias +------------ + +:eql-statement: +:eql-haswith: + +Define a new alias in the schema. + +.. eql:synopsis:: + + [ with [, ...] ] + create alias := ; + + [ with [, ...] ] + create alias "{" + using ; + [ create annotation := ; ... ] + "}" ; + + # where is: + + [ := ] module + +Parameters +^^^^^^^^^^ + +Most sub-commands and options of this command are identical to the +:ref:`SDL alias declaration `, with some +additional features listed below: + +:eql:synopsis:`[ := ] module ` + An optional list of module alias declarations to be used in the + alias definition. + +:eql:synopsis:`create annotation := ;` + An optional list of annotation values for the alias. + See :eql:stmt:`create annotation` for details. + +Example +^^^^^^^ + +Create a new alias: + +.. code-block:: edgeql + + create alias Superusers := ( + select User filter User.groups.name = 'Superusers' + ); + + +Drop alias +---------- + +:eql-statement: +:eql-haswith: + +Remove an alias from the schema. + +.. eql:synopsis:: + + [ with [, ...] ] + drop alias ; + +Parameters +^^^^^^^^^^ + +*alias-name* + The name (optionally qualified with a module name) of an existing + expression alias. + +Example +^^^^^^^ + +Remove an alias: -.. list-table:: - :class: seealso +.. code-block:: edgeql - * - **See also** - * - :ref:`SDL > Aliases ` - * - :ref:`DDL > Aliases ` - * - :ref:`Cheatsheets > Aliases ` + drop alias SuperUsers; diff --git a/docs/datamodel/annotations.rst b/docs/datamodel/annotations.rst index 4983081d205..1ddb58929cb 100644 --- a/docs/datamodel/annotations.rst +++ b/docs/datamodel/annotations.rst @@ -1,46 +1,51 @@ .. _ref_datamodel_annotations: +.. _ref_eql_sdl_annotations: =========== Annotations =========== -.. index:: annotation, title, description, deprecated +.. index:: annotation -*Annotations* are named values associated with schema items and -are designed to hold arbitrary schema-level metadata represented as a -:eql:type:`str`. +*Annotations* are named values associated with schema items and are +designed to hold arbitrary schema-level metadata represented as a +:eql:type:`str` (unstructured text). + +Users can store JSON-encoded data in annotations if they need to store +more complex metadata. Standard annotations --------------------- +==================== + +.. index:: title, description, deprecated -There are a number of annotations defined in the standard library. -The following are the annotations which can be set on any schema item: +There are a number of annotations defined in the standard library. The +following are the annotations which can be set on any schema item: -- ``title`` -- ``description`` -- ``deprecated`` +- ``std::title`` +- ``std::description`` +- ``std::deprecated`` For example, consider the following declaration: .. code-block:: sdl - type Status { - annotation title := 'Activity status'; - annotation description := 'All possible user activities'; + type Status { + annotation title := 'Activity status'; + annotation description := 'All possible user activities'; - required name: str { - constraint exclusive - } + required name: str { + constraint exclusive } + } -The ``deprecated`` annotation is used to mark deprecated items (e.g. -:eql:func:`str_rpad`) and to provide some information such as what +And the ``std::deprecated`` annotation can be used to mark deprecated items +(e.g., :eql:func:`str_rpad`) and to provide some information such as what should be used instead. - User-defined annotations ------------------------- +======================== .. index:: abstract annotation @@ -58,12 +63,328 @@ and code generation. } +.. _ref_eql_sdl_annotations_syntax: + +Declaring annotations +===================== + +This section describes the syntax to use annotations in your schema. + +Syntax +------ + +.. sdl:synopsis:: + + # Abstract annotation form: + abstract [ inheritable ] annotation + [ "{" ; [...] "}" ] ; + + # Concrete annotation (same as ) form: + annotation := ; + +Description +^^^^^^^^^^^ + +There are two forms of annotation declarations: abstract and concrete. +The *abstract annotation* form is used for declaring new kinds of +annotation in a module. The *concrete annotation* declarations are +used as sub-declarations for all other declarations in order to +actually annotate them. + +The annotation declaration options are as follows: + +:eql:synopsis:`abstract` + If specified, the annotation will be *abstract*. + +:eql:synopsis:`inheritable` + If specified, the annotation will be *inheritable*. The + annotations are non-inheritable by default. That is, if a schema + item has an annotation defined on it, the descendants of that + schema item will not automatically inherit the annotation. Normal + inheritance behavior can be turned on by declaring the annotation + with the ``inheritable`` qualifier. This is only valid for *abstract + annotation*. + +:eql:synopsis:`` + The name (optionally module-qualified) of the annotation. + +:eql:synopsis:`` + Any string value that the specified annotation is intended to have + for the given context. + +The only valid SDL sub-declarations are *concrete annotations*: + +:sdl:synopsis:`` + Annotations can also have annotations. Set the *annotation* of the + enclosing annotation to a specific value. + + +.. _ref_eql_ddl_annotations: + +DDL commands +============ + +This section describes the low-level DDL commands for creating, altering, +and dropping annotations and abstract annotations. You typically don't need to +use these commands directly, but knowing about them is useful for reviewing +migrations. + + +Create abstract annotation +-------------------------- + +:eql-statement: + +Define a new annotation. + +.. eql:synopsis:: + + [ with [, ...] ] + create abstract [ inheritable ] annotation + [ + "{" + create annotation := ; + [...] + "}" + ] ; + +Description +^^^^^^^^^^^ + +The command ``create abstract annotation`` defines a new annotation +for use in the current Gel database. + +If *name* is qualified with a module name, then the annotation is created +in that module, otherwise it is created in the current module. +The annotation name must be distinct from that of any existing schema item +in the module. + +The annotations are non-inheritable by default. That is, if a schema item +has an annotation defined on it, the descendants of that schema item will +not automatically inherit the annotation. Normal inheritance behavior can +be turned on by declaring the annotation with the ``inheritable`` qualifier. + +Most sub-commands and options of this command are identical to the +:ref:`SDL annotation declaration `. +There's only one subcommand that is allowed in the ``create +annotation`` block: + +:eql:synopsis:`create annotation := ` + Annotations can also have annotations. Set the + :eql:synopsis:`` of the + enclosing annotation to a specific :eql:synopsis:``. + See :eql:stmt:`create annotation` for details. + +Example +^^^^^^^ + +Declare an annotation ``extrainfo``: + +.. code-block:: edgeql + + create abstract annotation extrainfo; + + +Alter abstract annotation +------------------------- + +:eql-statement: + +Change the definition of an annotation. + +.. eql:synopsis:: + + alter abstract annotation + [ "{" ] ; [...] [ "}" ]; + + # where is one of + + rename to + create annotation := + alter annotation := + drop annotation + +Description +^^^^^^^^^^^ + +:eql:synopsis:`alter abstract annotation` changes the definition of an +abstract annotation. + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`` + The name (optionally module-qualified) of the annotation to alter. + +The following subcommands are allowed in the ``alter abstract annotation`` +block: + +:eql:synopsis:`rename to ` + Change the name of the annotation to :eql:synopsis:``. + +:eql:synopsis:`alter annotation := ` + Annotations can also have annotations. Change + :eql:synopsis:`` to a specific + :eql:synopsis:``. See :eql:stmt:`alter annotation` for + details. + +:eql:synopsis:`drop annotation ` + Annotations can also have annotations. Remove annotation + :eql:synopsis:``. + See :eql:stmt:`drop annotation` for details. + +All the subcommands allowed in the ``create abstract annotation`` +block are also valid subcommands for ``alter annotation`` block. + +Example +^^^^^^^ + +Rename an annotation: + +.. code-block:: edgeql + + alter abstract annotation extrainfo + rename to extra_info; + + +Drop abstract annotation +------------------------ + +:eql-statement: + +Drop a schema annotation. + +.. eql:synopsis:: + + [ with [, ...] ] + drop abstract annotation ; + +Description +^^^^^^^^^^^ + +The command ``drop abstract annotation`` removes an existing schema +annotation from the database schema. Note that the ``inheritable`` +qualifier is not necessary in this statement. + +Example +^^^^^^^ + +Drop the annotation ``extra_info``: + +.. code-block:: edgeql + + drop abstract annotation extra_info; + + +Create annotation +----------------- + +:eql-statement: + +Define an annotation value for a given schema item. + +.. eql:synopsis:: + + create annotation := + +Description +^^^^^^^^^^^ + +The command ``create annotation`` defines an annotation for a schema item. + +:eql:synopsis:`` refers to the name of a defined annotation, +and :eql:synopsis:`` must be a constant EdgeQL expression +evaluating into a string. + +This statement can only be used as a subcommand in another +DDL statement. + +Example +^^^^^^^ + +Create an object type ``User`` and set its ``title`` annotation to +``"User type"``. + +.. code-block:: edgeql + + create type User { + create annotation title := "User type"; + }; + + +Alter annotation +---------------- + +:eql-statement: + +Alter an annotation value for a given schema item. + +.. eql:synopsis:: + + alter annotation := + +Description +^^^^^^^^^^^ + +The command ``alter annotation`` alters an annotation value on a schema item. + +:eql:synopsis:`` refers to the name of a defined annotation, +and :eql:synopsis:`` must be a constant EdgeQL expression +evaluating into a string. + +This statement can only be used as a subcommand in another +DDL statement. + +Example +^^^^^^^ + +Alter an object type ``User`` and alter the value of its previously set +``title`` annotation to ``"User type"``. + +.. code-block:: edgeql + + alter type User { + alter annotation title := "User type"; + }; + + +Drop annotation +--------------- + +:eql-statement: + +Remove an annotation from a given schema item. + +.. eql:synopsis:: + + drop annotation ; + +Description +^^^^^^^^^^^ + +The command ``drop annotation`` removes an annotation value from a schema item. + +:eql:synopsis:`` refers to the name of a defined annotation. +The annotation value does not have to exist on a schema item. + +This statement can only be used as a subcommand in another +DDL statement. + +Example +^^^^^^^ + +Drop the ``title`` annotation from the ``User`` object type: + +.. code-block:: edgeql + + alter type User { + drop annotation title; + }; + + .. list-table:: :class: seealso * - **See also** - * - :ref:`SDL > Annotations ` - * - :ref:`DDL > Annotations ` * - :ref:`Cheatsheets > Annotations ` - * - :ref:`Introspection > Object types - ` + * - :ref:`Introspection > Object types ` diff --git a/docs/datamodel/constraints.rst b/docs/datamodel/constraints.rst index 676fca306a8..859d4349063 100644 --- a/docs/datamodel/constraints.rst +++ b/docs/datamodel/constraints.rst @@ -1,4 +1,5 @@ .. _ref_datamodel_constraints: +.. _ref_eql_sdl_constraints: =========== Constraints @@ -8,127 +9,139 @@ Constraints max_ex_value, min_value, min_ex_value, max_len_value, min_len_value, regexp, __subject__ -.. important:: +Constraints give users fine-grained control to ensure data consistency. +They can be defined on :ref:`properties `, +:ref:`links`, +:ref:`object types `, +and :ref:`custom scalars `. - This section assumes a basic understanding of EdgeQL. -Constraints give users fine-grained control over which data is considered -valid. They can be defined on :ref:`properties `, -:ref:`links `, :ref:`object types -`, and :ref:`custom scalars -`. +.. _ref_datamodel_constraints_builtin: + +Standard constraints +==================== + +|Gel| includes a number of standard ready-to-use constraints: + +.. include:: ../stdlib/constraint_table.rst -Below is a simple property constraint. + +Constraints on properties +========================= + +Example: enforce all ``User`` objects to have a unique ``username`` +no longer than 25 characters: .. code-block:: sdl - type User { - required username: str { - constraint exclusive; - } - } + type User { + required username: str { + # usernames must be unique + constraint exclusive; -.. _ref_datamodel_constraints_builtin: + # max length (built-in) + constraint max_len_value(25); + }; + } -This example uses a built-in constraint, ``exclusive``. Refer to the table -below for a complete list; click the name of a given constraint for the full -documentation. +.. _ref_datamodel_constraints_objects: -.. include:: ../stdlib/constraint_table.rst +Constraints on object types +=========================== -.. _ref_datamodel_constraints_properties: +.. index:: __subject__ -Constraints on properties -------------------------- +Constraints can be defined on object types. This is useful when the +constraint logic must reference multiple links or properties. -The ``max_len_value`` constraint below uses the built-in :eql:func:`len` -function, which returns the length of a string. +Example: enforce that the magnitude of ``ConstrainedVector`` objects +is no more than 5 .. code-block:: sdl - type User { - required username: str { - # usernames must be unique - constraint exclusive; + type ConstrainedVector { + required x: float64; + required y: float64; - # max length (built-in) - constraint max_len_value(25); - }; - } - -Custom constraints -^^^^^^^^^^^^^^^^^^ + constraint expression on ( + (.x ^ 2 + .y ^ 2) ^ 0.5 <= 5 + # or, long form: `(__subject__.x + __subject__.y) ^ 0.5 <= 5` + ); + } -The ``expression`` constraint is used to define custom constraint logic. Inside -custom constraints, the keyword ``__subject__`` can used to reference the +The ``expression`` constraint is used here to define custom constraint logic. +Inside constraints, the keyword ``__subject__`` can be used to reference the *value* being constrained. -.. code-block:: sdl +.. note:: + Note that inside an object type declaration, you can omit ``__subject__`` + and simply refer to properties with the + :ref:`leading dot notation ` (e.g. ``.property``). - type User { - required username: str { - # max length (as custom constraint) - constraint expression on (len(__subject__) <= 25); - }; - } +.. note:: -.. _ref_datamodel_constraints_objects: + Also note that the constraint expression are fairly restricted. Due + to how constraints are implemented, you can only reference ``single`` + (non-multi) properties and links defined on the object type: -Constraints on object types ---------------------------- + .. code-block:: sdl -Constraints can be defined on object types. This is useful when the -constraint logic must reference multiple links or properties. + # Not valid! + type User { + required username: str; + multi friends: User; + + # ❌ constraints cannot contain paths with more than one hop + constraint expression on ('bob' in .friends.username); + } -.. important:: +Abstract constraints +==================== - Inside an object type declaration, you can omit ``__subject__`` and simply - refer to properties with the :ref:`leading dot notation ` - (e.g. ``.``). +You can re-use constraints across multiple object types by declaring them as +abstract constraints. Example: .. code-block:: sdl - type ConstrainedVector { - required x: float64; - required y: float64; + abstract constraint min_value(min: anytype) { + errmessage := + 'Minimum allowed value for {__subject__} is {min}.'; - constraint expression on ( - .x ^ 2 + .y ^ 2 <= 25 - ); + using (__subject__ >= min); } -Note that the constraint expression cannot contain arbitrary EdgeQL! Due to -how constraints are implemented, you can only reference ``single`` (non-multi) -properties and links defined on the object type. + # use it like this: -.. code-block:: sdl + scalar type posint64 extending int64 { + constraint min_value(0); + } - # Not valid! - type User { - required username: str; - multi friends: User; + # or like this: - # ❌ constraints cannot contain paths with more than one hop - constraint expression on ('bob' in .friends.username); + type User { + required age: int16 { + constraint min_value(12); + }; } + Computed constraints -^^^^^^^^^^^^^^^^^^^^ +==================== -Constraints can be defined on computed properties. +Constraints can be defined on computed properties: .. code-block:: sdl - type User { - required username: str; - required clean_username := str_trim(str_lower(.username)); + type User { + required username: str; + required clean_username := str_trim(str_lower(.username)); - constraint exclusive on (.clean_username); - } + constraint exclusive on (.clean_username); + } Composite constraints -^^^^^^^^^^^^^^^^^^^^^ +===================== .. index:: tuple @@ -137,86 +150,83 @@ tuple of properties or links. .. code-block:: sdl - type User { - username: str; - } + type User { + username: str; + } - type BlogPost { - title: str; - author: User; + type BlogPost { + title: str; - constraint exclusive on ((.title, .author)); - } + author: User; + + constraint exclusive on ((.title, .author)); + } .. _ref_datamodel_constraints_partial: Partial constraints -^^^^^^^^^^^^^^^^^^^ +=================== .. index:: constraint exclusive on, except -Constraints on object types can be made partial, so that they don't apply -when some condition holds. +Constraints on object types can be made partial, so that they are not enforced +when the specified ``except`` condition is met. .. code-block:: sdl - type User { - required username: str; - deleted: bool; - - # Usernames must be unique unless marked deleted - constraint exclusive on (.username) except (.deleted); - } + type User { + required username: str; + deleted: bool; + # Usernames must be unique unless marked deleted + constraint exclusive on (.username) except (.deleted); + } -.. _ref_datamodel_constraints_links: Constraints on links --------------------- +==================== You can constrain links such that a given object can only be linked once by using :eql:constraint:`exclusive`: .. code-block:: sdl - type User { - required name: str; + type User { + required name: str; - # Make sure none of the "owned" items belong - # to any other user. - multi owns: Item { - constraint exclusive; - } + # Make sure none of the "owned" items belong + # to any other user. + multi owns: Item { + constraint exclusive; } + } + Link property constraints -^^^^^^^^^^^^^^^^^^^^^^^^^ +========================= You can also add constraints for :ref:`link properties `: .. code-block:: sdl - type User { - name: str; - multi friends: User { - strength: float64; - constraint expression on ( - @strength >= 0 - ); - } - } + type User { + name: str; -Link ``@source`` and ``@target`` constraints -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + multi friends: User { + strength: float64; -.. versionadded:: 4.0 + constraint expression on ( + @strength >= 0 + ); + } + } -.. index:: constraint exclusive on, @source, @target -.. note:: +Link's "@source" and "@target" +============================== - ``@source`` and ``@target`` are available starting with version 4.3. +.. index:: constraint exclusive on, @source, @target You can create a composite exclusive constraint on the object linking/linked *and* a link property by using ``@source`` or ``@target`` respectively. Here's @@ -225,18 +235,20 @@ checked them out: .. code-block:: sdl - type Book { - required title: str; - } - type User { - name: str; - multi checked_out: Book { - date: cal::local_date; - # Ensures a given Book can be checked out - # only once on a given day. - constraint exclusive on ((@target, @date)); - } + type Book { + required title: str; + } + + type User { + name: str; + multi checked_out: Book { + date: cal::local_date; + + # Ensures a given Book can be checked out + # only once on a given day. + constraint exclusive on ((@target, @date)); } + } Here, the constraint ensures that no book can be checked out to two ``User``\s on the same ``@date``. @@ -246,24 +258,26 @@ player picks in a color-based memory game: .. code-block:: sdl - type Player { - required name: str; - multi picks: Color { - order: int16; - constraint exclusive on ((@source, @order)); - } - } - type Color { - required name: str; + type Player { + required name: str; + + multi picks: Color { + order: int16; + + constraint exclusive on ((@source, @order)); } + } + + type Color { + required name: str; + } This constraint ensures that a single ``Player`` cannot pick two ``Color``\s at the same ``@order``. -.. _ref_datamodel_constraints_scalars: Constraints on custom scalars ------------------------------ +============================= Custom scalar types can be constrained. @@ -290,8 +304,8 @@ using arbitrary EdgeQL expressions. The example below uses the built-in } -Constraints and type inheritence --------------------------------- +Constraints and inheritance +=========================== .. index:: delegated constraint @@ -301,39 +315,38 @@ apply globally across all the types that inherited the constraint. .. code-block:: sdl - type User { - required name: str { - constraint exclusive; - } + type User { + required name: str { + constraint exclusive; } - type Administrator extending User; - type Moderator extending User; + } + type Administrator extending User; + type Moderator extending User; .. code-block:: edgeql-repl - db> insert Administrator { - ... name := 'Jan' - ... }; - {default::Administrator {id: 7aeaa146-f5a5-11ed-a598-53ddff476532}} - db> insert Moderator { - ... name := 'Jan' - ... }; - gel error: ConstraintViolationError: name violates exclusivity - constraint - Detail: value of property 'name' of object type 'default::Moderator' - violates exclusivity constraint - db> insert User { - ... name := 'Jan' - ... }; - gel error: ConstraintViolationError: name violates exclusivity - constraint - Detail: value of property 'name' of object type 'default::User' - violates exclusivity constraint - - -As this example demonstrates, this means if an object of one of the extending -types has a value for a property that is exclusive, an object of a different -extending type cannot have the same value. + gel> insert Administrator { + .... name := 'Jan' + .... }; + {default::Administrator {id: 7aeaa146-f5a5-11ed-a598-53ddff476532}} + + gel> insert Moderator { + .... name := 'Jan' + .... }; + gel error: ConstraintViolationError: name violates exclusivity constraint + Detail: value of property 'name' of object type 'default::Moderator' + violates exclusivity constraint + + gel> insert User { + .... name := 'Jan' + .... }; + gel error: ConstraintViolationError: name violates exclusivity constraint + Detail: value of property 'name' of object type 'default::User' + violates exclusivity constraint + +As this example demonstrates, if an object of one extending type has a value +for a property that is exclusive, an object of a *different* extending type +cannot have the same value. If that's not what you want, you can instead delegate the constraint to the inheriting types by prepending the ``delegated`` keyword to the constraint. @@ -342,48 +355,601 @@ on each of the inheriting types. .. code-block:: sdl - type User { - required name: str { - delegated constraint exclusive; - } + type User { + required name: str { + delegated constraint exclusive; } - type Administrator extending User; - type Moderator extending User; + } + type Administrator extending User; + type Moderator extending User; .. code-block:: edgeql-repl - db> insert Administrator { - ... name := 'Jan' - ... }; - {default::Administrator {id: 7aeaa146-f5a5-11ed-a598-53ddff476532}} - db> insert User { - ... name := 'Jan' - ... }; - {default::User {id: a6e3fdaf-c44b-4080-b39f-6a07496de66b}} - db> insert Moderator { - ... name := 'Jan' - ... }; - {default::Moderator {id: d3012a3f-0f16-40a8-8884-7203f393b63d}} - db> insert Moderator { - ... name := 'Jan' - ... }; - gel error: ConstraintViolationError: name violates exclusivity - constraint - Detail: value of property 'name' of object type 'default::Moderator' - violates exclusivity constraint + gel> insert Administrator { + .... name := 'Jan' + .... }; + {default::Administrator {id: 7aeaa146-f5a5-11ed-a598-53ddff476532}} + + gel> insert User { + .... name := 'Jan' + .... }; + {default::User {id: a6e3fdaf-c44b-4080-b39f-6a07496de66b}} + + gel> insert Moderator { + .... name := 'Jan' + .... }; + {default::Moderator {id: d3012a3f-0f16-40a8-8884-7203f393b63d}} + + gel> insert Moderator { + .... name := 'Jan' + .... }; + gel error: ConstraintViolationError: name violates exclusivity constraint + Detail: value of property 'name' of object type 'default::Moderator' + violates exclusivity constraint With the addition of ``delegated`` to the constraints, the inserts were -successful for each of the types. In this case, we did not hit a constraint -violation until we tried to insert a second ``Moderator`` object with the same -name as the one we had just inserted. +successful for each of the types. We did not hit a constraint violation +until we tried to insert a second ``Moderator`` object with the same +name as the existing one. + + +.. _ref_eql_sdl_constraints_syntax: + +Declaring constraints +===================== + +This section describes the syntax to declare constraints in your schema. + +Syntax +------ + +.. sdl:synopsis:: + + [{abstract | delegated}] constraint [ ( [] [, ...] ) ] + [ on ( ) ] + [ except ( ) ] + [ extending [, ...] ] + "{" + [ using ; ] + [ errmessage := ; ] + [ ] + [ ... ] + "}" ; + + # where is: + + [ : ] { | } + +Description +^^^^^^^^^^^ + +This declaration defines a new constraint with the following options: + +:eql:synopsis:`abstract` + If specified, the constraint will be *abstract*. + +:eql:synopsis:`delegated` + If specified, the constraint is defined as *delegated*, which means + that it will not be enforced on the type it's declared on, and the + enforcement will be delegated to the subtypes of this type. + This is particularly useful for :eql:constraint:`exclusive` + constraints in abstract types. This is only valid for *concrete + constraints*. + +:eql:synopsis:`` + The name (optionally module-qualified) of the new constraint. + +:eql:synopsis:`` + An optional list of constraint arguments. + + For an *abstract constraint* :eql:synopsis:`` optionally + specifies the argument name and :eql:synopsis:`` specifies + the argument type. + + For a *concrete constraint* :eql:synopsis:`` optionally + specifies the argument name and :eql:synopsis:`` specifies + the argument value. The argument value specification must match the + parameter declaration of the abstract constraint. + +:eql:synopsis:`on ( )` + An optional expression defining the *subject* of the constraint. + If not specified, the subject is the value of the schema item on which + the concrete constraint is defined. + + The expression must refer to the original subject of the constraint as + ``__subject__``. The expression must be + :ref:`Immutable `, but may refer to + ``__subject__`` and its properties and links. + + Note also that ```` itself has to + be parenthesized. + +:eql:synopsis:`except ( )` + An optional expression defining a condition to create exceptions to + the constraint. If ```` evaluates to ``true``, + the constraint is ignored for the current subject. If it evaluates + to ``false`` or ``{}``, the constraint applies normally. + + ``except`` may only be declared on object constraints, and otherwise + follows the same rules as ``on``. + +:eql:synopsis:`extending [, ...]` + If specified, declares the *parent* constraints for this abstract + constraint. + +The valid SDL sub-declarations are listed below: + +:eql:synopsis:`using ` + A boolean expression that returns ``true`` for valid data and + ``false`` for invalid data. The expression may refer to the + subject of the constraint as ``__subject__``. This declaration is + only valid for *abstract constraints*. + +:eql:synopsis:`errmessage := ` + An optional string literal defining the error message template + that is raised when the constraint is violated. The template is a + formatted string that may refer to constraint context variables in + curly braces. The template may refer to the following: + + - ``$argname`` -- the value of the specified constraint argument + - ``__subject__`` -- the value of the ``title`` annotation of the + scalar type, property or link on which the constraint is defined. + + If the content of curly braces does not match any variables, + the curly braces are emitted as-is. They can also be escaped by + using double curly braces. + +:sdl:synopsis:`` + Set constraint :ref:`annotation ` + to a given *value*. + + +.. _ref_eql_ddl_constraints: + +DDL commands +============ + +This section describes the low-level DDL commands for creating and dropping +constraints and abstract constraints. You typically don't need to use these +commands directly, but knowing about them is useful for reviewing migrations. + + +Create abstract constraint +-------------------------- + +:eql-statement: +:eql-haswith: + +Define a new abstract constraint. + +.. eql:synopsis:: + + [ with [ := ] module ] + create abstract constraint [ ( [] [, ...] ) ] + [ on ( ) ] + [ extending [, ...] ] + "{" ; [...] "}" ; + + # where is: + + [ : ] + + # where is one of + + using + set errmessage := + create annotation := + + +Description +^^^^^^^^^^^ +The command ``create abstract constraint`` defines a new abstract constraint. + +If *name* is qualified with a module name, then the constraint is created in +that module, otherwise it is created in the current module. The constraint +name must be distinct from that of any existing schema item in the module. + + +Parameters +^^^^^^^^^^ +Most sub-commands and options of this command are identical to the +:ref:`SDL constraint declaration `, +with some additional features listed below: + +:eql:synopsis:`[ := ] module ` + An optional list of module alias declarations to be used in the + migration definition. When *module-alias* is not specified, + *module-name* becomes the effective current module and is used + to resolve all unqualified names. + +:eql:synopsis:`set errmessage := ` + An optional string literal defining the error message template + that is raised when the constraint is violated. Other than a + slight syntactical difference this is the same as the + corresponding SDL declaration. + +:eql:synopsis:`create annotation := ;` + Set constraint annotation ```` to ````. + See :eql:stmt:`create annotation` for details. + + +Example +^^^^^^^ +Create an abstract constraint "uppercase" which checks if the subject +is a string in upper case: + +.. code-block:: edgeql + + create abstract constraint uppercase { + create annotation title := "Upper case constraint"; + + using (str_upper(__subject__) = __subject__); + + set errmessage := "{__subject__} is not in upper case"; + }; + + +Alter abstract constraint +------------------------- + +:eql-statement: +:eql-haswith: + +Alter the definition of an abstract constraint. + +.. eql:synopsis:: + + [ with [ := ] module ] + alter abstract constraint + "{" ; [...] "}" ; + + # where is one of + + rename to + using + set errmessage := + reset errmessage + create annotation := + alter annotation := + drop annotation + + +Description +^^^^^^^^^^^ + +The command ``alter abstract constraint`` changes the definition of an +abstract constraint item. *name* must be a name of an existing +abstract constraint, optionally qualified with a module name. + + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`[ := ] module ` + An optional list of module alias declarations to be used in the + migration definition. When *module-alias* is not specified, + *module-name* becomes the effective current module and is used + to resolve all unqualified names. + +:eql:synopsis:`` + The name (optionally module-qualified) of the constraint to alter. + +Subcommands allowed in the ``alter abstract constraint`` block: + +:eql:synopsis:`rename to ` + Change the name of the constraint to *newname*. All concrete + constraints inheriting from this constraint are also renamed. + +:eql:synopsis:`alter annotation := ` + Alter constraint annotation ````. + See :eql:stmt:`alter annotation` for details. + +:eql:synopsis:`drop annotation ` + Remove annotation ````. + See :eql:stmt:`drop annotation` for details. + +:eql:synopsis:`reset errmessage` + Remove the error message from this abstract constraint. The error message + specified in the base abstract constraint will be used instead. + +All subcommands allowed in a ``create abstract constraint`` block are also +valid here. + + +Example +^^^^^^^ + +Rename the abstract constraint "uppercase" to "upper_case": + +.. code-block:: edgeql + + alter abstract constraint uppercase rename to upper_case; + + +Drop abstract constraint +------------------------ + +:eql-statement: +:eql-haswith: + +Remove an abstract constraint from the schema. + +.. eql:synopsis:: + + [ with [ := ] module ] + drop abstract constraint ; + + +Description +^^^^^^^^^^^ + +The command ``drop abstract constraint`` removes an existing abstract +constraint item from the database schema. If any schema items depending +on this constraint exist, the operation is refused. + + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`[ := ] module ` + An optional list of module alias declarations to be used in the + migration definition. + +:eql:synopsis:`` + The name (optionally module-qualified) of the constraint to remove. + + +Example +^^^^^^^ + +Drop abstract constraint ``upper_case``: + +.. code-block:: edgeql + + drop abstract constraint upper_case; + + +Create constraint +----------------- + +:eql-statement: + +Define a concrete constraint on the specified schema item. + +.. eql:synopsis:: + + [ with [ := ] module ] + create [ delegated ] constraint + [ ( [] [, ...] ) ] + [ on ( ) ] + [ except ( ) ] + "{" ; [...] "}" ; + + # where is: + + [ : ] + + # where is one of + + set errmessage := + create annotation := + + +Description +^^^^^^^^^^^ + +The command ``create constraint`` defines a new concrete constraint. It can +only be used in the context of :eql:stmt:`create scalar`, +:eql:stmt:`alter scalar`, :eql:stmt:`create property`, +:eql:stmt:`alter property`, :eql:stmt:`create link`, or :eql:stmt:`alter link`. + +*name* must be a name (optionally module-qualified) of a previously defined +abstract constraint. + + +Parameters +^^^^^^^^^^ + +Most sub-commands and options of this command are identical to the +:ref:`SDL constraint declaration `, +with some additional features listed below: + +:eql:synopsis:`[ := ] module ` + An optional list of module alias declarations to be used in the + migration definition. + +:eql:synopsis:`set errmessage := ` + An optional string literal defining the error message template + that is raised when the constraint is violated. Other than a + slight syntactical difference, this is the same as the corresponding + SDL declaration. + +:eql:synopsis:`create annotation := ;` + An optional list of annotations for the constraint. See + :eql:stmt:`create annotation` for details. + + +Example +^^^^^^^ + +Create a "score" property on the "User" type with a minimum value +constraint: + +.. code-block:: edgeql + + alter type User create property score -> int64 { + create constraint min_value(0) + }; + +Create a Vector with a maximum magnitude: + +.. code-block:: edgeql + + create type Vector { + create required property x -> float64; + create required property y -> float64; + create constraint expression ON ( + __subject__.x^2 + __subject__.y^2 < 25 + ); + } + + +Alter constraint +---------------- + +:eql-statement: + +Alter the definition of a concrete constraint on the specified schema item. + +.. eql:synopsis:: + + [ with [ := ] module [, ...] ] + alter constraint + [ ( [] [, ...] ) ] + [ on ( ) ] + [ except ( ) ] + "{" ; [ ... ] "}" ; + + # -- or -- + + [ with [ := ] module [, ...] ] + alter constraint + [ ( [] [, ...] ) ] + [ on ( ) ] + ; + + # where is one of: + + set delegated + set not delegated + set errmessage := + reset errmessage + create annotation := + alter annotation + drop annotation + + +Description +^^^^^^^^^^^ + +The command ``alter constraint`` changes the definition of a concrete +constraint. Both single- and multi-command forms are supported. + + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`[ := ] module ` + An optional list of module alias declarations for the migration. + +:eql:synopsis:`` + The name (optionally module-qualified) of the concrete constraint + that is being altered. + +:eql:synopsis:`` + A list of constraint arguments as specified at the time of + ``create constraint``. + +:eql:synopsis:`on ( )` + An expression defining the *subject* of the constraint as specified + at the time of ``create constraint``. + +The following subcommands are allowed in the ``alter constraint`` block: + +:eql:synopsis:`set delegated` + Mark the constraint as *delegated*, which means it will + not be enforced on the type it's declared on, and enforcement is + delegated to subtypes. Useful for :eql:constraint:`exclusive` constraints. + +:eql:synopsis:`set not delegated` + Mark the constraint as *not delegated*, so it is enforced globally across + the type and any extending types. + +:eql:synopsis:`rename to ` + Change the name of the constraint to ````. + +:eql:synopsis:`alter annotation ` + Alter a constraint annotation. + +:eql:synopsis:`drop annotation ` + Remove a constraint annotation. + +:eql:synopsis:`reset errmessage` + Remove the error message from this constraint, reverting to that of the + abstract constraint, if any. + +All subcommands allowed in ``create constraint`` are also valid in +``alter constraint``. + +Example +^^^^^^^ + +Change the error message on the minimum value constraint on the property +"score" of the "User" type: + +.. code-block:: edgeql + + alter type User alter property score + alter constraint min_value(0) + set errmessage := 'Score cannot be negative'; + + +Drop constraint +--------------- + +:eql-statement: +:eql-haswith: + +Remove a concrete constraint from the specified schema item. + +.. eql:synopsis:: + + [ with [ := ] module [, ...] ] + drop constraint + [ ( [] [, ...] ) ] + [ on ( ) ] + [ except ( ) ] ; + + +Description +^^^^^^^^^^^ + +The command ``drop constraint`` removes the specified constraint from +its containing schema item. + + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`[ := ] module ` + Optional module alias declarations for the migration definition. + +:eql:synopsis:`` + The name (optionally module-qualified) of the concrete constraint + to remove. + +:eql:synopsis:`` + A list of constraint arguments as specified at the time of + ``create constraint``. + +:eql:synopsis:`on ( )` + Expression defining the *subject* of the constraint as specified + at the time of ``create constraint``. + +Example +^^^^^^^ + +Remove constraint "min_value" from the property "score" of the "User" type: + +.. code-block:: edgeql + + alter type User alter property score + drop constraint min_value(0); .. list-table:: :class: seealso * - **See also** - * - :ref:`SDL > Constraints ` - * - :ref:`DDL > Constraints ` - * - :ref:`Introspection > Constraints - ` + * - :ref:`Introspection > Constraints ` * - :ref:`Standard Library > Constraints ` diff --git a/docs/datamodel/extensions.rst b/docs/datamodel/extensions.rst index 53a4ab6d932..dc93c132ffe 100644 --- a/docs/datamodel/extensions.rst +++ b/docs/datamodel/extensions.rst @@ -6,12 +6,13 @@ Extensions .. index:: using extension -Extensions are the way |Gel| adds more functionality. In principle, -extensions could add new types, scalars, functions, etc., but, more +Extensions are the way |Gel| can be extended with more functionality. +They can add new types, scalars, functions, etc., but, more importantly, they can add new ways of interacting with the database. + Built-in extensions -------------------- +=================== .. index:: edgeql_http, graphql, auth, ai, pg_trgm, pg_unaccent, pgcrypto, pgvector @@ -21,7 +22,7 @@ There are a few built-in extensions available: - ``edgeql_http``: enables :ref:`EdgeQL over HTTP `, - ``graphql``: enables :ref:`GraphQL `, - ``auth``: enables :ref:`Gel Auth `, -- ``ai``: enables :ref:`ext::ai module `, +- ``ai``: enables :ref:`ext::ai module `, - ``pg_trgm``: enables ``ext::pg_trgm``, which re-exports `pgtrgm `__, @@ -37,76 +38,172 @@ There are a few built-in extensions available: .. _ref_datamodel_using_extension: -To enable these extensions, add a ``using`` statement at the top level of your schema: +To enable these extensions, add a ``using`` statement at the top level of +your schema: .. code-block:: sdl - using extension auth; - + using extension auth; + # or / and + using extension ai; Standalone extensions ---------------------- +===================== .. index:: postgis -Additionally, standalone extension packages can be installed via the CLI. +Additionally, standalone extension packages can be installed via the CLI, +with ``postgis`` being a notable example. List installed extensions: .. code-block:: bash - $ gel extension list -I my_instance - ┌─────────┬─────────┐ - │ Name │ Version │ - └─────────┴─────────┘ + $ gel extension list + ┌─────────┬─────────┐ + │ Name │ Version │ + └─────────┴─────────┘ List available extensions: .. code-block:: bash - $ gel extension list-available -I my_instance - ┌─────────┬───────────────┐ - │ Name │ Version │ - │ postgis │ 3.4.3+6b82d77 │ - └─────────┴───────────────┘ + $ gel extension list-available + ┌─────────┬───────────────┐ + │ Name │ Version │ + │ postgis │ 3.4.3+6b82d77 │ + └─────────┴───────────────┘ Install the ``postgis`` extension: .. code-block:: bash - $ gel extension install -I my_instance -E postgis - Found extension package: postgis version 3.4.3+6b82d77 - 00:00:03 [====================] 22.49 MiB/22.49 MiB - Extension 'postgis' installed successfully. + $ gel extension install -E postgis + Found extension package: postgis version 3.4.3+6b82d77 + 00:00:03 [====================] 22.49 MiB/22.49 MiB + Extension 'postgis' installed successfully. Check that extension is installed: .. code-block:: bash - $ gel extension list -I my_instance - ┌─────────┬───────────────┐ - │ Name │ Version │ - │ postgis │ 3.4.3+6b82d77 │ - └─────────┴───────────────┘ + $ gel extension list + ┌─────────┬───────────────┐ + │ Name │ Version │ + │ postgis │ 3.4.3+6b82d77 │ + └─────────┴───────────────┘ After installing extensions, make sure to restart your instance: .. code-block:: bash - $ gel instance restart -I my_instance + $ gel instance restart + +Standalone extensions can now be declared in the schema, same as +built-in extensions: + +.. code-block:: sdl + + using extension postgis; + +.. note:: + To restore a dump that uses a standalone extension, that extension must + be installed before the restore process. + +.. _ref_eql_sdl_extensions: + +Using extensions +================ + +Syntax +------ + +.. sdl:synopsis:: + + using extension ";" + + +Extension declaration must be outside any :ref:`module block +` since extensions affect the entire database and +not a specific module. + + + +.. _ref_eql_ddl_extensions: + +DDL commands +============ + +This section describes the low-level DDL commands for creating and +dropping extensions. You typically don't need to use these commands directly, +but knowing about them is useful for reviewing migrations. + + +create extension +---------------- + +:eql-statement: + +Enable a particular extension for the current schema. + +.. eql:synopsis:: + + create extension ";" + + +Description +^^^^^^^^^^^ + +The command ``create extension`` enables the specified extension for +the current :versionreplace:`database;5.0:branch`. + +Examples +^^^^^^^^ + +Enable :ref:`GraphQL ` extension for the current +schema: + +.. code-block:: edgeql + + create extension graphql; + +Enable :ref:`EdgeQL over HTTP ` extension for the +current :versionreplace:`database;5.0:branch`: + +.. code-block:: edgeql + + create extension edgeql_http; + + +drop extension +-------------- + +:eql-statement: + +Disable an extension. + +.. eql:synopsis:: + + drop extension ";" + + +The command ``drop extension`` disables a currently active extension for +the current |branch|. + +Examples +^^^^^^^^ + +Disable :ref:`GraphQL ` extension for the current +schema: -Standalone extensions can now be declared in the schema, same as :ref:`built-in -extensions `. +.. code-block:: edgeql -To restore a dump that uses a standalone extension, that extension must be installed -before the restore process. + drop extension graphql; +Disable :ref:`EdgeQL over HTTP ` extension for the +current :versionreplace:`database;5.0:branch`: -.. list-table:: - :class: seealso +.. code-block:: edgeql - * - **See also** - * - :ref:`SDL > Extensions ` - * - :eql:stmt:`DDL > CREATE EXTENSION ` - * - :eql:stmt:`DDL > DROP EXTENSION ` + drop extension edgeql_http; diff --git a/docs/datamodel/functions.rst b/docs/datamodel/functions.rst index 7f2030a00f1..7b9dcd4ec56 100644 --- a/docs/datamodel/functions.rst +++ b/docs/datamodel/functions.rst @@ -1,4 +1,5 @@ .. _ref_datamodel_functions: +.. _ref_eql_sdl_functions: ========= Functions @@ -12,12 +13,11 @@ Functions large library of built-in functions and operators. These are documented in :ref:`Standard Library `. -Functions are ways to transform one set of data into another. User-defined Functions ----------------------- +====================== -It is also possible to define custom functions. For example, consider +Gel allows you to define custom functions. For example, consider a function that adds an exclamation mark ``'!'`` at the end of the string: @@ -34,10 +34,11 @@ This function accepts a :eql:type:`str` as an argument and produces a test> select exclamation({'Hello', 'World'}); {'Hello!', 'World!'} + .. _ref_datamodel_functions_modifying: Sets as arguments -^^^^^^^^^^^^^^^^^ +================= Calling a user-defined function on a set will always apply it as :ref:`*element-wise* `. @@ -78,8 +79,9 @@ applied element-wise. db> select magnitude({[3, 4], [5, 12]}); {5, 13} + Modifying Functions -^^^^^^^^^^^^^^^^^^^ +=================== .. versionadded:: 6.0 @@ -150,12 +152,430 @@ aggregated into an array as described above: } ); + +.. _ref_eql_sdl_functions_syntax: + +Declaring functions +=================== + +This section describes the syntax to declare a function in your schema. + +Syntax +------ + +.. sdl:synopsis:: + + function ([ ] [, ... ]) -> + using ( ); + + function ([ ] [, ... ]) -> + using ; + + function ([ ] [, ... ]) -> + "{" + [ ] + [ volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'} ] + [ using ( ) ; ] + [ using ; ] + [ ... ] + "}" ; + + # where is: + + [ ] : [ ] [ = ] + + # is: + + [ { variadic | named only } ] + + # is: + + [ { set of | optional } ] + + # and is: + + [ ] + + +Description +^^^^^^^^^^^ + +This declaration defines a new **function** with the following options: + +:eql:synopsis:`` + The name (optionally module-qualified) of the function to create. + +:eql:synopsis:`` + The kind of an argument: ``variadic`` or ``named only``. + + If not specified, the argument is called *positional*. + + The ``variadic`` modifier indicates that the function takes an + arbitrary number of arguments of the specified type. The passed + arguments will be passed as an array of the argument type. + Positional arguments cannot follow a ``variadic`` argument. + ``variadic`` parameters cannot have a default value. + + The ``named only`` modifier indicates that the argument can only + be passed using that specific name. Positional arguments cannot + follow a ``named only`` argument. + +:eql:synopsis:`` + The name of an argument. If ``named only`` modifier is used this + argument *must* be passed using this name only. + +.. _ref_sdl_function_typequal: + +:eql:synopsis:`` + The type qualifier: ``set of`` or ``optional``. + + The ``set of`` qualifier indicates that the function is taking the + argument as a *whole set*, as opposed to being called on the input + product element-by-element. + + User defined functions can not use ``set of`` arguments. + + The ``optional`` qualifier indicates that the function will be called + if the argument is an empty set. The default behavior is to return + an empty set if the argument is not marked as ``optional``. + +:eql:synopsis:`` + The data type of the function's arguments + (optionally module-qualified). + +:eql:synopsis:`` + An expression to be used as default value if the parameter is not + specified. The expression has to be of a type compatible with the + type of the argument. + +.. _ref_sdl_function_rettype: + +:eql:synopsis:`` + The return data type (optionally module-qualified). + + The ``set of`` modifier indicates that the function will return + a non-singleton set. + + The ``optional`` qualifier indicates that the function may return + an empty set. + +The valid SDL sub-declarations are listed below: + +:eql:synopsis:`volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'}` + Function volatility determines how aggressively the compiler can + optimize its invocations. + + If not explicitly specified the function volatility is + :ref:`inferred ` from the function body. + + * An ``Immutable`` function cannot modify the database and is + guaranteed to return the same results given the same arguments + *in all statements*. + + * A ``Stable`` function cannot modify the database and is + guaranteed to return the same results given the same + arguments *within a single statement*. + + * A ``Volatile`` function cannot modify the database and can return + different results on successive calls with the same arguments. + + * A ``Modifying`` function can modify the database and can return + different results on successive calls with the same arguments. + +:eql:synopsis:`using ( )` + Specifies the body of the function. :eql:synopsis:`` is an + arbitrary EdgeQL expression. + +:eql:synopsis:`using ` + A verbose version of the :eql:synopsis:`using` clause that allows + specifying the language of the function body. + + * :eql:synopsis:`` is the name of the language that + the function is implemented in. Currently can only be ``edgeql``. + + * :eql:synopsis:`` is a string constant defining + the function. It is often helpful to use + :ref:`dollar quoting ` + to write the function definition string. + +:sdl:synopsis:`` + Set function :ref:`annotation ` + to a given *value*. + +The function name must be distinct from that of any existing function +with the same argument types in the same module. Functions of +different argument types can share a name, in which case the functions +are called *overloaded functions*. + + +.. _ref_eql_ddl_functions: + +DDL commands +============ + +This section describes the low-level DDL commands for creating, altering, and +dropping functions. You typically don't need to use these commands directly, but +knowing about them is useful for reviewing migrations. + +Create function +--------------- + +:eql-statement: +:eql-haswith: + +Define a new function. + +.. eql:synopsis:: + + [ with [, ...] ] + create function ([ ] [, ... ]) -> + using ( ); + + [ with [, ...] ] + create function ([ ] [, ... ]) -> + using ; + + [ with [, ...] ] + create function ([ ] [, ... ]) -> + "{" [, ...] "}" ; + + # where is: + + [ ] : [ ] [ = ] + + # is: + + [ { variadic | named only } ] + + # is: + + [ { set of | optional } ] + + # and is: + + [ ] + + # and is one of + + set volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'} ; + create annotation := ; + using ( ) ; + using ; + + +Description +^^^^^^^^^^^ + +The command ``create function`` defines a new function. If *name* is +qualified with a module name, then the function is created in that +module, otherwise it is created in the current module. + +The function name must be distinct from that of any existing function +with the same argument types in the same module. Functions of +different argument types can share a name, in which case the functions +are called *overloaded functions*. + + +Parameters +^^^^^^^^^^ + +Most sub-commands and options of this command are identical to the +:ref:`SDL function declaration `, with +some additional features listed below: + +:eql:synopsis:`set volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'}` + Function volatility determines how aggressively the compiler can + optimize its invocations. Other than a slight syntactical + difference this is the same as the corresponding SDL declaration. + +:eql:synopsis:`create annotation := ` + Set the function's :eql:synopsis:`` to + :eql:synopsis:``. + + See :eql:stmt:`create annotation` for details. + + +Examples +^^^^^^^^ + +Define a function returning the sum of its arguments: + +.. code-block:: edgeql + + create function mysum(a: int64, b: int64) -> int64 + using ( + select a + b + ); + +The same, but using a variadic argument and an explicit language: + +.. code-block:: edgeql + + create function mysum(variadic argv: int64) -> int64 + using edgeql $$ + select sum(array_unpack(argv)) + $$; + +Define a function using the block syntax: + +.. code-block:: edgeql + + create function mysum(a: int64, b: int64) -> int64 { + using ( + select a + b + ); + create annotation title := "My sum function."; + }; + + +Alter function +-------------- + +:eql-statement: +:eql-haswith: + +Change the definition of a function. + +.. eql:synopsis:: + + [ with [, ...] ] + alter function ([ ] [, ... ]) "{" + [, ...] + "}" + + # where is: + + [ ] : [ ] [ = ] + + # and is one of + + set volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'} ; + reset volatility ; + rename to ; + create annotation := ; + alter annotation := ; + drop annotation ; + using ( ) ; + using ; + + +Description +^^^^^^^^^^^ + +The command ``alter function`` changes the definition of a function. +The command allows changing annotations, the volatility level, and +other attributes. + + +Subcommands +^^^^^^^^^^^ + +The following subcommands are allowed in the ``alter function`` block +in addition to the commands common to the ``create function``: + +:eql:synopsis:`reset volatility` + Remove explicitly specified volatility in favor of the volatility + inferred from the function body. + +:eql:synopsis:`rename to ` + Change the name of the function to *newname*. + +:eql:synopsis:`alter annotation ;` + Alter function :eql:synopsis:``. + See :eql:stmt:`alter annotation` for details. + +:eql:synopsis:`drop annotation ;` + Remove function :eql:synopsis:``. + See :eql:stmt:`drop annotation` for details. + +:eql:synopsis:`reset errmessage;` + Remove the error message from this abstract constraint. + The error message specified in the base abstract constraint + will be used instead. + + +Example +^^^^^^^ + +.. code-block:: edgeql + + create function mysum(a: int64, b: int64) -> int64 { + using ( + select a + b + ); + create annotation title := "My sum function."; + }; + + alter function mysum(a: int64, b: int64) { + set volatility := 'Immutable'; + drop annotation title; + }; + + alter function mysum(a: int64, b: int64) { + using ( + select (a + b) * 100 + ) + }; + + +Drop function +------------- + +:eql-statement: +:eql-haswith: + + +Remove a function. + +.. eql:synopsis:: + + [ with [, ...] ] + drop function ([ ] [, ... ]); + + # where is: + + [ ] : [ ] [ = ] + + +Description +^^^^^^^^^^^ + +The command ``drop function`` removes the definition of an existing function. +The argument types to the function must be specified, since there +can be different functions with the same name. + + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`` + The name (optionally module-qualified) of an existing function. + +:eql:synopsis:`` + The name of an argument used in the function definition. + +:eql:synopsis:`` + The mode of an argument: ``set of`` or ``optional`` or ``variadic``. + +:eql:synopsis:`` + The data type(s) of the function's arguments + (optionally module-qualified), if any. + + +Example +^^^^^^^ + +Remove the ``mysum`` function: + +.. code-block:: edgeql + + drop function mysum(a: int64, b: int64); + + .. list-table:: :class: seealso * - **See also** - * - :ref:`SDL > Functions ` - * - :ref:`DDL > Functions ` * - :ref:`Reference > Function calls ` * - :ref:`Introspection > Functions ` * - :ref:`Cheatsheets > Functions ` diff --git a/docs/datamodel/future.rst b/docs/datamodel/future.rst index 4128715577c..b48bf5c12a5 100644 --- a/docs/datamodel/future.rst +++ b/docs/datamodel/future.rst @@ -1,51 +1,156 @@ .. _ref_datamodel_future: =============== -Future Behavior +Future behavior =============== .. index:: future, nonrecursive_access_policies -Any time that we add new functionality to |Gel| we strive to do it in the -least disruptive way possible. Deprecation warnings, documentation and guides -can help make these transitions smoother, but sometimes the changes are just -too big, especially if they affect already existing functionality. It is often -inconvenient dealing with these changes at the same time as upgrading to a new -major version of Gel. To help with this transition we introduce -:ref:`future ` specification. - -The purpose of this specification is to provide a way to try out and ease into -an upcoming feature before a major release. Sometimes enabling future behavior -is necessary to fix some current issues. Other times enabling future behavior -can simply provide a way to test out the feature before it gets released, to -make sure that the current project codebase is compatible and well-behaved. It -provides a longer timeframe for adopting a new feature and for catching bugs -that arise from the change in behavior. - -The ``future`` specification is intended to help with transitions between -major releases. Once a feature is released this specification is no longer -necessary to enable that feature and it will be removed from the schema during -the upgrade process. - -Once some behavior is available as a ``future`` all new :ref:`projects -` enable this behavior by default when initializing an -empty database. It is possible to explicitly disable the ``future`` feature by -removing it from the schema, but it is not recommended unless the feature is -causing some issues which cannot be fixed otherwise. Existing projects don't -change their behavior by default, the ``future`` specification needs to be -added to the schema by the developer in order to gain early access to it. - -At the moment there is only one ``future`` available: - -- ``nonrecursive_access_policies``: makes access policies :ref:`non-recursive - ` and simplifies policy - interactions. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`SDL > Future Behavior ` - * - :eql:stmt:`DDL > CREATE FUTURE ` - * - :eql:stmt:`DDL > DROP FUTURE ` +This article explains what the ``using future ...;`` statement means in your +schema. + +Our goal is to make |Gel| the best database system in the world, which requires +us to keep evolving. Usually, we can add new functionality while preserving +backward compatibility, but on rare occasions we must implement changes that +require elaborate transitions. + +To handle these cases, we introduce *future* behavior, which lets you try out +upcoming features before a major release. Sometimes enabling a future is +necessary to fix current issues; other times it offers a safe and easy way to +ensure your codebase remains compatible. This approach provides more time to +adopt a new feature and identify any resulting bugs. + +Any time a behavior is available as a ``future,`` all new :ref:`projects +` enable it by default for empty databases. You can remove +a ``future`` from your schema if absolutely necessary, but doing so is +discouraged. Existing projects are unaffected by default, so you must manually +add the ``future`` specification to gain early access. + +Flags +===== + +At the moment there are three ``future`` flags available: + +- ``simple_scoping`` + + Introduced in |Gel| 6.0, this flag simplifies the scoping rules for + path expressions. Read more about it and in great detail in + :ref:`ref_eql_path_resolution`. + +- ``warn_old_scoping`` + + Introduced in |Gel| 6.0, this flag will emit a warning when a query + is detected to depend on the old scoping rules. This is an intermediate + step towards enabling the ``simple_scoping`` flag in existing large + codebases. + + Read more about this flag in :ref:`ref_warn_old_scoping`. + +.. _ref_datamodel_access_policies_nonrecursive: +.. _nonrecursive: + +- ``nonrecursive_access_policies``: makes access policies non-recursive. + + This flag is no longer used becauae the behavior is enabled + by default since |EdgeDB| 4. The flag was helpful to ease transition + from EdgeDB 3.x to 4.x. + + Since |EdgeDB| 3.0, access policy restrictions do **not** apply + to any access policy expression. This means that when reasoning about access + policies it is no longer necessary to take other policies into account. + Instead, all data is visible for the purpose of *defining* an access + policy. + + This change was made to simplify reasoning about access policies and + to allow certain patterns to be expressed efficiently. Since those who have + access to modifying the schema can remove unwanted access policies, no + additional security is provided by applying access policies to each + other's expressions. + + +.. _ref_eql_sdl_future: + +Declaring future flags +====================== + +Syntax +------ + +Declare that the current schema enables a particular future behavior. + +.. sdl:synopsis:: + + using future ";" + +Description +^^^^^^^^^^^ + +Future behavior declaration must be outside any :ref:`module block +` since this behavior affects the entire database and not +a specific module. + +Example +^^^^^^^ + +.. code-block:: sdl-invalid + + using future simple_scoping; + + +.. _ref_eql_ddl_future: + +DDL commands +============ + +This section describes the low-level DDL commands for creating and +dropping future flags. You typically don't need to use these commands directly, +but knowing about them is useful for reviewing migrations. + +Create future +------------- + +:eql-statement: + +Enable a particular future behavior for the current schema. + +.. eql:synopsis:: + + create future ";" + + +The command ``create future`` enables the specified future behavior for +the current branch. + +Example +^^^^^^^ + +.. code-block:: edgeql + + create future simple_scoping; + + +Drop future +----------- + +:eql-statement: + +Disable a particular future behavior for the current schema. + +.. eql:synopsis:: + + drop future ";" + +Description +^^^^^^^^^^^ + +The command ``drop future`` disables a currently active future behavior for the +current branch. However, this is only possible for versions of |Gel| when the +behavior in question is not officially introduced. Once a particular behavior is +introduced as the standard behavior in a |Gel| release, it cannot be disabled. + +Example +^^^^^^^ + +.. code-block:: edgeql + + drop future warn_old_scoping; diff --git a/docs/datamodel/globals.rst b/docs/datamodel/globals.rst index 60d9fd5af2a..c94fbed6233 100644 --- a/docs/datamodel/globals.rst +++ b/docs/datamodel/globals.rst @@ -6,32 +6,58 @@ Globals .. index:: global, required global -Schemas can contain scalar-typed *global variables*. +Schemas in Gel can contain typed *global variables*. These create a mechanism +for specifying session-level context that can be referenced in queries, +access policies, triggers, and elsewhere with the ``global`` keyword. + +Here's a very common example of a global variable representing the current +user ID: .. code-block:: sdl - global current_user_id: uuid; + global current_user_id: uuid; -These provide a useful mechanism for specifying session-level data that can be -referenced in queries with the ``global`` keyword. +.. tabs:: -.. code-block:: edgeql + .. code-tab:: edgeql - select User { - id, - posts: { title, content } - } - filter .id = global current_user_id; + select User { + id, + posts: { title, content } + } + filter .id = global current_user_id; + .. code-tab:: python + + # In a non-trivial example, `global current_user_id` would + # be used indirectly in an access policy or some other context. + await client.with_globals({'user_id': user_id}).qeury(''' + select User { + id, + posts: { title, content } + } + filter .id = global current_user_id; + ''') + + .. code-tab:: typescript + + // In a non-trivial example, `global current_user_id` would + // be used indirectly in an access policy or some other context. + await client.withGlobals({user_id}).qeury(''' + select User { + id, + posts: { title, content } + } + filter .id = global current_user_id; + ''') -As in the example above, this is particularly useful for representing the -notion of a session or "current user". Setting global variables -^^^^^^^^^^^^^^^^^^^^^^^^ +======================== -Global variables are set when initializing a client. The exact API depends on -which client library you're using. +Global variables are set at session level or when initializing a client. +The exact API depends on which client library you're using, but the general +behavior and principles are the same across all libraries. .. tabs:: @@ -40,26 +66,34 @@ which client library you're using. import createClient from 'gel'; const baseClient = createClient(); - // returns a new Client instance that stores the provided - // globals and sends them along with all future queries: + + // returns a new Client instance, that shares the underlying + // network connection with `baseClient` , but sends the configured + // globals along with all queries run through it: const clientWithGlobals = baseClient.withGlobals({ current_user_id: '2141a5b4-5634-4ccc-b835-437863534c51', }); - await clientWithGlobals.query(`select global current_user_id;`); + const result = await clientWithGlobals.query( + `select global current_user_id;` + ); .. code-tab:: python from gel import create_client - client = create_client().with_globals({ + base_client = create_client() + + # returns a new Client instance, that shares the underlying + # network connection with `base_client` , but sends the configured + # globals along with all queries run through it: + client = base_client.with_globals({ 'current_user_id': '580cc652-8ab8-4a20-8db9-4c79a4b1fd81' }) result = client.query(""" select global current_user_id; """) - print(result) .. code-tab:: go @@ -125,101 +159,363 @@ which client library you're using. Cardinality ------------ +=========== -Global variables can be marked ``required``; in this case, you must specify a -default value. +A global variable can be declared with one of two cardinalities: -.. code-block:: sdl +- ``single`` (the default): At most one value. +- ``multi``: A set of values. Only valid for computed global variables. + +In addition, a global can be marked ``required`` or ``optional`` (the default). +If marked ``required``, a default value must be provided. - required global one_string: str { - default := "Hi Mom!" - }; Computed globals ----------------- +================ .. index:: global, := -Global variables can also be computed. The value of computed globals are +Global variables can also be computed. The value of computed globals is dynamically computed when they are referenced in queries. .. code-block:: sdl - required global random_global := datetime_of_transaction(); + required global now := datetime_of_transaction(); The provided expression will be computed at the start of each query in which -the global is referenced. There's no need to provide an explicit type; the -type is inferred from the computed expression. +the global is referenced. There's no need to provide an explicit type; the type +is inferred from the computed expression. -Computed globals are not subject to the same constraints as non-computed ones; -specifically, they can be object-typed and have a ``multi`` cardinality. +Computed globals can also be object-typed and have ``multi`` cardinality. +For example: .. code-block:: sdl - global current_user_id: uuid; + global current_user_id: uuid; - # object-typed global - global current_user := ( - select User filter .id = global current_user_id - ); + # object-typed global + global current_user := ( + select User filter .id = global current_user_id + ); - # multi global - global current_user_friends := (global current_user).friends; + # multi global + global current_user_friends := (global current_user).friends; -Usage in schema ---------------- +Referencing globals +=================== -.. You may be wondering what purpose globals serve that can't. -.. For instance, the simple ``current_user_id`` example above could easily -.. be rewritten like so: +Unlike query parameters, globals can be referenced *inside your schema +declarations*: -.. .. code-block:: edgeql-diff +.. code-block:: sdl -.. select User { -.. id, -.. posts: { title, content } -.. } -.. - filter .id = global current_user_id -.. + filter .id = $current_user_id + type User { + name: str; + is_self := (.id = global current_user_id) + }; -.. There is a subtle difference between these two in terms of -.. developer experience. When using parameters, you must provide a -.. value for ``$current_user_id`` on each *query execution*. By constrast, -.. the value of ``global current_user_id`` is defined when you initialize -.. the client; you can use this "sessionified" client to execute -.. user-specific queries without needing to keep pass around the -.. value of the user's UUID. +This is particularly useful when declaring :ref:`access policies +`: -.. But that's a comparatively marginal difference. +.. code-block:: sdl -Unlike query parameters, globals can be referenced -*inside your schema declarations*. + type Person { + required name: str; -.. code-block:: sdl + access policy my_policy allow all + using (.id = global current_user_id); + } - type User { - name: str; - is_self := (.id = global current_user_id) - }; +Refer to :ref:`Access Policies ` for complete +documentation. -This is particularly useful when declaring :ref:`access policies -`. +.. _ref_eql_sdl_globals: +.. _ref_eql_sdl_globals_syntax: + +Declaring globals +================= + +This section describes the syntax to declare a global variable in your schema. + +Syntax +------ + +Define a new global variable in SDL, corresponding to the more explicit DDL +commands described later: + +.. sdl:synopsis:: + + # Global variable declaration: + [{required | optional}] [single] + global -> + [ "{" + [ default := ; ] + [ ] + ... + "}" ] + + # Computed global variable declaration: + [{required | optional}] [{single | multi}] + global := ; + + +Description +^^^^^^^^^^^ + +There are two different forms of ``global`` declarations, as shown in the +syntax synopsis above: + +1. A *settable* global (defined with ``-> ``) which can be changed using + a session-level :ref:`set ` command. + +2. A *computed* global (defined with ``:= ``), which cannot be + directly set but instead derives its value from the provided expression. + +The following options are available: + +:eql:synopsis:`required` + If specified, the global variable is considered *required*. It is an + error for this variable to have an empty value. If a global variable is + declared *required*, it must also declare a *default* value. + +:eql:synopsis:`optional` + The global variable is considered *optional*, i.e. it is possible for the + variable to have an empty value. (This is the default.) + +:eql:synopsis:`multi` + Specifies that the global variable may have a set of values. Only + *computed* global variables can have this qualifier. + +:eql:synopsis:`single` + Specifies that the global variable must have at most a *single* value. It + is assumed that a global variable is ``single`` if neither ``multi`` nor + ``single`` is specified. All non-computed global variables must be *single*. + +:eql:synopsis:`` + The name of the global variable. It can be fully-qualified with the module + name, or it is assumed to belong to the module in which it appears. + +:eql:synopsis:`` + The type must be a valid :ref:`type expression ` denoting a + non-abstract scalar or a container type. + +:eql:synopsis:` := ` + Defines a *computed* global variable. The provided expression must be a + :ref:`Stable ` EdgeQL expression. It can refer + to other global variables. The type of a *computed* global variable is + not limited to scalar and container types; it can also be an object type. + +The valid SDL sub-declarations are: + +:eql:synopsis:`default := ` + Specifies the default value for the global variable as an EdgeQL + expression. The default value is used in a session if the value was not + explicitly specified by the client, or was reset with the :ref:`reset + ` command. + +:sdl:synopsis:`` + Set global variable :ref:`annotation ` + to a given *value*. + + +Examples +-------- + +Declare a new global variable: .. code-block:: sdl - type Person { - required name: str; - access policy my_policy allow all using (.id = global current_user_id); - } + global current_user_id -> uuid; + global current_user := ( + select User filter .id = global current_user_id + ); -Refer to :ref:`Access Policies ` for complete -documentation. +Set the global variable to a specific value using :ref:`session-level commands +`: + +.. code-block:: edgeql + + set global current_user_id := + '00ea8eaa-02f9-11ed-a676-6bd11cc6c557'; + +Use the computed global variable that is based on the value that was just set: + +.. code-block:: edgeql + + select global current_user { name }; + +:ref:`Reset ` the global variable to +its default value: -.. list-table:: - :class: seealso +.. code-block:: edgeql + + reset global user_id; + + +.. _ref_eql_ddl_globals: + + +DDL commands +============ + +This section describes the low-level DDL commands for creating, altering, and +dropping globals. You typically don't need to use these commands directly, but +knowing about them is useful for reviewing migrations. + + +Create global +------------- + +:eql-statement: +:eql-haswith: + +Declare a new global variable using DDL. + +.. eql:synopsis:: + + [ with [, ...] ] + create [{required | optional}] [single] + global -> + [ "{" ; [...] "}" ] ; + + # Computed global variable form: + + [ with [, ...] ] + create [{required | optional}] [{single | multi}] + global := ; + + # where is one of + + set default := + create annotation := + +Description +^^^^^^^^^^^ + +As with SDL, there are two different forms of ``global`` declaration: + +- A global variable that can be :ref:`set ` + in a session. +- A *computed* global that is derived from an expression (and so cannot be + directly set in a session). + +The subcommands mirror those in SDL: + +:eql:synopsis:`set default := ` + Specifies the default value for the global variable as an EdgeQL + expression. The default value is used by the session if the value was not + explicitly specified or was reset with the :ref:`reset + ` command. + +:eql:synopsis:`create annotation := ` + Assign an annotation to the global variable. See :eql:stmt:`create annotation` + for details. + + +Examples +^^^^^^^^ + +Define a new global property ``current_user_id``: + +.. code-block:: edgeql + + create global current_user_id -> uuid; + +Define a new *computed* global property ``current_user`` based on the +previously defined ``current_user_id``: + +.. code-block:: edgeql + + create global current_user := ( + select User filter .id = global current_user_id + ); + + +Alter global +------------ + +:eql-statement: +:eql-haswith: + +Change the definition of a global variable. + +.. eql:synopsis:: + + [ with [, ...] ] + alter global + [ "{" ; [...] "}" ] ; + + # where is one of + + set default := + reset default + rename to + set required + set optional + reset optionalily + set single + set multi + reset cardinality + set type reset to default + using () + create annotation := + alter annotation := + drop annotation + +Description +^^^^^^^^^^^ + +The command :eql:synopsis:`alter global` changes the definition of a global +variable. It can modify default values, rename the global, or change other +attributes like optionality, cardinality, computed expressions, etc. + +Examples +^^^^^^^^ + +Set the ``description`` annotation of global variable ``current_user``: + +.. code-block:: edgeql + + alter global current_user + create annotation description := + 'Current User as specified by the global ID'; + +Make the ``current_user_id`` global variable ``required``: + +.. code-block:: edgeql + + alter global current_user_id { + set required; + # A required global variable MUST have a default value. + set default := '00ea8eaa-02f9-11ed-a676-6bd11cc6c557'; + } + + +Drop global +----------- + +:eql-statement: +:eql-haswith: + +Remove a global variable from the schema. + +.. eql:synopsis:: + + [ with [, ...] ] + drop global ; + +Description +^^^^^^^^^^^ + +The command :eql:synopsis:`drop global` removes the specified global variable +from the schema. + +Example +^^^^^^^ + +Remove the ``current_user`` global variable: + +.. code-block:: edgeql - * - **See also** - * - :ref:`SDL > Globals ` - * - :ref:`DDL > Globals ` + drop global current_user; diff --git a/docs/datamodel/index.rst b/docs/datamodel/index.rst index 1a363e79a0f..54a9d372e1a 100644 --- a/docs/datamodel/index.rst +++ b/docs/datamodel/index.rst @@ -22,6 +22,7 @@ Schema annotations globals access_policies + modules functions triggers mutation_rewrites @@ -121,8 +122,6 @@ Instances can be branched when working on new features, similar to branches in your VCS. Each branch has its own schema and data. -.. _ref_datamodel_modules: - Module ^^^^^^ @@ -131,65 +130,4 @@ Each |branch| has a schema consisting of several schemas into logical units. In practice, though, most users put their entire schema inside a single module called ``default``. -.. code-block:: sdl - - module default { - # declare types here - } - -.. versionadded:: 3.0 - - You may define nested modules using the following syntax: - - .. code-block:: sdl - - module dracula { - type Person { - required property name -> str; - multi link places_visited -> City; - property strength -> int16; - } - - module combat { - function fight( - one: dracula::Person, - two: dracula::Person - ) -> str - using ( - (one.name ?? 'Fighter 1') ++ ' wins!' - IF (one.strength ?? 0) > (two.strength ?? 0) - ELSE (two.name ?? 'Fighter 2') ++ ' wins!' - ); - } - } - - Here we have a ``dracula`` module containing a ``Person`` type. Nested in - the ``dracula`` module we have a ``combat`` module. - -.. _ref_name_resolution: - -.. note:: Name resolution - - When referencing schema objects from another module, you must use - a *fully-qualified* name in the form ``module_name::object_name``. - -The following module names are reserved by |Gel| and contain pre-defined -types, utility functions, and operators. - -* ``std``: standard types, functions, and operators in the :ref:`standard - library ` -* ``math``: algebraic and statistical :ref:`functions ` -* ``cal``: local (non-timezone-aware) and relative date/time :ref:`types and - functions ` -* ``schema``: types describing the :ref:`introspection - ` schema -* ``sys``: system-wide entities, such as user roles and - :ref:`databases ` -* ``cfg``: configuration and settings - -.. versionadded:: 3.0 - - You can chain together module names in a fully-qualified name to traverse a - tree of nested modules. For example, to call the ``fight`` function in the - nested module example above, you would use - ``dracula::combat::fight()``. +Read more about modules in the :ref:`modules ` section. diff --git a/docs/datamodel/indexes.rst b/docs/datamodel/indexes.rst index 1b9def1b21f..18329cf5845 100644 --- a/docs/datamodel/indexes.rst +++ b/docs/datamodel/indexes.rst @@ -4,201 +4,355 @@ Indexes ======= -.. index:: index on, performance, postgres query planner +.. index:: + index on, performance, postgres query planner An index is a data structure used internally to speed up filtering, ordering, -and grouping operations. Indexes help accomplish this in two key ways: +and grouping operations in |Gel|. Indexes help accomplish this in two key ways: -- They are pre-sorted which saves time on costly sort operations on rows. +- They are pre-sorted, which saves time on costly sort operations on rows. - They can be used by the query planner to filter out irrelevant rows. .. note:: - The Postgres query planner decides when to use indexes for a query. In some - cases — for example, when tables are small and it would be faster to scan - the whole table than to use an index — an applicable index may be ignored. + The Postgres query planner decides when to use indexes for a query. In some + cases—e.g. when tables are small—it may be faster to scan the whole table + rather than use an index. In such scenarios, the index might be ignored. - For more information on how it does this, read `the Postgres query planner - documentation - `_. + For more information on how the planner decides this, see + `the Postgres query planner documentation + `_. -Most commonly, indexes are declared within object type declarations and -reference a particular property. The index can be used to speed up queries -which reference that property in a ``filter``, ``order by``, or ``group`` -clause. -.. note:: +Tradeoffs +========= + +While improving query performance, indexes also increase disk and memory usage +and can slow down insertions and updates. Creating too many indexes may be +detrimental; only index properties you often filter, order, or group by. + +.. important:: + **Foreign and primary keys** + + In SQL databases, indexes are commonly used to index *primary keys* and + *foreign keys*. Gel's analog to a SQL primary key is the ``id`` field + automatically created for each object, while a link in Gel is the analog + to a SQL foreign key. Both of these are automatically indexed. + + Moreover, any property with an :eql:constraint:`exclusive` constraint + is also automatically indexed. - While improving query performance, indexes also increase disk and memory - usage and slow down insertions and updates. Creating too many indexes may be - detrimental; only index properties you often filter, order, or group by. Index on a property -------------------- +=================== -Below, we are referencing the ``User.name`` property with the :ref:`dot -notation shorthand `: ``.name``. +Most commonly, indexes are declared within object type declarations and +reference a particular property. The index can be used to speed up queries +that reference that property in a filter, order by, or group by clause: .. code-block:: sdl - type User { - required name: str; - index on (.name); - } + type User { + required name: str; + index on (.name); + } By indexing on ``User.name``, the query planner will have access to that index -for use when planning queries containing the property in a filter, order, or -group by. This may result in better performance in these queries as the -database can look up a name in the index instead of scanning through all -``User`` objects sequentially, although whether or not to use the index is -ultimately up to the Postgres query planner. +when planning queries using the ``name`` property. This may result in better +performance as the database can look up a name in the index instead of scanning +through all ``User`` objects sequentially—though ultimately it's up to the +Postgres query planner whether to use the index. -To see if an index can help your query, try adding the :ref:`analyze -` keyword before a query with an index compared to one -without. +To see if an index helps, compare query plans by adding +:ref:`analyze ` to your queries. .. note:: - Even if your database is too small now to benefit from an index, it may - benefit from one as it continues to grow. + Even if your database is small now, you may benefit from an index as it grows. Index on an expression ----------------------- +====================== Indexes may be defined using an arbitrary *singleton* expression that references multiple properties of the enclosing object type. .. important:: + A singleton expression is an expression that's guaranteed to return + *at most one* element. As such, you can't index on a ``multi`` property. - A singleton expression is an expression that's guaranteed to return *at most - one* element. As such, you can't index on a ``multi`` property. +Example: .. code-block:: sdl - type User { - required first_name: str; - required last_name: str; - index on (str_lower(.firstname + ' ' + .lastname)); - } + type User { + required first_name: str; + required last_name: str; + index on (str_lower(.first_name + ' ' + .last_name)); + } + Index on multiple properties ----------------------------- +============================ .. index:: tuple -A *composite index* is an index that references multiple properties. This can -speed up queries that filter, order, or group on both properties. +A *composite index* references multiple properties. This can speed up queries +that filter, order, or group on multiple properties at once. .. note:: - An index on multiple properties may also be used in queries where only a - single property in the index is filtered, ordered, or grouped by. It is - best to have the properties most likely to be used in this way listed first - when you create the index on multiple properties. + An index on multiple properties may also be used in queries where only a + single property in the index is referenced. In many traditional database + systems, placing the most frequently used columns first in the composite + index can improve the likelihood of its use. - Read `the Postgres documentation on multicolumn indexes - `_ to - learn more about how the query planner uses these indexes. + Read `the Postgres documentation on multicolumn indexes + `_ to learn + more about how the query planner uses these indexes. -In Gel, this index is created by indexing on a ``tuple`` of properties. +In |Gel|, a composite index is created by indexing on a ``tuple`` of properties: .. code-block:: sdl - type User { - required name: str; - required email: str; - index on ((.name, .email)); - } + type User { + required name: str; + required email: str; + index on ((.name, .email)); + } Index on a link property ------------------------- +======================== .. index:: __subject__, linkprops -Link properties can also be indexed. +Link properties can also be indexed. The special placeholder +``__subject__`` refers to the source object in a link property expression: .. code-block:: sdl - abstract link friendship { - strength: float64; - index on (__subject__@strength); - } + abstract link friendship { + strength: float64; + index on (__subject__@strength); + } - type User { - multi friends: User { - extending friendship; - }; - } + type User { + multi friends: User { + extending friendship; + }; + } Specify a Postgres index type ------------------------------ +============================= .. index:: pg::hash, pg::btree, pg::gin, pg::gist, pg::spgist, pg::brin .. versionadded:: 3.0 -Gel exposes Postgres indexes that you can use in your schemas. These are -exposed through the ``pg`` module. +Gel exposes Postgres index types that can be used directly in schemas via +the ``pg`` module: -* ``pg::hash``- Index based on a 32-bit hash derived from the indexed value +- ``pg::hash`` : Index based on a 32-bit hash of the value +- ``pg::btree`` : B-tree index (can help with sorted data retrieval) +- ``pg::gin`` : Inverted index for multi-element data (arrays, JSON) +- ``pg::gist`` : Generalized Search Tree for range and geometric searches +- ``pg::spgist`` : Space-partitioned GiST +- ``pg::brin`` : Block Range INdex -* ``pg::btree``- B-tree index can be used to retrieve data in sorted order +Example: -* ``pg::gin``- GIN is an "inverted index" appropriate for data values that - contain multiple elements, such as arrays and JSON +.. code-block:: sdl -* ``pg::gist``- GIST index can be used to optimize searches involving ranges + type User { + required name: str; + index pg::spgist on (.name); + } -* ``pg::spgist``- SP-GIST index can be used to optimize searches involving - ranges and strings -* ``pg::brin``- BRIN (Block Range INdex) index works with summaries about the - values stored in consecutive physical block ranges in the database +Annotate an index +================= + +.. index:: annotation -You can use them like this: +Indexes can include annotations: .. code-block:: sdl - type User { - required name: str; - index pg::spgist on (.name); - }; + type User { + name: str; + index on (.name) { + annotation description := 'Indexing all users by name.'; + }; + } -Annotate an index ------------------ +.. _ref_eql_sdl_indexes: -.. index:: annotation +Declaring indexes +================= -Indexes can be augmented with annotations. +This section describes the syntax to use indexes in your schema. -.. code-block:: sdl +Syntax +------ - type User { - name: str; - index on (.name) { - annotation description := 'Indexing all users by name.'; - }; - } +.. sdl:synopsis:: -.. important:: + index on ( ) + [ except ( ) ] + [ "{" "}" ] ; + +.. rubric:: Description + +- :sdl:synopsis:`on ( )` + + The expression to index. It must be :ref:`Immutable ` + but may refer to the indexed object's properties/links. The expression itself + must be parenthesized. + +- :eql:synopsis:`except ( )` + + An optional condition. If ```` evaluates to ``true``, the object + is omitted from the index; if ``false`` or empty, it is included. + +- :sdl:synopsis:`` + + Allows setting index :ref:`annotation ` to a given + value. + + +.. _ref_eql_ddl_indexes: + +DDL commands +============ + +This section describes the low-level DDL commands for creating, altering, and +dropping indexes. You typically don't need to use these commands directly, but +knowing about them is useful for reviewing migrations. + + +Create index +------------ + +:eql-statement: + +.. eql:synopsis:: + + create index on ( ) + [ except ( ) ] + [ "{" ; [...] "}" ] ; + + # where is one of + + create annotation := + +Creates a new index for a given object type or link using *index-expr*. + +- Most parameters/options match those in + :ref:`Declaring indexes `. + +- Allowed subcommand: + + :eql:synopsis:`create annotation := ` + Assign an annotation to this index. + See :eql:stmt:`create annotation` for details. + +Example: + +.. code-block:: edgeql + + create type User { + create property name -> str { + set default := ''; + }; + + create index on (.name); + }; + + +Alter index +----------- + +:eql-statement: + +Alter the definition of an index. + +.. eql:synopsis:: + + alter index on ( ) [ except ( ) ] + [ "{" ; [...] "}" ] ; + + # where is one of + + create annotation := + alter annotation := + drop annotation + +The command ``alter index`` is used to change the :ref:`annotations +` of an index. The *index-expr* is used to +identify the index to be altered. + +:sdl:synopsis:`on ( )` + The specific expression for which the index is made. Note also + that ```` itself has to be parenthesized. + +The following subcommands are allowed in the ``alter index`` block: + +:eql:synopsis:`create annotation := ` + Set index :eql:synopsis:`` to + :eql:synopsis:``. + See :eql:stmt:`create annotation` for details. + +:eql:synopsis:`alter annotation ;` + Alter index :eql:synopsis:``. + See :eql:stmt:`alter annotation` for details. + +:eql:synopsis:`drop annotation ;` + Remove constraint :eql:synopsis:``. + See :eql:stmt:`drop annotation` for details. + + +Example: + +.. code-block:: edgeql + + alter type User { + alter index on (.name) { + create annotation title := 'User name index'; + }; + }; + + +Drop index +---------- + +:eql-statement: + +Remove an index from a given schema item. + +.. eql:synopsis:: + + drop index on ( ) [ except ( ) ] ; + +Removes an index from a schema item. + +- :sdl:synopsis:`on ( )` identifies the indexed expression. + +This statement can only be used as a subdefinition in another DDL statement. + +Example: + +.. code-block:: edgeql - **Foreign and primary keys** + alter type User { + drop index on (.name); + }; - In SQL databases, indexes are commonly used to index *primary keys* and - *foreign keys*. Gel's analog to SQL's primary key is the ``id`` field - that gets automatically created for each object, while a link in Gel - is the analog to SQL's foreign key. Both of these are automatically indexed. - Moreover, any property with an :eql:constraint:`exclusive` constraint - is also automatically indexed. .. list-table:: - :class: seealso + :class: seealso - * - **See also** - * - :ref:`SDL > Indexes ` - * - :ref:`DDL > Indexes ` - * - :ref:`Introspection > Indexes ` + * - **See also** + - :ref:`Introspection > Indexes ` diff --git a/docs/datamodel/links.rst b/docs/datamodel/links.rst index 89ee5b2823b..253bb454b6d 100644 --- a/docs/datamodel/links.rst +++ b/docs/datamodel/links.rst @@ -4,54 +4,56 @@ Links ===== -.. index:: link, relation, source, target, backlinks, foreign key +Links define a relationship between two +:ref:`object types ` in Gel. -Links define a specific relationship between two :ref:`object -types `. +Links in |Gel| are incredibly powerful and flexible. They can be used to model +relationships of any cardinality, can be traversed in both directions, +can be polymorphic, can have constraints, and many other things. -Defining links --------------- + +Links are directional +===================== + +Links are *directional*: they have a **source** (the type on which they are +declared) and a **target** (the type they point to). + +E.g. the following schema defines a link from ``Person`` to ``Person`` and +a link from ``Company`` to ``Person``: .. code-block:: sdl - type Person { - best_friend: Person; - } + type Person { + link best_friend: Person; + } + + type Company { + multi link employees: Person; + } + +The ``employees`` link's source is ``Company`` and its target is ``Person``. + +The ``link`` keyword is optional, and can be omitted. -Links are *directional*; they have a source (the object type on which they are -declared) and a *target* (the type they point to). Link cardinality ----------------- +================ .. index:: single, multi All links have a cardinality: either ``single`` or ``multi``. The default is ``single`` (a "to-one" link). Use the ``multi`` keyword to declare a "to-many" -link. +link: .. code-block:: sdl - type Person { - multi friends: Person; - } + type Person { + multi friends: Person; + } -On the other hand, backlinks work in reverse to find objects that link to the -object, and thus assume ``multi`` as a default. Use the ``single`` keyword to -declare a "to-one" backlink. - -.. code-block:: sdl - - type Author { - posts := .` to +``Shirts`` by traversing the ``Shirt.owner`` link *in reverse*, known as a +**backlink**. See the :ref:`select docs ` to learn more. + .. _ref_guide_one_to_many: One-to-many -^^^^^^^^^^^ +----------- Conceptually, one-to-many and many-to-one relationships are identical; the -"directionality" of a relation is just a matter of perspective. Here, the -same "shirt owner" relationship is represented with a ``multi`` link. +"directionality" is a matter of perspective. Here, the same "shirt owner" +relationship is represented with a ``multi`` link: .. code-block:: sdl - type Person { - required name: str; - multi shirts: Shirt { - # ensures a one-to-many relationship - constraint exclusive; - } + type Person { + required name: str; + multi shirts: Shirt { + # ensures a one-to-many relationship + constraint exclusive; } + } - type Shirt { - required color: str; - } + type Shirt { + required color: str; + } .. note:: - Don't forget the exclusive constraint! This is required to ensure that each - ``Shirt`` corresponds to a single ``Person``. Without it, the relationship - will be many-to-many. + Don't forget the ``exclusive`` constraint! Without it, the relationship + becomes many-to-many. Under the hood, a ``multi`` link is stored in an intermediate `association table `_, whereas a @@ -203,88 +284,84 @@ table `_, whereas a .. note:: - Choosing a link direction can be tricky when modeling these kinds of - relationships. Should you model the relationship as one-to-many using a - ``multi`` link, or as many-to-one using a ``single`` link with a - backlink to traverse in the other direction? A general rule of thumb - in this case is as follows. - - Use a ``multi`` link if: + Choosing a link direction can be tricky. Should you model this + relationship as one-to-many (with a ``multi`` link) or as many-to-one + (with a ``single`` link and a backlink)? A general rule of thumb: - - The relationship is relatively stable and thus not updated very - frequently. For example, a list of postal addresses in a - user profile. - - The number of elements in the link tends to be small. - - Otherwise, prefer a single link from one object type coupled with a - computed backlink on the other. This is marginally more efficient - and generally recommended when modeling 1:N relations: + - Use a ``multi`` link if the relationship is relatively stable and + not updated frequently, and the set of related objects is typically + small. For example, a list of postal addresses in a user profile. + - Otherwise, prefer a single link from one object type and a computed + backlink on the other. This can be more efficient and is generally + recommended for 1:N relations: .. code-block:: sdl - type Post { - required author: User; - } + type Post { + required author: User; + } + + type User { + multi posts := (.`. For instance: + + .. code-block:: edgeql + + # 🚫 Does not work + insert Movie { + title := 'The Incredible Hulk', + characters := { + ( + select Person { + @character_name := 'The Hulk' + } + filter .name = 'Mark Ruffalo' + ), + ( + select Person { + @character_name := 'Abomination' + } + filter .name = 'Tim Roth' + ) + } + }; - type Person { - name: str; - multi family_members: Person { - relationship: str; - } - } + will produce an error ``QueryError: invalid reference to link property in + top level shape``. + + One workaround is to insert them via a ``for`` loop, combined with + :eql:func:`assert_distinct`: + + .. code-block:: edgeql + + # ✅ Works! + insert Movie { + title := 'The Incredible Hulk', + characters := assert_distinct(( + with actors := { + ('The Hulk', 'Mark Ruffalo'), + ('Abomination', 'Tim Roth') + }, + for actor in actors union ( + select Person { + @character_name := actor.0 + } + filter .name = actor.1 + ) + )) + }; -.. note:: +Querying link properties +------------------------ + +To query a link property, add the link property's name (prefixed with ``@``) +in the shape: - The divide between "link" and "property" is important when it comes to - understanding what link properties can do. They are link **properties**, - not link **links**. This means link properties can contain only primitive - data — data of any of the :ref:`scalar types ` like - ``str``, ``int32``, or ``bool``, :ref:`enums `, - :ref:`arrays `, and :ref:`tuples - `. They cannot contain links to other objects. +.. code-block:: edgeql-repl - That means this would not work: + db> select Person { + ... name, + ... family_members: { + ... name, + ... @relationship + ... } + ... }; - .. code-block:: +.. note:: - type Person { - name: str; - multi friends: Person { - introduced_by: Person; - } - } + In the results above, Bob has a *step-sister* property on the link to + Alice, but Alice does not automatically have a property describing Bob. + Changes to link properties are not mirrored on the "backlink" side unless + explicitly updated, because link properties cannot be required. .. note:: - Link properties cannot be made required. They are always optional. + For a full guide on modeling, inserting, updating, and querying link + properties, see the :ref:`Using Link Properties ` + guide. -Above, we model a family tree with a single ``Person`` type. The ``Person. -family_members`` link is a many-to-many relation; each ``family_members`` link -can contain a string ``relationship`` describing the relationship of the two -individuals. -Due to how they're persisted under the hood, link properties must always be -``single`` and ``optional``. +.. _ref_datamodel_link_deletion: -In practice, link properties are most useful with many-to-many relationships. -In that situation there's a significant difference between the *relationship* -described by the link and the *target object*. Thus it makes sense to separate -properties of the relationships and properties of the target objects. On the -other hand, for one-to-one, one-to-many, and many-to-one relationships there's -an exact correspondence between the link and one of the objects being linked. -In these situations any property of the relationship can be equally expressed -as the property of the source object (for one-to-many and one-to-one cases) or -as the property of the target object (for many-to-one and one-to-one cases). -It is generally advisable to use object properties instead of link properties -in these cases due to better ergonomics of selecting, updating, and even -casting into :eql:type:`json` when keeping all data in the same place rather -than spreading it across link and object properties. +Deletion policies +================= +.. index:: on target delete, on source delete, restrict, delete source, allow, + deferred restrict, delete target, if orphan -Inserting and updating link properties -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Links can declare their own **deletion policy** for when the **target** or +**source** is deleted. -To add a link with a link property, add the link property to a shape on the -linked object being added. Be sure to prepend the link property's name with -``@``. +Target deletion +--------------- -.. code-block:: edgeql +The clause ``on target delete`` determines the action when the target object is +deleted: - insert Person { - name := "Bob", - family_members := ( - select detached Person { - @relationship := "sister" - } - filter .name = "Alice" - ) - }; +- ``restrict`` (default) — raises an exception if the target is deleted. +- ``delete source`` — deletes the source when the target is deleted (a cascade). +- ``allow`` — removes the target from the link if the target is deleted. +- ``deferred restrict`` — like ``restrict`` but defers the error until the + end of the transaction if the object remains linked. -The shape could alternatively be included on an insert if the object being -linked (the ``Person`` named "Alice" in this example) is being inserted as part -of the query. If the outer person ("Bob" in the example) already exists and -only the links need to be added, this can be done in an ``update`` query -instead of an ``insert`` as shown in the example above. +.. code-block:: sdl -Updating a link's property is similar to adding a new one except that you no -longer need to select from the object type being linked: you can instead select -the existing link on the object being updated because the link has already been -established. Here, we've discovered that Alice is actually Bob's *step*-sister, -so we want to change the link property on the already-established link between -the two: + type MessageThread { + title: str; + } -.. code-block:: edgeql + type Message { + content: str; + chat: MessageThread { + on target delete delete source; + } + } - update Person - filter .name = "Bob" - set { - family_members := ( - select .family_members { - @relationship := "step-sister" - } - filter .name = "Alice" - ) - }; -Using ``select .family_members`` here with the shape including the link -property allows us to modify the link property of the existing link. +.. _ref_datamodel_links_source_deletion: -.. warning:: +Source deletion +--------------- - A link property cannot be referenced in a set union *except* in the case of - a :ref:`for loop `. That means this will *not* work: - - .. code-block:: edgeql - - # 🚫 Does not work - insert Movie { - title := 'The Incredible Hulk', - characters := {( - select Person { - @character_name := 'The Hulk' - } filter .name = 'Mark Ruffalo' - ), - ( - select Person { - @character_name := 'Abomination' - } filter .name = 'Tim Roth' - )} - }; - - That query will produce an error: ``QueryError: invalid reference to link - property in top level shape`` - - You can use this workaround instead: - - .. code-block:: edgeql - - # ✅ Works! - insert Movie { - title := 'The Incredible Hulk', - characters := assert_distinct(( - with actors := { - ('The Hulk', 'Mark Ruffalo'), - ('Abomination', 'Tim Roth') - }, - for actor in actors union ( - select Person { - @character_name := actor.0 - } filter .name = actor.1 - ) - )) - }; - - Note that we are also required to wrap the ``actors`` query with - :eql:func:`assert_distinct` here to assure the compiler that the result set - is distinct. +The clause ``on source delete`` determines the action when the **source** is +deleted: +- ``allow`` — deletes the source, removing the link to the target. +- ``delete target`` — unconditionally deletes the target as well. +- ``delete target if orphan`` — deletes the target if and only if it's no + longer linked by any other object *via the same link*. -Querying link properties -^^^^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: sdl + + type MessageThread { + title: str; + multi messages: Message { + on source delete delete target; + } + } -To query a link property, add the link property's name prepended with ``@`` to -a shape on the link. + type Message { + content: str; + } -.. code-block:: edgeql-repl +You can add ``if orphan`` if you'd like to avoid deleting a target that remains +linked elsewhere via the **same** link name. - db> select Person { - ... name, - ... family_members: { - ... name, - ... @relationship - ... } - ... }; - { - default::Person {name: 'Alice', family_members: {}}, - default::Person { - name: 'Bob', - family_members: { - default::Person {name: 'Alice', @relationship: 'step-sister'} - } - }, +.. code-block:: sdl-diff + + type MessageThread { + title: str; + multi messages: Message { + - on source delete delete target; + + on source delete delete target if orphan; + } } .. note:: - In the query results above, Alice appears to have no family members even - though we know that, if she is Bob's step-sister, he must be her - step-brother. We would need to update Alice manually before this is - reflected in the database. Since link properties cannot be required, not - setting one is always allowed and results in the value being the empty set - (``{}``). + The ``if orphan`` qualifier **does not** apply globally across + all links in the database or even all links from the same type. If another + link *by a different name* or *with a different on-target-delete* policy + points at the same object, it *doesn't* prevent the object from being + considered "orphaned" for the link that includes ``if orphan``. -.. note:: - For a full guide on modeling, inserting, updating, and querying link - properties, see the :ref:`Using Link Properties ` - guide. +.. _ref_datamodel_link_polymorphic: -.. _ref_datamodel_link_deletion: +Polymorphic links +================= -Deletion policies ------------------ +.. index:: abstract, subtypes, polymorphic -.. index:: on target delete, on source delete, restrict, delete source, allow, - deferred restrict, delete target, if orphan +Links can be **polymorphic**, i.e., have an ``abstract`` target. In the +example below, we have an abstract type ``Person`` with concrete subtypes +``Hero`` and ``Villain``: + +.. code-block:: sdl -Links can declare their own **deletion policy**. There are two kinds of events -that might trigger these policies: *target deletion* and *source deletion*. + abstract type Person { + name: str; + } -Target deletion -^^^^^^^^^^^^^^^ + type Hero extending Person { + # additional fields + } -Target deletion policies determine what action should be taken when the -*target* of a given link is deleted. They are declared with the ``on target -delete`` clause. + type Villain extending Person { + # additional fields + } + +A polymorphic link can target any non-abstract subtype: .. code-block:: sdl - type MessageThread { - title: str; - } + type Movie { + title: str; + multi characters: Person; + } - type Message { - content: str; - chat: MessageThread { - on target delete delete source; - } - } +When querying a polymorphic link, you can filter by a specific subtype, cast +the link to a subtype, etc. See :ref:`Polymorphic Queries ` +for details. -The ``Message.chat`` link in the example uses the ``delete source`` policy. -There are 4 available target deletion policies. +Abstract links +============== -- ``restrict`` (default) - Any attempt to delete the target object immediately - raises an exception. -- ``delete source`` - when the target of a link is deleted, the source - is also deleted. This is useful for implementing cascading deletes. +.. index:: abstract + +It's possible to define ``abstract`` links that aren't tied to a particular +source or target, and then extend them in concrete object types. This can help +eliminate repetitive declarations: - .. note:: +.. code-block:: sdl - There is `a limit - `_ to the depth of a deletion - cascade due to an upstream stack size limitation. + abstract link link_with_strength { + strength: float64; + index on (__subject__@strength); + } -- ``allow`` - the target object is deleted and is removed from the - set of the link targets. -- ``deferred restrict`` - any attempt to delete the target object - raises an exception at the end of the transaction, unless by - that time this object is no longer in the set of link targets. + type Person { + multi friends: Person { + extending link_with_strength; + }; + } -.. _ref_datamodel_links_source_deletion: -Source deletion -^^^^^^^^^^^^^^^ +.. _ref_eql_sdl_links_overloading: -Source deletion policies determine what action should be taken when the -*source* of a given link is deleted. They are declared with the ``on source -delete`` clause. +Overloading +=========== -There are 3 available source deletion policies: +.. index:: overloaded -- ``allow`` - the source object is deleted and is removed from the set of the - link's source objects. -- ``delete target`` - when the source of a link is deleted, the target - is unconditionally deleted. -- ``delete target if orphan`` - the source object is deleted and the target - object is unconditionally deleted unless the target object is linked to by - another source object via the same link. +When an inherited link is modified (by adding more constraints or changing its +target type, etc.), the ``overloaded`` keyword is required. This prevents +unintentional overloading due to name clashes: .. code-block:: sdl - type MessageThread { - title: str; - multi messages: Message { - on source delete delete target; - } - } + abstract type Friendly { + # this type can have "friends" + multi friends: Friendly; + } + + type User extending Friendly { + # overload the link target to to be specifically User + overloaded multi friends: User; + + # ... other links and properties + } + + +.. _ref_eql_sdl_links: +.. _ref_eql_sdl_links_syntax: + +Declaring links +=============== + +This section describes the syntax to use links in your schema. + +Syntax +------ + +.. sdl:synopsis:: + + # Concrete link form used inside type declaration: + [ overloaded ] [{required | optional}] [{single | multi}] + [ link ] : + [ "{" + [ extending [, ...] ; ] + [ default := ; ] + [ readonly := {true | false} ; ] + [ on target delete ; ] + [ on source delete ; ] + [ ] + [ ] + [ ] + ... + "}" ] + + # Computed link form used inside type declaration: + [{required | optional}] [{single | multi}] + [ link ] := ; + + # Computed link form used inside type declaration (extended): + [ overloaded ] [{required | optional}] [{single | multi}] + link [: ] + [ "{" + using () ; + [ extending [, ...] ; ] + [ ] + [ ] + ... + "}" ] + + # Abstract link form: + abstract link + [ "{" + [ extending [, ...] ; ] + [ readonly := {true | false} ; ] + [ ] + [ ] + [ ] + [ ] + ... + "}" ] + +There are several forms of link declaration, as shown in the syntax synopsis +above: + +- the first form is the canonical definition form; +- the second form is used for defining a + :ref:`computed link `; +- and the last form is used to define an abstract link. + +The following options are available: + +:eql:synopsis:`overloaded` + If specified, indicates that the link is inherited and that some + feature of it may be altered in the current object type. It is an + error to declare a link as *overloaded* if it is not inherited. + +:eql:synopsis:`required` + If specified, the link is considered *required* for the parent + object type. It is an error for an object to have a required + link resolve to an empty value. Child links **always** inherit + the *required* attribute, i.e it is not possible to make a + required link non-required by extending it. + +:eql:synopsis:`optional` + This is the default qualifier assumed when no qualifier is + specified, but it can also be specified explicitly. The link is + considered *optional* for the parent object type, i.e. it is + possible for the link to resolve to an empty value. + +:eql:synopsis:`multi` + Specifies that there may be more than one instance of this link + in an object, in other words, ``Object.link`` may resolve to a set + of a size greater than one. + +:eql:synopsis:`single` + Specifies that there may be at most *one* instance of this link + in an object, in other words, ``Object.link`` may resolve to a set + of a size not greater than one. ``single`` is assumed if nether + ``multi`` nor ``single`` qualifier is specified. + +:eql:synopsis:`extending [, ...]` + Optional clause specifying the *parents* of the new link item. + + Use of ``extending`` creates a persistent schema relationship + between the new link and its parents. Schema modifications + to the parent(s) propagate to the child. + + If the same *property* name exists in more than one parent, or + is explicitly defined in the new link and at least one parent, + then the data types of the property targets must be *compatible*. + If there is no conflict, the link properties are merged to form a + single property in the new link item. + +:eql:synopsis:`` + The type must be a valid :ref:`type expression ` + denoting an object type. + +The valid SDL sub-declarations are listed below: + +:eql:synopsis:`default := ` + Specifies the default value for the link as an EdgeQL expression. + The default value is used in an ``insert`` statement if an explicit + value for this link is not specified. + + The expression must be :ref:`Stable `. + +:eql:synopsis:`readonly := {true | false}` + If ``true``, the link is considered *read-only*. Modifications + of this link are prohibited once an object is created. All of the + derived links **must** preserve the original *read-only* value. + +:sdl:synopsis:`` + Set link :ref:`annotation ` + to a given *value*. + +:sdl:synopsis:`` + Define a concrete :ref:`property ` on the link. + +:sdl:synopsis:`` + Define a concrete :ref:`constraint ` on the link. + +:sdl:synopsis:`` + Define an :ref:`index ` for this abstract + link. Note that this index can only refer to link properties. + + +.. _ref_eql_ddl_links: + +DDL commands +============ + +This section describes the low-level DDL commands for creating, altering, and +dropping links. You typically don't need to use these commands directly, but +knowing about them is useful for reviewing migrations. + +Create link +----------- - type Message { - content: str; - } +:eql-statement: +:eql-haswith: -Under this policy, deleting a ``MessageThread`` will *unconditionally* delete -its ``messages`` as well. +Define a new link. -To avoid deleting a ``Message`` that is linked to by other ``MessageThread`` -objects via their ``message`` link, append ``if orphan`` to that link's -deletion policy. +.. eql:synopsis:: + + [ with [, ...] ] + {create|alter} type "{" + [ ... ] + create [{required | optional}] [{single | multi}] + link + [ extending [, ...] ] -> + [ "{" ; [...] "}" ] ; + [ ... ] + "}" -.. code-block:: sdl-diff + # Computed link form: + + [ with [, ...] ] + {create|alter} type "{" + [ ... ] + create [{required | optional}] [{single | multi}] + link := ; + [ ... ] + "}" - type MessageThread { - title: str; - multi messages: Message { - - on source delete delete target; - + on source delete delete target if orphan; - } - } + # Abstract link form: -.. note:: + [ with [, ...] ] + create abstract link [::] [extending [, ...]] + [ "{" ; [...] "}" ] - The ``if orphan`` qualifier does not apply globally across all links in the - database or across any other links even if they're from the same type. - Deletion policies using ``if orphan`` will result in the target being - deleted unless + # where is one of - 1. it is linked by another object via **the same link the policy is on**, - or - 2. its deletion is restricted by another link's ``on target delete`` policy - (which defaults to ``restrict`` unless otherwise specified) + set default := + set readonly := {true | false} + create annotation := + create property ... + create constraint ... + on target delete + on source delete + reset on target delete + create index on - For example, a ``Message`` might be linked from both a ``MessageThread`` - and a ``Channel``, which is defined like this: +Description +^^^^^^^^^^^ - .. code-block:: sdl +The combinations of ``create type ... create link`` and ``alter type ... +create link`` define a new concrete link for a given object type, in DDL form. - type Channel { - title: str; - multi messages: Message { - on target delete allow; - } - } +There are three forms of ``create link``: - If the ``MessageThread`` linking to the ``Message`` is deleted, the source - deletion policy would still result in the ``Message`` being deleted as long - as no other ``MessageThread`` objects link to it on their ``messages`` link - and the deletion isn't otherwise restricted (e.g., the default policy of - ``on target delete restrict`` has been overridden, as in the schema above). - The object is deleted despite not being orphaned with respect to *all* - links because it *is* orphaned with respect to the ``MessageThread`` type's - ``messages`` field, which is the link governed by the deletion policy. - - If the ``Channel`` type's ``messages`` link had the default policy, the - outcome would change. - - .. code-block:: sdl-diff - - type Channel { - title: str; - multi messages: Message { - - on target delete allow; - } - } +1. The canonical definition form (specifying a target type). +2. The computed link form (declaring a link via an expression). +3. The abstract link form (declaring a module-level link). - With this schema change, the ``Message`` object would *not* be deleted, but - not because the message isn't globally orphaned. Deletion would be - prevented because of the default target deletion policy of ``restrict`` - which would now be in force on the linking ``Channel`` object's - ``messages`` link. +Parameters +^^^^^^^^^^^ - The limited scope of ``if orphan`` holds true even when the two links to an - object are from the same type. If ``MessageThread`` had two different links - both linking to messages — maybe the existing ``messages`` link and another - called ``related`` used to link other related ``Message`` objects that are - not in the thread — ``if orphan`` on a deletion policy on ``message`` could - result in linked messages being deleted even if they were also linked from - another ``MessageThread`` object's ``related`` link because they were - orphaned with respect to the ``messages`` link. +Most sub-commands and options mirror those found in the +:ref:`SDL link declaration `. In DDL form: +- ``set default := `` specifies a default value. +- ``set readonly := {true | false}`` makes the link read-only or not. +- ``create annotation := `` adds an annotation. +- ``create property ...`` defines a property on the link. +- ``create constraint ...`` defines a constraint on the link. +- ``on target delete `` and ``on source delete `` specify + deletion policies. +- ``reset on target delete`` resets the target deletion policy to default + or inherited. +- ``create index on `` creates an index on the link. -.. _ref_datamodel_link_polymorphic: +Examples +^^^^^^^^ -Polymorphic links ------------------ +.. code-block:: edgeql -.. index:: abstract, subtypes + alter type User { + create multi link friends -> User + }; -Links can have ``abstract`` targets, in which case the link is considered -**polymorphic**. Consider the following schema: +.. code-block:: edgeql -.. code-block:: sdl + alter type User { + create link special_group := ( + select __source__.friends + filter .town = __source__.town + ) + }; - abstract type Person { - name: str; - } +.. code-block:: edgeql - type Hero extending Person { - # additional fields - } + create abstract link orderable { + create property weight -> std::int64 + }; + + alter type User { + create multi link interests extending orderable -> Interest + }; + + +Alter link +---------- + +:eql-statement: +:eql-haswith: + +Changes the definition of a link. + +.. eql:synopsis:: + + [ with [, ...] ] + {create|alter} type "{" + [ ... ] + alter link + [ "{" ] ; [...] [ "}" ]; + [ ... ] + "}" + + [ with [, ...] ] + alter abstract link [::] + [ "{" ] ; [...] [ "}" ]; + + # where is one of + + set default := + reset default + set readonly := {true | false} + reset readonly + rename to + extending ... + set required + set optional + reset optionality + set single + set multi + reset cardinality + set type [using ()] + reset type + using () + create annotation := + alter annotation := + drop annotation + create property ... + alter property ... + drop property ... + create constraint ... + alter constraint ... + drop constraint ... + on target delete + on source delete + create index on + drop index on + +Description +^^^^^^^^^^^ - type Villain extending Person { - # additional fields - } +This command modifies an existing link on a type. It can also be used on +an abstract link at the module level. -The ``abstract`` type ``Person`` has two concrete subtypes: ``Hero`` and -``Villain``. Despite being abstract, ``Person`` can be used as a link target in -concrete object types. +Parameters +^^^^^^^^^^ -.. code-block:: sdl +- ``rename to `` changes the link's name. +- ``extending ...`` changes or adds link parents. +- ``set required`` / ``set optional`` changes the link optionality. +- ``reset optionality`` reverts optionality to default or inherited value. +- ``set single`` / ``set multi`` changes cardinality. +- ``reset cardinality`` reverts cardinality to default or inherited value. +- ``set type [using ()]`` changes the link's target type. +- ``reset type`` reverts the link's type to inherited. +- ``using ()`` changes the expression of a computed link. +- ``create annotation``, ``alter annotation``, ``drop annotation`` manage + annotations. +- ``create property``, ``alter property``, ``drop property`` manage link + properties. +- ``create constraint``, ``alter constraint``, ``drop constraint`` manage + link constraints. +- ``on target delete `` and ``on source delete `` manage + deletion policies. +- ``reset on target delete`` reverts the target deletion policy. +- ``create index on `` / ``drop index on `` manage + indexes on link properties. + +Examples +^^^^^^^^ - type Movie { - title: str; - multi characters: Person; - } +.. code-block:: edgeql -In practice, the ``Movie.characters`` link can point to a ``Hero``, -``Villain``, or any other non-abstract subtype of ``Person``. For details on -how to write queries on such a link, refer to the :ref:`Polymorphic Queries -docs ` + alter type User { + alter link friends create annotation title := "Friends"; + }; +.. code-block:: edgeql -Abstract links --------------- + alter abstract link orderable rename to sorted; -.. index:: abstract +.. code-block:: edgeql -It's possible to define ``abstract`` links that aren't tied to a particular -*source* or *target*. If you're declaring several links with the same set -of properties, annotations, constraints, or indexes, abstract links can be used -to eliminate repetitive SDL. + alter type User { + alter link special_group using ( + # at least one of the friend's interests + # must match the user's + select __source__.friends + filter .interests IN __source__.interests + ); + }; -.. code-block:: sdl +Drop link +--------- - abstract link link_with_strength { - strength: float64; - index on (__subject__@strength); - } +:eql-statement: +:eql-haswith: + +Removes the specified link from the schema. + +.. eql:synopsis:: + + [ with [, ...] ] + alter type "{" + [ ... ] + drop link + [ ... ] + "}" + + [ with [, ...] ] + drop abstract link []:: + +Description +^^^^^^^^^^^ + +- ``alter type ... drop link `` removes the link from an object type. +- ``drop abstract link `` removes an abstract link from the schema. + +Examples +^^^^^^^^ + +.. code-block:: edgeql + + alter type User drop link friends; + +.. code-block:: edgeql + + drop abstract link orderable; - type Person { - multi friends: Person { - extending link_with_strength; - }; - } .. list-table:: :class: seealso * - **See also** - * - :ref:`SDL > Links ` - * - :ref:`DDL > Links ` - * - :ref:`Introspection > Object types - ` + - :ref:`Introspection > Object types ` diff --git a/docs/datamodel/modules.rst b/docs/datamodel/modules.rst new file mode 100644 index 00000000000..d47151019c2 --- /dev/null +++ b/docs/datamodel/modules.rst @@ -0,0 +1,289 @@ +.. _ref_datamodel_modules: +.. _ref_eql_sdl_modules: + +======= +Modules +======= + +Each |branch| has a schema consisting of several **modules**, each with +a unique name. Modules can be used to organize large schemas into +logical units. In practice, though, most users put their entire +schema inside a single module called ``default``. + +.. code-block:: sdl + + module default { + # declare types here + } + +.. _ref_name_resolution: + +Name resolution +=============== + +When you define a module that references schema objects from another module, +you must use a *fully-qualified* name in the form +``other_module_name::object_name``: + +.. code-block:: sdl + + module A { + type User extending B::AbstractUser; + } + + module B { + abstract type AbstractUser { + required name: str; + } + } + +Reserved module names +===================== + +The following module names are reserved by |Gel| and contain pre-defined +types, utility functions, and operators: + +* ``std``: standard types, functions, and operators in the :ref:`standard + library ` +* ``math``: algebraic and statistical :ref:`functions ` +* ``cal``: local (non-timezone-aware) and relative date/time :ref:`types and + functions ` +* ``schema``: types describing the :ref:`introspection + ` schema +* ``sys``: system-wide entities, such as user roles and + :ref:`databases ` +* ``cfg``: configuration and settings + + +Modules are containers +====================== + +They can contain types, functions, and other modules. Here's an example of an +empty module: + +.. code-block:: sdl + + module my_module {} + +And here's an example of a module with a type: + +.. code-block:: sdl + + module my_module { + type User { + required name: str; + } + } + + +Nested modules +============== + +.. code-block:: sdl + + module dracula { + type Person { + required name: str; + multi places_visited: City; + strength: int16; + } + + module combat { + function fight( + one: dracula::Person, + two: dracula::Person + ) -> str + using ( + (one.name ?? 'Fighter 1') ++ ' wins!' + IF (one.strength ?? 0) > (two.strength ?? 0) + ELSE (two.name ?? 'Fighter 2') ++ ' wins!' + ); + } + } + +You can chain together module names in a fully-qualified name to traverse a +tree of nested modules. For example, to call the ``fight`` function in the +nested module example above, you would use +``dracula::combat::fight()``. + + +Declaring modules +================= + +This section describes the syntax to declare a module in your schema. + + +Syntax +------ + +.. sdl:synopsis:: + + module "{" + [ ] + ... + "}" + +Define a nested module: + +.. sdl:synopsis:: + + module "{" + [ ] + module "{" + [ ] + "}" + ... + "}" + + +Description +^^^^^^^^^^^ + +The module block declaration defines a new module similar to the +:eql:stmt:`create module` command, but it also allows putting the +module content as nested declarations: + +:sdl:synopsis:`` + Define various schema items that belong to this module. + +Unlike :eql:stmt:`create module`, a module block with the +same name can appear multiple times in an SDL document. In that case +all blocks with the same name are merged into a single module under +that name. For example: + +.. code-block:: sdl + + module my_module { + abstract type Named { + required name: str; + } + } + + module my_module { + type User extending Named; + } + +The above is equivalent to: + +.. code-block:: sdl + + module my_module { + abstract type Named { + required name: str; + } + + type User extending Named; + } + +Typically, in the documentation examples of SDL the *module block* is +omitted and instead its contents are described without assuming which +specific module they belong to. + +It's also possible to declare modules implicitly. In this style, SDL +declaration uses a :ref:`fully-qualified name ` for the +item that is being declared. The *module* part of the *fully-qualified* name +implies that a module by that name will be automatically created in the +schema. The following declaration is equivalent to the previous examples, +but it declares module ``my_module`` implicitly: + +.. code-block:: sdl + + abstract type my_module::Named { + required name: str; + } + + type my_module::User extending my_module::Named; + +A module block can be nested inside another module block to create a nested +module. If you want to reference an entity in a nested module by its +fully-qualified name, you will need to include all of the containing +modules' names: ``::::`` + +.. _ref_eql_ddl_modules: + +DDL commands +============ + +This section describes the low-level DDL commands for creating and dropping +modules. You typically don't need to use these commands directly, but +knowing about them is useful for reviewing migrations. + + +Create module +------------- + +:eql-statement: + +Create a new module. + +.. eql:synopsis:: + + create module [ :: ] + [ if not exists ]; + +There's a :ref:`corresponding SDL declaration ` +for a module, although in SDL a module declaration is likely to also +include that module's content. + + +Description +^^^^^^^^^^^ + +The command ``create module`` defines a new module for the current +:versionreplace:`database;5.0:branch`. The name of the new module must be +distinct from any existing module in the current +:versionreplace:`database;5.0:branch`. Unlike :ref:`SDL module declaration +` the ``create module`` command does not have sub-commands; +module contents are created separately. + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`if not exists` + Normally, creating a module that already exists is an error, but + with this flag the command will succeed. It is useful for scripts + that add something to a module or, if the module is missing, the + module is created as well. + +Examples +^^^^^^^^ + +Create a new module: + +.. code-block:: edgeql + + create module payments; + +Create a new nested module: + +.. code-block:: edgeql + + create module payments::currencies; + + +Drop module +----------- + +:eql-statement: + +Remove a module. + +.. eql:synopsis:: + + drop module ; + +Description +^^^^^^^^^^^ + +The command ``drop module`` removes an existing empty module from the +current :versionreplace:`database;5.0:branch`. If the module contains any +schema items, this command will fail. + +Examples +^^^^^^^^ + +Remove a module: + +.. code-block:: edgeql + + drop module payments; diff --git a/docs/datamodel/mutation_rewrites.rst b/docs/datamodel/mutation_rewrites.rst index b0acd581262..b014fd956fa 100644 --- a/docs/datamodel/mutation_rewrites.rst +++ b/docs/datamodel/mutation_rewrites.rst @@ -7,8 +7,6 @@ Mutation rewrites .. index:: rewrite, insert, update, using, __subject__, __specified__, __old__, modify, modification -.. edb:youtube-embed:: ImgMfb_jCJQ?end=41 - Mutation rewrites allow you to intercept database mutations (i.e., :ref:`inserts ` and/or :ref:`updates `) and set the value of a property or link to the result of an expression you define. They @@ -18,113 +16,116 @@ Mutation rewrites are complementary to :ref:`triggers `. While triggers are unable to modify the triggering object, mutation rewrites are built for that purpose. +Example: last modified +====================== + Here's an example of a mutation rewrite that updates a property of a ``Post`` type to reflect the time of the most recent modification: .. code-block:: sdl - type Post { - required title: str; - required body: str; - modified: datetime { - rewrite insert, update using (datetime_of_statement()) - } - } - + type Post { + required title: str; + required body: str; + modified: datetime { + rewrite insert, update using (datetime_of_statement()) + } + } Every time a ``Post`` is updated, the mutation rewrite will be triggered, updating the ``modified`` property: .. code-block:: edgeql-repl - db> insert Post { - ... title := 'One wierd trick to fix all your spelling errors' - ... }; - {default::Post {id: 19e024dc-d3b5-11ed-968c-37f5d0159e5f}} - db> select Post {title, modified}; - { - default::Post { - title: 'One wierd trick to fix all your spelling errors', - modified: '2023-04-05T13:23:49.488335Z', - }, - } - db> update Post - ... filter .id = '19e024dc-d3b5-11ed-968c-37f5d0159e5f' - ... set {title := 'One weird trick to fix all your spelling errors'}; - {default::Post {id: 19e024dc-d3b5-11ed-968c-37f5d0159e5f}} - db> select Post {title, modified}; - { - default::Post { - title: 'One weird trick to fix all your spelling errors', - modified: '2023-04-05T13:25:04.119641Z', - }, - } + db> insert Post { + ... title := 'One wierd trick to fix all your spelling errors' + ... }; + {default::Post {id: 19e024dc-d3b5-11ed-968c-37f5d0159e5f}} + db> select Post {title, modified}; + { + default::Post { + title: 'One wierd trick to fix all your spelling errors', + modified: '2023-04-05T13:23:49.488335Z', + }, + } + db> update Post + ... filter .id = '19e024dc-d3b5-11ed-968c-37f5d0159e5f' + ... set {title := 'One weird trick to fix all your spelling errors'}; + {default::Post {id: 19e024dc-d3b5-11ed-968c-37f5d0159e5f}} + db> select Post {title, modified}; + { + default::Post { + title: 'One weird trick to fix all your spelling errors', + modified: '2023-04-05T13:25:04.119641Z', + }, + } In some cases, you will want different rewrites depending on the type of query. Here, we will add an ``insert`` rewrite and an ``update`` rewrite: .. code-block:: sdl - type Post { - required title: str; - required body: str; - created: datetime { - rewrite insert using (datetime_of_statement()) - } - modified: datetime { - rewrite update using (datetime_of_statement()) - } - } + type Post { + required title: str; + required body: str; + created: datetime { + rewrite insert using (datetime_of_statement()) + } + modified: datetime { + rewrite update using (datetime_of_statement()) + } + } With this schema, inserts will set the ``Post`` object's ``created`` property while updates will set the ``modified`` property: .. code-block:: edgeql-repl - db> insert Post { - ... title := 'One wierd trick to fix all your spelling errors' - ... }; - {default::Post {id: 19e024dc-d3b5-11ed-968c-37f5d0159e5f}} - db> select Post {title, created, modified}; - { - default::Post { - title: 'One wierd trick to fix all your spelling errors', - created: '2023-04-05T13:23:49.488335Z', - modified: {}, - }, - } - db> update Post - ... filter .id = '19e024dc-d3b5-11ed-968c-37f5d0159e5f' - ... set {title := 'One weird trick to fix all your spelling errors'}; - {default::Post {id: 19e024dc-d3b5-11ed-968c-37f5d0159e5f}} - db> select Post {title, created, modified}; - { - default::Post { - title: 'One weird trick to fix all your spelling errors', - created: '2023-04-05T13:23:49.488335Z', - modified: '2023-04-05T13:25:04.119641Z', - }, - } + db> insert Post { + ... title := 'One wierd trick to fix all your spelling errors' + ... }; + {default::Post {id: 19e024dc-d3b5-11ed-968c-37f5d0159e5f}} + db> select Post {title, created, modified}; + { + default::Post { + title: 'One wierd trick to fix all your spelling errors', + created: '2023-04-05T13:23:49.488335Z', + modified: {}, + }, + } + db> update Post + ... filter .id = '19e024dc-d3b5-11ed-968c-37f5d0159e5f' + ... set {title := 'One weird trick to fix all your spelling errors'}; + {default::Post {id: 19e024dc-d3b5-11ed-968c-37f5d0159e5f}} + db> select Post {title, created, modified}; + { + default::Post { + title: 'One weird trick to fix all your spelling errors', + created: '2023-04-05T13:23:49.488335Z', + modified: '2023-04-05T13:25:04.119641Z', + }, + } .. note:: - Each property may have a single ``insert`` and a single ``update`` mutation - rewrite rule, or they may have a single rule that covers both. + Each property may have a single ``insert`` and a single ``update`` mutation + rewrite rule, or they may have a single rule that covers both. + -Available variables -=================== +Mutation context +================ .. index:: rewrite, __subject__, __specified__, __old__ Inside the rewrite rule's expression, you have access to a few special values: * ``__subject__`` refers to the object type with the new property and link - values + values. * ``__specified__`` is a named tuple with a key for each property or link in the type and a boolean value indicating whether this value was explicitly set - in the mutation + in the mutation. * ``__old__`` refers to the object type with the previous property and link - values (available for update-only mutation rewrites) + values (available for update-only mutation rewrites). Here are some examples of the special values in use. Maybe your blog hosts articles about particularly controversial topics. You could use ``__subject__`` @@ -132,16 +133,16 @@ to enforce a "cooling off" period before publishing a blog post: .. code-block:: sdl - type Post { - required title: str; - required body: str; - publish_time: datetime { - rewrite insert, update using ( - __subject__.publish_time ?? datetime_of_statement() + - cal::to_relative_duration(days := 10) - ) - } - } + type Post { + required title: str; + required body: str; + publish_time: datetime { + rewrite insert, update using ( + __subject__.publish_time ?? datetime_of_statement() + + cal::to_relative_duration(days := 10) + ) + } + } Here we take the post's ``publish_time`` if set or the time the statement is executed and add 10 days to it. That should give our authors time to consider @@ -151,17 +152,17 @@ You can omit ``__subject__`` in many cases and achieve the same thing: .. code-block:: sdl-diff - type Post { - required title: str; - required body: str; - publish_time: datetime { - rewrite insert, update using ( - - __subject__.publish_time ?? datetime_of_statement() + - + .publish_time ?? datetime_of_statement() + - cal::to_relative_duration(days := 10) - ) - } - } + type Post { + required title: str; + required body: str; + publish_time: datetime { + rewrite insert, update using ( + - __subject__.publish_time ?? datetime_of_statement() + + + .publish_time ?? datetime_of_statement() + + cal::to_relative_duration(days := 10) + ) + } + } but only if the path prefix has not changed. In the following schema, for example, the ``__subject__`` in the rewrite rule is required, because in the @@ -170,25 +171,25 @@ context of the nested ``select`` query, the leading dot resolves from the .. code-block:: sdl - type Post { - required title: str; - required body: str; - author_email: str; - author_name: str { - rewrite insert, update using ( - (select User {name} filter .email = __subject__.author_email).name - ) - } - } - type User { - name: str; - email: str; - } + type Post { + required title: str; + required body: str; + author_email: str; + author_name: str { + rewrite insert, update using ( + (select User {name} filter .email = __subject__.author_email).name + ) + } + } + type User { + name: str; + email: str; + } .. note:: - Learn more about how this works in our documentation on :ref:`path - resolution `. + Learn more about how this works in our documentation on :ref:`path + resolution `. Using ``__specified__``, we can determine which fields were specified in the mutation. This would allow us to track when a single property was last modified @@ -196,17 +197,17 @@ as in the ``title_modified`` property in this schema: .. code-block:: sdl - type Post { - required title: str; - required body: str; - title_modified: datetime { - rewrite update using ( - datetime_of_statement() - if __specified__.title - else __old__.title_modified - ) - } - } + type Post { + required title: str; + required body: str; + title_modified: datetime { + rewrite update using ( + datetime_of_statement() + if __specified__.title + else __old__.title_modified + ) + } + } ``__specified__.title`` will be ``true`` if that value was set as part of the update, and this rewrite mutation rule will update ``title_modified`` to @@ -216,19 +217,19 @@ Another way you might use this is to set a default value but allow overriding: .. code-block:: sdl - type Post { - required title: str; - required body: str; - modified: datetime { - rewrite update using ( - datetime_of_statement() - if not __specified__.modified - else .modified - ) - } - } - -Here, we rewrite ``modified`` on updates to ``datetime_of_statment()`` unless + type Post { + required title: str; + required body: str; + modified: datetime { + rewrite update using ( + datetime_of_statement() + if not __specified__.modified + else .modified + ) + } + } + +Here, we rewrite ``modified`` on updates to ``datetime_of_statement()`` unless ``modified`` was set in the update. In that case, we allow the specified value to be set. This is different from a :ref:`default ` value because the rewrite happens on each @@ -250,17 +251,17 @@ comparing them: .. code-block:: sdl - type Post { - required title: str; - required body: str; - modified: datetime { - rewrite update using ( - datetime_of_statement() - if __subject__ {**} != __old__ {**} - else __old__.modified - ) - } - } + type Post { + required title: str; + required body: str; + modified: datetime { + rewrite update using ( + datetime_of_statement() + if __subject__ {**} != __old__ {**} + else __old__.modified + ) + } + } Lastly, if we want to add an ``author`` property that can be set for each write and keep a history of all the authors, we can do this with the help of @@ -268,61 +269,228 @@ and keep a history of all the authors, we can do this with the help of .. code-block:: sdl - type Post { - required title: str; - required body: str; - author: str; - all_authors: array { - default := >[]; - rewrite update using ( - __old__.all_authors - ++ [__subject__.author] - ); - } - } + type Post { + required title: str; + required body: str; + author: str; + all_authors: array { + default := >[]; + rewrite update using ( + __old__.all_authors + ++ [__subject__.author] + ); + } + } On insert, our ``all_authors`` property will get initialized to an empty array of strings. We will rewrite updates to concatenate that array with an array containing the new author value. -Mutation rewrite as cached computed -=================================== +Cached computed +=============== -..index:: cached computeds, caching computeds +.. index:: cached computeds, caching computeds Mutation rewrites can be used to effectively create a cached computed value as demonstrated with the ``byline`` property in this schema: .. code-block:: sdl - type Post { - required title: str; - required body: str; - author: str; - created: datetime { - rewrite insert using (datetime_of_statement()) - } - byline: str { - rewrite insert, update using ( - 'by ' ++ - __subject__.author ++ - ' on ' ++ - to_str(__subject__.created, 'Mon DD, YYYY') - ) - } - } + type Post { + required title: str; + required body: str; + author: str; + created: datetime { + rewrite insert using (datetime_of_statement()) + } + byline: str { + rewrite insert, update using ( + 'by ' ++ + __subject__.author ++ + ' on ' ++ + to_str(__subject__.created, 'Mon DD, YYYY') + ) + } + } The ``byline`` property will be updated on each insert or update, but the value will not need to be calculated at read time like a proper :ref:`computed property `. +.. _ref_eql_sdl_mutation_rewrites: +.. _ref_eql_sdl_mutation_rewrites_syntax: + +Declaring mutation rewrites +=========================== + +This section describes the syntax to declare mutation rewrites in your schema. + +Syntax +------ + +Define a new mutation rewrite corresponding to the :ref:`more explicit DDL +commands `. + +.. sdl:synopsis:: + + rewrite {insert | update} [, ...] + using + +Mutation rewrites must be defined inside a property or link block. + +Description +^^^^^^^^^^^ + +This declaration defines a new trigger with the following options: + +:eql:synopsis:`insert | update [, ...]` + The query type (or types) the rewrite runs on. Separate multiple values + with commas to invoke the same rewrite for multiple types of queries. + +:eql:synopsis:`` + The expression to be evaluated to produce the new value of the property. + + +.. _ref_eql_ddl_mutation_rewrites: + +DDL commands +============ + +This section describes the low-level DDL commands for creatin and +dropping mutation rewrites. You typically don't need to use these commands +directly, but knowing about them is useful for reviewing migrations. + + +Create rewrite +-------------- + +:eql-statement: + +Define a new mutation rewrite. + +When creating a new property or link: + +.. eql:synopsis:: + + {create | alter} type "{" + create { property | link } -> "{" + create rewrite {insert | update} [, ...] + using + "}" ; + "}" ; + +When altering an existing property or link: + +.. eql:synopsis:: + + {create | alter} type "{" + alter { property | link } "{" + create rewrite {insert | update} [, ...] + using + "}" ; + "}" ; + + +Description +^^^^^^^^^^^ + +The command ``create rewrite`` nested under ``create type`` or ``alter type`` +and then under ``create property/link`` or ``alter property/link`` defines a +new mutation rewrite for the given property or link on the given object. + + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`` + The name (optionally module-qualified) of the type containing the rewrite. + +:eql:synopsis:`` + The name (optionally module-qualified) of the property or link being + rewritten. + +:eql:synopsis:`insert | update [, ...]` + The query type (or types) that are rewritten. Separate multiple values with + commas to invoke the same rewrite for multiple types of queries. + + +Examples +^^^^^^^^ + +Declare two mutation rewrites on new properties: one that sets a ``created`` +property when a new object is inserted and one that sets a ``modified`` +property on each update: + +.. code-block:: edgeql + + alter type User { + create property created -> datetime { + create rewrite insert using (datetime_of_statement()); + }; + create property modified -> datetime { + create rewrite update using (datetime_of_statement()); + }; + }; + + +Drop rewrite +------------ + +:eql-statement: + +Drop a mutation rewrite. + +.. eql:synopsis:: + + alter type "{" + alter property "{" + drop rewrite {insert | update} ; + "}" ; + "}" ; + + +Description +^^^^^^^^^^^ + +The command ``drop rewrite`` inside an ``alter type`` block and further inside +an ``alter property`` block removes the definition of an existing mutation +rewrite on the specified property or link of the specified type. + + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`` + The name (optionally module-qualified) of the type containing the rewrite. + +:eql:synopsis:`` + The name (optionally module-qualified) of the property or link being + rewritten. + +:eql:synopsis:`insert | update [, ...]` + The query type (or types) that are rewritten. Separate multiple values with + commas to invoke the same rewrite for multiple types of queries. + + +Example +^^^^^^^ + +Remove the ``insert`` rewrite of the ``created`` property on the ``User`` type: + +.. code-block:: edgeql + + alter type User { + alter property created { + drop rewrite insert; + }; + }; + + .. list-table:: :class: seealso * - **See also** - * - :ref:`SDL > Mutation rewrites ` - * - :ref:`DDL > Mutation rewrites ` * - :ref:`Introspection > Mutation rewrites ` diff --git a/docs/datamodel/objects.rst b/docs/datamodel/objects.rst index c641e2eba83..d803dd5424a 100644 --- a/docs/datamodel/objects.rst +++ b/docs/datamodel/objects.rst @@ -6,63 +6,95 @@ Object Types .. index:: type, tables, models -*Object types* are the primary components of an Gel schema. They are +*Object types* are the primary components of a Gel schema. They are analogous to SQL *tables* or ORM *models*, and consist of :ref:`properties ` and :ref:`links `. -Properties are used to attach primitive data to an object type. -For the full documentation on properties, -see :ref:`Properties `. +Properties +========== + +Properties are used to attach primitive/scalar data to an object type. +For the full documentation on properties, see :ref:`ref_datamodel_props`. .. code-block:: sdl - type Person { - email: str; - } + type Person { + email: str; + } + +Using in a query: + +.. code-block:: edgeql + + select Person { + email + }; + + +Links +===== Links are used to define relationships between object types. For the full -documentation on links, see :ref:`Links `. +documentation on links, see :ref:`ref_datamodel_links`. .. code-block:: sdl - type Person { - best_friend: Person; - } + type Person { + email: str; + best_friend: Person; + } + +Using in a query: + +.. code-block:: edgeql + select Person { + email, + best_friend: { + email + } + }; -IDs ---- +ID +== .. index:: uuid, primary key There's no need to manually declare a primary key on your object types. All object types automatically contain a property ``id`` of type ``UUID`` that's -*required*, *globally unique*, and *readonly*. This ``id`` is assigned upon -creation and never changes. +*required*, *globally unique*, *readonly*, and has an index on it. +The ``id`` is assigned upon creation and cannot be changed. + +Using in a query: + +.. code-block:: edgeql + + select Person { id }; + select Person { email } filter .id = '123e4567-e89b-...'; + Abstract types --------------- +============== .. index:: abstract, inheritance Object types can either be *abstract* or *non-abstract*. By default all object -types are non-abstract. You can't create or store instances of abstract types, -but they're a useful way to share functionality and structure among -other object types. +types are non-abstract. You can't create or store instances of abstract types +(a.k.a. mixins), but they're a useful way to share functionality and +structure among other object types. .. code-block:: sdl - abstract type HasName { - first_name: str; - last_name: str; - } - -Abstract types are commonly used in tandem with inheritance. + abstract type HasName { + first_name: str; + last_name: str; + } .. _ref_datamodel_objects_inheritance: +.. _ref_eql_sdl_object_types_inheritance: Inheritance ------------ +=========== .. index:: extending, extends, subtypes, supertypes @@ -72,55 +104,394 @@ Object types can *extend* other object types. The extending type (AKA the .. code-block:: sdl - abstract type Animal { - species: str; - } + abstract type HasName { + first_name: str; + last_name: str; + } + + type Person extending HasName { + email: str; + best_friend: Person; + } + +Using in a query: + +.. code-block:: edgeql + + select Person { + first_name, + email, + best_friend: { + last_name + } + }; - type Dog extending Animal { - breed: str; - } .. _ref_datamodel_objects_multiple_inheritance: Multiple Inheritance -^^^^^^^^^^^^^^^^^^^^ +==================== -Object types can :ref:`extend more -than one type ` — that's called +Object types can extend more than one type — that's called *multiple inheritance*. This mechanism allows building complex object types out of combinations of more basic types. -.. code-block:: sdl +.. note:: - abstract type HasName { - first_name: str; - last_name: str; - } + Gel's multiple inheritance should not be confused with the multiple + inheritance of C++ or Python, where the complexity usually arises + from fine-grained mixing of logic. Gel's multiple inheritance is + structural and allows for natural composition. - abstract type HasEmail { - email: str; - } +.. code-block:: sdl-diff - type Person extending HasName, HasEmail { - profession: str; - } + abstract type HasName { + first_name: str; + last_name: str; + } + + + abstract type HasEmail { + + email: str; + + } + + - type Person extending HasName { + + type Person extending HasName, HasEmail { + - email: str; + best_friend: Person; + } If multiple supertypes share links or properties, those properties must be of the same type and cardinality. -.. note:: - Refer to the dedicated pages on :ref:`Indexes `, - :ref:`Constraints `, :ref:`Access Policies - `, and :ref:`Annotations - ` for documentation on these concepts. +.. _ref_eql_sdl_object_types: +.. _ref_eql_sdl_object_types_syntax: + + +Defining object types +===================== + +This section describes the syntax to declare object types in your schema. + +Syntax +------ + +.. sdl:synopsis:: + + [abstract] type [extending [, ...] ] + [ "{" + [ ] + [ ] + [ ] + [ ] + [ ] + ... + "}" ] + +Description +^^^^^^^^^^^ + +This declaration defines a new object type with the following options: + +:eql:synopsis:`abstract` + If specified, the created type will be *abstract*. + +:eql:synopsis:`` + The name (optionally module-qualified) of the new type. + +:eql:synopsis:`extending [, ...]` + Optional clause specifying the *supertypes* of the new type. + + Use of ``extending`` creates a persistent type relationship + between the new subtype and its supertype(s). Schema modifications + to the supertype(s) propagate to the subtype. + + References to supertypes in queries will also include objects of + the subtype. + + If the same *link* name exists in more than one supertype, or + is explicitly defined in the subtype and at least one supertype, + then the data types of the link targets must be *compatible*. + If there is no conflict, the links are merged to form a single + link in the new type. + +These sub-declarations are allowed in the ``Type`` block: + +:sdl:synopsis:`` + Set object type :ref:`annotation ` + to a given *value*. + +:sdl:synopsis:`` + Define a concrete :ref:`property ` for this object type. + +:sdl:synopsis:`` + Define a concrete :ref:`link ` for this object type. + +:sdl:synopsis:`` + Define a concrete :ref:`constraint ` for this + object type. + +:sdl:synopsis:`` + Define an :ref:`index ` for this object type. + + +.. _ref_eql_ddl_object_types: + +DDL commands +============ + +This section describes the low-level DDL commands for creating, altering, and +dropping object types. You typically don't need to use these commands directly, +but knowing about them is useful for reviewing migrations. + +Create type +----------- + +:eql-statement: +:eql-haswith: + +Define a new object type. + +.. eql:synopsis:: + + [ with [, ...] ] + create [abstract] type [ extending [, ...] ] + [ "{" ; [...] "}" ] ; + + # where is one of + + create annotation := + create link ... + create property ... + create constraint ... + create index on + +Description +^^^^^^^^^^^ + +The command ``create type`` defines a new object type for use in the +current |branch|. + +If *name* is qualified with a module name, then the type is created +in that module, otherwise it is created in the current module. +The type name must be distinct from that of any existing schema item +in the module. + +Parameters +^^^^^^^^^^ + +Most sub-commands and options of this command are identical to the +:ref:`SDL object type declaration `, +with some additional features listed below: + +:eql:synopsis:`with [, ...]` + Alias declarations. + + The ``with`` clause allows specifying module aliases + that can be referenced by the command. See :ref:`ref_eql_statements_with` + for more information. + +The following subcommands are allowed in the ``create type`` block: + +:eql:synopsis:`create annotation := ` + Set object type :eql:synopsis:`` to + :eql:synopsis:``. + + See :eql:stmt:`create annotation` for details. + +:eql:synopsis:`create link ...` + Define a new link for this object type. See + :eql:stmt:`create link` for details. + +:eql:synopsis:`create property ...` + Define a new property for this object type. See + :eql:stmt:`create property` for details. + +:eql:synopsis:`create constraint ...` + Define a concrete constraint for this object type. See + :eql:stmt:`create constraint` for details. + +:eql:synopsis:`create index on ` + Define a new :ref:`index ` + using *index-expr* for this object type. See + :eql:stmt:`create index` for details. + +Example +^^^^^^^ + +Create an object type ``User``: + +.. code-block:: edgeql + + create type User { + create property name -> str; + }; + + +Alter type +---------- + +:eql-statement: +:eql-haswith: + +Change the definition of an object type. + +.. eql:synopsis:: + + [ with [, ...] ] + alter type + [ "{" ; [...] "}" ] ; + + [ with [, ...] ] + alter type ; + + # where is one of + + rename to + extending [, ...] + create annotation := + alter annotation := + drop annotation + create link ... + alter link ... + drop link ... + create property ... + alter property ... + drop property ... + create constraint ... + alter constraint ... + drop constraint ... + create index on + drop index on + +Description +^^^^^^^^^^^ + +The command ``alter type`` changes the definition of an object type. +*name* must be a name of an existing object type, optionally qualified +with a module name. + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`with [, ...]` + Alias declarations. + + The ``with`` clause allows specifying module aliases + that can be referenced by the command. See :ref:`ref_eql_statements_with` + for more information. + +:eql:synopsis:`` + The name (optionally module-qualified) of the type being altered. + +:eql:synopsis:`extending [, ...]` + Alter the supertype list. The full syntax of this subcommand is: + + .. eql:synopsis:: + + extending [, ...] + [ first | last | before | after ] + + This subcommand makes the type a subtype of the specified list + of supertypes. The requirements for the parent-child relationship + are the same as when creating an object type. + + It is possible to specify the position in the parent list + using the following optional keywords: + + * ``first`` -- insert parent(s) at the beginning of the + parent list, + * ``last`` -- insert parent(s) at the end of the parent list, + * ``before `` -- insert parent(s) before an + existing *parent*, + * ``after `` -- insert parent(s) after an existing + *parent*. + +:eql:synopsis:`alter annotation ;` + Alter object type annotation :eql:synopsis:``. + See :eql:stmt:`alter annotation` for details. + +:eql:synopsis:`drop annotation ` + Remove object type :eql:synopsis:``. + See :eql:stmt:`drop annotation` for details. + +:eql:synopsis:`alter link ...` + Alter the definition of a link for this object type. See + :eql:stmt:`alter link` for details. + +:eql:synopsis:`drop link ` + Remove a link item from this object type. See + :eql:stmt:`drop link` for details. + +:eql:synopsis:`alter property ...` + Alter the definition of a property item for this object type. + See :eql:stmt:`alter property` for details. + +:eql:synopsis:`drop property ` + Remove a property item from this object type. See + :eql:stmt:`drop property` for details. + +:eql:synopsis:`alter constraint ...` + Alter the definition of a constraint for this object type. See + :eql:stmt:`alter constraint` for details. + +:eql:synopsis:`drop constraint ;` + Remove a constraint from this object type. See + :eql:stmt:`drop constraint` for details. + +:eql:synopsis:`drop index on ` + Remove an :ref:`index ` defined as *index-expr* + from this object type. See :eql:stmt:`drop index` for details. + +All the subcommands allowed in the ``create type`` block are also +valid subcommands for the ``alter type`` block. + +Example +^^^^^^^ + +Alter the ``User`` object type to make ``name`` required: + +.. code-block:: edgeql + + alter type User { + alter property name { + set required; + } + }; + + +Drop type +--------- + +:eql-statement: +:eql-haswith: + +Remove the specified object type from the schema. + +.. eql:synopsis:: + + drop type ; + +Description +^^^^^^^^^^^ + +The command ``drop type`` removes the specified object type from the +schema. All subordinate schema items defined on this type, +such as links and indexes, are removed as well. + +Example +^^^^^^^ + +Remove the ``User`` object type: + +.. code-block:: edgeql + + drop type User; .. list-table:: :class: seealso * - **See also** - * - :ref:`SDL > Object types ` - * - :ref:`DDL > Object types ` * - :ref:`Introspection > Object types ` * - :ref:`Cheatsheets > Object types ` diff --git a/docs/datamodel/primitives.rst b/docs/datamodel/primitives.rst index 24b402c3adc..508325382e9 100644 --- a/docs/datamodel/primitives.rst +++ b/docs/datamodel/primitives.rst @@ -5,72 +5,102 @@ Primitives ========== |Gel| has a robust type system consisting of primitive and object types. -Below is a review of Gel's primitive types; later, these will be used to -declare *properties* on object types. - +types. Primitive types are used to declare *properties* on object types, +as query and function arguments, as as well as in other contexts. .. _ref_datamodel_scalars: -Scalar types -^^^^^^^^^^^^ +Built-in scalar types +===================== + +Gel comes with a range of built-in scalar types, such as: + +* String: :eql:type:`str` +* Boolean: :eql:type:`bool` +* Various numeric types: :eql:type:`int16`, :eql:type:`int32`, + :eql:type:`int64`, :eql:type:`float32`, :eql:type:`float64`, :eql:type:`bigint`, :eql:type:`decimal` +* JSON: :eql:type:`json`, +* UUID: :eql:type:`uuid`, +* Date/time: :eql:type:`datetime`, :eql:type:`duration` + :eql:type:`cal::local_datetime`, :eql:type:`cal::local_date`, + :eql:type:`cal::local_time`, :eql:type:`cal::relative_duration`, + :eql:type:`cal::date_duration` +* Miscellaneous: :eql:type:`sequence`, :eql:type:`bytes`, etc. + +Custom scalars +============== -.. include:: ../stdlib/scalar_table.rst +You can extend built-in scalars with additional constraints or annotations. +Here's an example of a non-negative custom ``int64`` variant: -Custom scalar types can also be declared. For full documentation, see :ref:`SDL -> Scalar types `. +.. code-block:: sdl + + scalar type posint64 extending int64 { + constraint min_value(0); + } .. _ref_datamodel_enums: Enums -^^^^^ +===== -To represent an enum, declare a custom scalar that extends the abstract -:ref:`enum ` type. +Enum types are created by extending the abstract :eql:type:`enum` type, e.g.: .. code-block:: sdl - scalar type Color extending enum; + scalar type Color extending enum; - type Shirt { - color: Color; - } + type Shirt { + color: Color; + } -.. important:: +which can be queries with: - To reference enum values inside EdgeQL queries, use dot notation, e.g. - ``Color.Green``. +.. code-block:: edgeql -For a full reference on enum types, see the :ref:`Enum docs `. + select Shirt filter .color = Color.Red; +For a full reference on enum types, see the :ref:`Enum docs `. .. _ref_datamodel_arrays: Arrays -^^^^^^ +====== Arrays store zero or more primitive values of the same type in an ordered list. -Arrays cannot contain object types or other arrays. +Arrays cannot contain object types or other arrays, but can contain virtually +any other type. .. code-block:: sdl - type Person { - str_array: array; - json_array: array; + type Person { + str_array: array; + json_array: array; + tuple_array: array>; - # INVALID: arrays of object types not allowed - # friends: array + # INVALID: arrays of object types not allowed: + # friends: array - # INVALID: arrays cannot be nested - # nested_array: array> - } + # INVALID: arrays cannot be nested: + # nested_array: array> -For a full reference on array types, see the :ref:`Array docs `. + # VALID: arrays can contain tuples with arrays in them + nested_array_via_tuple: array>> + } + +Array syntax in EdgeQL is very intuitive (indexing starts at ``0``): + +.. code-block:: edgeql + select [1, 2, 3]; + select [1, 2, 3][1] = 2; # true + +For a full reference on array types, see the :ref:`Array docs `. .. _ref_datamodel_tuples: Tuples -^^^^^^ +====== Like arrays, tuples are ordered sequences of primitive data. Unlike arrays, each element of a tuple can have a distinct type. Tuple elements can be *any @@ -78,13 +108,11 @@ type*, including primitives, objects, arrays, and other tuples. .. code-block:: sdl - type Person { - - unnamed_tuple: tuple; - nested_tuple: tuple>>; - tuple_of_arrays: tuple, array>; - - } + type Person { + unnamed_tuple: tuple; + nested_tuple: tuple>>; + tuple_of_arrays: tuple, array>; + } Optionally, you can assign a *key* to each element of the tuple. Tuples containing explicit keys are known as *named tuples*. You must assign keys to @@ -92,50 +120,48 @@ all elements (or none of them). .. code-block:: sdl - type BlogPost { - metadata: tuple; - } + type BlogPost { + metadata: tuple; + } Named and unnamed tuples are the same data structure under the hood. You can add, remove, and change keys in a tuple type after it's been declared. For -details, see :ref:`EdgeQL > Literals > Tuples `. +details, see :ref:`Tuples `. -.. important:: +.. note:: - When you query an *unnamed* tuple using one of EdgeQL's :ref:`client - libraries `, its value is converted to a list/array. When - you fetch a named tuple, it is converted into an object/dictionary/hashmap - depending on the language. + When you query an *unnamed* tuple using one of EdgeQL's + :ref:`client libraries `, its value is converted to a + list/array. When you fetch a named tuple, it is converted into an + object/dictionary/hashmap depending on the language. .. _ref_datamodel_ranges: Ranges -^^^^^^ +====== Ranges represent some interval of values. The intervals can be bound or unbound on either end. They can also be empty, containing no values. Only some scalar types have corresponding range types: -- ``range`` -- ``range`` -- ``range`` -- ``range`` -- ``range`` -- ``range`` -- ``range`` -- ``range`` +- Numeric ranges: ``range``, ``range``, ``range``, + ``range``, ``range`` +- Date/time ranges: ``range``, ``range``, + ``range`` + +Example: .. code-block:: sdl - type DieRoll { - values: range; - } + type DieRoll { + values: range; + } -For a full reference on ranges, functions and operators see the :ref:`Range -docs `. +For a full reference on ranges, functions and operators see the +:ref:`Range docs `. Sequences -^^^^^^^^^ +========= To represent an auto-incrementing integer property, declare a custom scalar that extends the abstract ``sequence`` type. Creating a sequence type @@ -145,10 +171,264 @@ share the counter. .. code-block:: sdl - scalar type ticket_number extending sequence; - type Ticket { - number: ticket_number; - } + scalar type ticket_number extending sequence; + type Ticket { + number: ticket_number; + rendered_number := 'TICKET-\(.number)'; + } + +For a full reference on sequences, see the :ref:`Sequence docs `. + +.. _ref_eql_sdl_scalars: +.. _ref_eql_sdl_scalars_syntax: + +Declaring scalars +================= + +This section describes the syntax to declare a custom scalar type in your +schema. + + +Syntax +------ + +.. sdl:synopsis:: + + [abstract] scalar type [extending [, ...] ] + [ "{" + [ ] + [ ] + ... + "}" ] + +Description +^^^^^^^^^^^ + +This declaration defines a new object type with the following options: + +:eql:synopsis:`abstract` + If specified, the created scalar type will be *abstract*. + +:eql:synopsis:`` + The name (optionally module-qualified) of the new scalar type. + +:eql:synopsis:`extending ` + Optional clause specifying the *supertype* of the new type. + + If :eql:synopsis:`` is an + :eql:type:`enumerated type ` declaration then + an enumerated scalar type is defined. + + Use of ``extending`` creates a persistent type relationship + between the new subtype and its supertype(s). Schema modifications + to the supertype(s) propagate to the subtype. + +The valid SDL sub-declarations are listed below: + +:sdl:synopsis:`` + Set scalar type :ref:`annotation ` + to a given *value*. + +:sdl:synopsis:`` + Define a concrete :ref:`constraint ` for + this scalar type. + + +.. _ref_eql_ddl_scalars: + +DDL commands +============ + +This section describes the low-level DDL commands for creating, altering, and +dropping scalar types. You typically don't need to use these commands directly, +but knowing about them is useful for reviewing migrations. + +Create scalar +------------- + +:eql-statement: +:eql-haswith: + +Define a new scalar type. + +.. eql:synopsis:: + + [ with [, ...] ] + create [abstract] scalar type [ extending ] + [ "{" ; [...] "}" ] ; + + # where is one of + + create annotation := + create constraint ... + +Description +^^^^^^^^^^^ + +The command ``create scalar type`` defines a new scalar type for use in the +current |branch|. + +If *name* is qualified with a module name, then the type is created +in that module, otherwise it is created in the current module. +The type name must be distinct from that of any existing schema item +in the module. + +If the ``abstract`` keyword is specified, the created type will be +*abstract*. + +All non-abstract scalar types must have an underlying core +implementation. For user-defined scalar types this means that +``create scalar type`` must have another non-abstract scalar type +as its *supertype*. + +The most common use of ``create scalar type`` is to define a scalar +subtype with constraints. + +Most sub-commands and options of this command are identical to the +:ref:`SDL scalar type declaration `. The +following subcommands are allowed in the ``create scalar type`` block: + +:eql:synopsis:`create annotation := ;` + Set scalar type's :eql:synopsis:`` to + :eql:synopsis:``. + + See :eql:stmt:`create annotation` for details. + +:eql:synopsis:`create constraint ...` + Define a new constraint for this scalar type. See + :eql:stmt:`create constraint` for details. + + +Examples +^^^^^^^^ + +Create a new non-negative integer type: + +.. code-block:: edgeql + + create scalar type posint64 extending int64 { + create constraint min_value(0); + }; + +Create a new enumerated type: + +.. code-block:: edgeql + + create scalar type Color + extending enum; + + +Alter scalar +------------ + +:eql-statement: +:eql-haswith: + +Alter the definition of a scalar type. + +.. eql:synopsis:: + + [ with [, ...] ] + alter scalar type + "{" ; [...] "}" ; + + # where is one of + + rename to + extending ... + create annotation := + alter annotation := + drop annotation + create constraint ... + alter constraint ... + drop constraint ... + +Description +^^^^^^^^^^^ + +The command ``alter scalar type`` changes the definition of a scalar type. +*name* must be a name of an existing scalar type, optionally qualified +with a module name. + +The following subcommands are allowed in the ``alter scalar type`` block: + +:eql:synopsis:`rename to ;` + Change the name of the scalar type to *newname*. + +:eql:synopsis:`extending ...` + Alter the supertype list. It works the same way as in + :eql:stmt:`alter type`. + +:eql:synopsis:`alter annotation ;` + Alter scalar type :eql:synopsis:``. + See :eql:stmt:`alter annotation` for details. + +:eql:synopsis:`drop annotation ` + Remove scalar type's :eql:synopsis:`` from + :eql:synopsis:``. + See :eql:stmt:`drop annotation` for details. + +:eql:synopsis:`alter constraint ...` + Alter the definition of a constraint for this scalar type. See + :eql:stmt:`alter constraint` for details. + +:eql:synopsis:`drop constraint ` + Remove a constraint from this scalar type. See + :eql:stmt:`drop constraint` for details. + +All the subcommands allowed in the ``create scalar type`` block are also +valid subcommands for ``alter scalar type`` block. + + +Examples +^^^^^^^^ + +Define a new constraint on a scalar type: + +.. code-block:: edgeql + + alter scalar type posint64 { + create constraint max_value(100); + }; + +Add one more label to an enumerated type: + +.. code-block:: edgeql + + alter scalar type Color + extending enum; + + +Drop scalar +----------- + +:eql-statement: +:eql-haswith: + +Remove a scalar type. + +.. eql:synopsis:: + + [ with [, ...] ] + drop scalar type ; + +Description +^^^^^^^^^^^ + +The command ``drop scalar type`` removes a scalar type. + +Parameters +^^^^^^^^^^ + +*name* + The name (optionally qualified with a module name) of an existing + scalar type. + +Example +^^^^^^^ + +Remove a scalar type: + +.. code-block:: edgeql -For a full reference on sequences, see the :ref:`Sequence docs -`. + drop scalar type posint64; diff --git a/docs/datamodel/triggers.rst b/docs/datamodel/triggers.rst index fb0f16a5823..10141f8191f 100644 --- a/docs/datamodel/triggers.rst +++ b/docs/datamodel/triggers.rst @@ -1,4 +1,5 @@ .. _ref_datamodel_triggers: +.. _ref_eql_sdl_triggers: ======== Triggers @@ -7,35 +8,76 @@ Triggers .. index:: trigger, after insert, after update, after delete, for each, for all, when, do, __new__, __old__ -.. edb:youtube-embed:: ImgMfb_jCJQ?start=41 - Triggers allow you to define an expression to be executed whenever a given query type is run on an object type. The original query will *trigger* your pre-defined expression to run in a transaction along with the original query. These can be defined in your schema. -.. note:: - Triggers cannot be used to *modify* the object that set off the trigger, - although they can be used with :eql:func:`assert` to do *validation* on - that object. If you need to modify the object, you can use :ref:`mutation - rewrites `. +Important notes +=============== + +Triggers are an advanced feature and have some caveats that +you should be aware of. + +Consider using mutation rewrites +-------------------------------- + +Triggers cannot be used to *modify* the object that set off the trigger, +although they can be used with :eql:func:`assert` to do *validation* on +that object. If you need to modify the object, you can use :ref:`mutation +rewrites `. + +Unified trigger query execution +------------------------------- + +All queries within triggers, along with the initial triggering query, are +compiled into a single combined SQL query under the hood. Keep this in mind +when designing triggers that modify existing records. If multiple ``update`` +queries within your triggers target the same object, only one of these +queries will ultimately be executed. To ensure all desired updates on an +object are applied, consolidate them into a single ``update`` query within +one trigger, instead of distributing them across multiple updates. + +Multi-stage trigger execution +----------------------------- + +In some cases, a trigger can cause another trigger to fire. When this +happens, Gel completes all the triggers fired by the initial query +before kicking off a new "stage" of triggers. In the second stage, any +triggers fired by the initial stage of triggers will fire. Gel will +continue adding trigger stages until all triggers are complete. -Here's an example that creates a simple audit log type so that we can keep +The exception to this is when triggers would cause a loop or would cause +the same trigger to be run in two different stages. These triggers will +generate an error. + +Data visibility +--------------- + +Any query in your trigger will return the state of the database *after* the +triggering query. If this query's results include the object that flipped +the trigger, the results will contain that object in the same state as +``__new__``. + + +Example: audit log +================== + +Here's an example that creates a simple **audit log** type so that we can keep track of what's happening to our users in a database. First, we will create a ``Log`` type: .. code-block:: sdl - type Log { - action: str; - timestamp: datetime { - default := datetime_current(); - } - target_name: str; - change: str; - } - + type Log { + action: str; + timestamp: datetime { + default := datetime_current(); + } + target_name: str; + change: str; + } With the ``Log`` type in place, we can write some triggers that will automatically create ``Log`` objects for any insert, update, or delete queries @@ -43,123 +85,93 @@ on the ``Person`` type: .. code-block:: sdl - type Person { - required name: str; - - trigger log_insert after insert for each do ( - insert Log { - action := 'insert', - target_name := __new__.name - } - ); - - trigger log_update after update for each do ( - insert Log { - action := 'update', - target_name := __new__.name, - change := __old__.name ++ '->' ++ __new__.name - } - ); - - trigger log_delete after delete for each do ( - insert Log { - action := 'delete', - target_name := __old__.name - } - ); - } + type Person { + required name: str; + + trigger log_insert after insert for each do ( + insert Log { + action := 'insert', + target_name := __new__.name + } + ); + + trigger log_update after update for each do ( + insert Log { + action := 'update', + target_name := __new__.name, + change := __old__.name ++ '->' ++ __new__.name + } + ); + + trigger log_delete after delete for each do ( + insert Log { + action := 'delete', + target_name := __old__.name + } + ); + } In a trigger's expression, we have access to the ``__old__`` and/or ``__new__`` variables which capture the object before and after the query. Triggers on ``update`` can use both variables. Triggers on ``delete`` can use ``__old__``. Triggers on ``insert`` can use ``__new__``. -.. note:: - - Any query in your trigger will return the state of the database *after* the - triggering query. If this query's results include the object that flipped - the trigger, the results will contain that object in the same state as - ``__new__``. - Now, whenever we run a query, we get a log entry as well: .. code-block:: edgeql-repl - db> insert Person {name := 'Jonathan Harker'}; - {default::Person {id: b4d4e7e6-bd19-11ed-8363-1737d8d4c3c3}} - db> select Log {action, timestamp, target_name, change}; - { - default::Log { - action: 'insert', - timestamp: '2023-03-07T18:56:02.403817Z', - target_name: 'Jonathan Harker', - change: {} - } - } - db> update Person filter .name = 'Jonathan Harker' - ... set {name := 'Mina Murray'}; - {default::Person {id: b4d4e7e6-bd19-11ed-8363-1737d8d4c3c3}} - db> select Log {action, timestamp, target_name, change}; - { - default::Log { - action: 'insert', - timestamp: '2023-03-07T18:56:02.403817Z', - target_name: 'Jonathan Harker', - change: {} - }, - default::Log { - action: 'update', - timestamp: '2023-03-07T18:56:39.520889Z', - target_name: 'Mina Murray', - change: 'Jonathan Harker->Mina Murray' - }, - } - db> delete Person filter .name = 'Mina Murray'; - {default::Person {id: b4d4e7e6-bd19-11ed-8363-1737d8d4c3c3}} - db> select Log {action, timestamp, target_name, change}; - { - default::Log { - action: 'insert', - timestamp: '2023-03-07T18:56:02.403817Z', - target_name: 'Jonathan Harker', - change: {} - }, - default::Log { - action: 'update', - timestamp: '2023-03-07T18:56:39.520889Z', - target_name: 'Mina Murray', - change: 'Jonathan Harker->Mina Murray' - }, - default::Log { - action: 'delete', - timestamp: '2023-03-07T19:00:52.636084Z', - target_name: 'Mina Murray', - change: {} - }, - } - -.. note:: - - All queries within triggers, along with the initial triggering query, are - compiled into a single combined query. Keep this in mind when - designing triggers that modify existing records. If multiple ``update`` - queries within your triggers target the same object, only one of these - queries will ultimately be executed. To ensure all desired updates on an - object are applied, consolidate them into a single ``update`` query within - one trigger, instead of distributing them across multiple updates. - -.. note:: - - In some cases, a trigger can cause another trigger to fire. When this - happens, Gel completes all the triggers fired by the initial query - before kicking off a new "stage" of triggers. In the second stage, any - triggers fired by the initial stage of triggers will fire. Gel will - continue adding trigger stages until all triggers are complete. - - The exception to this is when triggers would cause a loop or would cause - the same trigger to be run in two different stages. These triggers will - generate an error. - + db> insert Person {name := 'Jonathan Harker'}; + {default::Person {id: b4d4e7e6-bd19-11ed-8363-1737d8d4c3c3}} + db> select Log {action, timestamp, target_name, change}; + { + default::Log { + action: 'insert', + timestamp: '2023-03-07T18:56:02.403817Z', + target_name: 'Jonathan Harker', + change: {} + } + } + db> update Person filter .name = 'Jonathan Harker' + ... set {name := 'Mina Murray'}; + {default::Person {id: b4d4e7e6-bd19-11ed-8363-1737d8d4c3c3}} + db> select Log {action, timestamp, target_name, change}; + { + default::Log { + action: 'insert', + timestamp: '2023-03-07T18:56:02.403817Z', + target_name: 'Jonathan Harker', + change: {} + }, + default::Log { + action: 'update', + timestamp: '2023-03-07T18:56:39.520889Z', + target_name: 'Mina Murray', + change: 'Jonathan Harker->Mina Murray' + }, + } + db> delete Person filter .name = 'Mina Murray'; + {default::Person {id: b4d4e7e6-bd19-11ed-8363-1737d8d4c3c3}} + db> select Log {action, timestamp, target_name, change}; + { + default::Log { + action: 'insert', + timestamp: '2023-03-07T18:56:02.403817Z', + target_name: 'Jonathan Harker', + change: {} + }, + default::Log { + action: 'update', + timestamp: '2023-03-07T18:56:39.520889Z', + target_name: 'Mina Murray', + change: 'Jonathan Harker->Mina Murray' + }, + default::Log { + action: 'delete', + timestamp: '2023-03-07T19:00:52.636084Z', + target_name: 'Mina Murray', + change: {} + }, + } Our audit logging works, but the update logs have a major shortcoming: they log an update even when nothing changes. Any time an ``update`` query runs, @@ -169,30 +181,30 @@ rework of our ``update`` logging query: .. code-block:: sdl-invalid - trigger log_update after update for each - when (__old__.name != __new__.name) - do ( - insert Log { - action := 'update', - target_name := __new__.name, - change := __old__.name ++ '->' ++ __new__.name - } - ); + trigger log_update after update for each + when (__old__.name != __new__.name) + do ( + insert Log { + action := 'update', + target_name := __new__.name, + change := __old__.name ++ '->' ++ __new__.name + } + ); If this object were more complicated and we had many properties to compare, we could use a ``json`` cast to compare them all in one shot: .. code-block:: sdl-invalid - trigger log_update after update for each - when (__old__ {**} != __new__ {**}) - do ( - insert Log { - action := 'update', - target_name := __new__.name, - change := __old__.name ++ '->' ++ __new__.name - } - ); + trigger log_update after update for each + when (__old__ {**} != __new__ {**}) + do ( + insert Log { + action := 'update', + target_name := __new__.name, + change := __old__.name ++ '->' ++ __new__.name + } + ); You might find that one log entry per row is too granular or too noisy for your use case. In that case, a ``for all`` trigger may be a better fit. Here's a @@ -207,25 +219,25 @@ writes by making ``target_name`` and ``change`` :ref:`multi properties timestamp: datetime { default := datetime_current(); } - - target_name: str; - - change: str; - + multi target_name: str; - + multi change: str; + - target_name: str; + - change: str; + + multi target_name: str; + + multi change: str; } type Person { required name: str; - - trigger log_insert after insert for each do ( - + trigger log_insert after insert for all do ( + - trigger log_insert after insert for each do ( + + trigger log_insert after insert for all do ( insert Log { action := 'insert', target_name := __new__.name } ); - - trigger log_update after update for each do ( - + trigger log_update after update for all do ( + - trigger log_update after update for each do ( + + trigger log_update after update for all do ( insert Log { action := 'update', target_name := __new__.name, @@ -233,8 +245,8 @@ writes by making ``target_name`` and ``change`` :ref:`multi properties } ); - - trigger log_delete after delete for each do ( - + trigger log_delete after delete for all do ( + - trigger log_delete after delete for each do ( + + trigger log_delete after delete for all do ( insert Log { action := 'delete', target_name := __old__.name @@ -247,55 +259,55 @@ object instead of one ``Log`` object per row: .. code-block:: edgeql-repl - db> for name in {'Jonathan Harker', 'Mina Murray', 'Dracula'} - ... union ( - ... insert Person {name := name} - ... ); - { - default::Person {id: 3836f9c8-d393-11ed-9638-3793d3a39133}, - default::Person {id: 38370a8a-d393-11ed-9638-d3e9b92ca408}, - default::Person {id: 38370abc-d393-11ed-9638-5390f3cbd375}, - } - db> select Log {action, timestamp, target_name, change}; - { - default::Log { - action: 'insert', - timestamp: '2023-03-07T19:12:21.113521Z', - target_name: {'Jonathan Harker', 'Mina Murray', 'Dracula'}, - change: {}, - }, - } - db> for change in { - ... (old_name := 'Jonathan Harker', new_name := 'Jonathan'), - ... (old_name := 'Mina Murray', new_name := 'Mina') - ... } - ... union ( - ... update Person filter .name = change.old_name set { - ... name := change.new_name - ... } - ... ); - { - default::Person {id: 3836f9c8-d393-11ed-9638-3793d3a39133}, - default::Person {id: 38370a8a-d393-11ed-9638-d3e9b92ca408}, - } - db> select Log {action, timestamp, target_name, change}; - { - default::Log { - action: 'insert', - timestamp: '2023-04-05T09:21:17.514089Z', - target_name: {'Jonathan Harker', 'Mina Murray', 'Dracula'}, - change: {}, - }, - default::Log { - action: 'update', - timestamp: '2023-04-05T09:35:30.389571Z', - target_name: {'Jonathan', 'Mina'}, - change: {'Jonathan Harker->Jonathan', 'Mina Murray->Mina'}, - }, - } - -Validation using triggers -========================= + db> for name in {'Jonathan Harker', 'Mina Murray', 'Dracula'} + ... union ( + ... insert Person {name := name} + ... ); + { + default::Person {id: 3836f9c8-d393-11ed-9638-3793d3a39133}, + default::Person {id: 38370a8a-d393-11ed-9638-d3e9b92ca408}, + default::Person {id: 38370abc-d393-11ed-9638-5390f3cbd375}, + } + db> select Log {action, timestamp, target_name, change}; + { + default::Log { + action: 'insert', + timestamp: '2023-03-07T19:12:21.113521Z', + target_name: {'Jonathan Harker', 'Mina Murray', 'Dracula'}, + change: {}, + }, + } + db> for change in { + ... (old_name := 'Jonathan Harker', new_name := 'Jonathan'), + ... (old_name := 'Mina Murray', new_name := 'Mina') + ... } + ... union ( + ... update Person filter .name = change.old_name set { + ... name := change.new_name + ... } + ... ); + { + default::Person {id: 3836f9c8-d393-11ed-9638-3793d3a39133}, + default::Person {id: 38370a8a-d393-11ed-9638-d3e9b92ca408}, + } + db> select Log {action, timestamp, target_name, change}; + { + default::Log { + action: 'insert', + timestamp: '2023-04-05T09:21:17.514089Z', + target_name: {'Jonathan Harker', 'Mina Murray', 'Dracula'}, + change: {}, + }, + default::Log { + action: 'update', + timestamp: '2023-04-05T09:35:30.389571Z', + target_name: {'Jonathan', 'Mina'}, + change: {'Jonathan Harker->Jonathan', 'Mina Murray->Mina'}, + }, + } + +Example: validation +=================== .. index:: trigger, validate, assert @@ -307,50 +319,264 @@ common objects linked in both. .. code-block:: sdl - type Person { - required name: str; - multi friends: Person; - multi enemies: Person; - - trigger prohibit_frenemies after insert, update for each do ( - assert( - not exists (__new__.friends intersect __new__.enemies), - message := "Invalid frenemies", - ) - ) - } + type Person { + required name: str; + multi friends: Person; + multi enemies: Person; + + trigger prohibit_frenemies after insert, update for each do ( + assert( + not exists (__new__.friends intersect __new__.enemies), + message := "Invalid frenemies", + ) + ) + } With this trigger in place, it is impossible to link the same ``Person`` as both a friend and an enemy of any other person. .. code-block:: edgeql-repl - db> insert Person {name := 'Quincey Morris'}; - {default::Person {id: e4a55480-d2de-11ed-93bd-9f4224fc73af}} - db> insert Person {name := 'Dracula'}; - {default::Person {id: e7f2cff0-d2de-11ed-93bd-279780478afb}} - db> update Person - ... filter .name = 'Quincey Morris' - ... set { - ... enemies := ( - ... select detached Person filter .name = 'Dracula' - ... ) - ... }; - {default::Person {id: e4a55480-d2de-11ed-93bd-9f4224fc73af}} - db> update Person - ... filter .name = 'Quincey Morris' - ... set { - ... friends := ( - ... select detached Person filter .name = 'Dracula' - ... ) - ... }; - gel error: GelError: Invalid frenemies + db> insert Person {name := 'Quincey Morris'}; + {default::Person {id: e4a55480-d2de-11ed-93bd-9f4224fc73af}} + db> insert Person {name := 'Dracula'}; + {default::Person {id: e7f2cff0-d2de-11ed-93bd-279780478afb}} + db> update Person + ... filter .name = 'Quincey Morris' + ... set { + ... enemies := ( + ... select detached Person filter .name = 'Dracula' + ... ) + ... }; + {default::Person {id: e4a55480-d2de-11ed-93bd-9f4224fc73af}} + db> update Person + ... filter .name = 'Quincey Morris' + ... set { + ... friends := ( + ... select detached Person filter .name = 'Dracula' + ... ) + ... }; + gel error: GelError: Invalid frenemies + + +Example: logging +================ + +Declare a trigger that inserts a ``Log`` object for each new ``User`` object: + +.. code-block:: sdl + + type User { + required name: str; + + trigger log_insert after insert for each do ( + insert Log { + action := 'insert', + target_name := __new__.name + } + ); + } + +Declare a trigger that inserts a ``Log`` object conditionally when an update +query makes a change to a ``User`` object: + +.. code-block:: sdl + + type User { + required name: str; + + trigger log_update after update for each + when (__old__ {**} != __new__ {**}) + do ( + insert Log { + action := 'update', + target_name := __new__.name, + change := __old__.name ++ '->' ++ __new__.name + } + ); + } + + +.. _ref_eql_sdl_triggers_syntax: + + +Declaring triggers +================== + +This section describes the syntax to declare a trigger in your schema. + +Syntax +------ + +.. sdl:synopsis:: + + type "{" + trigger + after + {insert | update | delete} [, ...] + for {each | all} + [ when () ] + do + "}" + +Description +----------- + +This declaration defines a new trigger with the following options: + +:eql:synopsis:`` + The name (optionally module-qualified) of the type to be triggered on. + +:eql:synopsis:`` + The name of the trigger. + +:eql:synopsis:`insert | update | delete [, ...]` + The query type (or types) to trigger on. Separate multiple values with + commas to invoke the same trigger for multiple types of queries. + +:eql:synopsis:`each` + The expression will be evaluated once per modified object. ``__new__`` and + ``__old__`` in this context within the expression will refer to a single + object. + +:eql:synopsis:`all` + The expression will be evaluted once for the entire query, even if multiple + objects were modified. ``__new__`` and ``__old__`` in this context within + the expression refer to sets of the modified objects. + +.. versionadded:: 4.0 + + :eql:synopsis:`when ()` + Optionally provide a condition for the trigger. If the condition is + met, the trigger will run. If not, the trigger is skipped. + +:eql:synopsis:`` + The expression to be evaluated when the trigger is invoked. + +The trigger name must be distinct from that of any existing trigger +on the same type. + + +.. _ref_eql_ddl_triggers: + +DDL commands +============ + +This section describes the low-level DDL commands for creating and dropping +triggers. You typically don't need to use these commands directly, but +knowing about them is useful for reviewing migrations. + + +Create trigger +-------------- + +:eql-statement: + +:ref:`Define ` a new trigger. + +.. eql:synopsis:: + + {create | alter} type "{" + create trigger + after + {insert | update | delete} [, ...] + for {each | all} + [ when () ] + do + "}" + +Description +^^^^^^^^^^^ + +The command ``create trigger`` nested under ``create type`` or ``alter type`` +defines a new trigger for a given object type. + +The trigger name must be distinct from that of any existing trigger +on the same type. + +Parameters +^^^^^^^^^^ + +The options of this command are identical to the +:ref:`SDL trigger declaration `. + +Example +^^^^^^^ + +Declare a trigger that inserts a ``Log`` object for each new ``User`` object: + +.. code-block:: edgeql + + alter type User { + create trigger log_insert after insert for each do ( + insert Log { + action := 'insert', + target_name := __new__.name + } + ); + }; + +.. versionadded:: 4.0 + + Declare a trigger that inserts a ``Log`` object conditionally when an update + query makes a change to a ``User`` object: + + .. code-block:: edgeql + + alter type User { + create trigger log_update after update for each + when (__old__ {**} != __new__ {**}) + do ( + insert Log { + action := 'update', + target_name := __new__.name, + change := __old__.name ++ '->' ++ __new__.name + } + ); + } + +Drop trigger +------------ + +:eql-statement: + +Remove a trigger. + +.. eql:synopsis:: + + alter type "{" + drop trigger ; + "}" + +Description +^^^^^^^^^^^ + +The command ``drop trigger`` inside an ``alter type`` block removes the +definition of an existing trigger on the specified type. + +Parameters +^^^^^^^^^^ + +:eql:synopsis:`` + The name (optionally module-qualified) of the type being triggered on. + +:eql:synopsis:`` + The name of the trigger. + +Example +^^^^^^^ + +Remove the ``log_insert`` trigger on the ``User`` type: + +.. code-block:: edgeql + + alter type User { + drop trigger log_insert; + }; .. list-table:: :class: seealso * - **See also** - * - :ref:`SDL > Triggers ` - * - :ref:`DDL > Triggers ` * - :ref:`Introspection > Triggers ` diff --git a/docs/edgeql/path_resolution.rst b/docs/edgeql/path_resolution.rst index 3ed1eefe99d..324cafbd097 100644 --- a/docs/edgeql/path_resolution.rst +++ b/docs/edgeql/path_resolution.rst @@ -132,6 +132,8 @@ configuration value ``simple_scoping``: - No - Yes +.. _ref_warn_old_scoping: + Warning on old scoping ---------------------- diff --git a/docs/edgeql/select.rst b/docs/edgeql/select.rst index 68cbbb62dc6..8098038930a 100644 --- a/docs/edgeql/select.rst +++ b/docs/edgeql/select.rst @@ -623,6 +623,31 @@ traversing a backlink would look like this: .. _ref_eql_select_order: +Filtering, ordering, and limiting of links +========================================== + +Clauses like ``filter``, ``order by``, and ``limit`` can be used on links. +If no properties of a link are selected, you can place the clauses directly +inside the shape: + +.. code-block:: edgeql + + select User { + likes order by .title desc limit 10 + }; + +If properties are selected, place the clauses after the link's shape: + +.. code-block:: edgeql + + select User { + likes: { + id, + title + } order by .title desc limit 10 + }; + + Ordering -------- diff --git a/docs/intro/index.rst b/docs/intro/index.rst index d7769e991cf..db81490ce14 100644 --- a/docs/intro/index.rst +++ b/docs/intro/index.rst @@ -9,8 +9,8 @@ Get Started :maxdepth: 3 :hidden: - quickstart quickstart/fastapi/index + quickstart/index cli instances projects diff --git a/docs/intro/quickstart.rst b/docs/intro/quickstart.rst deleted file mode 100644 index 88e700073f3..00000000000 --- a/docs/intro/quickstart.rst +++ /dev/null @@ -1,536 +0,0 @@ -.. _ref_quickstart: - -========== -Quickstart -========== - -Welcome to |Gel|! - -This quickstart will walk you through the entire process of creating a simple -Gel-powered application: installation, defining your schema, adding some -data, and writing your first query. Let's jump in! - -.. _ref_quickstart_install: - -1. Installation -=============== - -First let's install the Gel CLI. Open a terminal and run the appropriate -command below. - -JavaScript and Python users ---------------------------- - -If you use ``npx`` or ``uvx`` you can skip the installation steps below -and use Gel CLI like this: - -.. code-block:: bash - - # JavaScript: - $ npx gel project init - - # Python - $ uvx gel project init - - -Linux ------ - -.. tabs:: - - .. code-tab:: bash - :caption: Script - - $ curl https://geldata.com/sh --proto '=https' -sSf1 | sh - - .. code-tab:: bash - :caption: APT - - $ # Import the Gel packaging key - $ sudo mkdir -p /usr/local/share/keyrings && \ - sudo curl --proto '=https' --tlsv1.2 -sSf \ - -o /usr/local/share/keyrings/gel-keyring.gpg \ - https://packages.geldata.com/keys/gel-keyring.gpg && \ - $ # Add the Gel package repository - $ echo deb [signed-by=/usr/local/share/keyrings/gel-keyring.gpg]\ - https://packages.geldata.com/apt \ - $(grep "VERSION_CODENAME=" /etc/os-release | cut -d= -f2) main \ - | sudo tee /etc/apt/sources.list.d/gel.list - $ # Install the Gel package - $ sudo apt-get update && sudo apt-get install gel-6 - - .. code-tab:: bash - :caption: YUM - - $ # Add the Gel package repository - $ sudo curl --proto '=https' --tlsv1.2 -sSfL \ - https://packages.geldata.com/rpm/gel-rhel.repo \ - > /etc/yum.repos.d/gel.repo - $ # Install the Gel package - $ sudo yum install gel-6 - -macOS ------ - -.. tabs:: - - .. code-tab:: bash - :caption: Script - - $ curl https://geldata.com/sh --proto '=https' -sSf1 | sh - - .. code-tab:: bash - :caption: Homebrew - - $ # Add the Gel tap to your Homebrew - $ brew tap geldata/tap - $ # Install Gel CLI - $ brew install gel-cli - -Windows (Powershell) --------------------- - -.. note:: - - Gel on Windows requires WSL 2 to create local instances because the - Gel server runs on Linux. It is *not* required if you will use the CLI - only to manage Gel Cloud and/or other remote instances. This quickstart - *does* create local instances, so WSL 2 is required to complete the - quickstart. - -.. code-block:: powershell - - PS> iwr https://geldata.com/ps1 -useb | iex - -.. note:: Command prompt installation - - To install Gel in the Windows Command prompt, follow these steps: - - 1. `Download the CLI `__ - - 2. Navigate to the download location in the command prompt - - 3. Run the installation command: - - .. code-block:: - - gel-cli.exe _self_install - -The script installation methods download and execute a bash script that -installs the |gelcmd| CLI on your machine. You may be asked for your -password. Once the installation completes, you may need to **restart your -terminal** before you can use the |gelcmd| command. - -Now let's set up your Gel project. - -.. _ref_quickstart_createdb: - -2. Initialize a project -======================= - -In a terminal, create a new directory and ``cd`` into it. - -.. code-block:: bash - - $ mkdir quickstart - $ cd quickstart - -Then initialize your Gel project: - -.. code-block:: bash - - $ gel project init - -This starts an interactive tool that walks you through the process of setting -up your first Gel instance. You should see something like this: - -.. code-block:: bash - - $ gel project init - No `tel.toml` found in `/path/to/quickstart` or above - Do you want to initialize a new project? [Y/n] - > Y - Specify the name of Gel instance to use with this project - [default: quickstart]: - > quickstart - Checking Gel versions... - Specify the version of Gel to use with this project [default: x.x]: - > x.x - Specify branch name: [default: main]: - > main - ┌─────────────────────┬───────────────────────────────────────────────┐ - │ Project directory │ ~/path/to/quickstart │ - │ Project config │ ~/path/to/quickstart/gel.toml │ - │ Schema dir (empty) │ ~/path/to/quickstart/dbschema │ - │ Installation method │ portable package │ - │ Version │ x.x+cc4f3b5 │ - │ Instance name │ quickstart │ - └─────────────────────┴───────────────────────────────────────────────┘ - Downloading package... - 00:00:01 [====================] 41.40 MiB/41.40 MiB 32.89MiB/s | ETA: 0s - Successfully installed x.x+cc4f3b5 - Initializing Gel instance... - Applying migrations... - Everything is up to date. Revision initial - Project initialized. - To connect to quickstart, run `gel` - - -This did a couple things. - -1. First, it scaffolded your project by creating an - :ref:`ref_reference_gel_toml` config file and a schema file - :dotgel:`dbschema/default`. In the next section, you'll define a schema in - :dotgel:`default`. - -2. Second, it spun up an Gel instance called ``quickstart`` and "linked" it - to the current directory. As long as you're inside the project - directory, all CLI commands will be executed against this - instance. For more details on how Gel projects work, check out the - :ref:`Managing instances ` guide. - -.. note:: - - Quick note! You can have several **instances** of Gel running on your - computer simultaneously. Each instance may be **branched** many times. Each - branch may have an independent schema consisting of a number of **modules** - (though commonly your schema will be entirely defined inside the ``default`` - module). - -Let's connect to our new instance! Run |gelcmd| in your terminal to open an -interactive REPL to your instance. You're now connected to a live Gel -instance running on your computer! Try executing a simple query (``select 1 + 1;``) after the -REPL prompt (``quickstart:main>``): - -.. code-block:: edgeql-repl - - quickstart:main> select 1 + 1; - {2} - -Run ``\q`` to exit the REPL. More interesting queries are coming soon, -promise! But first we need to set up a schema. - -.. _ref_quickstart_createdb_sdl: - -3. Set up your schema -===================== - -Open the ``quickstart`` directory in your IDE or editor of choice. You should -see the following file structure. - -.. code-block:: - - /path/to/quickstart - ├── gel.toml - ├── dbschema - │ ├── default.gel - │ ├── migrations - -|Gel| schemas are defined with a dedicated schema definition language called -(predictably) Gel SDL (or just **SDL** for short). It's an elegant, -declarative way to define your data model. - -SDL lives inside |.gel| files. Commonly, your entire schema will be -declared in a file called :dotgel:`default` but you can split your schema -across several |.gel| files if you prefer. - -.. note:: - - Syntax-highlighter packages/extensions for |.gel| files are available - for - `Visual Studio Code `_, - `Sublime Text `_, - `Atom `_, - and `Vim `_. - -Let's build a simple movie database. We'll need to define two **object types** -(equivalent to a *table* in SQL): Movie and Person. Open -:dotgel:`dbschema/default` in your editor of choice and paste the following: - -.. code-block:: sdl - - module default { - type Person { - required name: str; - } - - type Movie { - title: str; - multi actors: Person; - } - }; - - -A few things to note here. - -- Our types don't contain an ``id`` property; Gel automatically - creates this property and assigns a unique UUID to every object inserted - into the database. -- The ``Movie`` type includes a **link** named ``actors``. In Gel, links are - used to represent relationships between object types. They eliminate the need - for foreign keys; later, you'll see just how easy it is to write "deep" - queries without JOINs. -- The object types are inside a ``module`` called ``default``. You can split - up your schema into logical subunits called modules, though it's common to - define the entire schema in a single module called ``default``. - -Now we're ready to run a migration to apply this schema to the database. - -4. Run a migration -================== - -Generate a migration file with :gelcmd:`migration create`. This command -gathers up our :dotgel:`*` files and sends them to the database. The *database -itself* parses these files, compares them against its current schema, and -generates a migration plan! Then the database sends this plan back to the CLI, -which creates a migration file. - -.. code-block:: bash - - $ gel migration create - Created ./dbschema/migrations/00001.edgeql (id: ) - -.. note:: - - If you're interested, open this migration file to see what's inside! It's - a simple EdgeQL script consisting of :ref:`DDL ` commands like - ``create type``, ``alter type``, and ``create property``. - -The migration file has been *created* but we haven't *applied it* against the -database. Let's do that. - -.. code-block:: bash - - $ gel migrate - Applied m1k54jubcs62wlzfebn3pxwwngajvlbf6c6qfslsuagkylg2fzv2lq (00001.edgeql) - -Looking good! Let's make sure that worked by running :gelcmd:`list types` on -the command line. This will print a table containing all currently-defined -object types. - -.. code-block:: bash - - $ gel list types - ┌─────────────────┬──────────────────────────────┐ - │ Name │ Extending │ - ├─────────────────┼──────────────────────────────┤ - │ default::Movie │ std::BaseObject, std::Object │ - │ default::Person │ std::BaseObject, std::Object │ - └─────────────────┴──────────────────────────────┘ - - -.. _ref_quickstart_migrations: - -.. _Migrate your schema: - -Before we proceed, let's try making a small change to our schema: making the -``title`` property of ``Movie`` required. First, update the schema file: - -.. code-block:: sdl-diff - - type Movie { - - title: str; - + required title: str; - multi actors: Person; - } - -Then create another migration. Because this isn't the initial migration, we -see something a little different than before. - -.. code-block:: bash - - $ gel migration create - did you make property 'title' of object type 'default::Movie' - required? [y,n,l,c,b,s,q,?] - > - -As before, Gel parses the schema files and compared them against its -current internal schema. It correctly detects the change we made, and prompts -us to confirm it. This interactive process lets you sanity check every change -and provide guidance when a migration is ambiguous (e.g. when a property is -renamed). - -Enter ``y`` to confirm the change. - -.. code-block:: bash - - $ gel migration create - did you make property 'title' of object type 'default::Movie' - required? [y,n,l,c,b,s,q,?] - > y - Please specify an expression to populate existing objects in - order to make property 'title' of object type 'default::Movie' required: - fill_expr> {} - -Hm, now we're seeing another prompt. Because ``title`` is changing from -*optional* to *required*, Gel is asking us what to do for all the ``Movie`` -objects that don't currently have a value for ``title`` defined. We'll just -specify a placeholder value of "Untitled". Replace the ``{}`` value -with ``"Untitled"`` and press Enter. - -.. code-block:: - - fill_expr> "Untitled" - Created dbschema/migrations/00002.edgeql (id: ) - - -If we look at the generated migration file, we see it contains the following -lines: - -.. code-block:: edgeql - - ALTER TYPE default::Movie { - ALTER PROPERTY title { - SET REQUIRED USING ('Untitled'); - }; - }; - -Let's wrap up by applying the new migration. - -.. code-block:: bash - - $ gel migrate - Applied m1rd2ikgwdtlj5ws7ll6rwzvyiui2xbrkzig4adsvwy2sje7kxeh3a (00002.edgeql) - -.. _ref_quickstart_insert_data: - -.. _Insert data: - -.. _Run some queries: - -5. Write some queries -===================== - -Let's write some simple queries via *Gel UI*, the admin dashboard baked -into every Gel instance. To open the dashboard: - -.. code-block:: bash - - $ gel ui - Opening URL in browser: - http://localhost:107xx/ui?authToken= - -You should see a simple landing page, as below. You'll see a card for each -branch of your instance. Remember: each instance can be branched multiple -times! - -.. image:: images/ui_landing.jpg - :width: 100% - -Currently, there's only one branch, which is simply called |main| by -default. Click the |main| card. - -.. image:: images/ui_db.jpg - :width: 100% - -Then click ``Open Editor`` so we can start writing some queries. We'll start -simple: ``select "Hello world!";``. Click ``RUN`` to execute the query. - -.. image:: images/ui_hello.jpg - :width: 100% - -The result of the query will appear on the right. - -The query will also be added to your history of previous queries, which can be -accessed via the "HISTORY" tab located on the lower left side of the editor. - -Now let's actually ``insert`` an object into our database. Copy the following -query into the query textarea and hit ``Run``. - -.. code-block:: edgeql - - insert Movie { - title := "Dune" - }; - -Nice! You've officially inserted the first object into your database! Let's -add a couple cast members with an ``update`` query. - -.. code-block:: edgeql - - update Movie - filter .title = "Dune" - set { - actors := { - (insert Person { name := "Timothee Chalamet" }), - (insert Person { name := "Zendaya" }) - } - }; - -Finally, we can run a ``select`` query to fetch all the data we just inserted. - -.. code-block:: edgeql - - select Movie { - title, - actors: { - name - } - }; - -Click the outermost ``COPY`` button in the top right of the query result area -to copy the result of this query to your clipboard as JSON. The copied text -will look something like this: - -.. code-block:: json - - [ - { - "title": "Dune", - "actors": [ - { - "name": "Timothee Chalamet" - }, - { - "name": "Zendaya" - } - ] - } - ] - -|Gel| UI is a useful development tool, but in practice your application will -likely be using one of Gel's *client libraries* to execute queries. Gel -provides official libraries for many langauges: - -- :ref:`JavaScript/TypeScript ` -- :ref:`Go ` -- :ref:`Python ` - -.. XXX: link to third-party doc websites -.. - :ref:`Rust ` -.. - :ref:`C# and F# ` -.. - :ref:`Java ` -.. - :ref:`Dart ` -.. - :ref:`Elixir ` - -Check out the :ref:`Clients ` guide to get -started with the language of your choice. - -.. _ref_quickstart_onwards: - -.. _Computeds: - -Onwards and upwards -=================== - -You now know the basics of Gel! You've installed the CLI and database, set -up a local project, run a couple migrations, inserted and queried some data, -and used a client library. - -- For a more in-depth exploration of each topic covered here, continue reading - the other pages in the Getting Started section, which will cover important - topics like migrations, the schema language, and EdgeQL in greater detail. - -.. XXX: -.. - For guided tours of major concepts, check out the showcase pages for `Data -.. Modeling `_, `EdgeQL -.. `_, and `Migrations -.. `_. - -- To start building an application using the language of your choice, check out - our client libraries: - - - :ref:`JavaScript/TypeScript ` - - :ref:`Go ` - - :ref:`Python ` diff --git a/docs/intro/quickstart/connecting.rst b/docs/intro/quickstart/connecting.rst new file mode 100644 index 00000000000..36838dd4f01 --- /dev/null +++ b/docs/intro/quickstart/connecting.rst @@ -0,0 +1,97 @@ +.. _ref_quickstart_connecting: + +========================== +Connecting to the database +========================== + +.. edb:split-section:: + + Before diving into the application, let's take a quick look at how to connect to the database from your code. We will intialize a client and use it to make a simple, static query to the database, and log the result to the console. + + .. note:: + + Notice that the ``createClient`` function isn't being passed any connection details. With |Gel|, you do not need to come up with your own scheme for how to build the correct database connection credentials and worry about leaking them into your code. You simply use |Gel| "projects" for local development, and set the appropriate environment variables in your deployment environments, and the ``createClient`` function knows what to do! + + .. edb:split-point:: + + .. code-block:: typescript + :caption: ./test.ts + + import { createClient } from "gel"; + + const client = createClient(); + + async function main() { + console.log(await client.query("select 'Hello from Gel!';")); + } + + main().then( + () => process.exit(0), + (err) => { + console.error(err); + process.exit(1); + } + ); + + + .. code-block:: sh + + $ npx tsx test.ts + [ 'Hello from Gel!' ] + +.. edb:split-section:: + + + With TypeScript, there are three ways to run a query: use a string EdgeQL query, use the ``queries`` generator to turn a string of EdgeQL into a TypeScript function, or use the query builder API to build queries dynamically in a type-safe manner. In this tutorial, you will use the TypeScript query builder API. + + This query builder must be regenerated any time the schema changes, so a hook has been added to the ``gel.toml`` file to generate the query builder any time the schema is updated. Moving beyond this simple query, use the query builder API to insert a few ``Deck`` objects into the database, and then select them back. + + .. edb:split-point:: + + .. code-block:: typescript-diff + :caption: ./test.ts + + import { createClient } from "gel"; + + import e from "@/dbschema/edgeql-js"; + + const client = createClient(); + + async function main() { + console.log(await client.query("select 'Hello from Gel!';")); + + + await e.insert(e.Deck, { name: "I am one" }).run(client); + + + + await e.insert(e.Deck, { name: "I am two" }).run(client); + + + + const decks = await e + + .select(e.Deck, () => ({ + + id: true, + + name: true, + + })) + + .run(client); + + + + console.table(decks); + + + + await e.delete(e.Deck).run(client); + } + + main().then( + () => process.exit(0), + (err) => { + console.error(err); + process.exit(1); + } + ); + + .. code-block:: sh + + $ npx tsx test.ts + [ 'Hello from Gel!' ] + ┌─────────┬────────────────────────────────────────┬────────────┐ + │ (index) │ id │ name │ + ├─────────┼────────────────────────────────────────┼────────────┤ + │ 0 │ 'f4cd3e6c-ea75-11ef-83ec-037350ea8a6e' │ 'I am one' │ + │ 1 │ 'f4cf27ae-ea75-11ef-83ec-3f7b2fceab24' │ 'I am two' │ + └─────────┴────────────────────────────────────────┴────────────┘ + +Now that you know how to connect to the database, you will see that we have provided an initialized ``Client`` object in the ``/lib/gel.ts`` module. Throughout the rest of the tutorial, you will import this ``Client`` object and use it to make queries. diff --git a/docs/intro/quickstart/images/flashcards-import.png b/docs/intro/quickstart/images/flashcards-import.png new file mode 100644 index 00000000000..8b3d66a2f06 Binary files /dev/null and b/docs/intro/quickstart/images/flashcards-import.png differ diff --git a/docs/intro/quickstart/images/schema-ui.png b/docs/intro/quickstart/images/schema-ui.png new file mode 100644 index 00000000000..3f197e8baeb Binary files /dev/null and b/docs/intro/quickstart/images/schema-ui.png differ diff --git a/docs/intro/quickstart/images/timestamped.png b/docs/intro/quickstart/images/timestamped.png new file mode 100644 index 00000000000..5d3f523a36b Binary files /dev/null and b/docs/intro/quickstart/images/timestamped.png differ diff --git a/docs/intro/quickstart/index.rst b/docs/intro/quickstart/index.rst new file mode 100644 index 00000000000..a9dd7543131 --- /dev/null +++ b/docs/intro/quickstart/index.rst @@ -0,0 +1,62 @@ +.. _ref_quickstart: + +========== +Quickstart +========== + +.. toctree:: + :maxdepth: 1 + :hidden: + + setup + modeling + connecting + working + inheritance + + +Welcome to the quickstart tutorial! In this tutorial, you will update a simple Next.js application to use |Gel| as your data layer. The application will let users build and manage their own study decks, with each flashcard featuring customizable text on both sides - making it perfect for studying, memorization practice, or creating educational games. + +Don't worry if you're new to |Gel| - you will be up and running with a working Next.js application and a local |Gel| database in just about **5 minutes**. From there, you will replace the static mock data with a |Gel| powered data layer in roughly 30-45 minutes. + +By the end of this tutorial, you will be comfortable with: + +* Creating and updating a database schema +* Running migrations to evolve your data +* Writing EdgeQL queries in text and via a TypeScript query builder +* Building an app backed by |Gel| + +Features of the flashcards app +------------------------------ + +* Create, edit, and delete decks +* Add/remove cards with front/back content +* Simple Next.js + Tailwind UI +* Clean, type-safe schema with |Gel| + +Requirements +------------ + +Before you start, you need: + +* Basic familiarity with TypeScript, Next.js, and React +* Node.js 20+ on a Unix-like OS (Linux, macOS, or WSL) +* A code editor you love + +Why |Gel| for Next.js? +---------------------- + +* **Type Safety**: Catch data errors before runtime +* **Rich Modeling**: Use object types and links to model relations +* **Modern Tooling**: TypeScript-friendly schemas and migrations +* **Performance**: Efficient queries for complex data +* **Developer Experience**: An intuitive query language (EdgeQL) + +Need Help? +---------- + +If you run into issues while following this tutorial: + +* Check the `Gel documentation `_ +* Visit our `community Discord `_ +* File an issue on `GitHub `_ diff --git a/docs/intro/quickstart/inheritance.rst b/docs/intro/quickstart/inheritance.rst new file mode 100644 index 00000000000..520b4d33dad --- /dev/null +++ b/docs/intro/quickstart/inheritance.rst @@ -0,0 +1,113 @@ +.. _ref_quickstart_inheritance: + +======================== +Adding shared properties +======================== + +.. edb:split-section:: + + One common pattern in applications is to add shared properties to the schema that are used by multiple objects. For example, you might want to add a ``created_at`` and ``updated_at`` property to every object in your schema. You can do this by adding an abstract type and using it as a mixin for your other object types. + + .. code-block:: sdl-diff + :caption: dbschema/default.gel + + module default { + + abstract type Timestamped { + + required created_at: datetime { + + default := datetime_of_statement(); + + }; + + required updated_at: datetime { + + default := datetime_of_statement(); + + }; + + } + + + - type Deck { + + type Deck extending Timestamped { + required name: str; + description: str; + + multi cards: Card { + constraint exclusive; + on target delete allow; + }; + }; + + - type Card { + + type Card extending Timestamped { + required order: int64; + required front: str; + required back: str; + } + } + +.. edb:split-section:: + + Since you don't have historical data for when these objects were actually created or modified, the migration will fall back to the default values set in the ``Timestamped`` type. + + .. code-block:: sh + + $ npx gel migration create + did you create object type 'default::Timestamped'? [y,n,l,c,b,s,q,?] + > y + did you alter object type 'default::Card'? [y,n,l,c,b,s,q,?] + > y + did you alter object type 'default::Deck'? [y,n,l,c,b,s,q,?] + > y + Created /home/strinh/projects/flashcards/dbschema/migrations/00004-m1d2m5n.edgeql, id: m1d2m5n5ajkalyijrxdliioyginonqbtfzihvwdfdmfwodunszstya + + $ npx gel migrate + Applying m1d2m5n5ajkalyijrxdliioyginonqbtfzihvwdfdmfwodunszstya (00004-m1d2m5n.edgeql) + ... parsed + ... applied + Generating query builder... + Detected tsconfig.json, generating TypeScript files. + To override this, use the --target flag. + Run `npx @gel/generate --help` for full options. + Introspecting database schema... + Generating runtime spec... + Generating cast maps... + Generating scalars... + Generating object types... + Generating function types... + Generating operators... + Generating set impl... + Generating globals... + Generating index... + Writing files to ./dbschema/edgeql-js + Generation complete! 🤘 + +.. edb:split-section:: + + Update the ``getDecks`` query to sort the decks by ``updated_at`` in descending order. + + .. code-block:: typescript-diff + :caption: app/queries.ts + + import { client } from "@/lib/gel"; + import e from "@/dbschema/edgeql-js"; + + export async function getDecks() { + const decks = await e.select(e.Deck, (deck) => ({ + id: true, + name: true, + description: true, + cards: e.select(deck.cards, (card) => ({ + id: true, + front: true, + back: true, + order_by: card.order, + })), + + order_by: { + + expression: deck.updated_at, + + direction: e.DESC, + + }, + })).run(client); + + return decks; + } + +.. edb:split-section:: + + Now when you look at the data in the UI, you will see the new properties on each of your object types. + + .. image:: images/timestamped.png diff --git a/docs/intro/quickstart/modeling.rst b/docs/intro/quickstart/modeling.rst new file mode 100644 index 00000000000..ce54e63690d --- /dev/null +++ b/docs/intro/quickstart/modeling.rst @@ -0,0 +1,108 @@ +.. _ref_quickstart_modeling: + +================= +Modeling the data +================= + +.. edb:split-section:: + + The flashcards application has a simple data model, but it's interesting enough to utilize many unique features of the |Gel| schema language. + + Looking at the mock data in our example JSON file ``./deck-edgeql.json``, you can see this structure in the JSON. There is a ``Card`` type that describes a single flashcard, which contains two required string properties: ``front`` and ``back``. Each ``Deck`` object has a link to zero or more ``Card`` objects in an array. + + .. code-block:: typescript + + interface Card { + front: string; + back: string; + } + + interface Deck { + name: string; + description: string | null; + cards: Card[]; + } + +.. edb:split-section:: + + Starting with this simple model, add these types to the :dotgel:`dbschema/default` schema file. As you can see, the types closely mirror the JSON mock data. + + Also of note, the link between ``Card`` and ``Deck`` objects creates a "1-to-n" relationship, where each ``Deck`` object has a link to zero or more ``Card`` objects. When you query the ``Deck.cards`` link, the cards will be unordered, so the ``Card`` type needs an explicit ``order`` property to allow sorting them at query time. + + By default, when you try to delete an object that is linked to another object, the database will prevent you from doing so. We want to support removing a ``Card``, so we define a deletion policy on the ``cards`` link that allows deleting the target of this link. + + .. code-block:: sdl-diff + :caption: dbschema/default.gel + + module default { + + type Card { + + required order: int64; + + required front: str; + + required back: str; + + }; + + + + type Deck { + + required name: str; + + description: str; + + multi cards: Card { + + constraint exclusive; + + on target delete allow; + + }; + + }; + }; + +.. edb:split-section:: + + Congratulations! This first version of the data model's schema is *stored in a file on disk*. Now you need to signal the database to actually create types for ``Deck`` and ``Card`` in the database. + + To make |Gel| do that, you need to do two quick steps: + + 1. **Create a migration**: a "migration" is a file containing a set of low level instructions that define how the database schema should change. It records any additions, modifications, or deletions to your schema in a way that the database can understand. + + .. note:: + + When you are changing existing schema, the CLI migration tool might ask questions to ensure that it understands your changes exactly. Since the existing schema was empty, the CLI will skip asking any questions and simply create the migration file. + + 2. **Apply the migration**: This executes the migration file on the database, instructing |Gel| to implement the recorded changes in the database. Essentially, this step updates the database structure to match your defined schema, ensuring that the ``Deck`` and ``Card`` types are created and ready for use. + + .. note:: + + Notice that after the migration is applied, the CLI will automatically run the script to generate the query builder. This is a convenience feature that is enabled by the ``schema.update.after`` hook in the ``gel.toml`` file. + + .. code-block:: sh + + $ npx gel migration create + Created ./dbschema/migrations/00001-m125ajr.edgeql, id: m125ajrbqp7ov36s7aniefxc376ofxdlketzspy4yddd3hrh4lxmla + $ npx gel migrate + Applying m125ajrbqp7ov36s7aniefxc376ofxdlketzspy4yddd3hrh4lxmla (00001-m125ajr.edgeql) + ... parsed + ... applied + Generating query builder... + Detected tsconfig.json, generating TypeScript files. + To override this, use the --target flag. + Run `npx @gel/generate --help` for full options. + Introspecting database schema... + Generating runtime spec... + Generating cast maps... + Generating scalars... + Generating object types... + Generating function types... + Generating operators... + Generating set impl... + Generating globals... + Generating index... + Writing files to ./dbschema/edgeql-js + Generation complete! 🤘 + + +.. edb:split-section:: + + Take a look at the schema you've generated in the built-in database UI. Use this tool to visualize your data model and see the object types and links you've defined. + + .. edb:split-point:: + + .. code-block:: sh + + $ npx gel ui + + .. image:: images/schema-ui.png diff --git a/docs/intro/quickstart/setup.rst b/docs/intro/quickstart/setup.rst new file mode 100644 index 00000000000..371fb5bd57d --- /dev/null +++ b/docs/intro/quickstart/setup.rst @@ -0,0 +1,89 @@ +.. _ref_quickstart_setup: + +=========================== +Setting up your environment +=========================== + +.. edb:split-section:: + + Use git to clone `the Next.js starter template `_ into a new directory called ``flashcards``. This will create a fully configured Next.js project and a local |Gel| instance with an empty schema. You will see the database instance being created and the project being initialized. You are now ready to start building the application. + + .. code-block:: sh + + $ git clone \ + git@github.com:geldata/quickstart-nextjs.git \ + flashcards + $ cd flashcards + $ npm install + $ npx gel project init + + +.. edb:split-section:: + + Explore the empty database by starting our REPL from the project root. + + .. code-block:: sh + + $ npx gel + +.. edb:split-section:: + + Try the following queries which will work without any schema defined. + + .. code-block:: edgeql-repl + + db> select 42; + {42} + db> select sum({1, 2, 3}); + {6} + db> with cards := { + ... ( + ... front := "What is the highest mountain in the world?", + ... back := "Mount Everest", + ... ), + ... ( + ... front := "Which ocean contains the deepest trench on Earth?", + ... back := "The Pacific Ocean", + ... ), + ... } + ... select cards order by random() limit 1; + { + ( + front := "What is the highest mountain in the world?", + back := "Mount Everest", + ) + } + +.. edb:split-section:: + + Fun! You will create a proper data model for the application in the next step, but for now, take a look around the project you've just created. Most of the project files will be familiar if you've worked with Next.js before. Here are the files that integrate |Gel|: + + - ``gel.toml``: The configuration file for the Gel project instance. Notice that we have a ``hooks.migration.apply.after`` hook that will run ``npx @gel/generate edgeql-js`` after migrations are applied. This will generate the query builder code that you'll use to interact with the database. More details on that to come! + - ``dbschema/``: This directory contains the schema for the database, and later supporting files like migrations, and generated code. + - :dotgel:`dbschema/default`: The default schema file that you'll use to define your data model. It is empty for now, but you'll add your data model to this file in the next step. + - ``lib/gel.ts``: A utility module that exports the Gel client, which you'll use to interact with the database. + + .. tabs:: + + .. code-tab:: toml + :caption: gel.toml + + [instance] + server-version = 6.0 + + [hooks] + schema.update.after = "npx @gel/generate edgeql-js" + + .. code-tab:: sdl + :caption: dbschema/default.gel + + module default { + + } + + .. code-tab:: typescript + :caption: lib/gel.ts + + import { createClient } from "gel"; + + export const client = createClient(); diff --git a/docs/intro/quickstart/working.rst b/docs/intro/quickstart/working.rst new file mode 100644 index 00000000000..3aee348a036 --- /dev/null +++ b/docs/intro/quickstart/working.rst @@ -0,0 +1,635 @@ +.. _ref_quickstart_working: + +===================== +Working with the data +===================== + +In this section, you will update the existing application to use |Gel| to store and query data, instead of a static JSON file. Having a working application with mock data allows you to focus on learning how |Gel| works, without getting bogged down by the details of the application. + +Bulk importing of data +====================== + +.. edb:split-section:: + + Begin by updating the server action to import a deck with cards. Loop through each card in the deck and insert it, building an array of IDs as you go. This array of IDs will be used to set the ``cards`` link on the ``Deck`` object after all cards have been inserted. + + The array of card IDs is initially an array of strings. To satisfy the |Gel| type system, which expects the ``id`` property of ``Card`` objects to be a ``uuid`` rather than a ``str``, you need to cast the array of strings to an array of UUIDs. Use the ``e.literal(e.array(e.uuid), cardIds)`` function to perform this casting. + + The function ``e.includes(cardIdsLiteral, c.id)`` from our standard library checks if a value is present in an array and returns a boolean. When inserting the ``Deck`` object, set the ``cards`` to the result of selecting only the ``Card`` objects whose ``id`` is included in the ``cardIds`` array. + + .. code-block:: typescript-diff + :caption: app/actions.ts + + "use server"; + + - import { readFile, writeFile } from "node:fs/promises"; + + import { client } from "@/lib/gel"; + + import e from "@/dbschema/edgeql-js"; + import { revalidatePath } from "next/cache"; + - import { RawJSONDeck, Deck } from "@/lib/models"; + + import { RawJSONDeck } from "@/lib/models"; + + export async function importDeck(formData: FormData) { + const file = formData.get("file") as File; + const rawDeck = JSON.parse(await file.text()) as RawJSONDeck; + const deck = { + ...rawDeck, + - id: crypto.randomUUID(), + - cards: rawDeck.cards.map((card) => ({ + + cards: rawDeck.cards.map((card, index) => ({ + ...card, + - id: crypto.randomUUID(), + + order: index, + })), + }; + - + - const existingDecks = JSON.parse( + - await readFile("./decks.json", "utf-8") + - ) as Deck[]; + - + - await writeFile( + - "./decks.json", + - JSON.stringify([...existingDecks, deck], null, 2) + - ); + + const cardIds: string[] = []; + + for (const card of deck.cards) { + + const createdCard = await e + + .insert(e.Card, { + + front: card.front, + + back: card.back, + + order: card.order, + + }) + + .run(client); + + + + cardIds.push(createdCard.id); + + } + + + + const cardIdsLiteral = e.literal(e.array(e.uuid), cardIds); + + + + await e.insert(e.Deck, { + + name: deck.name, + + description: deck.description, + + cards: e.select(e.Card, (c) => ({ + + filter: e.contains(cardIdsLiteral, c.id), + + })), + + }).run(client); + + revalidatePath("/"); + } + +.. edb:split-section:: + + This works, but you might notice that it is not atomic. For instance, if one of the ``Card`` objects fails to insert, the entire operation will fail and the ``Deck`` will not be inserted, but some data will still linger. To make this operation atomic, update the ``importDeck`` action to use a transaction. + + .. code-block:: typescript-diff + :caption: app/actions.ts + + "use server"; + + import { client } from "@/lib/gel"; + import e from "@/dbschema/edgeql-js"; + import { revalidatePath } from "next/cache"; + import { RawJSONDeck } from "@/lib/models"; + + export async function importDeck(formData: FormData) { + const file = formData.get("file") as File; + const rawDeck = JSON.parse(await file.text()) as RawJSONDeck; + const deck = { + ...rawDeck, + cards: rawDeck.cards.map((card, index) => ({ + ...card, + order: index, + })), + }; + + await client.transaction(async (tx) => { + const cardIds: string[] = []; + for (const card of deck.cards) { + const createdCard = await e + .insert(e.Card, { + front: card.front, + back: card.back, + order: card.order, + }) + - .run(client); + + .run(tx); + + cardIds.push(createdCard.id); + } + + const cardIdsLiteral = e.literal(e.array(e.uuid), cardIds); + + await e.insert(e.Deck, { + name: deck.name, + description: deck.description, + cards: e.select(e.Card, (c) => ({ + filter: e.contains(cardIdsLiteral, c.id), + })), + - }).run(client); + + }).run(tx); + + }); + + revalidatePath("/"); + } + +.. edb:split-section:: + + You might think this is as good as it gets, and many ORMs will create a similar set of queries. However, with the query builder, you can improve this by crafting a single query that inserts the ``Deck`` and ``Card`` objects, along with their links, in one efficient query. + + The first thing to notice is that the ``e.params`` function is used to define parameters for your query instead of embedding literal values directly. This approach eliminates the need for casting, as was necessary with the ``cardIds`` array. By defining the ``cards`` parameter as an array of tuples, you ensure full type safety with both TypeScript and the database. + + Another key feature of this query builder expression is the ``e.for(e.array_unpack(params.cards), (card) => {...})`` construct. This expression converts the array of tuples into a set of tuples and generates a set containing an expression for each element. Essentially, you assign the ``Deck.cards`` set of ``Card`` objects to the result of inserting each element from the ``cards`` array. This is similar to what you were doing before by selecting all ``Card`` objects by their ``id``, but is more efficient since you are inserting the ``Deck`` and all ``Card`` objects in one query. + + .. code-block:: typescript-diff + :caption: app/actions.ts + + "use server"; + + import { client } from "@/lib/gel"; + import e from "@/dbschema/edgeql-js"; + import { revalidatePath } from "next/cache"; + import { RawJSONDeck } from "@/lib/models"; + + export async function importDeck(formData: FormData) { + const file = formData.get("file") as File; + const rawDeck = JSON.parse(await file.text()) as RawJSONDeck; + const deck = { + ...rawDeck, + cards: rawDeck.cards.map((card, index) => ({ + ...card, + order: index, + })), + }; + - await client.transaction(async (tx) => { + - const cardIds: string[] = []; + - for (const card of deck.cards) { + - const createdCard = await e + - .insert(e.Card, { + - front: card.front, + - back: card.back, + - order: card.order, + - }) + - .run(tx); + - + - cardIds.push(createdCard.id); + - } + - + - const cardIdsLiteral = e.literal(e.array(e.uuid), cardIds); + - + - await e.insert(e.Deck, { + - name: deck.name, + - description: deck.description, + - cards: e.select(e.Card, (c) => ({ + - filter: e.contains(cardIdsLiteral, c.id), + - })), + - }).run(tx); + - }); + + await e + + .params( + + { + + name: e.str, + + description: e.optional(e.str), + + cards: e.array(e.tuple({ front: e.str, back: e.str, order: e.int64 })), + + }, + + (params) => + + e.insert(e.Deck, { + + name: params.name, + + description: params.description, + + cards: e.for(e.array_unpack(params.cards), (card) => + + e.insert(e.Card, { + + front: card.front, + + back: card.back, + + order: card.order, + + }) + + ), + + }) + + ) + + .run(client, deck); + + revalidatePath("/"); + } + +Updating data +============= + +.. edb:split-section:: + + Next, you will update the Server Actions for each ``Deck`` object: ``updateDeck``, ``addCard``, and ``deleteCard``. Start with ``updateDeck``, which is the most complex because it is dynamic. You can set either the ``title`` or ``description`` fields in an update. Use the dynamic nature of the query builder to generate separate queries based on which fields are present in the form data. + + This may seem a bit intimidating at first, but the key to making this query dynamic is the ``nameSet`` and ``descriptionSet`` variables. These variables conditionally add the ``name`` or ``description`` fields to the ``set`` parameter of the ``update`` call. + + .. code-block:: typescript-diff + :caption: app/deck/[id]/actions.ts + + "use server"; + + import { revalidatePath } from "next/cache"; + import { readFile, writeFile } from "node:fs/promises"; + + import { client } from "@/lib/gel"; + + import e from "@/dbschema/edgeql-js"; + import { Deck } from "@/lib/models"; + + export async function updateDeck(formData: FormData) { + const id = formData.get("id"); + const name = formData.get("name"); + const description = formData.get("description"); + + if ( + typeof id !== "string" || + (typeof name !== "string" && + typeof description !== "string") + ) { + return; + } + + - const decks = JSON.parse( + - await readFile("./decks.json", "utf-8") + - ) as Deck[]; + - decks[index].name = name ?? decks[index].name; + + const nameSet = typeof name === "string" ? { name } : {}; + - decks[index].description = description ?? decks[index].description; + + const descriptionSet = + + typeof description === "string" ? { description: description || null } : {}; + + + await e + + .update(e.Deck, (d) => ({ + + filter_single: e.op(d.id, "=", e.uuid(id)), + + set: { + + ...nameSet, + + ...descriptionSet, + + }, + + })).run(client); + - await writeFile("./decks.json", JSON.stringify(decks, null, 2)); + revalidatePath(`/deck/${id}`); + } + + export async function addCard(formData: FormData) { + const deckId = formData.get("deckId"); + const front = formData.get("front"); + const back = formData.get("back"); + + if ( + typeof deckId !== "string" || + typeof front !== "string" || + typeof back !== "string" + ) { + return; + } + + const decks = JSON.parse(await readFile("./decks.json", "utf-8")) as Deck[]; + + const deck = decks.find((deck) => deck.id === deckId); + if (!deck) { + return; + } + + deck.cards.push({ front, back, id: crypto.randomUUID() }); + await writeFile("./decks.json", JSON.stringify(decks, null, 2)); + + revalidatePath(`/deck/${deckId}`); + } + + export async function deleteCard(formData: FormData) { + const cardId = formData.get("cardId"); + + if (typeof cardId !== "string") { + return; + } + + const decks = JSON.parse(await readFile("./decks.json", "utf-8")) as Deck[]; + const deck = decks.find((deck) => deck.cards.some((card) => card.id === cardId)); + if (!deck) { + return; + } + + deck.cards = deck.cards.filter((card) => card.id !== cardId); + await writeFile("./decks.json", JSON.stringify(decks, null, 2)); + + revalidatePath(`/`); + } + +Adding linked data +================== + +.. edb:split-section:: + + For the ``addCard`` action, you need to insert a new ``Card`` object and update the ``Deck.cards`` set to include the new ``Card`` object. Notice that the ``order`` property is set by selecting the maximum ``order`` property of this ``Deck.cards`` set and incrementing it by 1. + + The syntax for adding an object to a set of links is ``{ "+=": object }``. You can think of this as a shortcut for setting the link set to the current set plus the new object. + + .. code-block:: typescript-diff + :caption: app/deck/[id]/actions.ts + + "use server"; + + import { revalidatePath } from "next/cache"; + import { readFile, writeFile } from "node:fs/promises"; + import { client } from "@/lib/gel"; + import e from "@/dbschema/edgeql-js"; + import { Deck } from "@/lib/models"; + + export async function updateDeck(formData: FormData) { + const id = formData.get("id"); + const name = formData.get("name"); + const description = formData.get("description"); + + if ( + typeof id !== "string" || + (typeof name !== "string" && + typeof description !== "string") + ) { + return; + } + + const nameSet = typeof name === "string" ? { name } : {}; + const descriptionSet = + typeof description === "string" ? { description: description || null } : {}; + + await e + .update(e.Deck, (d) => ({ + filter_single: e.op(d.id, "=", e.uuid(id)), + set: { + ...nameSet, + ...descriptionSet, + }, + })).run(client); + revalidatePath(`/deck/${id}`); + } + + export async function addCard(formData: FormData) { + const deckId = formData.get("deckId"); + const front = formData.get("front"); + const back = formData.get("back"); + + if ( + typeof deckId !== "string" || + typeof front !== "string" || + typeof back !== "string" + ) { + return; + } + + - const decks = JSON.parse(await readFile("./decks.json", "utf-8")) as Deck[]; + - + - const deck = decks.find((deck) => deck.id === deckId); + - if (!deck) { + - return; + - } + - + - deck.cards.push({ front, back, id: crypto.randomUUID() }); + - await writeFile("./decks.json", JSON.stringify(decks, null, 2)); + + await e + + .params( + + { + + front: e.str, + + back: e.str, + + deckId: e.uuid, + + }, + + (params) => { + + const deck = e.assert_exists( + + e.select(e.Deck, (d) => ({ + + filter_single: e.op(d.id, "=", params.deckId), + + })) + + ); + + + + const order = e.cast(e.int64, e.max(deck.cards.order)); + + const card = e.insert(e.Card, { + + front: params.front, + + back: params.back, + + order: e.op(order, "+", 1), + + }); + + return e.update(deck, (d) => ({ + + set: { + + cards: { + + "+=": card + + }, + + }, + + })) + + } + + ) + + .run(client, { + + front, + + back, + + deckId, + + }); + + revalidatePath(`/deck/${deckId}`); + } + + export async function deleteCard(formData: FormData) { + const cardId = formData.get("cardId"); + + if (typeof cardId !== "string") { + return; + } + + const decks = JSON.parse(await readFile("./decks.json", "utf-8")) as Deck[]; + const deck = decks.find((deck) => deck.cards.some((card) => card.id === cardId)); + if (!deck) { + return; + } + + deck.cards = deck.cards.filter((card) => card.id !== cardId); + await writeFile("./decks.json", JSON.stringify(decks, null, 2)); + + revalidatePath(`/`); + } + +Deleting linked data +==================== + +.. edb:split-section:: + + For the ``deleteCard`` action, delete the ``Card`` object and based on the deletion policy we set up earlier in the schema, the object will be deleted from the database and removed from the ``Deck.cards`` set. + + .. code-block:: typescript-diff + :caption: app/deck/[id]/actions.ts + + "use server"; + + import { revalidatePath } from "next/cache"; + - import { readFile, writeFile } from "node:fs/promises"; + import { client } from "@/lib/gel"; + import e from "@/dbschema/edgeql-js"; + import { Deck } from "@/lib/models"; + + export async function updateDeck(formData: FormData) { + const id = formData.get("id"); + const name = formData.get("name"); + const description = formData.get("description"); + + if ( + typeof id !== "string" || + (typeof name !== "string" && + typeof description !== "string") + ) { + return; + } + + const nameSet = typeof name === "string" ? { name } : {}; + const descriptionSet = + typeof description === "string" ? { description: description || null } : {}; + + await e + .update(e.Deck, (d) => ({ + filter_single: e.op(d.id, "=", e.uuid(id)), + set: { + ...nameSet, + ...descriptionSet, + }, + })).run(client); + revalidatePath(`/deck/${id}`); + } + + export async function addCard(formData: FormData) { + const deckId = formData.get("deckId"); + const front = formData.get("front"); + const back = formData.get("back"); + + if ( + typeof deckId !== "string" || + typeof front !== "string" || + typeof back !== "string" + ) { + return; + } + + await e + .params( + { + front: e.str, + back: e.str, + deckId: e.uuid, + }, + (params) => { + const deck = e.assert_exists( + e.select(e.Deck, (d) => ({ + filter_single: e.op(d.id, "=", params.deckId), + })) + ); + + const order = e.cast(e.int64, e.max(deck.cards.order)); + const card = e.insert(e.Card, { + front: params.front, + back: params.back, + order: e.op(order, "+", 1), + }); + return e.update(deck, (d) => ({ + set: { + cards: { + "+=": card + }, + }, + })) + } + ) + .run(client, { + front, + back, + deckId, + }); + + revalidatePath(`/deck/${deckId}`); + } + + export async function deleteCard(formData: FormData) { + const cardId = formData.get("cardId"); + + if (typeof cardId !== "string") { + return; + } + + - const decks = JSON.parse(await readFile("./decks.json", "utf-8")) as Deck[]; + - const deck = decks.find((deck) => deck.cards.some((card) => card.id === cardId)); + - if (!deck) { + - return; + - } + - + - deck.cards = deck.cards.filter((card) => card.id !== cardId); + - await writeFile("./decks.json", JSON.stringify(decks, null, 2)); + + await e + + .params({ id: e.uuid }, (params) => + + e.delete(e.Card, (c) => ({ + + filter_single: e.op(c.id, "=", params.id), + + })) + + ) + + .run(client, { id: cardId }); + + + + revalidatePath(`/`); + } + +Querying data +============= + +.. edb:split-section:: + + Next, update the two ``queries.ts`` methods: ``getDecks`` and ``getDeck``. + + .. tabs:: + + .. code-tab:: typescript-diff + :caption: app/queries.ts + + - import { readFile } from "node:fs/promises"; + + import { client } from "@/lib/gel"; + + import e from "@/dbschema/edgeql-js"; + - + - import { Deck } from "@/lib/models"; + + export async function getDecks() { + - const decks = JSON.parse(await readFile("./decks.json", "utf-8")) as Deck[]; + + const decks = await e.select(e.Deck, (deck) => ({ + + id: true, + + name: true, + + description: true, + + cards: e.select(deck.cards, (card) => ({ + + id: true, + + front: true, + + back: true, + + order_by: card.order, + + })), + + })).run(client); + + return decks; + } + + .. code-tab:: typescript-diff + :caption: app/deck/[id]/queries.ts + + - import { readFile } from "node:fs/promises"; + - import { Deck } from "@/lib/models"; + + import { client } from "@/lib/gel"; + + import e from "@/dbschema/edgeql-js"; + + export async function getDeck({ id }: { id: string }) { + - const decks = JSON.parse(await readFile("./decks.json", "utf-8")) as Deck[]; + - return decks.find((deck) => deck.id === id) ?? null; + + return await e + + .select(e.Deck, (deck) => ({ + + filter_single: e.op(deck.id, "=", e.uuid(id)), + + id: true, + + name: true, + + description: true, + + cards: e.select(deck.cards, (card) => ({ + + id: true, + + front: true, + + back: true, + + order_by: card.order, + + })), + + })) + + .run(client); + } + +.. edb:split-section:: + + In a terminal, run the Next.js development server. + + .. code-block:: sh + + $ npm run dev + +.. edb:split-section:: + + A static JSON file to seed your database with a deck of trivia cards is included in the project. Open your browser and navigate to the app at ``_. Use the "Import JSON" button to import this JSON file into your database. + + .. image:: images/flashcards-import.png diff --git a/docs/reference/connection.rst b/docs/reference/connection.rst index 015aad50b33..23bd1db1246 100644 --- a/docs/reference/connection.rst +++ b/docs/reference/connection.rst @@ -342,10 +342,9 @@ instance-level configuration object. Override behavior ^^^^^^^^^^^^^^^^^ -When specified, the connection parameters (user, password, and -:versionreplace:`database;5.0:branch`) will *override* the corresponding -element of a DSN, credentials file, etc. For instance, consider the following -environment variables: +When specified, the connection parameters (user, password, and |branch|) +will *override* the corresponding element of a DSN, credentials file, etc. +For instance, consider the following environment variables: .. code-block:: diff --git a/docs/reference/ddl/access_policies.rst b/docs/reference/ddl/access_policies.rst deleted file mode 100644 index 5f4f7aab3c7..00000000000 --- a/docs/reference/ddl/access_policies.rst +++ /dev/null @@ -1,225 +0,0 @@ -.. _ref_eql_ddl_access_policies: - -=============== -Access Policies -=============== - -This section describes the DDL commands pertaining to access policies. - -Create access policy -==================== - -:eql-statement: - -:ref:`Declare ` a new object access policy. - -.. eql:synopsis:: - - [ with [, ...] ] - { create | alter } type "{" - [ ... ] - create access policy - [ when () ; ] - { allow | deny } action [, action ... ; ] - [ using () ; ] - [ "{" - [ set errmessage := value ; ] - [ create annotation annotation-name := value ; ] - "}" ] - "}" - - # where is one of - all - select - insert - delete - update [{ read | write }] - - -Description ------------ - -The combination :eql:synopsis:`{create | alter} type ... create access policy` -defines a new access policy for a given object type. - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL access policy declaration `. - -:eql:synopsis:`` - The name of the access policy. - -:eql:synopsis:`when ()` - Specifies which objects this policy applies to. The - :eql:synopsis:`` has to be a :eql:type:`bool` expression. - - When omitted, it is assumed that this policy applies to all objects of a - given type. - -:eql:synopsis:`allow` - Indicates that qualifying objects should allow access under this policy. - -:eql:synopsis:`deny` - Indicates that qualifying objects should *not* allow access under this - policy. This flavor supersedes any :eql:synopsis:`allow` policy and can - be used to selectively deny access to a subset of objects that otherwise - explicitly allows accessing them. - -:eql:synopsis:`all` - Apply the policy to all actions. It is exactly equivalent to listing - :eql:synopsis:`select`, :eql:synopsis:`insert`, :eql:synopsis:`delete`, - :eql:synopsis:`update` actions explicitly. - -:eql:synopsis:`select` - Apply the policy to all selection queries. Note that any object that - cannot be selected, cannot be modified either. This makes - :eql:synopsis:`select` the most basic "visibility" policy. - -:eql:synopsis:`insert` - Apply the policy to all inserted objects. If a newly inserted object would - violate this policy, an error is produced instead. - -:eql:synopsis:`delete` - Apply the policy to all objects about to be deleted. If an object does not - allow access under this kind of policy, it is not going to be considered - by any :eql:stmt:`delete` command. - - Note that any object that cannot be selected, cannot be modified either. - -:eql:synopsis:`update read` - Apply the policy to all objects selected for an update. If an object does - not allow access under this kind of policy, it is not visible cannot be - updated. - - Note that any object that cannot be selected, cannot be modified either. - -:eql:synopsis:`update write` - Apply the policy to all objects at the end of an update. If an updated - object violates this policy, an error is produced instead. - - Note that any object that cannot be selected, cannot be modified either. - -:eql:synopsis:`update` - This is just a shorthand for :eql:synopsis:`update read` and - :eql:synopsis:`update write`. - - Note that any object that cannot be selected, cannot be modified either. - -:eql:synopsis:`using ` - Specifies what the policy is with respect to a given eligible (based on - :eql:synopsis:`when` clause) object. The :eql:synopsis:`` has to be - a :eql:type:`bool` expression. The specific meaning of this value also - depends on whether this policy flavor is :eql:synopsis:`allow` or - :eql:synopsis:`deny`. - - When omitted, it is assumed that this policy applies to all eligible - objects of a given type. - -The following subcommands are allowed in the ``create access policy`` block: - -:eql:synopsis:`set errmessage := ` - Set a custom error message of :eql:synopsis:`` that is displayed - when this access policy prevents a write action. - -:eql:synopsis:`create annotation := ` - Set access policy annotation :eql:synopsis:`` to - :eql:synopsis:``. - - See :eql:stmt:`create annotation` for details. - -Alter access policy -==================== - -:eql-statement: - -:ref:`Declare ` a new object access policy. - -.. eql:synopsis:: - - [ with [, ...] ] - alter type "{" - [ ... ] - alter access policy "{" - [ when () ; ] - [ reset when ; ] - { allow | deny } [, ... ; ] - [ using () ; ] - [ set errmessage := value ; ] - [ reset expression ; ] - [ create annotation := ; ] - [ alter annotation := ; ] - [ drop annotation ; ] - "}" - "}" - - # where is one of - all - select - insert - delete - update [{ read | write }] - -Description ------------ - -The combination :eql:synopsis:`{create | alter} type ... create access policy` -defines a new access policy for a given object type. - -Parameters ----------- - -The parameters describing the action policy are identical to the parameters -used by ``create action policy``. There are a handful of additional -subcommands that are allowed in the ``create access policy`` block: - -:eql:synopsis:`reset when` - Clear the :eql:synopsis:`when ()` so that the policy applies to - all objects of a given type. This is equivalent to ``when (true)``. - -:eql:synopsis:`reset expression` - Clear the :eql:synopsis:`using ()` so that the policy always - passes. This is equivalent to ``using (true)``. - -:eql:synopsis:`alter annotation ;` - Alter access policy annotation :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ;` - Remove access policy annotation :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - - -All the subcommands allowed in the ``create access policy`` block are also -valid subcommands for ``alter access policy`` block. - - -Drop access policy -================== - -:eql-statement: - -Remove an access policy from an object type. - -.. eql:synopsis:: - - [ with [, ...] ] - alter type "{" - [ ... ] - drop access policy ; - "}" - -Description ------------ - -The combination :eql:synopsis:`alter type ... drop access policy` -removes the specified access policy from a given object type. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Access policies ` - * - :ref:`SDL > Access policies ` diff --git a/docs/reference/ddl/aliases.rst b/docs/reference/ddl/aliases.rst deleted file mode 100644 index 60404397836..00000000000 --- a/docs/reference/ddl/aliases.rst +++ /dev/null @@ -1,123 +0,0 @@ -.. _ref_eql_ddl_aliases: - -======= -Aliases -======= - -This section describes the DDL commands pertaining to -:ref:`expression aliases `. - - -Create alias -============ - -:eql-statement: -:eql-haswith: - -:ref:`Define ` a new expression alias in the schema. - -.. eql:synopsis:: - - [ with [, ...] ] - create alias := ; - - [ with [, ...] ] - create alias "{" - using ; - [ create annotation := ; ... ] - "}" ; - - # where is: - - [ := ] module - - -Description ------------ - -The command ``create alias`` defines a new expression alias in the schema. -The schema-level expression aliases are functionally equivalent -to expression aliases defined in a statement :ref:`with block -`, but are available to all queries using the schema -and can be introspected. - -If *name* is qualified with a module name, then the alias is created -in that module, otherwise it is created in the current module. -The alias name must be distinct from that of any existing schema item -in the module. - - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL alias declaration `, with some -additional features listed below: - -:eql:synopsis:`[ := ] module ` - An optional list of module alias declarations to be used in the - alias definition. - -:eql:synopsis:`create annotation := ;` - An optional list of annotation values for the alias. - See :eql:stmt:`create annotation` for details. - - -Example -------- - -Create a new alias: - -.. code-block:: edgeql - - create alias Superusers := ( - select User filter User.groups.name = 'Superusers' - ); - - -Drop alias -========== - -:eql-statement: -:eql-haswith: - - -Remove an expression alias from the schema. - -.. eql:synopsis:: - - [ with [, ...] ] - drop alias ; - - -Description ------------ - -The command ``drop alias`` removes an expression alias from the schema. - - -Parameters ----------- - -*alias-name* - The name (optionally qualified with a module name) of an existing - expression alias. - - -Example -------- - -Remove an alias: - -.. code-block:: edgeql - - drop alias SuperUsers; - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Aliases ` - * - :ref:`SDL > Aliases ` - * - :ref:`Cheatsheets > Aliases ` diff --git a/docs/reference/ddl/annotations.rst b/docs/reference/ddl/annotations.rst deleted file mode 100644 index 0a7cafe9a50..00000000000 --- a/docs/reference/ddl/annotations.rst +++ /dev/null @@ -1,279 +0,0 @@ -.. _ref_eql_ddl_annotations: - -=========== -Annotations -=========== - -This section describes the DDL commands pertaining to -:ref:`annotations `. - - -Create abstract annotation -========================== - -:eql-statement: - -:ref:`Define ` a new annotation. - -.. eql:synopsis:: - - [ with [, ...] ] - create abstract [ inheritable ] annotation - [ "{" - create annotation := ; - [...] - "}" ] ; - - -Description ------------ - -The command ``create abstract annotation`` defines a new annotation -for use in the current :versionreplace:`database;5.0:branch`. - -If *name* is qualified with a module name, then the annotation is created -in that module, otherwise it is created in the current module. -The annotation name must be distinct from that of any existing schema item -in the module. - -The annotations are non-inheritable by default. That is, if a schema item -has an annotation defined on it, the descendants of that schema item will -not automatically inherit the annotation. Normal inheritance behavior can -be turned on by declaring the annotation with the ``inheritable`` qualifier. - -Most sub-commands and options of this command are identical to the -:ref:`SDL annotation declaration `. -There's only one subcommand that is allowed in the ``create -annotation`` block: - -:eql:synopsis:`create annotation := ` - Annotations can also have annotations. Set the - :eql:synopsis:`` of the - enclosing annotation to a specific :eql:synopsis:``. - See :eql:stmt:`create annotation` for details. - - -Example -------- - -Declare an annotation ``extrainfo``. - -.. code-block:: edgeql - - create abstract annotation extrainfo; - - -Alter abstract annotation -========================= - -:eql-statement: - - -Change the definition of an :ref:`annotation `. - -.. eql:synopsis:: - - alter abstract annotation - [ "{" ] ; [...] [ "}" ]; - - # where is one of - - rename to - create annotation := - alter annotation := - drop annotation - - -Description ------------ - -:eql:synopsis:`alter abstract annotation` changes the definition of an abstract -annotation. - - -Parameters ----------- - -:eql:synopsis:`` - The name (optionally module-qualified) of the annotation to alter. - -The following subcommands are allowed in the ``alter abstract annotation`` -block: - -:eql:synopsis:`rename to ` - Change the name of the annotation to :eql:synopsis:``. - -:eql:synopsis:`alter annotation ;` - Annotations can also have annotations. Change - :eql:synopsis:`` to a specific - :eql:synopsis:``. See :eql:stmt:`alter annotation` for - details. - -:eql:synopsis:`drop annotation ;` - Annotations can also have annotations. Remove annotation - :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - -All the subcommands allowed in the ``create abstract annotation`` -block are also valid subcommands for ``alter annotation`` block. - - -Examples --------- - -Rename an annotation: - -.. code-block:: edgeql - - alter abstract annotation extrainfo - rename to extra_info; - - -Drop abstract annotation -======================== - -:eql-statement: - -Remove a :ref:`schema annotation `. - -.. eql:synopsis:: - - [ with [, ...] ] - drop abstract annotation ; - -Description ------------ - -The command ``drop abstract annotation`` removes an existing schema -annotation from the database schema. Note that the ``inheritable`` -qualifier is not necessary in this statement. - -Example -------- - -Drop the annotation ``extra_info``: - -.. code-block:: edgeql - - drop abstract annotation extra_info; - - -Create annotation -================= - -:eql-statement: - -Define an annotation value for a given schema item. - -.. eql:synopsis:: - - create annotation := - -Description ------------ - -The command ``create annotation`` defines an annotation for a schema item. - -:eql:synopsis:`` refers to the name of a defined annotation, -and :eql:synopsis:`` must be a constant EdgeQL expression -evaluating into a string. - -This statement can only be used as a subcommand in another -DDL statement. - - -Example -------- - -Create an object type ``User`` and set its ``title`` annotation to -``"User type"``. - -.. code-block:: edgeql - - create type User { - create annotation title := "User type"; - }; - - -Alter annotation -================ - -:eql-statement: - -Alter an annotation value for a given schema item. - -.. eql:synopsis:: - - alter annotation := - -Description ------------ - -The command ``alter annotation`` alters an annotation value on a schema item. - -:eql:synopsis:`` refers to the name of a defined annotation, -and :eql:synopsis:`` must be a constant EdgeQL expression -evaluating into a string. - -This statement can only be used as a subcommand in another -DDL statement. - - -Example -------- - -Alter an object type ``User`` and alter the value of its previously set -``title`` annotation to ``"User type"``. - -.. code-block:: edgeql - - alter type User { - alter annotation title := "User type"; - }; - - -Drop annotation -=============== - -:eql-statement: - - -Remove an annotation from a given schema item. - -.. eql:synopsis:: - - drop annotation ; - -Description ------------ - -The command ``drop annotation`` removes an annotation value from a schema item. - -:eql:synopsis:`` refers to the name of a defined annotation. -The annotation value does not have to exist on a schema item. - -This statement can only be used as a subcommand in another -DDL statement. - - -Example -------- - -Drop the ``title`` annotation from the ``User`` object type: - -.. code-block:: edgeql - - alter type User { - drop annotation title; - }; - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Annotations ` - * - :ref:`SDL > Annotations ` - * - :ref:`Cheatsheets > Annotations ` - * - :ref:`Introspection > Object types - ` diff --git a/docs/reference/ddl/constraints.rst b/docs/reference/ddl/constraints.rst deleted file mode 100644 index 30fe09fd252..00000000000 --- a/docs/reference/ddl/constraints.rst +++ /dev/null @@ -1,480 +0,0 @@ -.. _ref_eql_ddl_constraints: - -=========== -Constraints -=========== - -This section describes the DDL commands pertaining to -:ref:`constraints `. - - -Create abstract constraint -========================== - -:eql-statement: -:eql-haswith: - -:ref:`Define ` a new abstract constraint. - -.. eql:synopsis:: - - [ with [ := ] module ] - create abstract constraint [ ( [] [, ...] ) ] - [ on ( ) ] - [ extending [, ...] ] - "{" ; [...] "}" ; - - # where is: - - [ : ] - - # where is one of - - using - set errmessage := - create annotation := - - -Description ------------ - -The command ``create abstract constraint`` defines a new abstract constraint. - -If *name* is qualified with a module name, then the constraint is -created in that module, otherwise it is created in the current module. -The constraint name must be distinct from that of any existing schema item -in the module. - - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL constraint declaration `, -with some additional features listed below: - -:eql:synopsis:`[ := ] module ` - An optional list of module alias declarations to be used in the - migration definition. When *module-alias* is not specified, - *module-name* becomes the effective current module and is used - to resolve all unqualified names. - -:eql:synopsis:`set errmessage := ` - An optional string literal defining the error message template - that is raised when the constraint is violated. Other than a - slight syntactical difference this is the same as the - corresponding SDL declaration. - -:eql:synopsis:`create annotation := ;` - Set constraint :eql:synopsis:`` to - :eql:synopsis:``. - - See :eql:stmt:`create annotation` for details. - - -Example -------- - -Create an abstract constraint "uppercase" which checks if the subject -is a string in upper case. - -.. code-block:: edgeql - - create abstract constraint uppercase { - create annotation title := "Upper case constraint"; - using (str_upper(__subject__) = __subject__); - set errmessage := "{__subject__} is not in upper case"; - }; - - -Alter abstract constraint -========================= - -:eql-statement: -:eql-haswith: - -Alter the definition of an -:ref:`abstract constraint `. - -.. eql:synopsis:: - - [ with [ := ] module ] - alter abstract constraint - "{" ; [...] "}" ; - - # where is one of - - rename to - using - set errmessage := - reset errmessage - create annotation := - alter annotation := - drop annotation - - -Description ------------ - -The command ``alter abstract constraint`` changes the definition of an -abstract constraint item. *name* must be a name of an existing -abstract constraint, optionally qualified with a module name. - - -Parameters ----------- - -:eql:synopsis:`[ := ] module ` - An optional list of module alias declarations to be used in the - migration definition. When *module-alias* is not specified, - *module-name* becomes the effective current module and is used - to resolve all unqualified names. - -:eql:synopsis:`` - The name (optionally module-qualified) of the constraint to alter. - -The following subcommands are allowed in the ``alter abstract -constraint`` block: - -:eql:synopsis:`rename to ` - Change the name of the constraint to *newname*. All concrete - constraints inheriting from this constraint are also renamed. - -:eql:synopsis:`alter annotation ;` - Alter constraint :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ;` - Remove constraint :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - -:eql:synopsis:`reset errmessage;` - Remove the error message from this abstract constraint. - The error message specified in the base abstract constraint - will be used instead. - -All the subcommands allowed in a ``create abstract constraint`` block -are also valid subcommands for an ``alter abstract constraint`` block. - - -Example -------- - -Rename the abstract constraint "uppercase" to "upper_case": - -.. code-block:: edgeql - - alter abstract constraint uppercase rename to upper_case; - - -Drop abstract constraint -======================== - -:eql-statement: -:eql-haswith: - - -Remove an :ref:`abstract constraint ` -from the schema. - -.. eql:synopsis:: - - [ with [ := ] module ] - drop abstract constraint ; - - -Description ------------ - -The command ``drop abstract constraint`` removes an existing abstract -constraint item from the database schema. If any schema items -depending on this constraint exist, the operation is refused. - - -Parameters ----------- - -:eql:synopsis:`[ := ] module ` - An optional list of module alias declarations to be used in the - migration definition. When *module-alias* is not specified, - *module-name* becomes the effective current module and is used - to resolve all unqualified names. - -:eql:synopsis:`` - The name (optionally module-qualified) of the constraint to remove. - - -Example -------- - -Drop abstract constraint ``upper_case``: - -.. code-block:: edgeql - - drop abstract constraint upper_case; - - -Create constraint -================= - -:eql-statement: - -Define a concrete constraint on the specified schema item. - -.. eql:synopsis:: - - [ with [ := ] module ] - create [ delegated ] constraint - [ ( [] [, ...] ) ] - [ on ( ) ] - [ except ( ) ] - "{" ; [...] "}" ; - - # where is: - - [ : ] - - # where is one of - - set errmessage := - create annotation := - - -Description ------------ - -The command ``create constraint`` defines a new concrete constraint. -It can only be used in the context of :eql:stmt:`create scalar type`, -:eql:stmt:`alter scalar type`, :eql:stmt:`create property`, -:eql:stmt:`alter property`, :eql:stmt:`create link`, or -:eql:Stmt:`alter link`. - -*name* must be a name (optionally module-qualified) of previously defined -abstract constraint. - - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL constraint declaration `, -with some additional features listed below: - -:eql:synopsis:`[ := ] module ` - An optional list of module alias declarations to be used in the - migration definition. When *module-alias* is not specified, - *module-name* becomes the effective current module and is used - to resolve all unqualified names. - -:eql:synopsis:`set errmessage := ` - An optional string literal defining the error message template - that is raised when the constraint is violated. Other than a - slight syntactical difference this is the same as the - corresponding SDL declaration. - -:eql:synopsis:`create annotation := ;` - An optional list of annotations for the constraint. - See :eql:stmt:`create annotation` for details. - - -Example -------- - -Create a "score" property on the "User" type with a minimum value -constraint: - -.. code-block:: edgeql - - alter type User create property score -> int64 { - create constraint min_value(0) - }; - -Create a Vector with a maximum magnitude: - -.. code-block:: edgeql - - create type Vector { - create required property x -> float64; - create required property y -> float64; - create constraint expression ON ( - __subject__.x^2 + __subject__.y^2 < 25 - ); - } - - -Alter constraint -================ - -:eql-statement: - -Alter the definition of a concrete constraint on the specified schema item. - -.. eql:synopsis:: - - [ with [ := ] module [, ...] ] - alter constraint - [ ( [] [, ...] ) ] - [ on ( ) ] - [ except ( ) ] - "{" ; [ ... ] "}" ; - - # -- or -- - - [ with [ := ] module [, ...] ] - alter constraint - [ ( [] [, ...] ) ] - [ on ( ) ] - ; - - # where is one of: - - set delegated - set not delegated - set errmessage := - reset errmessage - create annotation := - alter annotation - drop annotation - - -Description ------------ - -The command ``alter constraint`` changes the definition of a concrete -constraint. As for most ``alter`` commands, both single- and -multi-command forms are supported. - - -Parameters ----------- - -:eql:synopsis:`[ := ] module ` - An optional list of module alias declarations to be used in the - migration definition. When *module-alias* is not specified, - *module-name* becomes the effective current module and is used - to resolve all unqualified names. - -:eql:synopsis:`` - The name (optionally module-qualified) of the concrete constraint - that is being altered. - -:eql:synopsis:`` - A list of constraint arguments as specified at the time of - ``create constraint``. - -:eql:synopsis:`on ( )` - A expression defining the *subject* of the constraint as specified - at the time of ``create constraint``. - - -The following subcommands are allowed in the ``alter constraint`` block: - -:eql:synopsis:`set delegated` - If set, the constraint is defined as *delegated*, which means that it will - not be enforced on the type it's declared on, and the enforcement will be - delegated to the subtypes of this type. This is particularly useful for - :eql:constraint:`exclusive` constraints in abstract types. This is only - valid for *concrete constraints*. - -:eql:synopsis:`set not delegated` - If set, the constraint is defined as *not delegated*, which means that it - will be enforced globally across the type it's declared on and any - extending types. - -:eql:synopsis:`rename to ` - Change the name of the constraint to :eql:synopsis:``. - -:eql:synopsis:`alter annotation ;` - Alter constraint :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ;` - Remove an *annotation*. See :eql:stmt:`drop annotation` for details. - -:eql:synopsis:`reset errmessage;` - Remove the error message from this constraint. The error message - specified in the abstract constraint will be used instead. - -All the subcommands allowed in the ``create constraint`` block are also -valid subcommands for ``alter constraint`` block. - -Example -------- - -Change the error message on the minimum value constraint on the property -"score" of the "User" type: - -.. code-block:: edgeql - - alter type User alter property score - alter constraint min_value(0) - set errmessage := 'Score cannot be negative'; - - -Drop constraint -=============== - -:eql-statement: -:eql-haswith: - -Remove a concrete constraint from the specified schema item. - -.. eql:synopsis:: - - [ with [ := ] module [, ...] ] - drop constraint - [ ( [] [, ...] ) ] - [ on ( ) ] - [ except ( ) ] ; - - -Description ------------ - -The command ``drop constraint`` removes the specified constraint from -its containing schema item. - - -Parameters ----------- - -:eql:synopsis:`[ := ] module ` - An optional list of module alias declarations to be used in the - migration definition. When *module-alias* is not specified, - *module-name* becomes the effective current module and is used - to resolve all unqualified names. - -:eql:synopsis:`` - The name (optionally module-qualified) of the concrete constraint - to remove. - -:eql:synopsis:`` - A list of constraint arguments as specified at the time of - ``create constraint``. - -:eql:synopsis:`on ( )` - A expression defining the *subject* of the constraint as specified - at the time of ``create constraint``. - - -Example -------- - -Remove constraint "min_value" from the property "score" of the -"User" type: - -.. code-block:: edgeql - - alter type User alter property score - drop constraint min_value(0); - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Constraints ` - * - :ref:`SDL > Constraints ` - * - :ref:`Introspection > Constraints - ` - * - :ref:`Standard Library > Constraints ` diff --git a/docs/reference/ddl/extensions.rst b/docs/reference/ddl/extensions.rst deleted file mode 100644 index 55c0213fa26..00000000000 --- a/docs/reference/ddl/extensions.rst +++ /dev/null @@ -1,86 +0,0 @@ -.. _ref_eql_ddl_extensions: - -========== -Extensions -========== - -This section describes the DDL commands pertaining to -:ref:`extensions `. - - -Create extension -================ - -:eql-statement: - -Enable a particular extension for the current schema. - -.. eql:synopsis:: - - create extension ";" - -There's a :ref:`corresponding SDL declaration ` -for enabling an extension, which is the recommended way of doing this. - -Description ------------ - -The command ``create extension`` enables the specified extension for -the current :versionreplace:`database;5.0:branch`. - -Examples --------- - -Enable :ref:`GraphQL ` extension for the current -schema: - -.. code-block:: edgeql - - create extension graphql; - -Enable :ref:`EdgeQL over HTTP ` extension for the -current :versionreplace:`database;5.0:branch`: - -.. code-block:: edgeql - - create extension edgeql_http; - - -drop extension -============== - -:eql-statement: - - -Disable an extension. - -.. eql:synopsis:: - - drop extension ";" - - -Description ------------ - -The command ``drop extension`` disables a currently active extension for the -current :versionreplace:`database;5.0:branch`. - - -Examples --------- - -Disable :ref:`GraphQL ` extension for the current -schema: - -.. code-block:: edgeql - - drop extension graphql; - -Disable :ref:`EdgeQL over HTTP ` extension for the -current :versionreplace:`database;5.0:branch`: - -.. code-block:: edgeql - - drop extension edgeql_http; - - diff --git a/docs/reference/ddl/functions.rst b/docs/reference/ddl/functions.rst deleted file mode 100644 index 57a1eabffe0..00000000000 --- a/docs/reference/ddl/functions.rst +++ /dev/null @@ -1,275 +0,0 @@ -.. _ref_eql_ddl_functions: - -========= -Functions -========= - -This section describes the DDL commands pertaining to -:ref:`functions `. - - -Create function -=============== - -:eql-statement: -:eql-haswith: - - -:ref:`Define ` a new function. - -.. eql:synopsis:: - - [ with [, ...] ] - create function ([ ] [, ... ]) -> - using ( ); - - [ with [, ...] ] - create function ([ ] [, ... ]) -> - using ; - - [ with [, ...] ] - create function ([ ] [, ... ]) -> - "{" [, ...] "}" ; - - # where is: - - [ ] : [ ] [ = ] - - # is: - - [ { variadic | named only } ] - - # is: - - [ { set of | optional } ] - - # and is: - - [ ] - - # and is one of - - set volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'} ; - create annotation := ; - using ( ) ; - using ; - - -Description ------------ - -The command ``create function`` defines a new function. If *name* is -qualified with a module name, then the function is created in that -module, otherwise it is created in the current module. - -The function name must be distinct from that of any existing function -with the same argument types in the same module. Functions of -different argument types can share a name, in which case the functions -are called *overloaded functions*. - - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL function declaration `, with -some additional features listed below: - -:eql:synopsis:`set volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'}` - Function volatility determines how aggressively the compiler can - optimize its invocations. Other than a slight syntactical - difference this is the same as the corresponding SDL declaration. - -:eql:synopsis:`create annotation := ` - Set the function's :eql:synopsis:`` to - :eql:synopsis:``. - - See :eql:stmt:`create annotation` for details. - - -Examples --------- - -Define a function returning the sum of its arguments: - -.. code-block:: edgeql - - create function mysum(a: int64, b: int64) -> int64 - using ( - select a + b - ); - -The same, but using a variadic argument and an explicit language: - -.. code-block:: edgeql - - create function mysum(variadic argv: int64) -> int64 - using edgeql $$ - select sum(array_unpack(argv)) - $$; - -Define a function using the block syntax: - -.. code-block:: edgeql - - create function mysum(a: int64, b: int64) -> int64 { - using ( - select a + b - ); - create annotation title := "My sum function."; - }; - - -Alter function -============== - -:eql-statement: -:eql-haswith: - -Change the definition of a function. - -.. eql:synopsis:: - - [ with [, ...] ] - alter function ([ ] [, ... ]) "{" - [, ...] - "}" - - # where is: - - [ ] : [ ] [ = ] - - # and is one of - - set volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'} ; - reset volatility ; - rename to ; - create annotation := ; - alter annotation := ; - drop annotation ; - using ( ) ; - using ; - - -Description ------------ - -The command ``alter function`` changes the definition of a function. -The command allows to change annotations, the volatility level, and -other attributes. - - -Subcommands ------------ - -The following subcommands are allowed in the ``alter function`` block -in addition to the commands common to the ``create function``: - -:eql:synopsis:`reset volatility` - Remove explicitly specified volatility in favor of the volatility - inferred from the function body. - -:eql:synopsis:`rename to ` - Change the name of the function to *newname*. - -:eql:synopsis:`alter annotation ;` - Alter function :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ;` - Remove function :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - -:eql:synopsis:`reset errmessage;` - Remove the error message from this abstract constraint. - The error message specified in the base abstract constraint - will be used instead. - - -Example -------- - -.. code-block:: edgeql - - create function mysum(a: int64, b: int64) -> int64 { - using ( - select a + b - ); - create annotation title := "My sum function."; - }; - - alter function mysum(a: int64, b: int64) { - set volatility := 'Immutable'; - DROP ANNOTATION title; - }; - - alter function mysum(a: int64, b: int64) { - using ( - select (a + b) * 100 - ) - }; - - -Drop function -============= - -:eql-statement: -:eql-haswith: - - -Remove a function. - -.. eql:synopsis:: - - [ with [, ...] ] - drop function ([ ] [, ... ]); - - # where is: - - [ ] : [ ] [ = ] - - -Description ------------ - -The command ``drop function`` removes the definition of an existing function. -The argument types to the function must be specified, since there -can be different functions with the same name. - - -Parameters ----------- - -:eql:synopsis:`` - The name (optionally module-qualified) of an existing function. - -:eql:synopsis:`` - The name of an argument used in the function definition. - -:eql:synopsis:`` - The mode of an argument: ``set of`` or ``optional`` or ``variadic``. - -:eql:synopsis:`` - The data type(s) of the function's arguments - (optionally module-qualified), if any. - - -Example -------- - -Remove the ``mysum`` function: - -.. code-block:: edgeql - - drop function mysum(a: int64, b: int64); - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Functions ` - * - :ref:`SDL > Functions ` - * - :ref:`Reference > Function calls ` - * - :ref:`Introspection > Functions ` - * - :ref:`Cheatsheets > Functions ` diff --git a/docs/reference/ddl/future.rst b/docs/reference/ddl/future.rst deleted file mode 100644 index 010dd057572..00000000000 --- a/docs/reference/ddl/future.rst +++ /dev/null @@ -1,81 +0,0 @@ -.. _ref_eql_ddl_future: - -=============== -Future Behavior -=============== - -This section describes the DDL commands pertaining to -:ref:`future `. - - -Create future -============= - -:eql-statement: - -Enable a particular future behavior for the current schema. - -.. eql:synopsis:: - - create future ";" - -There's a :ref:`corresponding SDL declaration ` -for enabling a future behavior, which is the recommended way of doing this. - -Description ------------ - -The command ``create future`` enables the specified future behavior for -the current :versionreplace:`database;5.0:branch`. - -Examples --------- - -Enable simpler non-recursive access policy behavior :ref:`non-recursive access -policy ` for the current schema: - -.. code-block:: edgeql - - create future nonrecursive_access_policies; - - -drop future -=========== - -:eql-statement: - - -Stop importing future behavior prior to the Gel version in which it appears. - -.. eql:synopsis:: - - drop future ";" - - -Description ------------ - -The command ``drop future`` disables a currently active future behavior for the -current :versionreplace:`database;5.0:branch`. However, this is only possible -for versions of Gel when the behavior in question is not officially -introduced. Once a particular behavior is introduced as the standard behavior -in an Gel release, it cannot be disabled. Running this command will simply -denote that no special action is needed to enable it in this case. - - -Examples --------- - -Disable simpler non-recursive access policy behavior :ref:`non-recursive -access policy ` for the current -schema. This will make access policy restrictions apply to the expressions -defining other access policies: - -.. code-block:: edgeql - - drop future nonrecursive_access_policies; - - -Since |EdgeDB| 3.0 was released there is no more need for enabling non-recursive -access policy behavior anymore. So the above command will simply indicate that -the database no longer does anything non-standard. diff --git a/docs/reference/ddl/globals.rst b/docs/reference/ddl/globals.rst deleted file mode 100644 index 1a3d1bff4a4..00000000000 --- a/docs/reference/ddl/globals.rst +++ /dev/null @@ -1,234 +0,0 @@ -.. _ref_eql_ddl_globals: - -======= -Globals -======= - -This section describes the DDL commands pertaining to global variables. - - -Create global -============= - -:eql-statement: -:eql-haswith: - -:ref:`Declare ` a new global variable. - -.. eql:synopsis:: - - [ with [, ...] ] - create [{required | optional}] [single] - global -> - [ "{" ; [...] "}" ] ; - - # Computed global variable form: - - [ with [, ...] ] - create [{required | optional}] [{single | multi}] - global := ; - - # where is one of - - set default := - create annotation := - -Description ------------ - -There two different forms of ``global`` declaration, as shown in the syntax -synopsis above. The first form is for defining a ``global`` variable that can -be :ref:`set ` in a session. The second -form is not directly set, but instead it is *computed* based on an expression, -potentially deriving its value from other global variables. - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL global variable declaration `. The -following subcommands are allowed in the ``create global`` block: - -:eql:synopsis:`set default := ` - Specifies the default value for the global variable as an EdgeQL - expression. The default value is used by the session if the value was not - explicitly specified or by the :ref:`reset - ` command. - -:eql:synopsis:`create annotation := ` - Set global variable :eql:synopsis:`` to - :eql:synopsis:``. - - See :eql:stmt:`create annotation` for details. - -Examples --------- - -Define a new global property ``current_user_id``: - -.. code-block:: edgeql - - create global current_user_id -> uuid; - -Define a new *computed* global property ``current_user`` based on the -previously defined ``current_user_id``: - -.. code-block:: edgeql - - create global current_user := ( - select User filter .id = global current_user_id - ); - - -Alter global -============ - -:eql-statement: -:eql-haswith: - -Change the definition of a global variable. - -.. eql:synopsis:: - - [ with [, ...] ] - alter global - [ "{" ; [...] "}" ] ; - - # where is one of - - set default := - reset default - rename to - set required - set optional - reset optionalily - set single - set multi - reset cardinality - set type reset to default - using () - create annotation := - alter annotation := - drop annotation - -Description ------------ - -The command :eql:synopsis:`alter global` changes the definition of a global -variable. - -Parameters ----------- - -:eql:synopsis:`` - The name of the global variable to modify. - -The following subcommands are allowed in the ``alter global`` block: - -:eql:synopsis:`reset default` - Remove the default value from this global variable. - -:eql:synopsis:`rename to ` - Change the name of the global variable to :eql:synopsis:``. - -:eql:synopsis:`set required` - Make the global variable *required*. - -:eql:synopsis:`set optional` - Make the global variable no longer *required* (i.e. make it *optional*). - -:eql:synopsis:`reset optionalily` - Reset the optionality of the global variable to the default value - (``optional``). - -:eql:synopsis:`set single` - Change the maximum cardinality of the global variable to *one*. - -:eql:synopsis:`set multi` - Change the maximum cardinality of the global variable set to - *greater than one*. Only valid for computed global variables. - -:eql:synopsis:`reset cardinality` - Reset the maximum cardinality of the global variable to the default value - (``single``), or, if the property is computed, to the value inferred - from its expression. - -:eql:synopsis:`set type reset to default` - Change the type of the global variable to the specified - :eql:synopsis:``. The ``reset to default`` clause is mandatory - and it specifies that the variable will be reset to its default value - after this command. - -:eql:synopsis:`using ()` - Change the expression of a computed global variable. Only valid for - computed variables. - -:eql:synopsis:`alter annotation ;` - Alter global variable annotation :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ;` - Remove global variable :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - -All the subcommands allowed in the ``create global`` block are also -valid subcommands for ``alter global`` block. - -Examples --------- - -Set the ``description`` annotation of global variable ``current_user``: - -.. code-block:: edgeql - - alter global current_user - create annotation description := - 'Current User as specified by the global ID'; - -Make the ``current_user_id`` global variable ``required``: - -.. code-block:: edgeql - - alter global current_user_id { - set required; - # A required global variable MUST have a default value. - set default := '00ea8eaa-02f9-11ed-a676-6bd11cc6c557'; - } - - -Drop global -=========== - -:eql-statement: -:eql-haswith: - -Remove a global variable from the schema. - -.. eql:synopsis:: - - [ with [, ...] ] - drop global ; - -Description ------------ - -The command :eql:synopsis:`drop global` removes the specified global variable -from the schema. - -Example -------- - -Remove the ``current_user`` global variable: - -.. code-block:: edgeql - - drop global current_user; - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Globals ` - * - :ref:`SDL > Globals ` - diff --git a/docs/reference/ddl/index.rst b/docs/reference/ddl/index.rst index 31562c526d9..b16578327c6 100644 --- a/docs/reference/ddl/index.rst +++ b/docs/reference/ddl/index.rst @@ -7,22 +7,7 @@ DDL :maxdepth: 3 :hidden: - modules - objects - scalars - links properties - aliases - indexes - constraints - annotations - globals - access_policies - functions - triggers - mutation_rewrites - extensions - future migrations :edb-alt-title: Data Definition Language diff --git a/docs/reference/ddl/indexes.rst b/docs/reference/ddl/indexes.rst deleted file mode 100644 index c33eda1ca54..00000000000 --- a/docs/reference/ddl/indexes.rst +++ /dev/null @@ -1,174 +0,0 @@ -.. _ref_eql_ddl_indexes: - -======= -Indexes -======= - -This section describes the DDL commands pertaining to -:ref:`indexes `. - - -Create index -============ - -:eql-statement: - - -:ref:`Define ` an new index for a given object -type or link. - -.. eql:synopsis:: - - create index on ( ) - [ except ( ) ] - [ "{" ; [...] "}" ] ; - - # where is one of - - create annotation := - - -Description ------------ - -The command ``create index`` constructs a new index for a given object type or -link using *index-expr*. - - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL index declaration `. There's -only one subcommand that is allowed in the ``create index`` block: - -:eql:synopsis:`create annotation := ` - Set object type :eql:synopsis:`` to - :eql:synopsis:``. - - See :eql:stmt:`create annotation` for details. - - -Example -------- - -Create an object type ``User`` with an indexed ``name`` property: - -.. code-block:: edgeql - - create type User { - create property name -> str { - set default := ''; - }; - - create index on (.name); - }; - - -Alter index -=========== - -:eql-statement: - - -Alter the definition of an :ref:`index `. - -.. eql:synopsis:: - - alter index on ( ) [ except ( ) ] - [ "{" ; [...] "}" ] ; - - # where is one of - - create annotation := - alter annotation := - drop annotation - - -Description ------------ - -The command ``alter index`` is used to change the :ref:`annotations -` of an index. The *index-expr* is used to -identify the index to be altered. - - -Parameters ----------- - -:sdl:synopsis:`on ( )` - The specific expression for which the index is made. Note also - that ```` itself has to be parenthesized. - -The following subcommands are allowed in the ``alter index`` block: - -:eql:synopsis:`create annotation := ` - Set index :eql:synopsis:`` to - :eql:synopsis:``. - See :eql:stmt:`create annotation` for details. - -:eql:synopsis:`alter annotation ;` - Alter index :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ;` - Remove constraint :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - - -Example -------- - -Add an annotation to the index on the ``name`` property of object type -``User``: - -.. code-block:: edgeql - - alter type User { - alter index on (.name) { - create annotation title := "User name index"; - }; - }; - - -Drop index -========== - -:eql-statement: - -Remove an index from a given schema item. - -.. eql:synopsis:: - - drop index on ( ) [ except ( ) ]; - -Description ------------ - -The command ``drop index`` removes an index from a schema item. - -:sdl:synopsis:`on ( )` - The specific expression for which the index was made. - -This statement can only be used as a subdefinition in another -DDL statement. - - -Example -------- - -Drop the ``name`` index from the ``User`` object type: - -.. code-block:: edgeql - - alter type User { - drop index on (.name); - }; - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Indexes ` - * - :ref:`SDL > Indexes ` - * - :ref:`Introspection > Indexes ` diff --git a/docs/reference/ddl/links.rst b/docs/reference/ddl/links.rst deleted file mode 100644 index 57d3d52d5a8..00000000000 --- a/docs/reference/ddl/links.rst +++ /dev/null @@ -1,435 +0,0 @@ -.. _ref_eql_ddl_links: - -===== -Links -===== - -This section describes the DDL commands pertaining to -:ref:`links `. - - -Create link -=========== - -:eql-statement: -:eql-haswith: - -:ref:`Define ` a new link. - -.. eql:synopsis:: - - [ with [, ...] ] - {create|alter} type "{" - [ ... ] - create [{required | optional}] [{single | multi}] - link - [ extending [, ...] ] -> - [ "{" ; [...] "}" ] ; - [ ... ] - "}" - - # Computed link form: - - [ with [, ...] ] - {create|alter} type "{" - [ ... ] - create [{required | optional}] [{single | multi}] - link := ; - [ ... ] - "}" - - # Abstract link form: - - [ with [, ...] ] - create abstract link [::] [extending [, ...]] - [ "{" ; [...] "}" ] - - # where is one of - - set default := - set readonly := {true | false} - create annotation := - create property ... - create constraint ... - on target delete - on source delete - reset on target delete - create index on - - -Description ------------ - -The combinations of ``create type ... create link`` and ``alter type -... create link`` define a new concrete link for a given object type. - -There are three forms of ``create link``, as shown in the syntax synopsis -above. The first form is the canonical definition form, the second -form is a syntax shorthand for defining a -:ref:`computed link `, and the third is a -form to define an abstract link item. The abstract form allows creating -the link in the specified :eql:synopsis:``. Concrete link forms -are always created in the same module as the containing object type. - - -.. _ref_eql_ddl_links_syntax: - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL link declaration `. The following -subcommands are allowed in the ``create link`` block: - -:eql:synopsis:`set default := ` - Specifies the default value for the link as an EdgeQL expression. - Other than a slight syntactical difference this is the same as the - corresponding SDL declaration. - -:eql:synopsis:`set readonly := {true | false}` - Specifies whether the link is considered *read-only*. Other than a - slight syntactical difference this is the same as the - corresponding SDL declaration. - -:eql:synopsis:`create annotation := ;` - Add an annotation :eql:synopsis:`` - set to :eql:synopsis:`` to the type. - - See :eql:stmt:`create annotation` for details. - -:eql:synopsis:`create property ...` - Define a concrete property item for this link. See - :eql:stmt:`create property` for details. - -:eql:synopsis:`create constraint ...` - Define a concrete constraint for this link. See - :eql:stmt:`create constraint` for details. - -:eql:synopsis:`on target delete ` - Valid values for *action* are: ``restrict``, ``DELETE - SOURCE``, ``allow``, and ``deferred restrict``. The details of - what ``on target delete`` options mean are described in - :ref:`this section `. - -:eql:synopsis:`reset on target delete` - Reset the delete policy to either the inherited value or to the - default ``restrict``. The details of what ``on target delete`` - options mean are described in :ref:`this section `. - -:eql:synopsis:`create index on ` - Define a new :ref:`index ` - using *index-expr* for this link. See - :eql:stmt:`create index` for details. - - -Examples --------- - -Define a new link ``friends`` on the ``User`` object type: - -.. code-block:: edgeql - - alter type User { - create multi link friends -> User - }; - -Define a new :ref:`computed link ` -``special_group`` on the ``User`` object type, which contains all the -friends from the same town: - -.. code-block:: edgeql - - alter type User { - create link special_group := ( - select __source__.friends - filter .town = __source__.town - ) - }; - -Define a new abstract link ``orderable`` and a concrete link -``interests`` that extends it, inheriting its ``weight`` property: - -.. code-block:: edgeql - - create abstract link orderable { - create property weight -> std::int64 - }; - - alter type User { - create multi link interests extending orderable -> Interest - }; - - - -Alter link -========== - -:eql-statement: -:eql-haswith: - - -Change the definition of a :ref:`link `. - -.. eql:synopsis:: - - [ with [, ...] ] - {create|alter} type "{" - [ ... ] - alter link - [ "{" ] ; [...] [ "}" ]; - [ ... ] - "}" - - - [ with [, ...] ] - alter abstract link [::] - [ "{" ] ; [...] [ "}" ]; - - # where is one of - - set default := - reset default - set readonly := {true | false} - reset readonly - rename to - extending ... - set required - set optional - reset optionality - set single - set multi - reset cardinality - set type [using () - create annotation := - alter annotation := - drop annotation - create property ... - alter property ... - drop property ... - create constraint ... - alter constraint ... - drop constraint ... - on target delete - on source delete - create index on - drop index on - -Description ------------ - -The combinations of``create type ... alter link`` and ``alter type ... -alter link`` change the definition of a concrete link for a given -object type. - -The command ``alter abstract link`` changes the definition of an -abstract link item. *name* must be the identity of an existing -abstract link, optionally qualified with a module name. - -Parameters ----------- - -The following subcommands are allowed in the ``alter link`` block: - -:eql:synopsis:`rename to ` - Change the name of the link item to *newname*. All concrete links - inheriting from this links are also renamed. - -:eql:synopsis:`extending ...` - Alter the link parent list. The full syntax of this subcommand is: - - .. eql:synopsis:: - - extending [, ...] - [ first | last | before | after ] - - This subcommand makes the link a child of the specified list - of parent links. The requirements for the parent-child - relationship are the same as when creating a link. - - It is possible to specify the position in the parent list - using the following optional keywords: - - * ``first`` -- insert parent(s) at the beginning of the - parent list, - * ``last`` -- insert parent(s) at the end of the parent list, - * ``before `` -- insert parent(s) before an - existing *parent*, - * ``after `` -- insert parent(s) after an existing - *parent*. - -:eql:synopsis:`set required` - Make the link *required*. - -:eql:synopsis:`set optional` - Make the link no longer *required* (i.e. make it *optional*). - -:eql:synopsis:`reset optionality` - Reset the optionality of the link to the default value (``optional``), - or, if the link is inherited, to the value inherited from links in - supertypes. - -:eql:synopsis:`set single` - Change the link set's maximum cardinality to *one*. Only - valid for concrete links. - -:eql:synopsis:`set multi` - Remove the upper limit on the link set's cardinality. Only valid for - concrete links. - -:eql:synopsis:`reset cardinality` - Reset the link set's maximum cardinality to the default value - (``single``), or to the value inherited from the link's supertypes. - -:eql:synopsis:`set type [using (`. The optional ``using`` clause specifies - a conversion expression that computes the new link value from the old. - The conversion expression must return a singleton set and is evaluated - on each element of ``multi`` links. A ``using`` clause must be provided - if there is no implicit or assignment cast from old to new type. - -:eql:synopsis:`reset type` - Reset the type of the link to be strictly the inherited type. This only - has an effect on links that have been :ref:`overloaded - ` in order to change their inherited - type. It is an error to ``reset type`` on a link that is not inherited. - -:eql:synopsis:`using ()` - Change the expression of a :ref:`computed link - `. Only valid for concrete links. - -:eql:synopsis:`alter annotation ;` - Alter link annotation :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ;` - Remove link item's annotation :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - -:eql:synopsis:`alter property ...` - Alter the definition of a property item for this link. See - :eql:stmt:`alter property` for details. - -:eql:synopsis:`drop property ;` - Remove a property item from this link. See - :eql:stmt:`drop property` for details. - -:eql:synopsis:`alter constraint ...` - Alter the definition of a constraint for this link. See - :eql:stmt:`alter constraint` for details. - -:eql:synopsis:`drop constraint ;` - Remove a constraint from this link. See - :eql:stmt:`drop constraint` for details. - -:eql:synopsis:`drop index on ` - Remove an :ref:`index ` defined on *index-expr* - from this link. See :eql:stmt:`drop index` for details. - -:eql:synopsis:`reset default` - Remove the default value from this link, or reset it to the value - inherited from a supertype, if the link is inherited. - -:eql:synopsis:`reset readonly` - Set link writability to the default value (writable), or, if the link is - inherited, to the value inherited from links in supertypes. - -All the subcommands allowed in the ``create link`` block are also -valid subcommands for ``alter link`` block. - - -Examples --------- - -On the object type ``User``, set the ``title`` annotation of its -``friends`` link to ``"Friends"``: - -.. code-block:: edgeql - - alter type User { - alter link friends create annotation title := "Friends"; - }; - -Rename the abstract link ``orderable`` to ``sorted``: - -.. code-block:: edgeql - - alter abstract link orderable rename to sorted; - -Redefine the :ref:`computed link ` -``special_group`` to be those who have some shared interests: - -.. code-block:: edgeql - - alter type User { - create link special_group := ( - select __source__.friends - # at least one of the friend's interests - # must match the user's - filter .interests IN __source__.interests - ) - }; - - -Drop link -========= - -:eql-statement: -:eql-haswith: - - -Remove the specified link from the schema. - -.. eql:synopsis:: - - [ with [, ...] ] - alter type "{" - [ ... ] - drop link - [ ... ] - "}" - - - [ with [, ...] ] - drop abstract link []:: - - -Description ------------ - -The combination of ``alter type`` and ``drop link`` removes the -specified link from its containing object type. All links that -inherit from this link are also removed. - -The command ``drop abstract link`` removes an existing link item from -the database schema. All subordinate schema items defined on this -link, such as link properties and constraints, are removed as well. - - -Examples --------- - -Remove link ``friends`` from object type ``User``: - -.. code-block:: edgeql - - alter type User drop link friends; - - -Drop abstract link ``orderable``: - -.. code-block:: edgeql - - drop abstract link orderable; - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Links ` - * - :ref:`SDL > Links ` - * - :ref:`Introspection > Object types - ` diff --git a/docs/reference/ddl/modules.rst b/docs/reference/ddl/modules.rst deleted file mode 100644 index afc578d85f9..00000000000 --- a/docs/reference/ddl/modules.rst +++ /dev/null @@ -1,99 +0,0 @@ -.. _ref_eql_ddl_modules: - -======= -Modules -======= - -This section describes the DDL commands pertaining to -:ref:`modules `. - - -Create module -============= - -:eql-statement: - -Create a new module. - -.. eql:synopsis:: - - create module [ if not exists ]; - -There's a :ref:`corresponding SDL declaration ` -for a module, although in SDL a module declaration is likely to also -include that module's content. - -.. versionadded:: 3.0 - - You may also create a nested module. - - .. eql:synopsis:: - - create module :: [ if not exists ]; - -Description ------------ - -The command ``create module`` defines a new module for the current -:versionreplace:`database;5.0:branch`. The name of the new module must be -distinct from any existing module in the current -:versionreplace:`database;5.0:branch`. Unlike :ref:`SDL module declaration -` the ``create module`` command does not have -sub-commands, as module contents are created separately. - -Parameters ----------- - -:eql:synopsis:`if not exists` - Normally creating a module that already exists is an error, but - with this flag the command will succeed. It is useful for scripts - that add something to a module or if the module is missing the - module is created as well. - -Examples --------- - -Create a new module: - -.. code-block:: edgeql - - create module payments; - -.. versionadded:: 3.0 - - Create a new nested module: - - .. code-block:: edgeql - - create module payments::currencies; - - -Drop module -=========== - -:eql-statement: - - -Remove a module. - -.. eql:synopsis:: - - drop module ; - - -Description ------------ - -The command ``drop module`` removes an existing empty module from the -current :versionreplace:`database;5.0:branch`. If the module contains any -schema items, this command will fail. - - -Examples --------- - -Remove a module: - -.. code-block:: edgeql - - drop module payments; diff --git a/docs/reference/ddl/mutation_rewrites.rst b/docs/reference/ddl/mutation_rewrites.rst deleted file mode 100644 index 4fbc0aad396..00000000000 --- a/docs/reference/ddl/mutation_rewrites.rst +++ /dev/null @@ -1,145 +0,0 @@ -.. _ref_eql_ddl_mutation_rewrites: - -================= -Mutation Rewrites -================= - -This section describes the DDL commands pertaining to -:ref:`mutation rewrites `. - - -Create rewrite -============== - -:eql-statement: - - -:ref:`Define ` a new mutation rewrite. - -When creating a new property or link: - -.. eql:synopsis:: - - {create | alter} type "{" - create { property | link } -> "{" - create rewrite {insert | update} [, ...] - using - "}" ; - "}" ; - -When altering an existing property or link: - -.. eql:synopsis:: - - {create | alter} type "{" - alter { property | link } "{" - create rewrite {insert | update} [, ...] - using - "}" ; - "}" ; - - -Description ------------ - -The command ``create rewrite`` nested under ``create type`` or ``alter type`` -and then under ``create property/link`` or ``alter property/link`` defines a -new mutation rewrite for the given property or link on the given object. - - -Parameters ----------- - -:eql:synopsis:`` - The name (optionally module-qualified) of the type containing the rewrite. - -:eql:synopsis:`` - The name (optionally module-qualified) of the property or link being - rewritten. - -:eql:synopsis:`insert | update [, ...]` - The query type (or types) that are rewritten. Separate multiple values with - commas to invoke the same rewrite for multiple types of queries. - - -Examples --------- - -Declare two mutation rewrites on new properties: one that sets a ``created`` -property when a new object is inserted and one that sets a ``modified`` -property on each update: - -.. code-block:: edgeql - - alter type User { - create property created -> datetime { - create rewrite insert using (datetime_of_statement()); - }; - create property modified -> datetime { - create rewrite update using (datetime_of_statement()); - }; - }; - - -Drop rewrite -============ - -:eql-statement: - - -Remove a mutation rewrite. - -.. eql:synopsis:: - - alter type "{" - alter property "{" - drop rewrite {insert | update} ; - "}" ; - "}" ; - - -Description ------------ - -The command ``drop rewrite`` inside an ``alter type`` block and further inside -an ``alter property`` block removes the definition of an existing mutation -rewrite on the specified property or link of the specified type. - - -Parameters ----------- - -:eql:synopsis:`` - The name (optionally module-qualified) of the type containing the rewrite. - -:eql:synopsis:`` - The name (optionally module-qualified) of the property or link being - rewritten. - -:eql:synopsis:`insert | update [, ...]` - The query type (or types) that are rewritten. Separate multiple values with - commas to invoke the same rewrite for multiple types of queries. - - -Example -------- - -Remove the ``insert`` rewrite of the ``created`` property on the ``User`` type: - -.. code-block:: edgeql - - alter type User { - alter property created { - drop rewrite insert; - }; - }; - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Mutation rewrites ` - * - :ref:`SDL > Mutation rewrites ` - * - :ref:`Introspection > Mutation rewrites - ` diff --git a/docs/reference/ddl/objects.rst b/docs/reference/ddl/objects.rst deleted file mode 100644 index b855fe56762..00000000000 --- a/docs/reference/ddl/objects.rst +++ /dev/null @@ -1,272 +0,0 @@ -.. _ref_eql_ddl_object_types: - -============ -Object Types -============ - -This section describes the DDL commands pertaining to -:ref:`object types `. - - -Create type -=========== - -:eql-statement: -:eql-haswith: - - -:ref:`Define ` a new object type. - -.. eql:synopsis:: - - [ with [, ...] ] - create [abstract] type [ extending [, ...] ] - [ "{" ; [...] "}" ] ; - - # where is one of - - create annotation := - create link ... - create property ... - create constraint ... - create index on - -Description ------------ - -The command ``create type`` defines a new object type for use in the -current :versionreplace:`database;5.0:branch`. - -If *name* is qualified with a module name, then the type is created -in that module, otherwise it is created in the current module. -The type name must be distinct from that of any existing schema item -in the module. - -Parameters ----------- - -Most sub-commands and options of this command are identical to the -:ref:`SDL object type declaration `, -with some additional features listed below: - -:eql:synopsis:`with [, ...]` - Alias declarations. - - The ``with`` clause allows specifying module aliases - that can be referenced by the command. See :ref:`ref_eql_statements_with` - for more information. - -The following subcommands are allowed in the ``create type`` block: - -:eql:synopsis:`create annotation := ` - Set object type :eql:synopsis:`` to - :eql:synopsis:``. - - See :eql:stmt:`create annotation` for details. - -:eql:synopsis:`create link ...` - Define a new link for this object type. See - :eql:stmt:`create link` for details. - -:eql:synopsis:`create property ...` - Define a new property for this object type. See - :eql:stmt:`create property` for details. - -:eql:synopsis:`create constraint ...` - Define a concrete constraint for this object type. See - :eql:stmt:`create constraint` for details. - -:eql:synopsis:`create index on ` - Define a new :ref:`index ` - using *index-expr* for this object type. See - :eql:stmt:`create index` for details. - -Examples --------- - -Create an object type ``User``: - -.. code-block:: edgeql - - create type User { - create property name -> str; - }; - - -.. _ref_eql_ddl_object_types_alter: - -Alter type -========== - -:eql-statement: -:eql-haswith: - - -Change the definition of an -:ref:`object type `. - -.. eql:synopsis:: - - [ with [, ...] ] - alter type - [ "{" ; [...] "}" ] ; - - [ with [, ...] ] - alter type ; - - # where is one of - - rename to - extending [, ...] - create annotation := - alter annotation := - drop annotation - create link ... - alter link ... - drop link ... - create property ... - alter property ... - drop property ... - create constraint ... - alter constraint ... - drop constraint ... - create index on - drop index on - - -Description ------------ - -The command ``alter type`` changes the definition of an object type. -*name* must be a name of an existing object type, optionally qualified -with a module name. - -Parameters ----------- - -The following subcommands are allowed in the ``alter type`` block: - -:eql:synopsis:`with [, ...]` - Alias declarations. - - The ``with`` clause allows specifying module aliases - that can be referenced by the command. See :ref:`ref_eql_statements_with` - for more information. - -:eql:synopsis:`` - The name (optionally module-qualified) of the type being altered. - -:eql:synopsis:`extending [, ...]` - Alter the supertype list. The full syntax of this subcommand is: - - .. eql:synopsis:: - - extending [, ...] - [ first | last | before | after ] - - This subcommand makes the type a subtype of the specified list - of supertypes. The requirements for the parent-child relationship - are the same as when creating an object type. - - It is possible to specify the position in the parent list - using the following optional keywords: - - * ``first`` -- insert parent(s) at the beginning of the - parent list, - * ``last`` -- insert parent(s) at the end of the parent list, - * ``before `` -- insert parent(s) before an - existing *parent*, - * ``after `` -- insert parent(s) after an existing - *parent*. - -:eql:synopsis:`alter annotation ;` - Alter object type annotation :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ` - Remove object type :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - -:eql:synopsis:`alter link ...` - Alter the definition of a link for this object type. See - :eql:stmt:`alter link` for details. - -:eql:synopsis:`drop link ` - Remove a link item from this object type. See - :eql:stmt:`drop link` for details. - -:eql:synopsis:`alter property ...` - Alter the definition of a property item for this object type. - See :eql:stmt:`alter property` for details. - -:eql:synopsis:`drop property ` - Remove a property item from this object type. See - :eql:stmt:`drop property` for details. - -:eql:synopsis:`alter constraint ...` - Alter the definition of a constraint for this object type. See - :eql:stmt:`alter constraint` for details. - -:eql:synopsis:`drop constraint ;` - Remove a constraint from this object type. See - :eql:stmt:`drop constraint` for details. - -:eql:synopsis:`drop index on ` - Remove an :ref:`index ` defined as *index-expr* - from this object type. See :eql:stmt:`drop index` for details. - -All the subcommands allowed in the ``create type`` block are also -valid subcommands for ``alter type`` block. - -Examples --------- - -Alter the ``User`` object type to make ``name`` required: - -.. code-block:: edgeql - - alter type User { - alter property name { - set required; - } - }; - - -Drop type -========= - -:eql-statement: -:eql-haswith: - - -Remove the specified object type from the schema. - -.. eql:synopsis:: - - drop type ; - -Description ------------ - -The command ``drop type`` removes the specified object type from the -schema. schema. All subordinate schema items defined on this type, -such as links and indexes, are removed as well. - -Examples --------- - -Remove the ``User`` object type: - -.. code-block:: edgeql - - drop type User; - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Object types ` - * - :ref:`SDL > Object types ` - * - :ref:`Introspection > Object types - ` - * - :ref:`Cheatsheets > Object types ` diff --git a/docs/reference/ddl/scalars.rst b/docs/reference/ddl/scalars.rst deleted file mode 100644 index 6c52796eb69..00000000000 --- a/docs/reference/ddl/scalars.rst +++ /dev/null @@ -1,208 +0,0 @@ -.. _ref_eql_ddl_scalars: - -============ -Scalar Types -============ - -This section describes the DDL commands pertaining to -:ref:`scalar types `. - - -Create scalar type -================== - -:eql-statement: -:eql-haswith: - -:ref:`Define ` a new scalar type. - -.. eql:synopsis:: - - [ with [, ...] ] - create [abstract] scalar type [ extending ] - [ "{" ; [...] "}" ] ; - - # where is one of - - create annotation := - create constraint ... - - -Description ------------ - -The command ``create scalar type`` defines a new scalar type for use in the -current :versionreplace:`database;5.0:branch`. - -If *name* is qualified with a module name, then the type is created -in that module, otherwise it is created in the current module. -The type name must be distinct from that of any existing schema item -in the module. - -If the ``abstract`` keyword is specified, the created type will be -*abstract*. - -All non-abstract scalar types must have an underlying core -implementation. For user-defined scalar types this means that -``create scalar type`` must have another non-abstract scalar type -as its *supertype*. - -The most common use of ``create scalar type`` is to define a scalar -subtype with constraints. - -Most sub-commands and options of this command are identical to the -:ref:`SDL scalar type declaration `. The -following subcommands are allowed in the ``create scalar type`` -block: - -:eql:synopsis:`create annotation := ;` - Set scalar type's :eql:synopsis:`` to - :eql:synopsis:``. - - See :eql:stmt:`create annotation` for details. - -:eql:synopsis:`create constraint ...` - Define a new constraint for this scalar type. See - :eql:stmt:`create constraint` for details. - - -Examples --------- - -Create a new non-negative integer type: - -.. code-block:: edgeql - - create scalar type posint64 extending int64 { - create constraint min_value(0); - }; - - -Create a new enumerated type: - -.. code-block:: edgeql - - create scalar type Color - extending enum; - - -Alter scalar type -================= - -:eql-statement: -:eql-haswith: - - -Alter the definition of a :ref:`scalar type `. - -.. eql:synopsis:: - - [ with [, ...] ] - alter scalar type - "{" ; [...] "}" ; - - # where is one of - - rename to - extending ... - create annotation := - alter annotation := - drop annotation - create constraint ... - alter constraint ... - drop constraint ... - - -Description ------------ - -The command ``alter scalar type`` changes the definition of a scalar type. -*name* must be a name of an existing scalar type, optionally qualified -with a module name. - -The following subcommands are allowed in the ``alter scalar type`` block: - -:eql:synopsis:`rename to ;` - Change the name of the scalar type to *newname*. - -:eql:synopsis:`extending ...` - Alter the supertype list. It works the same way as in - :eql:stmt:`alter type`. - -:eql:synopsis:`alter annotation ;` - Alter scalar type :eql:synopsis:``. - See :eql:stmt:`alter annotation` for details. - -:eql:synopsis:`drop annotation ` - Remove scalar type's :eql:synopsis:`` from - :eql:synopsis:``. - See :eql:stmt:`drop annotation` for details. - -:eql:synopsis:`alter constraint ...` - Alter the definition of a constraint for this scalar type. See - :eql:stmt:`alter constraint` for details. - -:eql:synopsis:`drop constraint ` - Remove a constraint from this scalar type. See - :eql:stmt:`drop constraint` for details. - -All the subcommands allowed in the ``create scalar type`` block are also -valid subcommands for ``alter scalar type`` block. - - -Examples --------- - -Define a new constraint on a scalar type: - -.. code-block:: edgeql - - alter scalar type posint64 { - create constraint max_value(100); - }; - -Add one more label to an enumerated type: - -.. code-block:: edgeql - - alter scalar type Color - extending enum; - - -Drop scalar type -================ - -:eql-statement: -:eql-haswith: - - -Remove a scalar type. - -.. eql:synopsis:: - - [ with [, ...] ] - drop scalar type ; - - -Description ------------ - -The command ``drop scalar type`` removes a scalar type. - - -Parameters ----------- - -*name* - The name (optionally qualified with a module name) of an existing - scalar type. - - -Example -------- - -Remove a scalar type: - -.. code-block:: edgeql - - drop scalar type posint64; diff --git a/docs/reference/ddl/triggers.rst b/docs/reference/ddl/triggers.rst deleted file mode 100644 index 6fae93596b1..00000000000 --- a/docs/reference/ddl/triggers.rst +++ /dev/null @@ -1,131 +0,0 @@ -.. _ref_eql_ddl_triggers: - -======== -Triggers -======== - -This section describes the DDL commands pertaining to -:ref:`triggers `. - - -Create trigger -============== - -:eql-statement: - - -:ref:`Define ` a new trigger. - -.. eql:synopsis:: - - {create | alter} type "{" - create trigger - after - {insert | update | delete} [, ...] - for {each | all} - [ when () ] - do - "}" - - -Description ------------ - -The command ``create trigger`` nested under ``create type`` or ``alter type`` -defines a new trigger for a given object type. - -The trigger name must be distinct from that of any existing trigger -on the same type. - -Parameters ----------- - -The options of this command are identical to the -:ref:`SDL trigger declaration `. - - -Example -------- - -Declare a trigger that inserts a ``Log`` object for each new ``User`` object: - -.. code-block:: edgeql - - alter type User { - create trigger log_insert after insert for each do ( - insert Log { - action := 'insert', - target_name := __new__.name - } - ); - }; - -Declare a trigger that inserts a ``Log`` object conditionally when an update -query makes a change to a ``User`` object: - -.. code-block:: edgeql - - alter type User { - create trigger log_update after update for each - when (__old__ {**} != __new__ {**}) - do ( - insert Log { - action := 'update', - target_name := __new__.name, - change := __old__.name ++ '->' ++ __new__.name - } - ); - } - - -Drop trigger -============ - -:eql-statement: - - -Remove a trigger. - -.. eql:synopsis:: - - alter type "{" - drop trigger ; - "}" - - -Description ------------ - -The command ``drop trigger`` inside an ``alter type`` block removes the -definition of an existing trigger on the specified type. - - -Parameters ----------- - -:eql:synopsis:`` - The name (optionally module-qualified) of the type being triggered on. - -:eql:synopsis:`` - The name of the trigger. - - -Example -------- - -Remove the ``log_insert`` trigger on the ``User`` type: - -.. code-block:: edgeql - - alter type User { - drop trigger log_insert; - }; - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Triggers ` - * - :ref:`SDL > Triggers ` - * - :ref:`Introspection > Triggers ` diff --git a/docs/reference/environment.rst b/docs/reference/environment.rst index 0fb71192bf6..ad125dec861 100644 --- a/docs/reference/environment.rst +++ b/docs/reference/environment.rst @@ -91,8 +91,8 @@ GEL_SERVER_BOOTSTRAP_COMMAND_FILE ................................. Run the script when initializing the database. The script is run by the default -user within the default :versionreplace:`database;5.0:branch`. May be used with -or without :gelenv:`SERVER_BOOTSTRAP_ONLY`. +user within the default |branch|. May be used with or without +:gelenv:`SERVER_BOOTSTRAP_ONLY`. GEL_SERVER_BOOTSTRAP_SCRIPT_FILE @@ -103,7 +103,7 @@ GEL_SERVER_BOOTSTRAP_SCRIPT_FILE Use :gelenv:`SERVER_BOOTSTRAP_COMMAND_FILE` instead. Run the script when initializing the database. The script is run by the default -user within the default :versionreplace:`database;5.0:branch`. +user within the default |branch|. GEL_SERVER_COMPILER_POOL_MODE diff --git a/docs/reference/http.rst b/docs/reference/http.rst index 998f41a7ce8..d8545b212d1 100644 --- a/docs/reference/http.rst +++ b/docs/reference/http.rst @@ -25,8 +25,7 @@ Here's how to determine your *local* Gel instance's HTTP server URL: To determine the URL of a remote instance you have linked with the CLI, you can get both the hostname and port of the instance from the "Port" column of the :gelcmd:`instance list` table (formatted as ``:``). -The same guidance on local :versionreplace:`database;5.0:branch` names -applies to remote instances. +The same guidance on local |branch| names applies to remote instances. .. _ref_reference_health_checks: diff --git a/docs/reference/protocol/index.rst b/docs/reference/protocol/index.rst index cb24c7ff4e8..c7c86a32c9b 100644 --- a/docs/reference/protocol/index.rst +++ b/docs/reference/protocol/index.rst @@ -382,7 +382,7 @@ to finish implicit transaction. Restore Flow ------------ -Restore procedure fills up the :versionreplace:`database;5.0:branch` the +Restore procedure fills up the |branch| the client is connected to with the schema and data from the dump file. Flow is the following: diff --git a/docs/reference/protocol/messages.rst b/docs/reference/protocol/messages.rst index 4b61d8d7eac..f6dac4ffdbd 100644 --- a/docs/reference/protocol/messages.rst +++ b/docs/reference/protocol/messages.rst @@ -292,8 +292,7 @@ Restore Sent by: client. -Initiate restore to the current :versionreplace:`database;5.0:branch`. -See :ref:`ref_protocol_restore_flow`. +Initiate restore to the current |branch|. See :ref:`ref_protocol_restore_flow`. Format: diff --git a/docs/reference/sdl/access_policies.rst b/docs/reference/sdl/access_policies.rst deleted file mode 100644 index d99038d6777..00000000000 --- a/docs/reference/sdl/access_policies.rst +++ /dev/null @@ -1,165 +0,0 @@ -.. _ref_eql_sdl_access_policies: - -=============== -Access Policies -=============== - -This section describes the SDL declarations pertaining to access policies. - -Examples --------- - -Declare a schema where users can only see their own profiles: - -.. code-block:: sdl - - # Declare some global variables to store "current user" - # information. - global current_user_id: uuid; - global current_user := ( - select User filter .id = global current_user_id - ); - - type User { - required name: str; - } - - type Profile { - owner: User; - - # Only allow reading to the owner, but also - # ensure that a user cannot set the "owner" link - # to anything but themselves. - access policy owner_only - allow all using (.owner = global current_user); - } - -.. _ref_eql_sdl_access_policies_syntax: - -Syntax ------- - -Define a new access policy corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - # Access policy used inside a type declaration: - access policy - [ when () ] - { allow | deny } [, ... ] - [ using () ] - [ "{" - [ errmessage := value ; ] - [ ] - "}" ] ; - - # where is one of - all - select - insert - delete - update [{ read | write }] - -Description ------------ - -Access policies are used to implement object-level security and as such they -are defined on object types. In practice the access policies often work -together with :ref:`global variables `. - -Access policies are an opt-in feature, so once at least one access policy is -defined for a given type, all access not explicitly allowed by that policy -becomes forbidden. - -Any sub-type :ref:`extending ` a base type also -inherits all the access policies of the base type. - -The access policy declaration options are as follows: - -:eql:synopsis:`` - The name of the access policy. - -:eql:synopsis:`when ()` - Specifies which objects this policy applies to. The - :eql:synopsis:`` has to be a :eql:type:`bool` expression. - - When omitted, it is assumed that this policy applies to all objects of a - given type. - -:eql:synopsis:`allow` - Indicates that qualifying objects should allow access under this policy. - -:eql:synopsis:`deny` - Indicates that qualifying objects should *not* allow access under this - policy. This flavor supersedes any :eql:synopsis:`allow` policy and can - be used to selectively deny access to a subset of objects that otherwise - explicitly allows accessing them. - -:eql:synopsis:`all` - Apply the policy to all actions. It is exactly equivalent to listing - :eql:synopsis:`select`, :eql:synopsis:`insert`, :eql:synopsis:`delete`, - :eql:synopsis:`update` actions explicitly. - -:eql:synopsis:`select` - Apply the policy to all selection queries. Note that any object that - cannot be selected, cannot be modified either. This makes - :eql:synopsis:`select` the most basic "visibility" policy. - -:eql:synopsis:`insert` - Apply the policy to all inserted objects. If a newly inserted object would - violate this policy, an error is produced instead. - -:eql:synopsis:`delete` - Apply the policy to all objects about to be deleted. If an object does not - allow access under this kind of policy, it is not going to be considered - by any :eql:stmt:`delete` command. - - Note that any object that cannot be selected, cannot be modified either. - -:eql:synopsis:`update read` - Apply the policy to all objects selected for an update. If an object does - not allow access under this kind of policy, it is not visible cannot be - updated. - - Note that any object that cannot be selected, cannot be modified either. - -:eql:synopsis:`update write` - Apply the policy to all objects at the end of an update. If an updated - object violates this policy, an error is produced instead. - - Note that any object that cannot be selected, cannot be modified either. - -:eql:synopsis:`update` - This is just a shorthand for :eql:synopsis:`update read` and - :eql:synopsis:`update write`. - - Note that any object that cannot be selected, cannot be modified either. - -:eql:synopsis:`using ` - Specifies what the policy is with respect to a given eligible (based on - :eql:synopsis:`when` clause) object. The :eql:synopsis:`` has to be - a :eql:type:`bool` expression. The specific meaning of this value also - depends on whether this policy flavor is :eql:synopsis:`allow` or - :eql:synopsis:`deny`. - - The expression must be :ref:`Stable `. - - When omitted, it is assumed that this policy applies to all eligible - objects of a given type. - -:eql:synopsis:`set errmessage := ` - Set a custom error message of :eql:synopsis:`` that is displayed - when this access policy prevents a write action. - -:sdl:synopsis:`` - Set access policy :ref:`annotation ` - to a given *value*. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Access policies ` - * - :ref:`DDL > Access policies ` diff --git a/docs/reference/sdl/aliases.rst b/docs/reference/sdl/aliases.rst deleted file mode 100644 index e0520f98d9c..00000000000 --- a/docs/reference/sdl/aliases.rst +++ /dev/null @@ -1,66 +0,0 @@ -.. _ref_eql_sdl_aliases: - -================== -Expression Aliases -================== - -This section describes the SDL declarations pertaining to -:ref:`expression aliases `. - -Example -------- - -Declare a "UserAlias" that provides additional information for a "User" -via a :ref:`computed link ` "friend_of": - -.. code-block:: sdl - - alias UserAlias := User { - # declare a computed link - friend_of := User.`. - -.. sdl:synopsis:: - - alias := ; - - alias "{" - using ; - [ ] - "}" ; - - -Description ------------ - -This declaration defines a new alias with the following options: - -:eql:synopsis:`` - The name (optionally module-qualified) of an alias to be created. - -:eql:synopsis:`` - The aliased expression. Must be a :ref:`Stable ` - EdgeQL expression. - -The valid SDL sub-declarations are listed below: - -:sdl:synopsis:`` - Set alias :ref:`annotation ` - to a given *value*. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Aliases ` - * - :ref:`DDL > Aliases ` - * - :ref:`Cheatsheets > Aliases ` diff --git a/docs/reference/sdl/annotations.rst b/docs/reference/sdl/annotations.rst deleted file mode 100644 index d208dc5aebb..00000000000 --- a/docs/reference/sdl/annotations.rst +++ /dev/null @@ -1,93 +0,0 @@ -.. _ref_eql_sdl_annotations: - -=========== -Annotations -=========== - -This section describes the SDL declarations pertaining to -:ref:`annotations `. - - -Examples --------- - -Declare a new annotation: - -.. code-block:: sdl - - abstract annotation admin_note; - -Specify the value of an annotation for a type: - -.. code-block:: sdl - - type Status { - annotation admin_note := 'system-critical'; - required name: str { - constraint exclusive - } - } - -.. _ref_eql_sdl_annotations_syntax: - -Syntax ------- - -Define a new annotation corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - # Abstract annotation form: - abstract [ inheritable ] annotation - [ "{" ; [...] "}" ] ; - - # Concrete annotation (same as ) form: - annotation := ; - - -Description ------------ - -There are two forms of annotation declarations: abstract and concrete. -The *abstract annotation* form is used for declaring new kinds of -annotation in a module. The *concrete annotation* declarations are -used as sub-declarations for all other declarations in order to -actually annotate them. - -The annotation declaration options are as follows: - -:eql:synopsis:`abstract` - If specified, the annotation will be *abstract*. - -:eql:synopsis:`inheritable` - If specified, the annotation will be *inheritable*. The - annotations are non-inheritable by default. That is, if a schema - item has an annotation defined on it, the descendants of that - schema item will not automatically inherit the annotation. Normal - inheritance behavior can be turned on by declaring the annotation - with the ``inheritable`` qualifier. This is only valid for *abstract - annotation*. - -:eql:synopsis:`` - The name (optionally module-qualified) of the annotation. - -:eql:synopsis:`` - Any string value that the specified annotation is intended to have - for the given context. - -The only valid SDL sub-declarations are *concrete annotations*: - -:sdl:synopsis:`` - Annotations can also have annotations. Set the *annotation* of the - enclosing annotation to a specific value. - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Annotations ` - * - :ref:`DDL > Annotations ` - * - :ref:`Cheatsheets > Annotations ` - * - :ref:`Introspection > Object types - ` diff --git a/docs/reference/sdl/constraints.rst b/docs/reference/sdl/constraints.rst deleted file mode 100644 index facf917c7ad..00000000000 --- a/docs/reference/sdl/constraints.rst +++ /dev/null @@ -1,164 +0,0 @@ -.. _ref_eql_sdl_constraints: - -=========== -Constraints -=========== - -This section describes the SDL declarations pertaining to -:ref:`constraints `. - - -Examples --------- - -Declare an *abstract* constraint: - -.. code-block:: sdl - - abstract constraint min_value(min: anytype) { - errmessage := - 'Minimum allowed value for {__subject__} is {min}.'; - - using (__subject__ >= min); - } - -Declare a *concrete* constraint on an integer type: - -.. code-block:: sdl - - scalar type posint64 extending int64 { - constraint min_value(0); - } - -Declare a *concrete* constraint on an object type: - -.. code-block:: sdl - - type Vector { - required x: float64; - required y: float64; - constraint expression on ( - __subject__.x^2 + __subject__.y^2 < 25 - ); - } - -.. _ref_eql_sdl_constraints_syntax: - -Syntax ------- - -Define a constraint corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - [{abstract | delegated}] constraint [ ( [] [, ...] ) ] - [ on ( ) ] - [ except ( ) ] - [ extending [, ...] ] - "{" - [ using ; ] - [ errmessage := ; ] - [ ] - [ ... ] - "}" ; - - # where is: - - [ : ] { | } - - -Description ------------ - -This declaration defines a new constraint with the following options: - -:eql:synopsis:`abstract` - If specified, the constraint will be *abstract*. - -:eql:synopsis:`delegated` - If specified, the constraint is defined as *delegated*, which - means that it will not be enforced on the type it's declared on, - and the enforcement will be delegated to the subtypes of this - type. This is particularly useful for :eql:constraint:`exclusive` - constraints in abstract types. This is only valid for *concrete - constraints*. - -:eql:synopsis:`` - The name (optionally module-qualified) of the new constraint. - -:eql:synopsis:`` - An optional list of constraint arguments. - - For an *abstract constraint* :eql:synopsis:`` optionally - specifies the argument name and :eql:synopsis:`` - specifies the argument type. - - For a *concrete constraint* :eql:synopsis:`` optionally - specifies the argument name and :eql:synopsis:`` - specifies the argument value. The argument value specification must - match the parameter declaration of the abstract constraint. - -:eql:synopsis:`on ( )` - An optional expression defining the *subject* of the constraint. - If not specified, the subject is the value of the schema item on - which the concrete constraint is defined. - - The expression must refer to the original subject of the constraint as - ``__subject__``. The expression must be - :ref:`Immutable `, but may refer to - ``__subject__`` and its properties and links. - - Note also that ```` itself has to - be parenthesized. - -:eql:synopsis:`except ( )` - An optional expression defining a condition to create exceptions - to the constraint. If ```` evaluates to ``true``, - the constraint is ignored for the current subject. If it evaluates - to ``false`` or ``{}``, the constraint applies normally. - - ``except`` may only be declared on object constraints, and is - otherwise follows the same rules as ``on``, above. - -:eql:synopsis:`extending [, ...]` - If specified, declares the *parent* constraints for this abstract - constraint. - -The valid SDL sub-declarations are listed below: - -:eql:synopsis:`using ` - A boolean expression that returns ``true`` for valid data and - ``false`` for invalid data. The expression may refer to the - subject of the constraint as ``__subject__``. This declaration is - only valid for *abstract constraints*. - -:eql:synopsis:`errmessage := ` - An optional string literal defining the error message template - that is raised when the constraint is violated. The template is a - formatted string that may refer to constraint context variables in - curly braces. The template may refer to the following: - - - ``$argname`` -- the value of the specified constraint argument - - ``__subject__`` -- the value of the ``title`` annotation of the - scalar type, property or link on which the constraint is - defined. - - If the content of curly braces does not match any variables, - the curly braces are emitted as-is. They can also be escaped by - using double curly braces. - -:sdl:synopsis:`` - Set constraint :ref:`annotation ` - to a given *value*. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Constraints ` - * - :ref:`DDL > Constraints ` - * - :ref:`Introspection > Constraints - ` - * - :ref:`Standard Library > Constraints ` diff --git a/docs/reference/sdl/extensions.rst b/docs/reference/sdl/extensions.rst deleted file mode 100644 index d73ded8d7c9..00000000000 --- a/docs/reference/sdl/extensions.rst +++ /dev/null @@ -1,44 +0,0 @@ -.. _ref_eql_sdl_extensions: - -========== -Extensions -========== - -This section describes the SDL commands pertaining to -:ref:`extensions `. - - -Syntax ------- - -Declare that the current schema enables a particular extension. - -.. sdl:synopsis:: - - using extension ";" - - -Description ------------ - -Extension declaration must be outside any :ref:`module block -` since extensions affect the entire database and -not a specific module. - - -Examples --------- - -Enable :ref:`GraphQL ` extension for the current -schema: - -.. code-block:: sdl - - using extension graphql; - -Enable :ref:`EdgeQL over HTTP ` extension for the -current database: - -.. code-block:: sdl - - using extension edgeql_http; diff --git a/docs/reference/sdl/functions.rst b/docs/reference/sdl/functions.rst deleted file mode 100644 index 7f755784a47..00000000000 --- a/docs/reference/sdl/functions.rst +++ /dev/null @@ -1,188 +0,0 @@ -.. _ref_eql_sdl_functions: - -========= -Functions -========= - -This section describes the SDL declarations pertaining to -:ref:`functions `. - - -Example -------- - -Declare a custom function that concatenates the length of a string to -the end of the that string: - -.. code-block:: sdl - - function foo(s: str) -> str - using ( - select s ++ len(a) - ); - -.. _ref_eql_sdl_functions_syntax: - -Syntax ------- - -Define a new function corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - function ([ ] [, ... ]) -> - using ( ); - - function ([ ] [, ... ]) -> - using ; - - function ([ ] [, ... ]) -> - "{" - [ ] - [ volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'} ] - [ using ( ) ; ] - [ using ; ] - [ ... ] - "}" ; - - # where is: - - [ ] : [ ] [ = ] - - # is: - - [ { variadic | named only } ] - - # is: - - [ { set of | optional } ] - - # and is: - - [ ] - - -Description ------------ - - - -This declaration defines a new constraint with the following options: - -:eql:synopsis:`` - The name (optionally module-qualified) of the function to create. - -:eql:synopsis:`` - The kind of an argument: ``variadic`` or ``named only``. - - If not specified, the argument is called *positional*. - - The ``variadic`` modifier indicates that the function takes an - arbitrary number of arguments of the specified type. The passed - arguments will be passed as as array of the argument type. - Positional arguments cannot follow a ``variadic`` argument. - ``variadic`` parameters cannot have a default value. - - The ``named only`` modifier indicates that the argument can only - be passed using that specific name. Positional arguments cannot - follow a ``named only`` argument. - -:eql:synopsis:`` - The name of an argument. If ``named only`` modifier is used this - argument *must* be passed using this name only. - -.. _ref_sdl_function_typequal: - -:eql:synopsis:`` - The type qualifier: ``set of`` or ``optional``. - - The ``set of`` qualifier indicates that the function is taking the - argument as a *whole set*, as opposed to being called on the input - product element-by-element. - - User defined functions can not use ``set of`` arguments. - - The ``optional`` qualifier indicates that the function will be called - if the argument is an empty set. The default behavior is to return - an empty set if the argument is not marked as ``optional``. - -:eql:synopsis:`` - The data type of the function's arguments - (optionally module-qualified). - -:eql:synopsis:`` - An expression to be used as default value if the parameter is not - specified. The expression has to be of a type compatible with the - type of the argument. - -.. _ref_sdl_function_rettype: - -:eql:synopsis:`` - The return data type (optionally module-qualified). - - The ``set of`` modifier indicates that the function will return - a non-singleton set. - - The ``optional`` qualifier indicates that the function may return - an empty set. - -The valid SDL sub-declarations are listed below: - -:eql:synopsis:`volatility := {'Immutable' | 'Stable' | 'Volatile' | 'Modifying'}` - Function volatility determines how aggressively the compiler can - optimize its invocations. - - If not explicitly specified the function volatility is - :ref:`inferred ` from the function body. - - * An ``Immutable`` function cannot modify the database and is - guaranteed to return the same results given the same arguments - *in all statements*. - - * A ``Stable`` function cannot modify the database and is - guaranteed to return the same results given the same - arguments *within a single statement*. - - * A ``Volatile`` function cannot modify the database and can return - different results on successive calls with the same arguments. - - * A ``Modifying`` function can modify the database and can return - different results on successive calls with the same arguments. - -:eql:synopsis:`using ( )` - Specified the body of the function. :eql:synopsis:`` is an - arbitrary EdgeQL expression. - -:eql:synopsis:`using ` - A verbose version of the :eql:synopsis:`using` clause that allows - to specify the language of the function body. - - * :eql:synopsis:`` is the name of the language that - the function is implemented in. Currently can only be ``edgeql``. - - * :eql:synopsis:`` is a string constant defining - the function. It is often helpful to use - :ref:`dollar quoting ` - to write the function definition string. - -:sdl:synopsis:`` - Set function :ref:`annotation ` - to a given *value*. - -The function name must be distinct from that of any existing function -with the same argument types in the same module. Functions of -different argument types can share a name, in which case the functions -are called *overloaded functions*. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Functions ` - * - :ref:`DDL > Functions ` - * - :ref:`Reference > Function calls ` - * - :ref:`Introspection > Functions ` - * - :ref:`Cheatsheets > Functions ` - diff --git a/docs/reference/sdl/future.rst b/docs/reference/sdl/future.rst deleted file mode 100644 index ad0673d2fbf..00000000000 --- a/docs/reference/sdl/future.rst +++ /dev/null @@ -1,37 +0,0 @@ -.. _ref_eql_sdl_future: - -=============== -Future Behavior -=============== - -This section describes the SDL commands pertaining to -:ref:`future `. - - -Syntax ------- - -Declare that the current schema enables a particular future behavior. - -.. sdl:synopsis:: - - using future ";" - - -Description ------------ - -Future behavior declaration must be outside any :ref:`module block -` since this behavior affects the entire database and not -a specific module. - - -Examples --------- - -Enable simpler non-recursive access policy behavior :ref:`non-recursive access -policy ` for the current schema: - -.. code-block:: sdl - - using extension nonrecursive_access_policies; diff --git a/docs/reference/sdl/globals.rst b/docs/reference/sdl/globals.rst deleted file mode 100644 index ac246826a39..00000000000 --- a/docs/reference/sdl/globals.rst +++ /dev/null @@ -1,149 +0,0 @@ -.. _ref_eql_sdl_globals: - -======= -Globals -======= - -This section describes the SDL commands pertaining to global variables. - -Examples --------- - -Declare a new global variable: - -.. code-block:: sdl - - global current_user_id -> uuid; - global current_user := ( - select User filter .id = global current_user_id - ); - -Set the global variable to a specific value using :ref:`session-level commands -`: - -.. code-block:: edgeql - - set global current_user_id := - '00ea8eaa-02f9-11ed-a676-6bd11cc6c557'; - -Use the computed global variable that is based on the value that was just set: - -.. code-block:: edgeql - - select global current_user { name }; - -:ref:`Reset ` the global variable to -its default value: - -.. code-block:: edgeql - - reset global user_id; - - -.. _ref_eql_sdl_globals_syntax: - -Syntax ------- - -Define a new global variable corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - - # Global variable declaration: - [{required | optional}] [single] - global -> - [ "{" - [ default := ; ] - [ ] - ... - "}" ] - - # Computed global variable declaration: - [{required | optional}] [{single | multi}] - global := ; - - -Description ------------ - -There two different forms of ``global`` declaration, as shown in the syntax -synopsis above. The first form is for defining a ``global`` variable that can -be :ref:`set ` in a session. The second -form is not directly set, but instead it is *computed* based on an expression, -potentially deriving its value from other global variables. - -The following options are available: - -:eql:synopsis:`required` - If specified, the global variable is considered *required*. It is an - error for this variable to have an empty value. If a global variable is - declared *required*, it must also declare a *default* value. - -:eql:synopsis:`optional` - This is the default qualifier assumed when no qualifier is specified, but - it can also be specified explicitly. The global variable is considered - *optional*, i.e. it is possible for the variable to have an empty value. - -:eql:synopsis:`multi` - Specifies that the global variable may have a set of values. Only - *computed* global variables can have this qualifier. - -:eql:synopsis:`single` - Specifies that the global variable must have at most a *single* value. It - is assumed that a global variable is ``single`` if nether ``multi`` nor - ``single`` qualifier is specified. All non-computed global variables must - be *single*. - -:eql:synopsis:`` - Specifies the name of the global variable. The name has to be either - fully-qualified with the module name it belongs to or it will be assumed - to belong to the module in which it appears. - -:eql:synopsis:`` - The type must be a valid :ref:`type expression ` - denoting a non-abstract scalar or a container type. - -:eql:synopsis:` := ` - Defines a *computed* global variable. - - The provided expression must be a :ref:`Stable ` - EdgeQL expression. It can refer to other global variables. - - The type of a *computed* global variable is not limited to - scalar and container types, but also includes object types. So it is - possible to use that to define a global object variable based on an - another global scalar variable. - - For example: - - .. code-block:: sdl - - # Global scalar variable that can be set in a session: - global current_user_id -> uuid; - # Global computed object based on that: - global current_user := ( - select User filter .id = global current_user_id - ); - - -The valid SDL sub-declarations are listed below: - -:eql:synopsis:`default := ` - Specifies the default value for the global variable as an EdgeQL - expression. The default value is used by the session if the value was not - explicitly specified or by the client or was reset with the :ref:`reset - ` command. - -:sdl:synopsis:`` - Set global variable :ref:`annotation ` - to a given *value*. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Globals ` - * - :ref:`DDL > Globals ` diff --git a/docs/reference/sdl/index.rst b/docs/reference/sdl/index.rst index b008a32eebb..266329183d2 100644 --- a/docs/reference/sdl/index.rst +++ b/docs/reference/sdl/index.rst @@ -22,38 +22,6 @@ keywords omitted. The typical SDL structure is to use :ref:`module blocks ` with the rest of the declarations being nested in their respective modules. -.. versionadded:: 3.0 - - |EdgeDB| 3.0 introduces a new SDL syntax which diverges slightly from DDL. - The old SDL syntax is still fully supported, but the new syntax allows for - cleaner and less verbose expression of your schemas. - - * Pointers no longer require an arrow (``->``). You may instead use a colon - after the name of the link or property. - * The ``link`` and ``property`` keywords are now optional for non-computed - pointers when the target type is explicitly specified. - - That means that this type definition: - - .. code-block:: sdl - - type User { - required property email -> str; - } - - could be replaced with this equivalent one in |EdgeDB| 3+ / Gel: - - .. code-block:: sdl - - type User { - required email: str; - } - - When reading our documentation, the version selection dropdown will update - the syntax of most SDL examples to the preferred syntax for the version - selected. This is only true for versioned sections of the documentation. - - Since SDL is declarative in nature, the specific order of declarations of module blocks or individual items does not matter. @@ -102,19 +70,4 @@ to the previous migration: :maxdepth: 3 :hidden: - modules - objects - scalars - links properties - aliases - indexes - constraints - annotations - globals - access_policies - functions - triggers - mutation_rewrites - extensions - future diff --git a/docs/reference/sdl/indexes.rst b/docs/reference/sdl/indexes.rst deleted file mode 100644 index 7347dea6efd..00000000000 --- a/docs/reference/sdl/indexes.rst +++ /dev/null @@ -1,76 +0,0 @@ -.. _ref_eql_sdl_indexes: - -======= -Indexes -======= - -This section describes the SDL declarations pertaining to -:ref:`indexes `. - - -Example -------- - -Declare an index for a "User" based on the "name" property: - -.. code-block:: sdl - - type User { - required name: str; - address: str; - - multi friends: User; - - # define an index for User based on name - index on (.name) { - annotation title := 'User name index'; - } - } - -.. _ref_eql_sdl_indexes_syntax: - -Syntax ------- - -Define a new index corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - index on ( ) - [ except ( ) ] - [ "{" "}" ] ; - - -Description ------------ - -This declaration defines a new index with the following options: - -:sdl:synopsis:`on ( )` - The specific expression for which the index is made. - - The expression must be :ref:`Immutable ` but may - refer to the indexed object's properties and links. - - Note also that ```` itself has to be parenthesized. - -:eql:synopsis:`except ( )` - An optional expression defining a condition to create exceptions - to the index. If ```` evaluates to ``true``, - the object is omitted from the index. If it evaluates - to ``false`` or ``{}``, it appears in the index. - -The valid SDL sub-declarations are listed below: - -:sdl:synopsis:`` - Set index :ref:`annotation ` - to a given *value*. - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Indexes ` - * - :ref:`DDL > Indexes ` - * - :ref:`Introspection > Indexes ` diff --git a/docs/reference/sdl/links.rst b/docs/reference/sdl/links.rst deleted file mode 100644 index 52bba673eb9..00000000000 --- a/docs/reference/sdl/links.rst +++ /dev/null @@ -1,210 +0,0 @@ -.. _ref_eql_sdl_links: - -===== -Links -===== - -This section describes the SDL declarations pertaining to -:ref:`links `. - - -Examples --------- - -Declare an *abstract* link "friends_base" with a helpful title: - -.. code-block:: sdl - - abstract link friends_base { - # declare a specific title for the link - annotation title := 'Close contacts'; - } - -Declare a *concrete* link "friends" within a "User" type: - -.. code-block:: sdl - - type User { - required name: str; - address: str; - # define a concrete link "friends" - multi friends: User { - extending friends_base; - }; - - index on (__subject__.name); - } - -.. _ref_eql_sdl_links_overloading: - -Overloading -~~~~~~~~~~~ - -Any time that the SDL declaration refers to an inherited link that is -being overloaded (by adding more constraints or changing the target -type, for example), the ``overloaded`` keyword must be used. This is -to prevent unintentional overloading due to name clashes: - -.. code-block:: sdl - - abstract type Friendly { - # this type can have "friends" - multi friends: Friendly; - } - - type User extending Friendly { - # overload the link target to be User, specifically - overloaded multi friends: User; - # ... other links and properties - } - -.. _ref_eql_sdl_links_syntax: - -Syntax ------- - -Define a new link corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - # Concrete link form used inside type declaration: - [ overloaded ] [{required | optional}] [{single | multi}] - [ link ] : - [ "{" - [ extending [, ...] ; ] - [ default := ; ] - [ readonly := {true | false} ; ] - [ on target delete ; ] - [ on source delete ; ] - [ ] - [ ] - [ ] - ... - "}" ] - - - # Computed link form used inside type declaration: - [{required | optional}] [{single | multi}] - [ link ] := ; - - # Computed link form used inside type declaration (extended): - [ overloaded ] [{required | optional}] [{single | multi}] - link [: ] - [ "{" - using () ; - [ extending [, ...] ; ] - [ ] - [ ] - ... - "}" ] - - # Abstract link form: - abstract link - [ "{" - [extending [, ...] ; ] - [ readonly := {true | false} ; ] - [ ] - [ ] - [ ] - [ ] - ... - "}" ] - - -Description ------------ - -There are several forms of link declaration, as shown in the syntax synopsis -above. The first form is the canonical definition form, the second form is used -for defining a :ref:`computed link `, and the last form -is used to define an abstract link. The abstract form allows declaring the link -directly inside a :ref:`module `. Concrete link forms are -always used as sub-declarations of an :ref:`object type -`. - -The following options are available: - -:eql:synopsis:`overloaded` - If specified, indicates that the link is inherited and that some - feature of it may be altered in the current object type. It is an - error to declare a link as *overloaded* if it is not inherited. - -:eql:synopsis:`required` - If specified, the link is considered *required* for the parent - object type. It is an error for an object to have a required - link resolve to an empty value. Child links **always** inherit - the *required* attribute, i.e it is not possible to make a - required link non-required by extending it. - -:eql:synopsis:`optional` - This is the default qualifier assumed when no qualifier is - specified, but it can also be specified explicitly. The link is - considered *optional* for the parent object type, i.e. it is - possible for the link to resolve to an empty value. - -:eql:synopsis:`multi` - Specifies that there may be more than one instance of this link - in an object, in other words, ``Object.link`` may resolve to a set - of a size greater than one. - -:eql:synopsis:`single` - Specifies that there may be at most *one* instance of this link - in an object, in other words, ``Object.link`` may resolve to a set - of a size not greater than one. ``single`` is assumed if nether - ``multi`` nor ``single`` qualifier is specified. - -:eql:synopsis:`extending [, ...]` - Optional clause specifying the *parents* of the new link item. - - Use of ``extending`` creates a persistent schema relationship - between the new link and its parents. Schema modifications - to the parent(s) propagate to the child. - - If the same *property* name exists in more than one parent, or - is explicitly defined in the new link and at least one parent, - then the data types of the property targets must be *compatible*. - If there is no conflict, the link properties are merged to form a - single property in the new link item. - -:eql:synopsis:`` - The type must be a valid :ref:`type expression ` - denoting an object type. - -The valid SDL sub-declarations are listed below: - -:eql:synopsis:`default := ` - Specifies the default value for the link as an EdgeQL expression. - The default value is used in an ``insert`` statement if an explicit - value for this link is not specified. - - The expression must be :ref:`Stable `. - -:eql:synopsis:`readonly := {true | false}` - If ``true``, the link is considered *read-only*. Modifications - of this link are prohibited once an object is created. All of the - derived links **must** preserve the original *read-only* value. - -:sdl:synopsis:`` - Set link :ref:`annotation ` - to a given *value*. - -:sdl:synopsis:`` - Define a concrete :ref:`property ` on the link. - -:sdl:synopsis:`` - Define a concrete :ref:`constraint ` on the link. - -:sdl:synopsis:`` - Define an :ref:`index ` for this abstract - link. Note that this index can only refer to link properties. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Links ` - * - :ref:`DDL > Links ` - * - :ref:`Introspection > Object types - ` diff --git a/docs/reference/sdl/modules.rst b/docs/reference/sdl/modules.rst deleted file mode 100644 index 81e3dff197b..00000000000 --- a/docs/reference/sdl/modules.rst +++ /dev/null @@ -1,123 +0,0 @@ -.. _ref_eql_sdl_modules: - -======= -Modules -======= - -This section describes the SDL commands pertaining to -:ref:`modules `. - - -Example -------- - -Declare an empty module: - -.. code-block:: sdl - - module my_module {} - - -Declare a module with some content: - -.. code-block:: sdl - - module my_module { - type User { - required name: str; - } - } - -Syntax ------- - -Define a module corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - module "{" - [ ] - ... - "}" - -.. versionadded:: 3.0 - - Define a nested module. - - .. sdl:synopsis:: - - module "{" - [ ] - module "{" - [ ] - "}" - ... - "}" - - -Description ------------ - -The module block declaration defines a new module similar to the -:eql:stmt:`create module` command, but it also allows putting the -module content as nested declarations: - -:sdl:synopsis:`` - Define various schema items that belong to this module. - -Unlike :eql:stmt:`create module` command, a module block with the -same name can appear multiple times in an SDL document. In that case -all blocks with the same name are merged into a single module under -that name. For example: - -.. code-block:: sdl - - module my_module { - abstract type Named { - required name: str; - } - } - - module my_module { - type User extending Named; - } - -The above is equivalent to: - -.. code-block:: sdl - - module my_module { - abstract type Named { - required name: str; - } - - type User extending Named; - } - -Typically, in the documentation examples of SDL the *module block* is -omitted and instead its contents are described without assuming which -specific module they belong to. - -It's also possible to declare modules implicitly. In this style SDL -declaration uses :ref:`fully-qualified -name` for the item that is being -declared. The *module* part of the *fully-qualified* name implies -that a module by that name will be automatically created in the -schema. The following declaration is equivalent to the previous -examples, but it declares module ``my_module`` implicitly: - -.. code-block:: sdl - - abstract type my_module::Named { - required name: str; - } - - type my_module::User extending my_module::Named; - -.. versionadded:: 3.0 - - A module block can be nested inside another module block to create a nested - module. If you want reference an entity in a nested module by its - fully-qualified name, you will need to reference all of the containing - modules' names: ``::::`` diff --git a/docs/reference/sdl/mutation_rewrites.rst b/docs/reference/sdl/mutation_rewrites.rst deleted file mode 100644 index 2a791130d77..00000000000 --- a/docs/reference/sdl/mutation_rewrites.rst +++ /dev/null @@ -1,64 +0,0 @@ -.. _ref_eql_sdl_mutation_rewrites: - -================= -Mutation rewrites -================= - -This section describes the SDL declarations pertaining to -:ref:`mutation rewrites `. - - -Example -------- - -Declare two mutation rewrites: one that sets a ``created`` property when a new -object is inserted and one that sets a ``modified`` property on each update: - -.. code-block:: sdl - - type User { - created: datetime { - rewrite insert using (datetime_of_statement()); - } - modified: datetime { - rewrite update using (datetime_of_statement()); - } - }; - -.. _ref_eql_sdl_mutation_rewrites_syntax: - -Syntax ------- - -Define a new mutation rewrite corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - rewrite {insert | update} [, ...] - using - -Mutation rewrites must be defined inside a property or link block. - - -Description ------------ - -This declaration defines a new trigger with the following options: - -:eql:synopsis:`insert | update [, ...]` - The query type (or types) the rewrite runs on. Separate multiple values - with commas to invoke the same rewrite for multiple types of queries. - -:eql:synopsis:`` - The expression to be evaluated to produce the new value of the property. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Mutation rewrites ` - * - :ref:`DDL > Mutation rewrites ` - * - :ref:`Introspection > Mutation rewrites - ` diff --git a/docs/reference/sdl/objects.rst b/docs/reference/sdl/objects.rst deleted file mode 100644 index 97f362876a1..00000000000 --- a/docs/reference/sdl/objects.rst +++ /dev/null @@ -1,132 +0,0 @@ -.. _ref_eql_sdl_object_types: - -============ -Object Types -============ - -This section describes the SDL declarations pertaining to -:ref:`object types `. - - -Example -------- - -Consider a ``User`` type with a few properties: - -.. code-block:: sdl - - type User { - # define some properties and a link - required name: str; - address: str; - - multi friends: User; - - # define an index for User based on name - index on (__subject__.name); - } - -.. _ref_eql_sdl_object_types_inheritance: - -An alternative way to define the same ``User`` type could be by using -abstract types. These abstract types can then be re-used in other type -definitions as well: - -.. code-block:: sdl - - abstract type Named { - required name: str; - } - - abstract type HasAddress { - address: str; - } - - type User extending Named, HasAddress { - # define some user-specific properties and a link - multi friends: User; - - # define an index for User based on name - index on (__subject__.name); - } - -Introducing abstract types opens up the possibility of -:ref:`polymorphic queries `. - - -.. _ref_eql_sdl_object_types_syntax: - -Syntax ------- - -Define a new object type corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - [abstract] type [extending [, ...] ] - [ "{" - [ ] - [ ] - [ ] - [ ] - [ ] - ... - "}" ] - -Description ------------ - -This declaration defines a new object type with the following options: - -:eql:synopsis:`abstract` - If specified, the created type will be *abstract*. - -:eql:synopsis:`` - The name (optionally module-qualified) of the new type. - -:eql:synopsis:`extending [, ...]` - Optional clause specifying the *supertypes* of the new type. - - Use of ``extending`` creates a persistent type relationship - between the new subtype and its supertype(s). Schema modifications - to the supertype(s) propagate to the subtype. - - References to supertypes in queries will also include objects of - the subtype. - - If the same *link* name exists in more than one supertype, or - is explicitly defined in the subtype and at least one supertype, - then the data types of the link targets must be *compatible*. - If there is no conflict, the links are merged to form a single - link in the new type. - -These sub-declarations are allowed in the ``Type`` block: - -:sdl:synopsis:`` - Set object type :ref:`annotation ` - to a given *value*. - -:sdl:synopsis:`` - Define a concrete :ref:`property ` for this object type. - -:sdl:synopsis:`` - Define a concrete :ref:`link ` for this object type. - -:sdl:synopsis:`` - Define a concrete :ref:`constraint ` for this - object type. - -:sdl:synopsis:`` - Define an :ref:`index ` for this object type. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Object types ` - * - :ref:`DDL > Object types ` - * - :ref:`Introspection > Object types - ` - * - :ref:`Cheatsheets > Object types ` diff --git a/docs/reference/sdl/scalars.rst b/docs/reference/sdl/scalars.rst deleted file mode 100644 index de31b6f1187..00000000000 --- a/docs/reference/sdl/scalars.rst +++ /dev/null @@ -1,70 +0,0 @@ -.. _ref_eql_sdl_scalars: - -============ -Scalar Types -============ - -This section describes the SDL declarations pertaining to -:ref:`scalar types `. - - -Example -------- - -Declare a new non-negative integer type: - -.. code-block:: sdl - - scalar type posint64 extending int64 { - constraint min_value(0); - } - -.. _ref_eql_sdl_scalars_syntax: - -Syntax ------- - -Define a new scalar type corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - [abstract] scalar type [extending [, ...] ] - [ "{" - [ ] - [ ] - ... - "}" ] - - -Description ------------ - -This declaration defines a new object type with the following options: - -:eql:synopsis:`abstract` - If specified, the created scalar type will be *abstract*. - -:eql:synopsis:`` - The name (optionally module-qualified) of the new scalar type. - -:eql:synopsis:`extending ` - Optional clause specifying the *supertype* of the new type. - - If :eql:synopsis:`` is an - :eql:type:`enumerated type ` declaration then - an enumerated scalar type is defined. - - Use of ``extending`` creates a persistent type relationship - between the new subtype and its supertype(s). Schema modifications - to the supertype(s) propagate to the subtype. - -The valid SDL sub-declarations are listed below: - -:sdl:synopsis:`` - Set scalar type :ref:`annotation ` - to a given *value*. - -:sdl:synopsis:`` - Define a concrete :ref:`constraint ` for - this scalar type. diff --git a/docs/reference/sdl/triggers.rst b/docs/reference/sdl/triggers.rst deleted file mode 100644 index b498d5fb54f..00000000000 --- a/docs/reference/sdl/triggers.rst +++ /dev/null @@ -1,112 +0,0 @@ -.. _ref_eql_sdl_triggers: - -======== -Triggers -======== - -This section describes the SDL declarations pertaining to -:ref:`triggers `. - - -Example -------- - -Declare a trigger that inserts a ``Log`` object for each new ``User`` object: - -.. code-block:: sdl - - type User { - required name: str; - - trigger log_insert after insert for each do ( - insert Log { - action := 'insert', - target_name := __new__.name - } - ); - } - -.. versionadded:: 4.0 - -Declare a trigger that inserts a ``Log`` object conditionally when an update -query makes a change to a ``User`` object: - -.. code-block:: sdl - - type User { - required name: str; - - trigger log_update after update for each - when (__old__ {**} != __new__ {**}) - do ( - insert Log { - action := 'update', - target_name := __new__.name, - change := __old__.name ++ '->' ++ __new__.name - } - ); - } - -.. _ref_eql_sdl_triggers_syntax: - -Syntax ------- - -Define a new trigger corresponding to the :ref:`more explicit DDL -commands `. - -.. sdl:synopsis:: - - type "{" - trigger - after - {insert | update | delete} [, ...] - for {each | all} - [ when () ] - do - "}" - - -Description ------------ - -This declaration defines a new trigger with the following options: - -:eql:synopsis:`` - The name (optionally module-qualified) of the type to be triggered on. - -:eql:synopsis:`` - The name of the trigger. - -:eql:synopsis:`insert | update | delete [, ...]` - The query type (or types) to trigger on. Separate multiple values with - commas to invoke the same trigger for multiple types of queries. - -:eql:synopsis:`each` - The expression will be evaluated once per modified object. ``__new__`` and - ``__old__`` in this context within the expression will refer to a single - object. - -:eql:synopsis:`all` - The expression will be evaluted once for the entire query, even if multiple - objects were modified. ``__new__`` and ``__old__`` in this context within - the expression refer to sets of the modified objects. - -:eql:synopsis:`when ()` - Optionally provide a condition for the trigger. If the condition is - met, the trigger will run. If not, the trigger is skipped. - -:eql:synopsis:`` - The expression to be evaluated when the trigger is invoked. - -The trigger name must be distinct from that of any existing trigger -on the same type. - - -.. list-table:: - :class: seealso - - * - **See also** - * - :ref:`Schema > Triggers ` - * - :ref:`DDL > Triggers ` - * - :ref:`Introspection > Triggers ` diff --git a/docs/reference/sql_adapter.rst b/docs/reference/sql_adapter.rst index d1255dd759c..74c2a1bc68f 100644 --- a/docs/reference/sql_adapter.rst +++ b/docs/reference/sql_adapter.rst @@ -20,9 +20,8 @@ managed in |.gel| files using Gel Schema Definition Language and migration commands. Any Postgres-compatible client can connect to an Gel database by using the -same port that is used for the Gel protocol and the -:versionreplace:`database;5.0:branch` name, username, and password already used -for the database. +same port that is used for the Gel protocol and the |branch| name, username, +and password already used for the database. .. versionchanged:: 5.0 diff --git a/docs/stdlib/cfg.rst b/docs/stdlib/cfg.rst index 34f9f2042e7..2cf4eadd91b 100644 --- a/docs/stdlib/cfg.rst +++ b/docs/stdlib/cfg.rst @@ -317,35 +317,15 @@ Client connections ---------- -.. eql:type:: cfg::DatabaseConfig - - The :versionreplace:`database;5.0:branch`-level configuration object type. - - This type will have only one object instance. The ``cfg::DatabaseConfig`` - object represents the state of :versionreplace:`database;5.0:branch` and - instance-level Gel configuration. - - For overall configuration state please refer to the :eql:type:`cfg::Config` - instead. - - .. versionadded:: 5.0 - - As of |EdgeDB| 5.0, this config object represents database *branch* - and instance-level configuration. - - ----------- - - .. eql:type:: cfg::BranchConfig .. versionadded:: 5.0 - The :versionreplace:`database;5.0:branch`-level configuration object type. + The branch-level configuration object type. This type will have only one object instance. The ``cfg::BranchConfig`` - object represents the state of :versionreplace:`database;5.0:branch` and - instance-level Gel configuration. + object represents the state of the branch and instance-level Gel + configuration. For overall configuration state please refer to the :eql:type:`cfg::Config` instead. diff --git a/docs/stdlib/deprecated.rst b/docs/stdlib/deprecated.rst index a26722da047..46cde1f5082 100644 --- a/docs/stdlib/deprecated.rst +++ b/docs/stdlib/deprecated.rst @@ -135,3 +135,14 @@ Deprecated {'data'} db> select str_rtrim('data.:.:.', '.:'); {'data'} + +---------- + +.. eql:type:: cfg::DatabaseConfig + + The branch-level configuration object type. + + As of |EdgeDB| 5.0, this config object represents database *branch* + and instance-level configuration. + + **Use the identical** :eql:type:`cfg::BranchConfig` instead. diff --git a/docs/stdlib/scalar_table.rst b/docs/stdlib/scalar_table.rst deleted file mode 100644 index 31dc6f6b7a4..00000000000 --- a/docs/stdlib/scalar_table.rst +++ /dev/null @@ -1,63 +0,0 @@ -.. list-table:: - - * - :eql:type:`str` - - A variable-length string - - * - :eql:type:`bool` - - Logical boolean (true/false) - - * - :eql:type:`int16` - - 16-bit integer - - * - :eql:type:`int32` - - 32-bit integer - - * - :eql:type:`int64` - - 64-bit integer - - * - :eql:type:`float32` - - 32-bit floating point number - - * - :eql:type:`float64` - - 64-bit floating point number - - * - :eql:type:`bigint` - - Arbitrary precision integer - - * - :eql:type:`decimal` - - Arbitrary precision number - - * - :eql:type:`json` - - Arbitrary JSON data - - * - :eql:type:`uuid` - - UUID type - - * - :eql:type:`bytes` - - Raw binary data - - * - :eql:type:`datetime` - - Timezone-aware point in time - - * - :eql:type:`duration` - - Absolute time span - - * - :eql:type:`cal::local_datetime` - - Date and time without timezone - - * - :eql:type:`cal::local_date` - - Date type - - * - :eql:type:`cal::local_time` - - Time type - - * - :eql:type:`cal::relative_duration` - - Relative time span (in months, days, and seconds) - - * - :eql:type:`cal::date_duration` - - Relative time span (in months and days only) - - * - :eql:type:`sequence` - - Auto-incrementing sequence of ``int64`` - - diff --git a/edb/tools/docs/edb.py b/edb/tools/docs/edb.py index 7761d86ec33..8ae53de86de 100644 --- a/edb/tools/docs/edb.py +++ b/edb/tools/docs/edb.py @@ -151,7 +151,7 @@ def apply(self): for node in self.document.traverse(d_nodes.substitution_reference): nt = node.astext() if nt.lower() in { - "gel", "gel's","edgedb", "gelcmd", ".gel", "gel.toml", + "gel", "gel's", "edgedb", "gelcmd", ".gel", "gel.toml", "gel-server", "geluri", "admin", "main", "branch", "branches" }: @@ -182,6 +182,7 @@ def apply(self): else: node.replace_self(d_nodes.Text(nt)) + class GelCmdRole: def __call__(