Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worker Domain Refactor #5338

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Worker Domain Refactor #5338

wants to merge 10 commits into from

Conversation

hugoghx
Copy link
Collaborator

@hugoghx hugoghx commented Dec 6, 2024

No description provided.

This comment has been minimized.

This comment has been minimized.

irenarindos and others added 9 commits December 6, 2024 15:38
This commit moves all the logic to select a worker to handle a session
to the internal/server package. Most functions were moved to this
package with no changes, however, there are a couple of notable
exceptions:

- Given that we don't use GRPC errors in repos, all errors were adjusted
  to use the internal/errors package instead.
- StorageBucketFilterCredIdFn is now used to prevent the internal/server
  package from importing internal/storage/plugin, which would cause a
  dep cycle. Callers of SelectSessionWorkers must implement this func
  which is responsible for returning a storage bucket's worker filter as
  well as its associated credential id.
- authorizeSessionWithWorkerFilter now receives an
  internal/server.(*Repository) object when called. This is transparent
  to SelectSessionWorkers callers as it is handled internally.
@hugoghx hugoghx force-pushed the oss/llb-worker-domain-refactor branch from 9ada784 to 2ccccb6 Compare December 6, 2024 15:38
Copy link

github-actions bot commented Dec 6, 2024

Database schema diff between main and oss/llb-worker-domain-refactor @ 2ccccb6

To understand how these diffs are generated and some limitations see the
documentation of the script.

Functions

Unchanged

Tables

diff --git a/.schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/public server_worker_tag.sql b/.schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/public server_worker_api_tag.sql
similarity index 100%
rename from .schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/public server_worker_tag.sql
rename to .schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/public server_worker_api_tag.sql
diff --git a/.schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/public server_worker_tag_enm.sql b/.schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/public server_worker_config_tag.sql
similarity index 100%
rename from .schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/public server_worker_tag_enm.sql
rename to .schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/public server_worker_config_tag.sql
diff --git a/.schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag.sql b/.schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_api_tag.sql
similarity index 64%
rename from .schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag.sql
rename to .schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_api_tag.sql
index eaaf39ea2..600ea2f15 100644
--- a/.schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag.sql
+++ b/.schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_api_tag.sql
@@ -21,17 +21,23 @@ set default_tablespace = '';
 set default_table_access_method = heap;
 
 --
--- name: server_worker_tag; type: table; schema: public; owner: -
+-- name: server_worker_api_tag; type: table; schema: public; owner: -
 --
 
-create table public.server_worker_tag (
+create table public.server_worker_api_tag (
     worker_id public.wt_public_id not null,
     key public.wt_tagpair not null,
-    value public.wt_tagpair not null,
-    source text not null
+    value public.wt_tagpair not null
 );
 
 
+--
+-- name: table server_worker_api_tag; type: comment; schema: public; owner: -
+--
+
+comment on table public.server_worker_api_tag is 'server_worker_api_tag is a table where each row represents a worker api tag.';
+
+
 --
 -- postgresql database dump complete
 --
diff --git a/.schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm.sql b/.schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_config_tag.sql
similarity index 56%
rename from .schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm.sql
rename to .schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_config_tag.sql
index 7a28a8530..6395667ec 100644
--- a/.schema-diff/tables_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm.sql
+++ b/.schema-diff/tables_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_config_tag.sql
@@ -21,15 +21,23 @@ set default_tablespace = '';
 set default_table_access_method = heap;
 
 --
--- name: server_worker_tag_enm; type: table; schema: public; owner: -
+-- name: server_worker_config_tag; type: table; schema: public; owner: -
 --
 
-create table public.server_worker_tag_enm (
-    source text not null,
-    constraint only_predefined_server_worker_tag_sources_allowed check ((source = any (array['configuration'::text, 'api'::text])))
+create table public.server_worker_config_tag (
+    worker_id public.wt_public_id not null,
+    key public.wt_tagpair not null,
+    value public.wt_tagpair not null
 );
 
 
+--
+-- name: table server_worker_config_tag; type: comment; schema: public; owner: -
+--
+
+comment on table public.server_worker_config_tag is 'server_worker_config_tag is a table where each row represents a worker config tag.';
+
+
 --
 -- postgresql database dump complete
 --

Views

diff --git a/.schema-diff/views_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_aggregate.sql b/.schema-diff/views_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_aggregate.sql
deleted file mode 100644
index 789833e78..000000000
--- a/.schema-diff/views_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_aggregate.sql
+++ /dev/null
@@ -1,69 +0,0 @@
---
--- postgresql database dump
---
-
--- dumped from database version 13.18
--- dumped by pg_dump version 16.6 (ubuntu 16.6-1.pgdg24.04+1)
-
-set statement_timeout = 0;
-set lock_timeout = 0;
-set idle_in_transaction_session_timeout = 0;
-set client_encoding = 'utf8';
-set standard_conforming_strings = on;
-select pg_catalog.set_config('search_path', '', false);
-set check_function_bodies = false;
-set xmloption = content;
-set client_min_messages = warning;
-set row_security = off;
-
---
--- name: server_worker_aggregate; type: view; schema: public; owner: -
---
-
-create view public.server_worker_aggregate as
- with worker_config_tags(worker_id, source, tags) as (
-         select ct_1.worker_id,
-            ct_1.source,
-            string_agg(distinct concat_ws('y'::text, ct_1.key, ct_1.value), 'z'::text) as tags
-           from public.server_worker_tag ct_1
-          group by ct_1.worker_id, ct_1.source
-        ), connection_count(worker_id, count) as (
-         select session_connection.worker_id,
-            count(1) as count
-           from public.session_connection
-          where (session_connection.closed_reason is null)
-          group by session_connection.worker_id
-        )
- select w.public_id,
-    w.scope_id,
-    w.description,
-    w.name,
-    w.address,
-    w.create_time,
-    w.update_time,
-    w.version,
-    w.last_status_time,
-    w.type,
-    w.release_version,
-    w.operational_state,
-    w.local_storage_state,
-    cc.count as active_connection_count,
-    wt.tags as api_tags,
-    ct.tags as worker_config_tags
-   from (((public.server_worker w
-     left join worker_config_tags wt on ((((w.public_id)::text = (wt.worker_id)::text) and (wt.source = 'api'::text))))
-     left join worker_config_tags ct on ((((w.public_id)::text = (ct.worker_id)::text) and (ct.source = 'configuration'::text))))
-     left join connection_count cc on (((w.public_id)::text = (cc.worker_id)::text)));
-
-
---
--- name: view server_worker_aggregate; type: comment; schema: public; owner: -
---
-
-comment on view public.server_worker_aggregate is 'server_worker_aggregate contains the worker resource with its worker provided config values and its configuration and api provided tags.';
-
-
---
--- postgresql database dump complete
---
-

Triggers

Unchanged

Indexes

Unchanged

Constraints

diff --git a/.schema-diff/constraints_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_api_tag_pkey.sql b/.schema-diff/constraints_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_api_tag_pkey.sql
new file mode 100644
index 000000000..079266134
--- /dev/null
+++ b/.schema-diff/constraints_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_api_tag_pkey.sql
@@ -0,0 +1,2 @@
+-- name: server_worker_api_tag server_worker_api_tag_pkey; type: constraint; schema: public; owner: -
+    add constraint server_worker_api_tag_pkey primary key (worker_id, key, value);
diff --git a/.schema-diff/constraints_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_config_tag_pkey.sql b/.schema-diff/constraints_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_config_tag_pkey.sql
new file mode 100644
index 000000000..50e2bb0bf
--- /dev/null
+++ b/.schema-diff/constraints_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_config_tag_pkey.sql
@@ -0,0 +1,2 @@
+-- name: server_worker_config_tag server_worker_config_tag_pkey; type: constraint; schema: public; owner: -
+    add constraint server_worker_config_tag_pkey primary key (worker_id, key, value);
diff --git a/.schema-diff/constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm_pkey.sql b/.schema-diff/constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm_pkey.sql
deleted file mode 100644
index 5663d8b58..000000000
--- a/.schema-diff/constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm_pkey.sql
+++ /dev/null
@@ -1,2 +0,0 @@
--- name: server_worker_tag_enm server_worker_tag_enm_pkey; type: constraint; schema: public; owner: -
-    add constraint server_worker_tag_enm_pkey primary key (source);
diff --git a/.schema-diff/constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_pkey.sql b/.schema-diff/constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_pkey.sql
deleted file mode 100644
index d8ece81fe..000000000
--- a/.schema-diff/constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_pkey.sql
+++ /dev/null
@@ -1,2 +0,0 @@
--- name: server_worker_tag server_worker_tag_pkey; type: constraint; schema: public; owner: -
-    add constraint server_worker_tag_pkey primary key (worker_id, key, value, source);

Foreign Key Constraints

diff --git a/.schema-diff/fk_constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_fkey.sql b/.schema-diff/fk_constraints_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_fkey.sql
index 7723af6cf..6fe8e49c9 100644
--- a/.schema-diff/fk_constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_fkey.sql
+++ b/.schema-diff/fk_constraints_c73574f3e9f1f1f18a3e09675f58b14a2d276ada/server_worker_fkey.sql
@@ -1,5 +1,3 @@
--- name: server_worker_tag server_worker_fkey; type: fk constraint; schema: public; owner: -
-    add constraint server_worker_fkey foreign key (worker_id) references public.server_worker(public_id) on update cascade on delete cascade;
 -- name: session_connection server_worker_fkey; type: fk constraint; schema: public; owner: -
     add constraint server_worker_fkey foreign key (worker_id) references public.server_worker(public_id) on update cascade on delete set null;
 -- name: worker_auth_authorized server_worker_fkey; type: fk constraint; schema: public; owner: -
@@ -10,3 +8,7 @@
     add constraint server_worker_fkey foreign key (worker_id) references public.server_worker(public_id) on update cascade on delete cascade;
 -- name: worker_storage_bucket_credential_state server_worker_fkey; type: fk constraint; schema: public; owner: -
     add constraint server_worker_fkey foreign key (worker_id) references public.server_worker(public_id) on update cascade on delete cascade;
+-- name: server_worker_config_tag server_worker_fkey; type: fk constraint; schema: public; owner: -
+    add constraint server_worker_fkey foreign key (worker_id) references public.server_worker(public_id) on update cascade on delete cascade;
+-- name: server_worker_api_tag server_worker_fkey; type: fk constraint; schema: public; owner: -
+    add constraint server_worker_fkey foreign key (worker_id) references public.server_worker(public_id) on update cascade on delete cascade;
diff --git a/.schema-diff/fk_constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm_fkey.sql b/.schema-diff/fk_constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm_fkey.sql
deleted file mode 100644
index 65d35df9f..000000000
--- a/.schema-diff/fk_constraints_5896a3b44b650875b9bad08d5b0c7d8d8fe2ba4c/server_worker_tag_enm_fkey.sql
+++ /dev/null
@@ -1,2 +0,0 @@
--- name: server_worker_tag server_worker_tag_enm_fkey; type: fk constraint; schema: public; owner: -
-    add constraint server_worker_tag_enm_fkey foreign key (source) references public.server_worker_tag_enm(source) on update cascade on delete restrict;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants