Airflow startup takes long time due to large ab_view_menu and ab_permission_view tables

[nadgirsanket]

Hi Team,

Airflow Version: 1.10.10
Infrastructure: Kubernetes on-prem
Airflow DB: Postgres
Executor: KubernetesExecutor

I have an airflow running with thousands of dynamic dags. First of all, since the number of dags is in thousands, it makes the size of the airflow DB grow rapidly. I am using Postgres DB. I do cleanup dag, dag_run, task_instance, log, job, etc tables regularly to keep the size small. No issues there.

Here's a problem I've noticed. Since the dags are dynamic and names can be different on each run, it has created hundreds of thousands of dags names that are now sitting in ab_view_menu and ab_permission_view tables.

Whenever the webserver pod dies and restarts (may be due to insufficient resources on k8s or some other reason), it takes a huge amount of time to come up. Webserver gets stuck for minutes on end at the below steps.

{security.py:475} INFO - Start syncing user roles.
{security.py:385} INFO - Fetching a set of all permission, view_menu from FAB meta-table
{security.py:328} INFO - Cleaning faulty perms

The code is trying to update roles and menu view permissions for each of those dynamic dags generated which is taking a lot of time to complete. In my case, this seems wasteful as the dags are dynamic and may no longer exist.

Is there a solution to gracefully delete the dynamic dag entries from these tables (ab_view_menu and ab_permission_view)to reduce the size and speedup restart times? For e.g, let's try to delete example_dag_* from these tables.

[yacine-del]

Le même problème au nivau view_menu

sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <PermissionView at 0x7f368c6e6d30> is not bound to a Session; lazy load operation of attribute 'view_menu' cannot proceed (Background on this error at: http://sqlalche.me/e/14/bhk3)