feat(ingestion/glue): Implement unified AWS Glue table representation

[svdimchenko]

Problem statement

We are currently using following tech stack:

• kafka
• s3 + external tables defined in glue catalog.
• dbt
• airflow
• tableau

We want to have the full data lineage, however it's broken on glue tables -> dbt stage.
Actually glue ingestor ignores athena's catalog name and stores entities in format database.table, however actual representation in case when platform='athena' should be catalog.database.table.

As I dig deeper, I understand that currently catalog entity is missed for glue ingest type.
So adding catalog at database level and database at datahub's schema level seems logical to me. However this brings breaking changes to the current implementation.
WDYT regarding the issue ? Once we finalise the vision, I can improve the current PR.

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues: Glue ingested tables in lineage view shows fullname with arn #6182
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

[codecov]

Codecov Report

Attention: Patch coverage is 83.78378% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ingestion/src/datahub/ingestion/source/aws/glue.py	84.84%	5 Missing ⚠️
...ion/src/datahub/ingestion/source/aws/aws_common.py	75.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
...ion/src/datahub/ingestion/source/aws/aws_common.py	66.19% <75.00%> (ø)
...ingestion/src/datahub/ingestion/source/aws/glue.py	87.82% <84.84%> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef36837...5382a60. Read the comment docs.

[svdimchenko]

closing due to more complex reengineering is required #12410