Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(pipeline) : Corriger l'intégration des contacts Brevo #281

Merged
merged 3 commits into from
Aug 30, 2024
Merged

chore(pipeline) : Corriger l'intégration des contacts Brevo #281

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d218563
chore(pipeline) : Make the Dora contact source work again
vperron Aug 28, 2024
9e729df
chore(pipeline) : Document Brevo fields better
vperron Aug 28, 2024
595bfcb
chore(pipeline) : Impove Dora blocked contacts resolution
vperron Aug 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions pipeline/dags/import_brevo.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,24 @@ def _load_rgpd_contacts(run_id: str, stream_id: str, source_id: str, logical_dat
op_kwargs={"stream_id": STREAM_ID, "source_id": SOURCE_ID},
)

dbt_build_brevo = dbt_operator_factory(
dbt_build_brevo_tmp = dbt_operator_factory(
task_id="dbt_build_brevo",
command="build",
select="path:models/staging/brevo path:models/intermediate/brevo",
dbt_vars={"build_intermediate_tmp": True},
)

(start >> extract_rgpd_contacts >> load_rgpd_contacts >> dbt_build_brevo >> end)
dbt_run_brevo = dbt_operator_factory(
task_id="dbt_run_brevo",
command="run",
select="path:models/staging/brevo path:models/intermediate/brevo",
)

(
start
>> extract_rgpd_contacts
>> load_rgpd_contacts
>> dbt_build_brevo_tmp
>> dbt_run_brevo
>> end
)
10 changes: 10 additions & 0 deletions pipeline/dbt/models/intermediate/brevo/_brevo__models.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ models:
- name: int_brevo__contacts
columns:
- name: id
description: L'ID interne Brevo de ce "contact". Largement inutilisé mais sert de clé primaire.
data_tests:
- unique
- not_null
Expand All @@ -14,7 +15,16 @@ models:
- dbt_utils.not_constant
- dbt_utils.not_empty_string
- name: est_interdit
description: |
Signifie que cet e-mail est considéré par Brevo comme:
- un "hard bounce" (n'existe pas)
- a fait l'objet d'une demande de désincription explicite.
- a classé le mail en spam
- a été classé "bloqué" par un administrateur Brevo
data_tests:
- not_null
- accepted_values:
values: [true, false]

- name: date_di_rgpd_opposition
description: Date d'opposition au RGPD telle qu'entrée par notre DPO dans Brevo lorsque signalé.
5 changes: 4 additions & 1 deletion pipeline/dbt/models/intermediate/sources/dora/int_dora__services.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,17 @@ di_profil_by_dora_profil AS (
),

blocked_contacts AS (
SELECT DISTINCT UNNEST(contact_uids) AS id FROM {{ ref('int_brevo__contacts') }} WHERE est_interdit = TRUE OR date_di_rgpd_opposition IS NOT NULL
SELECT DISTINCT UNNEST(contact_uids) AS id
FROM {{ ref('int_brevo__contacts') }}
WHERE est_interdit = TRUE OR date_di_rgpd_opposition IS NOT NULL
),

blocked_contact_uids AS (
SELECT
SPLIT_PART(id, ':', 3) AS id,
SPLIT_PART(id, ':', 2) AS kind
FROM blocked_contacts
WHERE SPLIT_PART(id, ':', 1) = 'dora'
),

blocked_services_uids AS (
Expand Down
Loading