Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(DENG-6890): Update mobile kpi support metric aggregates to support dma analysis #6783

Merged
merged 10 commits into from
Jan 15, 2025
Merged

feat(DENG-6890): Update mobile kpi support metric aggregates to support dma analysis #6783

merged 10 commits into from
Jan 15, 2025

Conversation

kik-kik
Copy link
Contributor

@kik-kik kik-kik commented Jan 9, 2025 •
edited by data-sync-user
Loading

feat(DENG-6890): Update mobile kpi support metric aggregates to support dma analysis

Description

This change aims to add the following two fields to engagement, retention and new_profiles KPI support metric aggregated datasets to support DMA related analysis needed.

┆Issue is synchronized with this Jira Task

@kik-kik kik-kik added the enhancement New feature or request label Jan 9, 2025
@kik-kik kik-kik self-assigned this Jan 9, 2025
@dataops-ci-bot

This comment has been minimized.

Sign in to view
@kik-kik kik-kik force-pushed the feat/DENG-6890/update-mobile-aggregates-to-support-dma-analysis branch from b235de4 to a59527c Compare January 9, 2025 18:09
@dataops-ci-bot

This comment has been minimized.

Sign in to view
@kik-kik kik-kik force-pushed the feat/DENG-6890/update-mobile-aggregates-to-support-dma-analysis branch from 03f6bd3 to 96811a0 Compare January 10, 2025 08:54
@kik-kik kik-kik requested a review from bani January 10, 2025 08:55
@dataops-ci-bot

This comment has been minimized.

Sign in to view
@kik-kik kik-kik force-pushed the feat/DENG-6890/update-mobile-aggregates-to-support-dma-analysis branch from 7b5b181 to c1afd0f Compare January 14, 2025 15:29
@dataops-ci-bot

This comment has been minimized.

Sign in to view
@kik-kik kik-kik force-pushed the feat/DENG-6890/update-mobile-aggregates-to-support-dma-analysis branch from c1afd0f to 14b2235 Compare January 14, 2025 16:51
@kik-kik kik-kik requested a review from bani January 14, 2025 17:02
@kik-kik kik-kik marked this pull request as ready for review January 14, 2025 17:02
kik-kik added 10 commits January 14, 2025 18:05
@kik-kik
feat: add device_type and device_manufacturer to KPI support metrics …
b35f7d0 
…datasets
@kik-kik
feat: ensure all mobile kpi support metric aggregated dataset have de…
ae3ba93 
…vice_type and device_manufacturer dimensions
@kik-kik
feat: add device_type and device_manfacturer to the mobile kpi suppor…
b32f130 
…t metric aggregate queries
@kik-kik
feat: tweak active_users view device_type value setting logic
69dde84
@kik-kik
feat: lower case device_manufacturer for some cases where the same ma…
8f4fc91 
…nufacturer value has different casing
@kik-kik
feat: bucket device_manufacturers into "other" bucket in case when th…
09c05dd 
…ey have a lower occurance to prevent from exploding the aggregate tables
@kik-kik
fix: retention_clients some selection fields being ambiguous
f76df7a
@kik-kik
feat: Use dense rank instead of count for bucketing all device_manufa…
2322162 
…cturers into other bucket if they are rank 151 or more
@kik-kik
fix: using incorrect source table for ranking device_manufacturers
84bc5b0
@kik-kik
feat: make sure that device_manufacturer is lowered prior to joining
b3b5b5a
@kik-kik kik-kik force-pushed the feat/DENG-6890/update-mobile-aggregates-to-support-dma-analysis branch from 14b2235 to b3b5b5a Compare January 14, 2025 17:05
kik-kik
kik-kik commented Jan 14, 2025
View reviewed changes
sql_generators/mobile_kpi_support_metrics/templates/active_users.view.sql
* EXCEPT (isp),
* EXCEPT (isp) REPLACE(
-- Lower device_manufacturer as in some cases the same manufacturer value has different casing.
LOWER(device_manufacturer) AS device_manufacturer
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lowering device_manufacturer here so that everything downstream inherits the lowered values.

@dataops-ci-bot
Copy link

Integration report for "feat: make sure that device_manufacturer is lowered prior to joining"

sql.diff

Click to expand! 
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/new_profile_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_android/engagement_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_android/new_profile_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_android/retention_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_ios/engagement_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_ios/new_profile_clients: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_ios/retention_clients: schema.yaml
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-01-14 17:06:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-01-14 17:16:29.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-01-14 17:06:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-01-14 17:16:24.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/active_users/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/active_users/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/active_users/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/active_users/schema.yaml	2025-01-14 17:17:15.000000000 +0000
@@ -137,3 +137,6 @@
 - name: isp
   type: STRING
   mode: NULLABLE
+- name: device_type
+  type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/active_users/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/active_users/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/active_users/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/active_users/view.sql	2025-01-14 17:16:11.000000000 +0000
@@ -3,7 +3,10 @@
   `moz-fx-data-shared-prod.fenix.active_users`
 AS
 SELECT
-  * EXCEPT (isp),
+  * EXCEPT (isp) REPLACE(
+    -- Lower device_manufacturer as in some cases the same manufacturer value has different casing.
+    LOWER(device_manufacturer) AS device_manufacturer
+  ),
   CASE
     WHEN LOWER(isp) = "browserstack"
       THEN CONCAT("Fenix", " ", isp)
@@ -43,5 +46,16 @@
   -- Adding isp at the end because it's in different column index in baseline table for some products.
   -- This is to make sure downstream union works as intended.
   isp,
+  CASE
+    WHEN normalized_os = "iOS"
+      AND STARTS_WITH(device_model, "iPad")
+      THEN "iPad"
+    WHEN normalized_os = "iOS"
+      AND STARTS_WITH(device_model, "iPhone")
+      THEN "iPhone"
+    WHEN normalized_os = "Android"
+      THEN "Android"
+    ELSE CAST(NULL AS STRING)
+  END AS device_type,
 FROM
   `moz-fx-data-shared-prod.fenix.baseline_clients_last_seen`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,97 +0,0 @@
-fields:
-- name: submission_date
-  type: DATE
-  mode: NULLABLE
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: sample_id
-  type: INTEGER
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: is_dau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_wau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_mau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: play_store_attribution_campaign
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_medium
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_source
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: play_store_attribution_content
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_term
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_install_referrer_response
-  type: STRING
-  mode: NULLABLE
-- name: meta_attribution_app
-  type: STRING
-  mode: NULLABLE
-- name: meta_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: install_source
-  type: STRING
-  mode: NULLABLE
-- name: adjust_ad_group
-  type: STRING
-  mode: NULLABLE
-- name: adjust_campaign
-  type: STRING
-  mode: NULLABLE
-- name: adjust_creative
-  type: STRING
-  mode: NULLABLE
-- name: adjust_network
-  type: STRING
-  mode: NULLABLE
-- name: adjust_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: distribution_id
-  type: STRING
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
-- name: lifecycle_stage
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql	2025-01-14 17:16:12.000000000 +0000
@@ -18,6 +18,8 @@
     is_wau,
     is_mau,
     is_mobile,
+    device_type,
+    device_manufacturer,
   FROM
     `moz-fx-data-shared-prod.fenix.active_users`
 ),
@@ -88,6 +90,8 @@
       THEN 'existing_user'
     ELSE 'Unknown'
   END AS lifecycle_stage,
+  device_type,
+  device_manufacturer,
 FROM
   active_users
 LEFT JOIN
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-01-14 17:06:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-01-14 17:17:10.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-01-14 17:06:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-01-14 17:17:18.000000000 +0000
@@ -48,6 +48,10 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
+  description: null
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml	2025-01-14 17:21:37.000000000 +0000
@@ -1,11 +1,11 @@
 fields:
 - name: normalized_app_id
-  type: STRING
   mode: NULLABLE
+  type: STRING
   description: App ID of the channel data was received from
 - name: normalized_channel
-  type: STRING
   mode: NULLABLE
+  type: STRING
   description: Normalized channel name
 - name: additional_properties
   type: STRING
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,91 +0,0 @@
-fields:
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: os
-  type: STRING
-  mode: NULLABLE
-- name: os_version
-  type: STRING
-  mode: NULLABLE
-- name: device_model
-  type: STRING
-  mode: NULLABLE
-- name: device_manufacturer
-  type: STRING
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: play_store_attribution_campaign
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_medium
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_source
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: play_store_attribution_content
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_term
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_install_referrer_response
-  type: STRING
-  mode: NULLABLE
-- name: meta_attribution_app
-  type: STRING
-  mode: NULLABLE
-- name: meta_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: install_source
-  type: STRING
-  mode: NULLABLE
-- name: adjust_ad_group
-  type: STRING
-  mode: NULLABLE
-- name: adjust_campaign
-  type: STRING
-  mode: NULLABLE
-- name: adjust_creative
-  type: STRING
-  mode: NULLABLE
-- name: adjust_network
-  type: STRING
-  mode: NULLABLE
-- name: adjust_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: distribution_id
-  type: STRING
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/new_profile_clients/view.sql	2025-01-14 17:16:12.000000000 +0000
@@ -33,6 +33,7 @@
   attribution.adjust_attribution_timestamp,
   attribution.distribution_id,
   attribution.paid_vs_organic,
+  device_type,
 FROM
   `moz-fx-data-shared-prod.fenix.active_users` AS active_users
 LEFT JOIN
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,118 +0,0 @@
-fields:
-- name: submission_date
-  type: DATE
-  mode: NULLABLE
-- name: metric_date
-  type: DATE
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: sample_id
-  type: INTEGER
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: play_store_attribution_campaign
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_medium
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_source
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: play_store_attribution_content
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_term
-  type: STRING
-  mode: NULLABLE
-- name: play_store_attribution_install_referrer_response
-  type: STRING
-  mode: NULLABLE
-- name: meta_attribution_app
-  type: STRING
-  mode: NULLABLE
-- name: meta_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: install_source
-  type: STRING
-  mode: NULLABLE
-- name: adjust_ad_group
-  type: STRING
-  mode: NULLABLE
-- name: adjust_campaign
-  type: STRING
-  mode: NULLABLE
-- name: adjust_creative
-  type: STRING
-  mode: NULLABLE
-- name: adjust_network
-  type: STRING
-  mode: NULLABLE
-- name: adjust_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: distribution_id
-  type: STRING
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
-- name: ping_sent_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: ping_sent_week_4
-  type: BOOLEAN
-  mode: NULLABLE
-- name: active_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: retained_week_4
-  type: BOOLEAN
-  mode: NULLABLE
-- name: new_profile_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: retained_week_4_new_profile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: repeat_profile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: days_seen_bits
-  type: INTEGER
-  mode: NULLABLE
-- name: days_active_bits
-  type: INTEGER
-  mode: NULLABLE
-- name: lifecycle_stage
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql	2025-01-14 17:16:11.000000000 +0000
@@ -14,6 +14,8 @@
     days_seen_bits,
     days_active_bits,
     is_mobile,
+    device_type,
+    device_manufacturer,
   FROM
     `moz-fx-data-shared-prod.fenix.active_users`
 ),
@@ -107,6 +109,8 @@
       THEN 'existing_user'
     ELSE 'Unknown'
   END AS lifecycle_stage,
+  active_users.device_type,
+  active_users.device_manufacturer,
 FROM
   `moz-fx-data-shared-prod.fenix.baseline_clients_daily` AS clients_daily
 INNER JOIN
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql	2025-01-14 17:16:12.000000000 +0000
@@ -1,4 +1,21 @@
 -- Query generated via `mobile_kpi_support_metrics` SQL generator.
+WITH device_manufacturer_counts AS (
+  SELECT
+    submission_date,
+    device_manufacturer,
+    RANK() OVER (PARTITION BY submission_date ORDER BY COUNT(*) DESC) AS manufacturer_rank,
+  FROM
+    `moz-fx-data-shared-prod.fenix.engagement_clients`
+  WHERE
+    {% if is_init() %}
+      submission_date < CURRENT_DATE
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    device_manufacturer
+)
 SELECT
   submission_date,
   first_seen_date,
@@ -21,8 +38,14 @@
   COUNTIF(is_dau) AS dau,
   COUNTIF(is_wau) AS wau,
   COUNTIF(is_mau) AS mau,
+  device_type,
+  -- Bucket device manufacturers with low count prior to aggregation
+  IF(manufacturer_rank <= 150, device_manufacturer, "other") AS device_manufacturer,
 FROM
   `moz-fx-data-shared-prod.fenix.engagement_clients`
+LEFT JOIN
+  device_manufacturer_counts
+  USING (submission_date, device_manufacturer)
 WHERE
   {% if is_init() %}
     submission_date < CURRENT_DATE
@@ -37,6 +60,8 @@
   app_version,
   country,
   locale,
+  device_type,
+  device_manufacturer,
   is_mobile,
   play_store_attribution_campaign,
   play_store_attribution_medium,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml	2025-01-14 17:16:12.000000000 +0000
@@ -103,3 +103,15 @@
   type: INTEGER
   mode: NULLABLE
   description: MAU - Monthly Active Users
+
+- name: device_type
+  type: STRING
+  mode: NULLABLE
+  description: |
+    On Apple devices allows us to differentiate between iPhone and iPad. On Android devices the value is always "Android".
+
+- name: device_manufacturer
+  type: STRING
+  mode: NULLABLE
+  description: |
+    Manufacturer of the device where the client is installed.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profiles_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profiles_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profiles_v1/query.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profiles_v1/query.sql	2025-01-14 17:16:12.000000000 +0000
@@ -1,4 +1,21 @@
 -- Query generated via `mobile_kpi_support_metrics` SQL generator.
+WITH device_manufacturer_counts AS (
+  SELECT
+    first_seen_date,
+    device_manufacturer,
+    RANK() OVER (PARTITION BY first_seen_date ORDER BY COUNT(*) DESC) AS manufacturer_rank,
+  FROM
+    `moz-fx-data-shared-prod.fenix.new_profile_clients`
+  WHERE
+    {% if is_init() %}
+      first_seen_date < CURRENT_DATE
+    {% else %}
+      first_seen_date = @submission_date
+    {% endif %}
+  GROUP BY
+    first_seen_date,
+    device_manufacturer
+)
 SELECT
   first_seen_date,
   normalized_channel,
@@ -8,7 +25,8 @@
   locale,
   os,
   os_version,
-  device_manufacturer,
+  -- Bucket device manufacturers with low count prior to aggregation
+  IF(manufacturer_rank <= 150, device_manufacturer, "other") AS device_manufacturer,
   is_mobile,
   play_store_attribution_campaign,
   play_store_attribution_medium,
@@ -21,8 +39,12 @@
   adjust_network,
   distribution_id,
   COUNT(*) AS new_profiles,
+  device_type,
 FROM
   `moz-fx-data-shared-prod.fenix.new_profile_clients`
+LEFT JOIN
+  device_manufacturer_counts
+  USING (first_seen_date, device_manufacturer)
 WHERE
   {% if is_init() %}
     first_seen_date < CURRENT_DATE
@@ -38,6 +60,7 @@
   locale,
   os,
   os_version,
+  device_type,
   device_manufacturer,
   is_mobile,
   play_store_attribution_campaign,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profiles_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profiles_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profiles_v1/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profiles_v1/schema.yaml	2025-01-14 17:16:12.000000000 +0000
@@ -103,3 +103,9 @@
   type: INTEGER
   mode: NULLABLE
   description: Number of new profiles recorded for the first seen date.
+
+- name: device_type
+  type: STRING
+  mode: NULLABLE
+  description: |
+    On Apple devices allows us to differentiate between iPhone and iPad. On Android devices the value is always "Android".
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql	2025-01-14 17:16:12.000000000 +0000
@@ -1,4 +1,23 @@
 -- Query generated via `mobile_kpi_support_metrics` SQL generator.
+WITH device_manufacturer_counts AS (
+  SELECT
+    submission_date,
+    device_manufacturer,
+    RANK() OVER (PARTITION BY submission_date ORDER BY COUNT(*) DESC) AS manufacturer_rank,
+  FROM
+    `moz-fx-data-shared-prod.fenix.retention_clients`
+  WHERE
+    {% if is_init() %}
+      metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY)
+      AND submission_date < CURRENT_DATE
+    {% else %}
+      metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY)
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    device_manufacturer
+)
 SELECT
   metric_date,
   first_seen_date,
@@ -25,8 +44,14 @@
   COUNTIF(retained_week_4_new_profile) AS retained_week_4_new_profiles,
   COUNTIF(new_profile_metric_date) AS new_profiles_metric_date,
   COUNTIF(repeat_profile) AS repeat_profiles,
+  device_type,
+  -- Bucket device manufacturers with low count prior to aggregation
+  IF(manufacturer_rank <= 150, device_manufacturer, "other") AS device_manufacturer,
 FROM
   `moz-fx-data-shared-prod.fenix.retention_clients`
+LEFT JOIN
+  device_manufacturer_counts
+  USING (submission_date, device_manufacturer)
 WHERE
   {% if is_init() %}
     metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY)
@@ -43,6 +68,8 @@
   country,
   app_version,
   locale,
+  device_type,
+  device_manufacturer,
   is_mobile,
   play_store_attribution_campaign,
   play_store_attribution_medium,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml	2025-01-14 17:16:12.000000000 +0000
@@ -123,3 +123,15 @@
   type: INTEGER
   mode: NULLABLE
   description: Number of new profiles on the metric date that were DAU at least twice in the next 28 days.
+
+- name: device_type
+  type: STRING
+  mode: NULLABLE
+  description: |
+    On Apple devices allows us to differentiate between iPhone and iPad. On Android devices the value is always "Android".
+
+- name: device_manufacturer
+  type: STRING
+  mode: NULLABLE
+  description: |
+    Manufacturer of the device where the client is installed.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2025-01-14 17:06:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2025-01-14 17:16:35.000000000 +0000
@@ -1,64 +1,66 @@
 fields:
-- description: Unique ID for the client installation.
-  mode: NULLABLE
-  name: client_id
+- name: client_id
   type: STRING
-- description: Sample ID - A number ranging from 0 - 99 based on client ID; used to pull a small sample of data related to a subset of clients over time
   mode: NULLABLE
-  name: sample_id
-  type: INT64
-- description: Submission Date
+  description: Unique ID for the client installation.
+- name: sample_id
+  type: INTEGER
   mode: NULLABLE
-  name: submission_date
+  description: Sample ID - A number ranging from 0 - 99 based on client ID; used to
+    pull a small sample of data related to a subset of clients over time
+- name: submission_date
   type: DATE
-- description: First Seen Date - The date this client was first seen
   mode: NULLABLE
-  name: first_seen_date
+  description: Submission Date
+- name: first_seen_date
   type: DATE
-- description: Days Since First Seen - The number of days since the client was first seen
   mode: NULLABLE
-  name: days_since_first_seen
-  type: INT64
-- description: Days Since Active
+  description: First Seen Date - The date this client was first seen
+- name: days_since_first_seen
+  type: INTEGER
   mode: NULLABLE
-  name: days_since_active
-  type: INT64
-- description: First Reported Country - The country this client ID was first reported from
+  description: Days Since First Seen - The number of days since the client was first
+    seen
+- name: days_since_active
+  type: INTEGER
   mode: NULLABLE
-  name: first_reported_country
+  description: Days Since Active
+- name: first_reported_country
   type: STRING
-- description: Attribution
   mode: NULLABLE
-  name: attribution
+  description: First Reported Country - The country this client ID was first reported
+    from
+- name: attribution
   type: RECORD
+  mode: NULLABLE
   fields:
-  - mode: NULLABLE
-    name: source
+  - name: source
     type: STRING
+    mode: NULLABLE
     description: Attribution Source
-  - mode: NULLABLE
-    name: medium
+  - name: medium
     type: STRING
+    mode: NULLABLE
     description: Attribution Medium
-  - mode: NULLABLE
-    name: campaign
+  - name: campaign
     type: STRING
+    mode: NULLABLE
     description: Attribution Campaign
-  - mode: NULLABLE
-    name: content
+  - name: content
     type: STRING
+    mode: NULLABLE
     description: Attribution Content
-  - mode: NULLABLE
-    name: experiment
+  - name: experiment
     type: STRING
+    mode: NULLABLE
     description: Attribution Experiment
-  - mode: NULLABLE
-    name: variation
+  - name: variation
     type: STRING
+    mode: NULLABLE
     description: Attribution Variation
-  - mode: NULLABLE
-    name: dltoken
+  - name: dltoken
     type: STRING
+    mode: NULLABLE
     description: Attribution Download Token
   - name: dlsource
     type: STRING
@@ -68,40 +70,43 @@
     type: STRING
     mode: NULLABLE
     description: Attribution UA
-- description: Active
-  mode: NULLABLE
-  name: active
-  type: INT64
-- description: Ad Clicks - The number of ad clicks from this client on the submission date
+  description: Attribution
+- name: active
+  type: INTEGER
   mode: NULLABLE
-  name: ad_clicks
-  type: INT64
-- description: Total Historic Ad Clicks - The number of ad clicks from this client on or before the submission date
+  description: Active
+- name: ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: total_historic_ad_clicks
-  type: INT64
-- description: Days Seen Bytes
+  description: Ad Clicks - The number of ad clicks from this client on the submission
+    date
+- name: total_historic_ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: days_seen_bytes
+  description: Total Historic Ad Clicks - The number of ad clicks from this client
+    on or before the submission date
+- name: days_seen_bytes
   type: BYTES
-- description: Pattern
   mode: NULLABLE
-  name: pattern
+  description: Days Seen Bytes
+- name: pattern
   type: INTEGER
-- description: Death Time
   mode: NULLABLE
-  name: death_time
+  description: Pattern
+- name: death_time
   type: INTEGER
-- description: Max Days
   mode: NULLABLE
-  name: max_days
+  description: Death Time
+- name: max_days
   type: INTEGER
-- description: Markov States
   mode: NULLABLE
-  name: markov_states
+  description: Max Days
+- name: markov_states
   type: RECORD
-  fields:
-  - description: Desktop States V1
     mode: NULLABLE
-    name: desktop_states_v1
+  fields:
+  - name: desktop_states_v1
     type: STRING
+    mode: NULLABLE
+    description: Desktop States V1
+  description: Markov States
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2025-01-14 17:06:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2025-01-14 17:16:20.000000000 +0000
@@ -1,48 +1,47 @@
 fields:
-- description: Submission Timestamp
-  mode: NULLABLE
-  name: submission_timestamp
+- name: submission_timestamp
   type: TIMESTAMP
-- description: Normalized Country Code, Examples - US, AR, BR, etc.
   mode: NULLABLE
-  name: normalized_country_code
+  description: Submission Timestamp
+- name: normalized_country_code
   type: STRING
-- description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
   mode: NULLABLE
-  name: normalized_channel
+  description: Normalized Country Code, Examples - US, AR, BR, etc.
+- name: normalized_channel
   type: STRING
-- description: Document ID
   mode: NULLABLE
-  name: document_id
+  description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
+- name: document_id
   type: STRING
-- description: Pocket Enabled
   mode: NULLABLE
-  name: pocket_enabled
+  description: Document ID
+- name: pocket_enabled
   type: BOOLEAN
-- description: Pocket Sponsored Stories Enabled
   mode: NULLABLE
-  name: pocket_sponsored_stories_enabled
+  description: Pocket Enabled
+- name: pocket_sponsored_stories_enabled
   type: BOOLEAN
-- description: Newtab Locale
   mode: NULLABLE
-  name: newtab_locale
+  description: Pocket Sponsored Stories Enabled
+- name: newtab_locale
   type: STRING
-- description: App Build
   mode: NULLABLE
-  name: app_build
+  description: Newtab Locale
+- name: app_build
   type: STRING
-- description: App Display Version
   mode: NULLABLE
-  name: app_display_version
+  description: App Build
+- name: app_display_version
   type: STRING
-- description: Client ID
   mode: NULLABLE
-  name: client_id
+  description: App Display Version
+- name: client_id
   type: STRING
+  mode: NULLABLE
+  description: Client ID
 - name: events
   type: RECORD
   mode: REPEATED
-  description: Events
   fields:
   - name: category
     type: STRING
@@ -51,7 +50,6 @@
   - name: extra
     type: RECORD
     mode: REPEATED
-    description: Extras
     fields:
     - name: key
       type: STRING
@@ -61,6 +59,7 @@
       type: STRING
       mode: NULLABLE
       description: Value
+    description: Extras
   - name: name
     type: STRING
     mode: NULLABLE
@@ -69,3 +68,4 @@
     type: INTEGER
     mode: NULLABLE
     description: Event Timestamp
+  description: Events
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/active_users/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/active_users/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/active_users/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/active_users/schema.yaml	2025-01-14 17:16:42.000000000 +0000
@@ -137,3 +137,6 @@
 - name: isp
   type: STRING
   mode: NULLABLE
+- name: device_type
+  type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/active_users/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/active_users/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/active_users/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/active_users/view.sql	2025-01-14 17:16:11.000000000 +0000
@@ -3,7 +3,10 @@
   `moz-fx-data-shared-prod.firefox_ios.active_users`
 AS
 SELECT
-  * EXCEPT (isp),
+  * EXCEPT (isp) REPLACE(
+    -- Lower device_manufacturer as in some cases the same manufacturer value has different casing.
+    LOWER(device_manufacturer) AS device_manufacturer
+  ),
   CASE
     WHEN LOWER(isp) = "browserstack"
       THEN CONCAT("Firefox iOS", " ", isp)
@@ -43,5 +46,16 @@
   -- Adding isp at the end because it's in different column index in baseline table for some products.
   -- This is to make sure downstream union works as intended.
   isp,
+  CASE
+    WHEN normalized_os = "iOS"
+      AND STARTS_WITH(device_model, "iPad")
+      THEN "iPad"
+    WHEN normalized_os = "iOS"
+      AND STARTS_WITH(device_model, "iPhone")
+      THEN "iPhone"
+    WHEN normalized_os = "Android"
+      THEN "Android"
+    ELSE CAST(NULL AS STRING)
+  END AS device_type,
 FROM
   `moz-fx-data-shared-prod.firefox_ios.baseline_clients_last_seen`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,67 +0,0 @@
-fields:
-- name: submission_date
-  type: DATE
-  mode: NULLABLE
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: sample_id
-  type: INTEGER
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: is_dau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_wau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_mau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_suspicious_device_client
-  type: BOOLEAN
-  mode: NULLABLE
-- name: adjust_ad_group
-  type: STRING
-  mode: NULLABLE
-- name: adjust_campaign
-  type: STRING
-  mode: NULLABLE
-- name: adjust_creative
-  type: STRING
-  mode: NULLABLE
-- name: adjust_network
-  type: STRING
-  mode: NULLABLE
-- name: adjust_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
-- name: lifecycle_stage
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql	2025-01-14 17:16:12.000000000 +0000
@@ -18,6 +18,8 @@
     is_wau,
     is_mau,
     is_mobile,
+    device_type,
+    device_manufacturer,
   FROM
     `moz-fx-data-shared-prod.firefox_ios.active_users`
 ),
@@ -68,6 +70,8 @@
       THEN 'existing_user'
     ELSE 'Unknown'
   END AS lifecycle_stage,
+  device_type,
+  device_manufacturer,
 FROM
   active_users
 LEFT JOIN
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,61 +0,0 @@
-fields:
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: os
-  type: STRING
-  mode: NULLABLE
-- name: os_version
-  type: STRING
-  mode: NULLABLE
-- name: device_model
-  type: STRING
-  mode: NULLABLE
-- name: device_manufacturer
-  type: STRING
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_suspicious_device_client
-  type: BOOLEAN
-  mode: NULLABLE
-- name: adjust_ad_group
-  type: STRING
-  mode: NULLABLE
-- name: adjust_campaign
-  type: STRING
-  mode: NULLABLE
-- name: adjust_creative
-  type: STRING
-  mode: NULLABLE
-- name: adjust_network
-  type: STRING
-  mode: NULLABLE
-- name: adjust_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/new_profile_clients/view.sql	2025-01-14 17:16:12.000000000 +0000
@@ -23,6 +23,7 @@
   attribution.adjust_network,
   attribution.adjust_attribution_timestamp,
   attribution.paid_vs_organic,
+  device_type,
 FROM
   `moz-fx-data-shared-prod.firefox_ios.active_users` AS active_users
 LEFT JOIN
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,88 +0,0 @@
-fields:
-- name: submission_date
-  type: DATE
-  mode: NULLABLE
-- name: metric_date
-  type: DATE
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: sample_id
-  type: INTEGER
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_suspicious_device_client
-  type: BOOLEAN
-  mode: NULLABLE
-- name: adjust_ad_group
-  type: STRING
-  mode: NULLABLE
-- name: adjust_campaign
-  type: STRING
-  mode: NULLABLE
-- name: adjust_creative
-  type: STRING
-  mode: NULLABLE
-- name: adjust_network
-  type: STRING
-  mode: NULLABLE
-- name: adjust_attribution_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
-- name: ping_sent_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: ping_sent_week_4
-  type: BOOLEAN
-  mode: NULLABLE
-- name: active_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: retained_week_4
-  type: BOOLEAN
-  mode: NULLABLE
-- name: new_profile_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: retained_week_4_new_profile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: repeat_profile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: days_seen_bits
-  type: INTEGER
-  mode: NULLABLE
-- name: days_active_bits
-  type: INTEGER
-  mode: NULLABLE
-- name: lifecycle_stage
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql	2025-01-14 17:16:11.000000000 +0000
@@ -14,6 +14,8 @@
     days_seen_bits,
     days_active_bits,
     is_mobile,
+    device_type,
+    device_manufacturer,
   FROM
     `moz-fx-data-shared-prod.firefox_ios.active_users`
 ),
@@ -87,6 +89,8 @@
       THEN 'existing_user'
     ELSE 'Unknown'
   END AS lifecycle_stage,
+  active_users.device_type,
+  active_users.device_manufacturer,
 FROM
   `moz-fx-data-shared-prod.firefox_ios.baseline_clients_daily` AS clients_daily
 INNER JOIN
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql	2025-01-14 17:16:12.000000000 +0000
@@ -1,4 +1,21 @@
 -- Query generated via `mobile_kpi_support_metrics` SQL generator.
+WITH device_manufacturer_counts AS (
+  SELECT
+    submission_date,
+    device_manufacturer,
+    RANK() OVER (PARTITION BY submission_date ORDER BY COUNT(*) DESC) AS manufacturer_rank,
+  FROM
+    `moz-fx-data-shared-prod.firefox_ios.engagement_clients`
+  WHERE
+    {% if is_init() %}
+      submission_date < CURRENT_DATE
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    device_manufacturer
+)
 SELECT
   submission_date,
   first_seen_date,
@@ -16,8 +33,14 @@
   COUNTIF(is_dau) AS dau,
   COUNTIF(is_wau) AS wau,
   COUNTIF(is_mau) AS mau,
+  device_type,
+  -- Bucket device manufacturers with low count prior to aggregation
+  IF(manufacturer_rank <= 150, device_manufacturer, "other") AS device_manufacturer,
 FROM
   `moz-fx-data-shared-prod.firefox_ios.engagement_clients`
+LEFT JOIN
+  device_manufacturer_counts
+  USING (submission_date, device_manufacturer)
 WHERE
   {% if is_init() %}
     submission_date < CURRENT_DATE
@@ -32,6 +55,8 @@
   app_version,
   country,
   locale,
+  device_type,
+  device_manufacturer,
   is_mobile,
   is_suspicious_device_client,
   adjust_ad_group,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml	2025-01-14 17:16:12.000000000 +0000
@@ -78,3 +78,15 @@
   type: INTEGER
   mode: NULLABLE
   description: MAU - Monthly Active Users
+
+- name: device_type
+  type: STRING
+  mode: NULLABLE
+  description: |
+    On Apple devices allows us to differentiate between iPhone and iPad. On Android devices the value is always "Android".
+
+- name: device_manufacturer
+  type: STRING
+  mode: NULLABLE
+  description: |
+    Manufacturer of the device where the client is installed.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profiles_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profiles_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profiles_v1/query.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profiles_v1/query.sql	2025-01-14 17:16:12.000000000 +0000
@@ -1,4 +1,21 @@
 -- Query generated via `mobile_kpi_support_metrics` SQL generator.
+WITH device_manufacturer_counts AS (
+  SELECT
+    first_seen_date,
+    device_manufacturer,
+    RANK() OVER (PARTITION BY first_seen_date ORDER BY COUNT(*) DESC) AS manufacturer_rank,
+  FROM
+    `moz-fx-data-shared-prod.firefox_ios.new_profile_clients`
+  WHERE
+    {% if is_init() %}
+      first_seen_date < CURRENT_DATE
+    {% else %}
+      first_seen_date = @submission_date
+    {% endif %}
+  GROUP BY
+    first_seen_date,
+    device_manufacturer
+)
 SELECT
   first_seen_date,
   normalized_channel,
@@ -8,7 +25,8 @@
   locale,
   os,
   os_version,
-  device_manufacturer,
+  -- Bucket device manufacturers with low count prior to aggregation
+  IF(manufacturer_rank <= 150, device_manufacturer, "other") AS device_manufacturer,
   is_mobile,
   is_suspicious_device_client,
   adjust_ad_group,
@@ -16,8 +34,12 @@
   adjust_creative,
   adjust_network,
   COUNT(*) AS new_profiles,
+  device_type,
 FROM
   `moz-fx-data-shared-prod.firefox_ios.new_profile_clients`
+LEFT JOIN
+  device_manufacturer_counts
+  USING (first_seen_date, device_manufacturer)
 WHERE
   {% if is_init() %}
     first_seen_date < CURRENT_DATE
@@ -33,6 +55,7 @@
   locale,
   os,
   os_version,
+  device_type,
   device_manufacturer,
   is_mobile,
   is_suspicious_device_client,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profiles_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profiles_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profiles_v1/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profiles_v1/schema.yaml	2025-01-14 17:16:12.000000000 +0000
@@ -78,3 +78,9 @@
   type: INTEGER
   mode: NULLABLE
   description: Number of new profiles recorded for the first seen date.
+
+- name: device_type
+  type: STRING
+  mode: NULLABLE
+  description: |
+    On Apple devices allows us to differentiate between iPhone and iPad. On Android devices the value is always "Android".
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql	2025-01-14 17:16:12.000000000 +0000
@@ -1,4 +1,23 @@
 -- Query generated via `mobile_kpi_support_metrics` SQL generator.
+WITH device_manufacturer_counts AS (
+  SELECT
+    submission_date,
+    device_manufacturer,
+    RANK() OVER (PARTITION BY submission_date ORDER BY COUNT(*) DESC) AS manufacturer_rank,
+  FROM
+    `moz-fx-data-shared-prod.firefox_ios.retention_clients`
+  WHERE
+    {% if is_init() %}
+      metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY)
+      AND submission_date < CURRENT_DATE
+    {% else %}
+      metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY)
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    device_manufacturer
+)
 SELECT
   metric_date,
   first_seen_date,
@@ -20,8 +39,14 @@
   COUNTIF(retained_week_4_new_profile) AS retained_week_4_new_profiles,
   COUNTIF(new_profile_metric_date) AS new_profiles_metric_date,
   COUNTIF(repeat_profile) AS repeat_profiles,
+  device_type,
+  -- Bucket device manufacturers with low count prior to aggregation
+  IF(manufacturer_rank <= 150, device_manufacturer, "other") AS device_manufacturer,
 FROM
   `moz-fx-data-shared-prod.firefox_ios.retention_clients`
+LEFT JOIN
+  device_manufacturer_counts
+  USING (submission_date, device_manufacturer)
 WHERE
   {% if is_init() %}
     metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY)
@@ -38,6 +63,8 @@
   country,
   app_version,
   locale,
+  device_type,
+  device_manufacturer,
   is_mobile,
   is_suspicious_device_client,
   adjust_ad_group,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml	2025-01-14 17:16:12.000000000 +0000
@@ -98,3 +98,15 @@
   type: INTEGER
   mode: NULLABLE
   description: Number of new profiles on the metric date that were DAU at least twice in the next 28 days.
+
+- name: device_type
+  type: STRING
+  mode: NULLABLE
+  description: |
+    On Apple devices allows us to differentiate between iPhone and iPad. On Android devices the value is always "Android".
+
+- name: device_manufacturer
+  type: STRING
+  mode: NULLABLE
+  description: |
+    Manufacturer of the device where the client is installed.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/active_users/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/active_users/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/active_users/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/active_users/schema.yaml	2025-01-14 17:18:17.000000000 +0000
@@ -137,3 +137,6 @@
 - name: isp
   type: STRING
   mode: NULLABLE
+- name: device_type
+  type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/active_users/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/active_users/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/active_users/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/active_users/view.sql	2025-01-14 17:16:11.000000000 +0000
@@ -3,7 +3,10 @@
   `moz-fx-data-shared-prod.focus_android.active_users`
 AS
 SELECT
-  * EXCEPT (isp),
+  * EXCEPT (isp) REPLACE(
+    -- Lower device_manufacturer as in some cases the same manufacturer value has different casing.
+    LOWER(device_manufacturer) AS device_manufacturer
+  ),
   CASE
     WHEN LOWER(isp) = "browserstack"
       THEN CONCAT("Focus Android", " ", isp)
@@ -43,5 +46,16 @@
   -- Adding isp at the end because it's in different column index in baseline table for some products.
   -- This is to make sure downstream union works as intended.
   isp,
+  CASE
+    WHEN normalized_os = "iOS"
+      AND STARTS_WITH(device_model, "iPad")
+      THEN "iPad"
+    WHEN normalized_os = "iOS"
+      AND STARTS_WITH(device_model, "iPhone")
+      THEN "iPhone"
+    WHEN normalized_os = "Android"
+      THEN "Android"
+    ELSE CAST(NULL AS STRING)
+  END AS device_type,
 FROM
   `moz-fx-data-shared-prod.focus_android.baseline_clients_last_seen`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,49 +0,0 @@
-fields:
-- name: submission_date
-  type: DATE
-  mode: NULLABLE
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: sample_id
-  type: INTEGER
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: is_dau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_wau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_mau
-  type: BOOLEAN
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
-- name: lifecycle_stage
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql	2025-01-14 17:16:12.000000000 +0000
@@ -18,6 +18,8 @@
     is_wau,
     is_mau,
     is_mobile,
+    device_type,
+    device_manufacturer,
   FROM
     `moz-fx-data-shared-prod.focus_android.active_users`
 ),
@@ -56,6 +58,8 @@
       THEN 'existing_user'
     ELSE 'Unknown'
   END AS lifecycle_stage,
+  device_type,
+  device_manufacturer,
 FROM
   active_users
 LEFT JOIN
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,43 +0,0 @@
-fields:
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: os
-  type: STRING
-  mode: NULLABLE
-- name: os_version
-  type: STRING
-  mode: NULLABLE
-- name: device_model
-  type: STRING
-  mode: NULLABLE
-- name: device_manufacturer
-  type: STRING
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/new_profile_clients/view.sql	2025-01-14 17:16:12.000000000 +0000
@@ -17,6 +17,7 @@
   device_manufacturer,
   is_mobile,
   attribution.paid_vs_organic,
+  device_type,
 FROM
   `moz-fx-data-shared-prod.focus_android.active_users` AS active_users
 LEFT JOIN
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/schema.yaml	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,70 +0,0 @@
-fields:
-- name: submission_date
-  type: DATE
-  mode: NULLABLE
-- name: metric_date
-  type: DATE
-  mode: NULLABLE
-- name: first_seen_date
-  type: DATE
-  mode: NULLABLE
-- name: client_id
-  type: STRING
-  mode: NULLABLE
-- name: sample_id
-  type: INTEGER
-  mode: NULLABLE
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-- name: country
-  type: STRING
-  mode: NULLABLE
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-- name: locale
-  type: STRING
-  mode: NULLABLE
-- name: isp
-  type: STRING
-  mode: NULLABLE
-- name: is_mobile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: paid_vs_organic
-  type: STRING
-  mode: NULLABLE
-- name: ping_sent_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: ping_sent_week_4
-  type: BOOLEAN
-  mode: NULLABLE
-- name: active_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: retained_week_4
-  type: BOOLEAN
-  mode: NULLABLE
-- name: new_profile_metric_date
-  type: BOOLEAN
-  mode: NULLABLE
-- name: retained_week_4_new_profile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: repeat_profile
-  type: BOOLEAN
-  mode: NULLABLE
-- name: days_seen_bits
-  type: INTEGER
-  mode: NULLABLE
-- name: days_active_bits
-  type: INTEGER
-  mode: NULLABLE
-- name: lifecycle_stage
-  type: STRING
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql	2025-01-14 17:07:16.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql	2025-01-14 17:16:11.000000000 +0000
@@ -14,6 +14,8 @@
     days_seen_bits,
     days_active_bits,
     is

⚠️ Only part of the diff is displayed.

Link to full diff

@kik-kik kik-kik added this pull request to the merge queue Jan 15, 2025
Merged via the queue into main with commit b81a7a0 Jan 15, 2025
21 checks passed
@kik-kik kik-kik deleted the feat/DENG-6890/update-mobile-aggregates-to-support-dma-analysis branch January 15, 2025 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bani bani bani approved these changes

Assignees

@kik-kik kik-kik

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kik-kik @dataops-ci-bot @bani