Skip to content

Analytics routes #461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Jan 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d91948e
initial progress
simzzz Nov 23, 2021
0394416
Merge branch 'aip-61-adex-v5' into analytics-routes
simzzz Nov 23, 2021
cbc4cd6
initial implementation for analytics v5 routes finished
simzzz Nov 25, 2021
c1e90a0
code clean up
simzzz Nov 29, 2021
e10ceaa
(still WIP) requested changes + test coverage
simzzz Dec 3, 2021
00397f6
modified output + output is now split by timeframe
simzzz Dec 6, 2021
5be038a
primitives - analytics - rename_all=camelCase
elpiel Dec 7, 2021
150b1dd
improved query, small changes to logic
simzzz Dec 7, 2021
4346906
Merge branch 'analytics-routes' of https://github.com/AmbireTech/adex…
simzzz Dec 7, 2021
989f66d
improvements on analytics
elpiel Dec 7, 2021
fb6c8cf
fix impl Display for DateHour
elpiel Dec 8, 2021
dc97852
analytics query can now accept both timestamps and date strings
simzzz Dec 8, 2021
526b708
tests almost ready + changes to logic
simzzz Dec 9, 2021
381533d
Additional tests
simzzz Dec 10, 2021
743c1c3
test progress
simzzz Dec 13, 2021
e11962f
fixed failing tests and logic + added myself as author
simzzz Dec 14, 2021
35942fb
output is represented correctly + other minor changes
simzzz Dec 14, 2021
38ccc03
Cargo - patch postgres-types
elpiel Dec 16, 2021
69a2448
primitives - senty:
elpiel Dec 31, 2021
96fec64
Cargo patch postgres-types:
elpiel Dec 31, 2021
8e842ee
primitives - analytics - enum AllowedKeys & struct Time
elpiel Dec 31, 2021
1fea534
adapter - fix dummy warning
elpiel Jan 4, 2022
7d6a276
Analytics changes:
elpiel Jan 10, 2022
04fe52b
sentry - Event improvements & new enum EventType
elpiel Jan 14, 2022
69f4acf
sentry - db - analytics testing:
elpiel Jan 17, 2022
7e8d449
sentry - routes - analytics - finish route & tests
elpiel Jan 17, 2022
b647999
Merge branch 'aip-61-adex-v5' into analytics-routes
elpiel Jan 17, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 43 additions & 8 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,6 @@ members = [
"sentry",
"test_harness",
]

[patch.crates-io]
postgres-types = { git = "https://github.com/elpiel/rust-postgres", branch = "boxed-dyn-ToSql"}
1 change: 1 addition & 0 deletions docs/config/dev.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ ethereum_network = 'http://localhost:8545'

creators_whitelist = []
validators_whitelist = []
admins = ['0xce07CbB7e054514D590a0262C93070D838bFBA2e']

[[token_address_whitelist]]
# DAI
Expand Down
2 changes: 1 addition & 1 deletion docs/config/ganache.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ ethereum_network = 'http://localhost:8545'

creators_whitelist = []
validators_whitelist = []

admins = ['0x80690751969B234697e9059e04ed72195c3507fa']

[[token_address_whitelist]]
# Mocked TOKEN
Expand Down
2 changes: 1 addition & 1 deletion docs/config/prod.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ ethereum_network = 'http://localhost:8545'

creators_whitelist = []
validators_whitelist = []

admins = ['0x5d6A3F1AD7b124ecDFDf4841D9bB246eD5fBF04c']

[[token_address_whitelist]]
# DAI
Expand Down
1 change: 1 addition & 0 deletions primitives/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ authors = [
"Ambire <[email protected]>",
"Lachezar Lechev <[email protected]>",
"Omidiora Samuel <[email protected]>",
"Simeon Nakov <[email protected]>",
]
edition = "2021"
license = "AGPL-3.0"
Expand Down
206 changes: 132 additions & 74 deletions primitives/src/analytics.rs
Original file line number Diff line number Diff line change
@@ -1,40 +1,43 @@
use crate::{ChannelId, DomainError};
use crate::{
sentry::{EventType, IMPRESSION},
Address, CampaignId, ValidatorId, IPFS,
};
use parse_display::Display;
use serde::{Deserialize, Serialize};

pub const ANALYTICS_QUERY_LIMIT: u32 = 200;

#[derive(Debug, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct AnalyticsData {
pub time: f64,
pub value: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub channel_id: Option<ChannelId>,
}
use self::query::{AllowedKey, Time};

#[derive(Debug, Serialize, Deserialize)]
pub struct AnalyticsResponse {
pub aggr: Vec<AnalyticsData>,
pub limit: u32,
}
pub const ANALYTICS_QUERY_LIMIT: u32 = 200;

#[cfg(feature = "postgres")]
pub mod postgres {
use super::{AnalyticsData, OperatingSystem};
use super::{query::AllowedKey, AnalyticsQuery, OperatingSystem};
use bytes::BytesMut;
use std::error::Error;
use tokio_postgres::{
types::{accepts, to_sql_checked, FromSql, IsNull, ToSql, Type},
Row,
};

impl From<&Row> for AnalyticsData {
fn from(row: &Row) -> Self {
Self {
time: row.get("time"),
value: row.get("value"),
channel_id: row.try_get("channel_id").ok(),
use tokio_postgres::types::{accepts, to_sql_checked, FromSql, IsNull, ToSql, Type};

impl AnalyticsQuery {
pub fn get_key(&self, key: AllowedKey) -> Option<Box<dyn ToSql + Sync + Send>> {
match key {
AllowedKey::CampaignId => self
.campaign_id
.map(|campaign_id| Box::new(campaign_id) as _),
AllowedKey::AdUnit => self.ad_unit.map(|ad_unit| Box::new(ad_unit) as _),
AllowedKey::AdSlot => self.ad_slot.map(|ad_slot| Box::new(ad_slot) as _),
AllowedKey::AdSlotType => self
.ad_slot_type
.clone()
.map(|ad_slot_type| Box::new(ad_slot_type) as _),
AllowedKey::Advertiser => {
self.advertiser.map(|advertiser| Box::new(advertiser) as _)
}
AllowedKey::Publisher => self.publisher.map(|publisher| Box::new(publisher) as _),
AllowedKey::Hostname => self
.hostname
.clone()
.map(|hostname| Box::new(hostname) as _),
AllowedKey::Country => self.country.clone().map(|country| Box::new(country) as _),
AllowedKey::OsName => self.os_name.clone().map(|os_name| Box::new(os_name) as _),
}
}
}
Expand Down Expand Up @@ -66,20 +69,54 @@ pub mod postgres {
accepts!(TEXT, VARCHAR);
to_sql_checked!();
}

impl ToSql for AllowedKey {
fn to_sql(
&self,
ty: &Type,
w: &mut BytesMut,
) -> Result<IsNull, Box<dyn Error + Sync + Send>> {
self.to_string().to_sql(ty, w)
}

accepts!(TEXT, VARCHAR);
to_sql_checked!();
}

impl<'a> FromSql<'a> for AllowedKey {
fn from_sql(ty: &Type, raw: &'a [u8]) -> Result<Self, Box<dyn Error + Sync + Send>> {
let allowed_key_str = <&'a str as FromSql>::from_sql(ty, raw)?;

Ok(allowed_key_str.parse()?)
}

accepts!(TEXT, VARCHAR);
}
}

#[derive(Debug, Deserialize)]
pub mod query;

#[derive(Debug, Serialize, Deserialize, Clone)]
#[serde(rename_all = "camelCase")]
pub struct AnalyticsQuery {
#[serde(default = "default_limit")]
pub limit: u32,
#[serde(default = "default_event_type")]
pub event_type: String,
pub event_type: EventType,
#[serde(default = "default_metric")]
pub metric: String,
#[serde(default = "default_timeframe")]
pub timeframe: String,
pub segment_by_channel: Option<String>,
pub metric: Metric,
pub segment_by: Option<AllowedKey>,
#[serde(flatten)]
pub time: Time,
pub campaign_id: Option<CampaignId>,
pub ad_unit: Option<IPFS>,
pub ad_slot: Option<IPFS>,
pub ad_slot_type: Option<String>,
pub advertiser: Option<Address>,
pub publisher: Option<Address>,
pub hostname: Option<String>,
pub country: Option<String>,
pub os_name: Option<OperatingSystem>,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Display, Hash, Eq)]
Expand All @@ -91,6 +128,63 @@ pub enum OperatingSystem {
Other,
}

#[derive(Debug, Clone, Serialize, Deserialize, Display, PartialEq, Eq)]
#[serde(rename_all = "camelCase")]
pub enum Timeframe {
/// [`Timeframe::Year`] returns analytics grouped by month.
Year,
/// [`Timeframe::Month`] returns analytics grouped by day.
Month,
/// [`Timeframe::Week`] returns analytics grouped by hour.
/// Same as [`Timeframe::Day`].
Week,
/// [`Timeframe::Day`] returns analytics grouped by hour.
/// Same as [`Timeframe::Week`].
Day,
}

#[derive(Debug, Clone, Copy, Serialize, Deserialize, Display)]
#[serde(rename_all = "camelCase")]
pub enum Metric {
Count,
Paid,
}

#[derive(Debug, Clone, Copy, Serialize, Deserialize, Display)]
pub enum AuthenticateAs {
#[display("{0}")]
Advertiser(ValidatorId),
#[display("{0}")]
Publisher(ValidatorId),
}

impl Metric {
#[cfg(feature = "postgres")]
/// Returns the query column name of the [`Metric`].
///
/// Available only when the `postgres` feature is enabled.
pub fn column_name(self) -> &'static str {
match self {
Metric::Count => "payout_count",
Metric::Paid => "payout_amount",
}
}
}

impl Timeframe {
pub fn to_hours(&self) -> i64 {
let hour = 1;
let day = 24 * hour;
let year = 365 * day;
match self {
Timeframe::Day => day,
Timeframe::Week => 7 * day,
Timeframe::Month => year / 12,
Timeframe::Year => year,
}
}
}

impl Default for OperatingSystem {
fn default() -> Self {
Self::Other
Expand Down Expand Up @@ -171,52 +265,16 @@ impl OperatingSystem {
}
}

impl AnalyticsQuery {
pub fn is_valid(&self) -> Result<(), DomainError> {
let valid_event_types = ["IMPRESSION", "CLICK"];
let valid_metric = ["eventPayouts", "eventCounts"];
let valid_timeframe = ["year", "month", "week", "day", "hour"];

if !valid_event_types.contains(&self.event_type.as_str()) {
Err(DomainError::InvalidArgument(format!(
"invalid event_type, possible values are: {}",
valid_event_types.join(" ,")
)))
} else if !valid_metric.contains(&self.metric.as_str()) {
Err(DomainError::InvalidArgument(format!(
"invalid metric, possible values are: {}",
valid_metric.join(" ,")
)))
} else if !valid_timeframe.contains(&self.timeframe.as_str()) {
Err(DomainError::InvalidArgument(format!(
"invalid timeframe, possible values are: {}",
valid_timeframe.join(" ,")
)))
} else if self.limit > ANALYTICS_QUERY_LIMIT {
Err(DomainError::InvalidArgument(format!(
"invalid limit {}, maximum value 200",
self.limit
)))
} else {
Ok(())
}
}
}

fn default_limit() -> u32 {
100
}

fn default_event_type() -> String {
"IMPRESSION".into()
}

fn default_metric() -> String {
"eventCounts".into()
fn default_event_type() -> EventType {
IMPRESSION
}

fn default_timeframe() -> String {
"hour".into()
fn default_metric() -> Metric {
Metric::Count
}

#[cfg(test)]
Expand Down
Loading