Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Private Preferences DB #946

Merged
merged 14 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 2 additions & 11 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
-- This file should undo anything in `up.sql`
DROP TABLE IF EXISTS "consent_records";
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
CREATE TABLE "consent_records"(
-- Enum of the CONSENT_TYPE (GROUP_ID, INBOX_ID, etc..)
"entity_type" int NOT NULL,
-- Enum of CONSENT_STATE (ALLOWED, DENIED, etc..)
"state" int NOT NULL,
-- The entity of what has consent (0x00 etc..)
"entity" text NOT NULL,
PRIMARY KEY (entity_type, entity)
);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to make sure I'm reasoning about this correctly we will want a way to get the current consent state for a group or inbox id performantly so the primary key should be the value and the valueType so that you can fetch for type group and value group_id and get what the state is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That does feel like what we'll want at the end of the process. Very cheap to look up, has all the information a

One thing that was a struggle in the past is just making sure that we processed all the events in the correct order, so that if you allowed and then blocked the same group you'd get a consistent value in the end. What happens if you received a preferences event that you couldn't read before, but now you can? How do we rewind things so that you can rebuild the state? Maybe the right answer is to truncate this table and start from scratch in that scenario?

Copy link
Contributor

@insipx insipx Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it make sense to make an index here, like we did for inbox id?

CREATE INDEX idx_value_type_value ON private_preferences(valueType, value ASC);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that the primary key made an index by default so we shouldn't need to add another index.

20 changes: 12 additions & 8 deletions xmtp_mls/src/groups/group_metadata.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ pub enum GroupMetadataError {
InvalidConversationType,
#[error("missing extension")]
MissingExtension,
#[error("missing a dm member")]
MissingDmMember,
}

#[derive(Debug, Clone, PartialEq)]
Expand Down Expand Up @@ -72,14 +74,10 @@ impl TryFrom<GroupMetadataProto> for GroupMetadata {
type Error = GroupMetadataError;

fn try_from(value: GroupMetadataProto) -> Result<Self, Self::Error> {
let dm_members = if value.dm_members.is_some() {
Some(DmMembers::try_from(value.dm_members.unwrap())?)
} else {
None
};
let dm_members = value.dm_members.map(DmMembers::try_from).transpose()?;
Ok(Self::new(
value.conversation_type.try_into()?,
value.creator_inbox_id.clone(),
value.creator_inbox_id,
dm_members,
))
}
Expand Down Expand Up @@ -150,8 +148,14 @@ impl TryFrom<DmMembersProto> for DmMembers {

fn try_from(value: DmMembersProto) -> Result<Self, Self::Error> {
Ok(Self {
member_one_inbox_id: value.dm_member_one.unwrap().inbox_id.clone(),
member_two_inbox_id: value.dm_member_two.unwrap().inbox_id.clone(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These unwraps were just on the mls_dms feature branch already. Just cleaning up the linter in this PR. cc @cameronvoell

member_one_inbox_id: value
.dm_member_one
.ok_or(GroupMetadataError::MissingDmMember)?
.inbox_id,
member_two_inbox_id: value
.dm_member_two
.ok_or(GroupMetadataError::MissingDmMember)?
.inbox_id,
})
}
}
Expand Down
19 changes: 10 additions & 9 deletions xmtp_mls/src/groups/group_permissions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -863,15 +863,16 @@ impl PolicySet {
);

// We can always add DM member's inboxId to a DM
if commit.dm_members.is_some()
&& commit.added_inboxes.len() == 1
&& (commit.added_inboxes[0].inbox_id
== commit.dm_members.as_ref().unwrap().member_one_inbox_id
|| commit.added_inboxes[0].inbox_id
== commit.dm_members.as_ref().unwrap().member_two_inbox_id)
&& commit.added_inboxes[0].inbox_id != commit.actor_inbox_id()
{
added_inboxes_valid = true;
if let Some(dm_members) = &commit.dm_members {
if commit.added_inboxes.len() == 1 {
let added_inbox_id = &commit.added_inboxes[0].inbox_id;
if (added_inbox_id == &dm_members.member_one_inbox_id
|| added_inbox_id == &dm_members.member_two_inbox_id)
&& added_inbox_id != &commit.actor_inbox_id()
{
added_inboxes_valid = true;
}
}
}

// Verify remove member policy was not violated
Expand Down
8 changes: 4 additions & 4 deletions xmtp_mls/src/groups/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3068,16 +3068,16 @@ mod tests {
amal_dm.sync(&amal).await.unwrap();
bola_dm.sync(&bola).await.unwrap();
let is_amal_admin = amal_dm
.is_admin(amal.inbox_id(), &amal.mls_provider().unwrap())
.is_admin(amal.inbox_id(), amal.mls_provider().unwrap())
.unwrap();
let is_bola_admin = amal_dm
.is_admin(bola.inbox_id(), &bola.mls_provider().unwrap())
.is_admin(bola.inbox_id(), bola.mls_provider().unwrap())
.unwrap();
let is_amal_super_admin = amal_dm
.is_super_admin(amal.inbox_id(), &amal.mls_provider().unwrap())
.is_super_admin(amal.inbox_id(), amal.mls_provider().unwrap())
.unwrap();
let is_bola_super_admin = amal_dm
.is_super_admin(bola.inbox_id(), &bola.mls_provider().unwrap())
.is_super_admin(bola.inbox_id(), bola.mls_provider().unwrap())
.unwrap();
assert!(!is_amal_admin);
assert!(!is_bola_admin);
Expand Down
189 changes: 189 additions & 0 deletions xmtp_mls/src/storage/encrypted_store/consent_record.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
use crate::{impl_store, storage::StorageError};

use super::{
db_connection::DbConnection,
schema::consent_records::{self, dsl},
};
use diesel::{
backend::Backend,
deserialize::{self, FromSql, FromSqlRow},
expression::AsExpression,
prelude::*,
serialize::{self, IsNull, Output, ToSql},
sql_types::Integer,
sqlite::Sqlite,
upsert::excluded,
};
use serde::{Deserialize, Serialize};

/// StoredConsentRecord holds a serialized ConsentRecord
#[derive(Insertable, Queryable, Debug, Clone, PartialEq, Eq)]
#[diesel(table_name = consent_records)]
#[diesel(primary_key(entity_type, entity))]
pub struct StoredConsentRecord {
/// Enum, [`ConsentType`] representing the type of consent (group_id inbox_id, etc..)
pub entity_type: ConsentType,
/// Enum, [`ConsentState`] representing the state of consent (allowed, denied, etc..)
pub state: ConsentState,
/// The entity of what was consented (0x00 etc..)
pub entity: String,
}

impl StoredConsentRecord {
pub fn new(entity_type: ConsentType, state: ConsentState, entity: String) -> Self {
Self {
entity_type,
state,
entity,
}
}
}

impl_store!(StoredConsentRecord, consent_records);

impl DbConnection {
/// Returns the consent_records for the given entity up
pub fn get_consent_record(
&self,
entity: String,
entity_type: ConsentType,
) -> Result<Option<StoredConsentRecord>, StorageError> {
Ok(self.raw_query(|conn| {
dsl::consent_records
.filter(dsl::entity.eq(entity))
.filter(dsl::entity_type.eq(entity_type))
.first(conn)
.optional()
})?)
}

/// Insert consent_record, and replace existing entries
pub fn insert_or_replace_consent_record(
&self,
record: StoredConsentRecord,
) -> Result<(), StorageError> {
self.raw_query(|conn| {
diesel::insert_into(dsl::consent_records)
.values(&record)
.on_conflict((dsl::entity_type, dsl::entity))
.do_update()
.set(dsl::state.eq(excluded(dsl::state)))
.execute(conn)?;
Ok(())
})?;

Ok(())
}
}

#[repr(i32)]
#[derive(Debug, Copy, Clone, Serialize, Deserialize, Eq, PartialEq, AsExpression, FromSqlRow)]
#[diesel(sql_type = Integer)]
/// Type of consent record stored
pub enum ConsentType {
/// Consent is for a group
GroupId = 1,
/// Consent is for an inbox
InboxId = 2,
/// Consent is for an address
Address = 3,
}

impl ToSql<Integer, Sqlite> for ConsentType
where
i32: ToSql<Integer, Sqlite>,
{
fn to_sql<'b>(&'b self, out: &mut Output<'b, '_, Sqlite>) -> serialize::Result {
out.set_value(*self as i32);
Ok(IsNull::No)
}
}

impl FromSql<Integer, Sqlite> for ConsentType
where
i32: FromSql<Integer, Sqlite>,
{
fn from_sql(bytes: <Sqlite as Backend>::RawValue<'_>) -> deserialize::Result<Self> {
match i32::from_sql(bytes)? {
1 => Ok(ConsentType::GroupId),
2 => Ok(ConsentType::InboxId),
3 => Ok(ConsentType::Address),
x => Err(format!("Unrecognized variant {}", x).into()),
}
}
}

#[repr(i32)]
#[derive(Debug, Copy, Clone, Serialize, Deserialize, Eq, PartialEq, AsExpression, FromSqlRow)]
#[diesel(sql_type = Integer)]
/// The state of the consent
pub enum ConsentState {
/// Consent is allowed
Allowed = 1,
/// Consent is denied
Denied = 2,
}

impl ToSql<Integer, Sqlite> for ConsentState
where
i32: ToSql<Integer, Sqlite>,
{
fn to_sql<'b>(&'b self, out: &mut Output<'b, '_, Sqlite>) -> serialize::Result {
out.set_value(*self as i32);
Ok(IsNull::No)
}
}

impl FromSql<Integer, Sqlite> for ConsentState
where
i32: FromSql<Integer, Sqlite>,
{
fn from_sql(bytes: <Sqlite as Backend>::RawValue<'_>) -> deserialize::Result<Self> {
match i32::from_sql(bytes)? {
1 => Ok(ConsentState::Allowed),
2 => Ok(ConsentState::Denied),
x => Err(format!("Unrecognized variant {}", x).into()),
}
}
}

#[cfg(test)]
mod tests {
use crate::storage::encrypted_store::tests::with_connection;

use super::*;

fn generate_consent_record(
entity_type: ConsentType,
state: ConsentState,
entity: String,
) -> StoredConsentRecord {
StoredConsentRecord {
entity_type,
state,
entity,
}
}

#[test]
fn insert_and_read() {
with_connection(|conn| {
let inbox_id = "inbox_1";
let consent_record = generate_consent_record(
ConsentType::InboxId,
ConsentState::Denied,
inbox_id.to_string(),
);
let consent_record_entity = consent_record.entity.clone();

conn.insert_or_replace_consent_record(consent_record)
.expect("should store without error");

let consent_record = conn
.get_consent_record(inbox_id.to_owned(), ConsentType::InboxId)
.expect("query should work");

assert_eq!(consent_record.unwrap().entity, consent_record_entity);
});
}
}
1 change: 1 addition & 0 deletions xmtp_mls/src/storage/encrypted_store/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
//! `diesel print-schema` or use `cargo run update-schema` which will update the files for you.

pub mod association_state;
pub mod consent_record;
pub mod db_connection;
pub mod group;
pub mod group_intent;
Expand Down
9 changes: 9 additions & 0 deletions xmtp_mls/src/storage/encrypted_store/schema.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ diesel::table! {
}
}

diesel::table! {
consent_records (entity_type, entity) {
entity_type -> Integer,
state -> Integer,
entity -> Text,
}
}

diesel::table! {
group_intents (id) {
id -> Integer,
Expand Down Expand Up @@ -94,6 +102,7 @@ diesel::joinable!(group_messages -> groups (group_id));

diesel::allow_tables_to_appear_in_same_query!(
association_state,
consent_records,
group_intents,
group_messages,
groups,
Expand Down