Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CV2-2648 Refactor Tipline requests #1743

Merged
merged 60 commits into from
Jan 28, 2024
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
03b1e5e
CV2-2648: add tipline request model
melsawy Nov 27, 2023
7fb3280
CV2-2648: replace smooch_data queries with TiplineRequest
melsawy Nov 27, 2023
d20081c
CV2-2648: fix cached fields
melsawy Nov 27, 2023
fb2b0be
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Nov 27, 2023
0c12a55
CV2-2648: remove unneeded migration
melsawy Nov 27, 2023
0349c1a
CV2-2648: fix tests 1/2
melsawy Nov 28, 2023
fd11aad
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Nov 28, 2023
0034747
CV2-2648: fix syntex
melsawy Nov 28, 2023
40b2594
CV2-2648: fix tests 2/2
melsawy Nov 29, 2023
8471ea6
CV2-2648: rename methods and args
melsawy Nov 29, 2023
8d06298
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Nov 30, 2023
87d6df2
CV2-2648: fix typo
melsawy Nov 30, 2023
2032739
CV2-2648: fix seed file and change TiplineRequest column types
melsawy Dec 3, 2023
f15e1d7
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Dec 3, 2023
a3c4517
CV2-2648: fix check statistics
melsawy Dec 4, 2023
5edda56
CV2-2648: fix tests
melsawy Dec 4, 2023
3649148
CV2-2648: add more tests
melsawy Dec 4, 2023
3d2f03d
CV2-2648
melsawy Dec 4, 2023
22c523b
CV2-2648: test coverage
melsawy Dec 5, 2023
281a434
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Dec 5, 2023
c1be772
CV2-2648: cleanup
melsawy Dec 5, 2023
fdc268d
CV2-2648: apply PR comments
melsawy Dec 10, 2023
d392352
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Dec 10, 2023
8ec8d40
CV2-2648: fix tests
melsawy Dec 11, 2023
ebcdd07
CV2-2648: fix tests
melsawy Dec 11, 2023
7f154c4
CV2-2648: fix tests 1/2
melsawy Dec 12, 2023
1b83b61
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Dec 12, 2023
6f1652a
CV2-2648: add rake task to migrate item requests
melsawy Dec 13, 2023
78fdd39
CV2-2648: add more rake tasks
melsawy Dec 14, 2023
d852f2c
CV2-2648: cleanup
melsawy Dec 14, 2023
ad78372
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Dec 16, 2023
4f1a393
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
caiosba Dec 20, 2023
d455579
Debug hanging test
caiosba Dec 20, 2023
0f0cbc0
Better debugging for hanging test
caiosba Dec 20, 2023
213dfde
Looks like this test is hanging - commenting to be sure
caiosba Dec 20, 2023
9201a4d
CV2-2648: set start value for TiplineRequest.id and fix tests
melsawy Dec 25, 2023
c6e7ee3
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Dec 25, 2023
c89eb6f
Merge branch 'refactor/CV2-2648-refactor-tipline-requests' of github.…
melsawy Dec 25, 2023
0126d76
Trying to fix test
caiosba Dec 26, 2023
127b1f3
Trying to fix test
caiosba Dec 26, 2023
fc2a2e7
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
caiosba Dec 26, 2023
45946b8
Trying to fix tests
caiosba Dec 26, 2023
b30cb07
Trying to debug failing test
caiosba Dec 26, 2023
326cc2c
Revert "Trying to debug failing test"
caiosba Dec 26, 2023
e31716a
Revert "Trying to fix tests"
caiosba Dec 26, 2023
a4f0ce7
Fixing tests
caiosba Dec 28, 2023
5b0d7c2
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
caiosba Jan 3, 2024
a99b61d
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Jan 8, 2024
d683076
CV2-2648: apply PR comments
melsawy Jan 9, 2024
16f4ca2
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Jan 9, 2024
90ad25b
Merge branch 'refactor/CV2-2648-refactor-tipline-requests' of github.…
melsawy Jan 9, 2024
3df445e
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Jan 18, 2024
26ed154
CV2-2648: allow rake task to accept args(slug and batch size)
melsawy Jan 21, 2024
c15b4b6
CV2-2648: Get reports received based on smooch_report_received_at field
melsawy Jan 24, 2024
ce239b6
Fixing a bug
caiosba Jan 24, 2024
45b779f
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Jan 25, 2024
fb75157
Merge branch 'refactor/CV2-2648-refactor-tipline-requests' of github.…
melsawy Jan 25, 2024
7a9c11b
CV2-2648: fix average time for statistics data
melsawy Jan 27, 2024
39fc1ff
Merge branch 'develop' into refactor/CV2-2648-refactor-tipline-requests
melsawy Jan 27, 2024
69b6f16
CV2-2648: remove unneeded file
melsawy Jan 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions app/graph/mutations/tipline_message_mutations.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ class Send < Mutations::BaseMutation
field :success, GraphQL::Types::Boolean, null: true

def resolve(in_reply_to_id: nil, message: nil)
request = Annotation.find(in_reply_to_id).load
request = TiplineRequest.find(in_reply_to_id)
ability = context[:ability] || Ability.new
success = false
if Team.current&.id && User.current&.id && ability.can?(:send, TiplineMessage.new(team: Team.current)) && request.annotated.team_id == Team.current.id
if Team.current&.id && User.current&.id && ability.can?(:send, TiplineMessage.new(team: Team.current)) && request.team_id == Team.current.id
success = Bot::Smooch.reply_to_request_with_custom_message(request, message)
end
{ success: success }
Expand Down
8 changes: 4 additions & 4 deletions app/graph/types/tipline_request_type.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,9 @@ class TiplineRequestType < DefaultObject
implements GraphQL::Types::Relay::Node

field :dbid, GraphQL::Types::Int, null: true
field :value_json, JsonStringType, null: true
field :annotation, AnnotationType, null: true
field :annotation_id, GraphQL::Types::Int, null: true
field :associated_graphql_id, GraphQL::Types::String, null: true
field :associated_id, GraphQL::Types::Int, null: true
field :associated_type, GraphQL::Types::String, null: true
field :smooch_data, JsonStringType, null: true
field :smooch_user_slack_channel_url, GraphQL::Types::String, null: true
field :smooch_user_external_identifier, GraphQL::Types::String, null: true
field :smooch_report_received_at, GraphQL::Types::Int, null: true
Expand All @@ -16,4 +15,5 @@ class TiplineRequestType < DefaultObject
field :smooch_report_sent_at, GraphQL::Types::Int, null: true
field :smooch_report_correction_sent_at, GraphQL::Types::Int, null: true
field :smooch_request_type, GraphQL::Types::String, null: true
field :associated_graphql_id, GraphQL::Types::String, null: true
end
6 changes: 5 additions & 1 deletion app/models/ability.rb
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ def editor_perms
can [:cud], DynamicAnnotation::Field do |obj|
obj.annotation.team&.id == @context_team.id
end
can [:create, :update, :read, :destroy], [Account, Source, TiplineNewsletter, TiplineResource], :team_id => @context_team.id
can [:create, :update, :read, :destroy], [Account, Source, TiplineNewsletter, TiplineResource, TiplineRequest], :team_id => @context_team.id
can [:cud], AccountSource, source: { team: { team_users: { team_id: @context_team.id }}}
%w(annotation comment dynamic task tag).each do |annotation_type|
can [:cud], annotation_type.classify.constantize do |obj|
Expand Down Expand Up @@ -135,6 +135,10 @@ def collaborator_perms
obj.team&.id == @context_team.id && !obj.annotated_is_trashed?
end
end
can [:cud], TiplineRequest do |obj|
is_trashed = obj.associated.respond_to?(:archived) && obj.associated.archived == CheckArchivedFlags::FlagCodes::TRASHED
obj.team_id == @context_team.id && !is_trashed
end
can [:create, :destroy], Assignment do |obj|
type = obj.assigned_type
obj = obj.assigned
Expand Down
50 changes: 25 additions & 25 deletions app/models/bot/smooch.rb
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ class CapiUnhandledMessageWarning < MessageDeliveryError; end
include SmoochCapi
include SmoochStrings
include SmoochMenus
include SmoochFields
include SmoochLanguage
include SmoochBlocking

Expand All @@ -40,19 +39,18 @@ def report_image
self.get_dynamic_annotation('report_design')&.report_design_image_url
end

def get_deduplicated_smooch_annotations
def get_deduplicated_tipline_requests
uids = []
annotations = []
tipline_requests = []
ProjectMedia.where(id: self.related_items_ids).each do |pm|
pm.get_annotations('smooch').find_each do |annotation|
data = JSON.parse(annotation.load.get_field_value('smooch_data'))
uid = data['authorId']
pm.tipline_requests.find_each do |tr|
uid = tr.tipline_user_uid
next if uids.include?(uid)
uids << uid
annotations << annotation
tipline_requests << tr
end
end
annotations
tipline_requests
end
end

Expand Down Expand Up @@ -125,7 +123,9 @@ def self.inherit_status_and_send_report(rid)
'_id': Digest::MD5.hexdigest([self.action.to_s, Time.now.to_f.to_s].join(':')),
authorId: id,
type: 'text',
text: message[1]
text: message[1],
source: { type: "whatsapp" },
language: 'en'
}.with_indifferent_access
Bot::Smooch.save_message_later(payload, app_id)
end
Expand Down Expand Up @@ -602,10 +602,11 @@ def self.user_received_report(message)
original = begin JSON.parse(original) rescue {} end
if original['fallback_template'] =~ /report/
pmids = ProjectMedia.find(original['project_media_id']).related_items_ids
DynamicAnnotation::Field.joins(:annotation).where(field_name: 'smooch_data', 'annotations.annotated_type' => 'ProjectMedia', 'annotations.annotated_id' => pmids).where("value_json ->> 'authorId' = ?", message['appUser']['_id']).each do |f|
a = f.annotation.load
a.set_fields = { smooch_report_received: Time.now.to_i }.to_json
a.save!
TiplineRequest.where(associated_type: 'ProjectMedia', associated_id: pmids, tipline_user_uid: message['appUser']['_id']).find_each do |tr|
field_name = tr.smooch_report_received_at == 0 ? 'smooch_report_received_at' : 'smooch_report_update_received_at'
tr.send("#{field_name}=", Time.now.to_i)
tr.skip_check_ability = true
tr.save!
end
end
end
Expand Down Expand Up @@ -934,15 +935,15 @@ def self.send_report_to_users(pm, action)
report = parent.get_annotations('report_design').last&.load
return if report.nil?
last_published_at = report.get_field_value('last_published').to_i
parent.get_deduplicated_smooch_annotations.each do |annotation|
data = JSON.parse(annotation.load.get_field_value('smooch_data'))
parent.get_deduplicated_tipline_requests.each do |tipline_request|
data = tipline_request.smooch_data
self.get_installation(self.installation_setting_id_keys, data['app_id']) if self.config.blank?
self.send_correction_to_user(data, parent, annotation, last_published_at, action, report.get_field_value('published_count').to_i) unless self.config['smooch_disabled']
self.send_correction_to_user(data, parent, tipline_request, last_published_at, action, report.get_field_value('published_count').to_i) unless self.config['smooch_disabled']
end
end

def self.send_correction_to_user(data, pm, annotation, last_published_at, action, published_count = 0)
subscribed_at = annotation.created_at
def self.send_correction_to_user(data, pm, tipline_request, last_published_at, action, published_count = 0)
subscribed_at = tipline_request.created_at
self.get_platform_from_message(data)
uid = data['authorId']
lang = data['language']
Expand All @@ -959,10 +960,9 @@ def self.send_correction_to_user(data, pm, annotation, last_published_at, action
self.send_report_to_user(uid, data, pm, lang, 'fact_check_report')
end
unless field_name.blank?
annotation = annotation.load
annotation.skip_check_ability = true
annotation.set_fields = { "#{field_name}": Time.now.to_i }.to_json
annotation.save!
tipline_request.skip_check_ability = true
tipline_request.send("#{field_name}=", Time.now.to_i)
tipline_request.save!
end
end

Expand Down Expand Up @@ -1017,11 +1017,11 @@ def self.send_report_from_parent_to_child(parent_id, target_id)
parent = ProjectMedia.where(id: parent_id).last
child = ProjectMedia.where(id: target_id).last
return if parent.nil? || child.nil?
child.get_annotations('smooch').find_each do |annotation|
data = JSON.parse(annotation.load.get_field_value('smooch_data'))
child.tipline_requests.find_each do |tr|
data = tr.smooch_data
self.get_platform_from_message(data)
self.get_installation(self.installation_setting_id_keys, data['app_id']) if self.config.blank?
self.send_report_to_user(data['authorId'], data, parent, data['language'], 'fact_check_report')
self.send_report_to_user(tr.tipline_user_uid, data, parent, tr.language, 'fact_check_report')
end
end

Expand Down
17 changes: 3 additions & 14 deletions app/models/cluster.rb
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,9 @@ def claim_descriptions
recalculate: :recalculate_requests_count,
update_on: [
{
model: Dynamic,
if: proc { |d| d.annotation_type == 'smooch' && d.annotated_type == 'ProjectMedia' },
affected_ids: proc { |d| ProjectMedia.where(id: d.annotated.related_items_ids).group(:cluster_id).count.keys.reject{ |cid| cid.nil? } },
model: TiplineRequest,
if: proc { |tr| tr.associated_type == 'ProjectMedia' },
affected_ids: proc { |tr| ProjectMedia.where(id: tr.associated.related_items_ids).group(:cluster_id).count.keys.reject{ |cid| cid.nil? } },
events: {
create: :cached_field_cluster_requests_count_create,
destroy: :cached_field_cluster_requests_count_destroy
Expand Down Expand Up @@ -154,14 +154,3 @@ def update_elasticsearch
ElasticSearchWorker.perform_in(1.second, YAML::dump(model), YAML::dump(options), 'update_doc')
end
end


Dynamic.class_eval do
def cached_field_cluster_requests_count_create(target)
target.requests_count + 1
end

def cached_field_cluster_requests_count_destroy(target)
target.requests_count - 1
end
end
1 change: 1 addition & 0 deletions app/models/concerns/project_media_associations.rb
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ module ProjectMediaAssociations
has_one :claim_description, dependent: :destroy
belongs_to :cluster, counter_cache: :project_medias_count, optional: true
belongs_to :source, optional: true
has_many :tipline_requests, as: :associated
has_annotations
end
end
48 changes: 22 additions & 26 deletions app/models/concerns/project_media_cached_fields.rb
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,9 @@ def title_or_description_update
recalculate: :recalculate_requests_count,
update_on: [
{
model: Dynamic,
if: proc { |d| d.annotation_type == 'smooch' && d.annotated_type == 'ProjectMedia' },
affected_ids: proc { |d| [d.annotated_id] },
model: TiplineRequest,
if: proc { |tr| tr.associated_type == 'ProjectMedia' },
affected_ids: proc { |tr| [tr.associated_id] },
events: {
create: :recalculate,
destroy: :recalculate,
Expand All @@ -133,9 +133,9 @@ def title_or_description_update
recalculate: :recalculate_demand,
update_on: [
{
model: Dynamic,
if: proc { |d| d.annotation_type == 'smooch' && d.annotated_type == 'ProjectMedia' },
affected_ids: proc { |d| d.annotated.related_items_ids },
model: TiplineRequest,
if: proc { |tr| tr.associated_type == 'ProjectMedia' },
affected_ids: proc { |tr| tr.associated.related_items_ids },
events: {
create: :recalculate,
}
Expand All @@ -158,9 +158,9 @@ def title_or_description_update
recalculate: :recalculate_last_seen,
update_on: [
{
model: Dynamic,
if: proc { |d| d.annotation_type == 'smooch' && d.annotated_type == 'ProjectMedia' },
affected_ids: proc { |d| d.annotated&.related_items_ids.to_a },
model: TiplineRequest,
if: proc { |tr| tr.associated_type == 'ProjectMedia' },
affected_ids: proc { |tr| tr.associated&.related_items_ids.to_a },
events: {
create: :recalculate,
}
Expand Down Expand Up @@ -459,9 +459,9 @@ def title_or_description_update
recalculate: :recalculate_positive_tipline_search_results_count,
update_on: [
{
model: DynamicAnnotation::Field,
if: proc { |f| f.field_name == 'smooch_request_type' && f.value == 'relevant_search_result_requests' },
affected_ids: proc { |f| [f.annotation&.annotated_id.to_i] },
model: TiplineRequest,
if: proc { |tr| tr.smooch_request_type == 'relevant_search_result_requests' },
affected_ids: proc { |tr| [tr.associated_id] },
events: {
save: :recalculate,
destroy: :recalculate,
Expand All @@ -474,9 +474,9 @@ def title_or_description_update
recalculate: :recalculate_tipline_search_results_count,
update_on: [
{
model: DynamicAnnotation::Field,
if: proc { |f| f.field_name == 'smooch_request_type' && ['relevant_search_result_requests', 'irrelevant_search_result_requests', 'timeout_search_requests'].include?(f.value) },
affected_ids: proc { |f| [f.annotation&.annotated_id.to_i] },
model: TiplineRequest,
if: proc { |tr| ['relevant_search_result_requests', 'irrelevant_search_result_requests', 'timeout_search_requests'].include?(tr.smooch_request_type) },
affected_ids: proc { |tr| [tr.associated_id] },
events: {
save: :recalculate,
destroy: :recalculate,
Expand Down Expand Up @@ -507,7 +507,7 @@ def recalculate_related_count
end

def recalculate_requests_count
Dynamic.where(annotation_type: 'smooch', annotated_id: self.id).count
TiplineRequest.where(associated_type: 'ProjectMedia', associated_id: self.id).count
end

def recalculate_demand
Expand All @@ -517,10 +517,10 @@ def recalculate_demand
end

def recalculate_last_seen
# If it’s a main/parent item, last_seen is related to any request (smooch annotation) to that own ProjectMedia or any similar/child ProjectMedia
# If it’s not a main item (so, single or child, a.k.a. “confirmed match” or “suggestion”), then last_seen is related only to smooch annotations (requests) related to that ProjectMedia.
# If it’s a main/parent item, last_seen is related to any tipline request to that own ProjectMedia or any similar/child ProjectMedia
# If it’s not a main item (so, single or child, a.k.a. “confirmed match” or “suggestion”), then last_seen is related only to tipline requests related to that ProjectMedia.
ids = self.is_parent ? self.related_items_ids : self.id
v1 = Dynamic.where(annotation_type: 'smooch', annotated_id: ids).order('created_at DESC').first&.created_at || 0
v1 = TiplineRequest.where(associated_type: 'ProjectMedia', associated_id: ids).order('created_at DESC').first&.created_at || 0
v2 = ProjectMedia.where(id: ids).order('created_at DESC').first&.created_at || 0
[v1, v2].max.to_i
end
Expand Down Expand Up @@ -655,16 +655,12 @@ def cached_field_published_by_es(value)
end

def recalculate_positive_tipline_search_results_count
DynamicAnnotation::Field.where(annotation_type: 'smooch',field_name: 'smooch_request_type', value: 'relevant_search_result_requests')
.joins('INNER JOIN annotations a ON a.id = dynamic_annotation_fields.annotation_id')
.where('a.annotated_type = ? AND a.annotated_id = ?', 'ProjectMedia', self.id).count
TiplineRequest.where(associated_type: 'ProjectMedia', associated_id: self.id, smooch_request_type: 'relevant_search_result_requests').count
end

def recalculate_tipline_search_results_count
DynamicAnnotation::Field.where(annotation_type: 'smooch',field_name: 'smooch_request_type')
.where('value IN (?)', ['"relevant_search_result_requests"', '"irrelevant_search_result_requests"', '"timeout_search_requests"'])
.joins('INNER JOIN annotations a ON a.id = dynamic_annotation_fields.annotation_id')
.where('a.annotated_type = ? AND a.annotated_id = ?', 'ProjectMedia', self.id).count
types = ["relevant_search_result_requests", "irrelevant_search_result_requests", "timeout_search_requests"]
TiplineRequest.where(associated_type: 'ProjectMedia', associated_id: self.id, smooch_request_type: types).count
end
end

Expand Down
Loading
Loading