Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

6064 – Refactor duplicated media creation methods #2223

Merged
merged 23 commits into from
Feb 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
ed377e0
[WIP] First part: move methods to Media
vasconsaurus Feb 17, 2025
fa98310
[WIP] make create_media and create_original_claim similar
vasconsaurus Feb 17, 2025
ac8664e
merge methods to create claim into one
vasconsaurus Feb 17, 2025
574319b
extract setting the media_type
vasconsaurus Feb 17, 2025
9bc9271
merge methods to create uplodaded media and fix a variable name
vasconsaurus Feb 17, 2025
f194c1f
merge create_link methods
vasconsaurus Feb 17, 2025
371bf03
do some cleanup
vasconsaurus Feb 17, 2025
e5f58d6
update methods names
vasconsaurus Feb 18, 2025
4442c7b
fix undefined method `match?'
vasconsaurus Feb 18, 2025
d914b85
move set_media_type back and make create_media and create_original_cl…
vasconsaurus Feb 18, 2025
007bc76
trying to combine methods and remove set_media_type
vasconsaurus Feb 18, 2025
fe841ae
Fix failing tests
vasconsaurus Feb 19, 2025
da90f9e
update condition on when to set_media_type
vasconsaurus Feb 19, 2025
4a2be9e
update to use ProjectMedia.create instead of create_project_media
vasconsaurus Feb 19, 2025
3d2f767
update to keep create_media only, instead of create_original_claim
vasconsaurus Feb 19, 2025
a2a97db
add ! to method name
vasconsaurus Feb 20, 2025
9c49d01
Update so we only send the necessary data to media
vasconsaurus Feb 20, 2025
d4ebac0
try to deal with codeclimate complains
vasconsaurus Feb 21, 2025
7507a0a
fix indentation
vasconsaurus Feb 21, 2025
00dd298
try to make this a bit clearer
vasconsaurus Feb 21, 2025
09625ae
remove deprecated tests
vasconsaurus Feb 24, 2025
d47e1ef
clean up
vasconsaurus Feb 26, 2025
5b99440
replace ProjectMedia.create by ProjectMedia.create!
vasconsaurus Feb 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions app/models/concerns/project_association.rb
Original file line number Diff line number Diff line change
Expand Up @@ -158,8 +158,8 @@ def is_unique

def set_media
unless self.url.blank? && self.quote.blank? && self.file.blank? && self.media_type != 'Blank'
m = self.create_media
self.media_id = m.id unless m.nil?
self.create_media!
self.media_id unless self.media_id.nil?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: this line should return self.media_id or assign a value to self.media_id

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The create_media method changes the object and assigns the value, if I'm not mistaken. So I think we should be able to just return self.media_id.

end
end
end
Expand Down
93 changes: 50 additions & 43 deletions app/models/concerns/project_media_creators.rb
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,11 @@ def create_annotation
end
end

def create_original_claim
claim = self.set_original_claim.strip
self.media = Media.find_or_create_from_original_claim(claim, self.team)
def create_media!
self.set_media_type if self.set_original_claim || self.media_type.blank?

media_type, media_content, additional_args = *self.media_arguments
self.media = Media.find_or_create_media_from_content(media_type, media_content, additional_args)
end

def set_quote_metadata
Expand Down Expand Up @@ -114,49 +116,54 @@ def add_source_creation_log

protected

def create_with_file(media_type = 'UploadedImage')
klass = media_type.constantize
m = klass.find_by(file: Media.filename(self.file)) || klass.new(file: self.file)
m.save! if m.new_record?
m
end

def create_claim
m = Claim.new
m.quote = self.quote
m.quote_attributions = self.quote_attributions
m.save!
m
end

def create_link
team = self.team || Team.current
pender_key = team.get_pender_key if team
url_from_pender = Link.normalized(self.url, pender_key)
Link.find_by(url: url_from_pender) || Link.create(url: self.url, pender_key: pender_key)
end

def create_media
m = nil
self.set_media_type if self.media_type.blank?
case self.media_type
when 'UploadedImage', 'UploadedVideo', 'UploadedAudio'
m = self.create_with_file(media_type)
when 'Claim'
m = self.create_claim
when 'Link'
m = self.create_link
when 'Blank'
m = Blank.create!
def set_media_type
original_claim = self.set_original_claim&.strip

if original_claim && original_claim.match?(/\A#{URI::DEFAULT_PARSER.make_regexp(['http', 'https'])}\z/)
uri = URI.parse(original_claim)
content_type = Net::HTTP.get_response(uri)['content-type']

case content_type
when /^image\//
self.media_type = 'UploadedImage'
when /^video\//
self.media_type = 'UploadedVideo'
when /^audio\//
self.media_type = 'UploadedAudio'
else
self.media_type = 'Link'
end
elsif original_claim || self.quote.present?
self.media_type = 'Claim'
elsif self.url.present?
self.media_type = 'Link'
end
m
end

def set_media_type
if !self.url.blank?
self.media_type = 'Link'
elsif !self.quote.blank?
self.media_type = 'Claim'
def media_arguments
media_type = self.media_type
original_claim = self.set_original_claim&.strip

if original_claim
case media_type
when 'UploadedImage', 'UploadedVideo', 'UploadedAudio'
[media_type, original_claim, { has_original_claim: true }]
when 'Claim'
[media_type, original_claim, { has_original_claim: true }]
when 'Link'
[media_type, original_claim, { team: self.team, has_original_claim: true }]
end
else
case media_type
when 'UploadedImage', 'UploadedVideo', 'UploadedAudio'
[media_type, self.file]
when 'Claim'
[media_type, self.quote, { quote_attributions: self.quote_attributions }]
when 'Link'
[media_type, self.url, { team: self.team }]
when 'Blank'
[media_type]
end
end
end

Expand Down
90 changes: 54 additions & 36 deletions app/models/media.rb
Original file line number Diff line number Diff line change
Expand Up @@ -77,24 +77,16 @@ def domain
''
end

def self.find_or_create_from_original_claim(claim, project_media_team)
if claim.match?(/\A#{URI::DEFAULT_PARSER.make_regexp(['http', 'https'])}\z/)
uri = URI.parse(claim)
content_type = Net::HTTP.get_response(uri)['content-type']
ext = File.extname(uri.path)

case content_type
when /^image\//
find_or_create_uploaded_file_media_from_original_claim('UploadedImage', claim, ext)
when /^video\//
find_or_create_uploaded_file_media_from_original_claim('UploadedVideo', claim, ext)
when /^audio\//
find_or_create_uploaded_file_media_from_original_claim('UploadedAudio', claim, ext)
else
find_or_create_link_media_from_original_claim(claim, project_media_team)
end
else
find_or_create_claim_media_from_original_claim(claim)
def self.find_or_create_media_from_content(media_type, media_content = nil, additional_args = {})
case media_type
when 'UploadedImage', 'UploadedVideo', 'UploadedAudio'
find_or_create_uploaded_file_media(media_content, media_type, additional_args)
when 'Claim'
find_or_create_claim_media(media_content, additional_args)
when 'Link'
find_or_create_link_media(media_content, additional_args)
when 'Blank'
Blank.create!
end
end

Expand Down Expand Up @@ -130,18 +122,6 @@ def set_original_claim_hash
self.original_claim_hash = Digest::MD5.hexdigest(original_claim) unless self.original_claim.blank?
end

def self.find_or_create_uploaded_file_media_from_original_claim(media_type, url, ext)
klass = media_type.constantize
existing_media = klass.find_by(original_claim_hash: Digest::MD5.hexdigest(url))

if existing_media
existing_media
else
file = download_file(url, ext)
klass.create!(file: file, original_claim: url)
end
end

def self.download_file(url, ext)
raise "Invalid URL when creating media from original claim attribute" unless url =~ /\A#{URI::DEFAULT_PARSER.make_regexp(['http', 'https'])}\z/

Expand All @@ -152,13 +132,51 @@ def self.download_file(url, ext)
file
end

def self.find_or_create_claim_media_from_original_claim(text)
Claim.find_by(original_claim_hash: Digest::MD5.hexdigest(text)) || Claim.create!(quote: text, original_claim: text)
def self.find_or_create_uploaded_file_media(file_media, media_type, additional_args = {})
has_original_claim = additional_args&.fetch(:has_original_claim, nil)
klass = media_type.constantize

if has_original_claim
existing_media = klass.find_by(original_claim_hash: Digest::MD5.hexdigest(file_media))

if existing_media
existing_media
else
uri = URI.parse(file_media)
ext = File.extname(uri.path)
file = download_file(file_media, ext)

klass.create!(file: file, original_claim: file_media)
end
else
m = klass.find_by(file: Media.filename(file_media)) || klass.new(file: file_media)
m.save! if m.new_record?
m
end
end

def self.find_or_create_claim_media(claim_media, additional_args = {})
has_original_claim = additional_args.fetch(:has_original_claim, nil)
quote_attributions = additional_args.fetch(:quote_attributions, nil)

if has_original_claim
Claim.find_by(original_claim_hash: Digest::MD5.hexdigest(claim_media)) || Claim.create!(quote: claim_media, original_claim: claim_media)
else
Claim.create!(quote: claim_media, quote_attributions: quote_attributions)
end
end

def self.find_or_create_link_media_from_original_claim(url, project_media_team)
pender_key = project_media_team.get_pender_key if project_media_team
url_from_pender = Link.normalized(url, pender_key)
Link.find_by(url: url_from_pender) || Link.find_by(original_claim_hash: Digest::MD5.hexdigest(url)) || Link.create!(url: url, pender_key: pender_key, original_claim: url)
def self.find_or_create_link_media(link_media, additional_args = {})
has_original_claim = additional_args.fetch(:has_original_claim, nil)
project_media_team = additional_args.fetch(:team, nil)

pender_key = project_media_team&.get_pender_key if project_media_team
url_from_pender = Link.normalized(link_media, pender_key)

if has_original_claim
Link.find_by(url: url_from_pender) || Link.find_by(original_claim_hash: Digest::MD5.hexdigest(link_media)) || Link.create!(url: link_media, pender_key: pender_key, original_claim: link_media)
else
Link.find_by(url: url_from_pender) || Link.create(url: link_media, pender_key: pender_key)
end
end
end
2 changes: 1 addition & 1 deletion app/models/project_media.rb
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ class ProjectMedia < ApplicationRecord
validates_presence_of :custom_title, if: proc { |pm| pm.title_field == 'custom_title' }

before_validation :set_team_id, :set_channel, :set_project_id, on: :create
before_validation :create_original_claim, if: proc { |pm| pm.set_original_claim.present? }, on: :create
before_validation :create_media!, if: proc { |pm| pm.set_original_claim.present? }, on: :create
after_create :create_annotation, :create_metrics_annotation, :send_slack_notification, :create_relationship, :create_team_tasks, :create_claim_description_and_fact_check, :create_tags_in_background
after_create :add_source_creation_log, unless: proc { |pm| pm.source_id.blank? }
after_commit :apply_rules_and_actions_on_create, :set_quote_metadata, :notify_team_bots_create, on: [:create]
Expand Down
6 changes: 3 additions & 3 deletions test/controllers/graphql_controller_12_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -745,7 +745,7 @@ def teardown

t = create_team
p = create_project team: t
pm = create_project_media team: t, set_original_claim: url
pm = ProjectMedia.create!(team: t, set_original_claim: url)

assert_not_nil pm

Expand Down Expand Up @@ -807,7 +807,7 @@ def teardown
t.settings[:languages] << 'pt'
t.save!
p = create_project team: t
pm = create_project_media team: t, set_original_claim: url
pm = ProjectMedia.create!(team: t, set_original_claim: url)
c = create_claim_description project_media: pm
fc_1 = create_fact_check claim_description: c

Expand Down Expand Up @@ -870,7 +870,7 @@ def teardown
WebMock.stub_request(:get, pender_url).with({ query: { url: url } }).to_return(body: response_body)

t = create_team
pm = create_project_media team: t, set_original_claim: url
pm = ProjectMedia.create!(team: t, set_original_claim: url)
cd = create_claim_description project_media: pm
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest you replace ProjectMedia.create by ProjectMedia.create! - just to make it more verbose if it fails.

fc = create_fact_check claim_description: cd

Expand Down
48 changes: 0 additions & 48 deletions test/models/concerns/project_media_creators_test.rb

This file was deleted.

14 changes: 6 additions & 8 deletions test/models/media_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -630,7 +630,7 @@ def setup

test "Claim Media: should save the original_claim and original_claim_hash when created from original claim" do
claim = 'This is a claim.'
claim_media = Media.find_or_create_claim_media_from_original_claim(claim)
claim_media = Media.find_or_create_claim_media(claim, { has_original_claim: true })

assert_not_nil claim_media.original_claim_hash
assert_not_nil claim_media.original_claim
Expand All @@ -645,7 +645,7 @@ def setup

test "Claim Media: should not create duplicate media if media with original_claim_hash exists" do
assert_difference 'Claim.count', 1 do
2.times { Media.find_or_create_claim_media_from_original_claim('This is a claim.') }
2.times { Media.find_or_create_claim_media('This is a claim.', { has_original_claim: true }) }
end
end

Expand All @@ -664,7 +664,7 @@ def setup
}.to_json
WebMock.stub_request(:get, pender_url).with(query: { url: link_url }).to_return(body: link_response)

link_media = Media.find_or_create_link_media_from_original_claim(link_url, team)
link_media = Media.find_or_create_link_media(link_url, { team: team, has_original_claim: true })

assert_not_nil link_media.original_claim_hash
assert_not_nil link_media.original_claim
Expand Down Expand Up @@ -706,7 +706,7 @@ def setup
WebMock.stub_request(:get, pender_url).with(query: { url: link_url }).to_return(body: link_response)

assert_difference 'Link.count', 1 do
2.times { Media.find_or_create_link_media_from_original_claim(link_url, team) }
2.times { Media.find_or_create_link_media(link_url, { team: team, has_original_claim: true }) }
end
end

Expand All @@ -715,10 +715,9 @@ def setup
file.write(File.read(File.join(Rails.root, 'test', 'data', 'rails.mp3')))
file.rewind
audio_url = "http://example.com/#{file.path.split('/').last}"
ext = File.extname(URI.parse(audio_url).path)
WebMock.stub_request(:get, audio_url).to_return(body: file.read, headers: { 'Content-Type' => 'audio/mp3' })

uploaded_media = Media.find_or_create_uploaded_file_media_from_original_claim('UploadedAudio', audio_url, ext)
uploaded_media = Media.find_or_create_uploaded_file_media(audio_url, 'UploadedAudio', { has_original_claim: true })

assert_not_nil uploaded_media.original_claim_hash
assert_not_nil uploaded_media.original_claim
Expand All @@ -738,11 +737,10 @@ def setup
file.write(File.read(File.join(Rails.root, 'test', 'data', 'rails.mp3')))
file.rewind
audio_url = "http://example.com/#{file.path.split('/').last}"
ext = File.extname(URI.parse(audio_url).path)
WebMock.stub_request(:get, audio_url).to_return(body: file.read, headers: { 'Content-Type' => 'audio/mp3' })

assert_difference 'UploadedAudio.count', 1 do
2.times { Media.find_or_create_uploaded_file_media_from_original_claim('UploadedAudio', audio_url, ext) }
2.times { Media.find_or_create_uploaded_file_media(audio_url, 'UploadedAudio', { has_original_claim: true }) }
end
end
end
Expand Down
Loading
Loading