Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Outreachy Round 27] Stop storing liftwing features for non-wikidata wikis. #5682

Merged
merged 3 commits into from
Mar 11, 2024
Merged

[Outreachy Round 27] Stop storing liftwing features for non-wikidata wikis. #5682

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5c462e8
Do not save liftwing features in the revision features field for non-…
gabina Feb 27, 2024
c1c1478
Make the RevisionFeedbackController use the LiftWingApi directly (ins…
gabina Feb 28, 2024
1d99030
Improve comments
gabina Feb 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions app/controllers/revision_feedback_controller.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# frozen_string_literal: true

require_dependency "#{Rails.root}/lib/revision_feedback_service"
require_dependency "#{Rails.root}/lib/importers/revision_score_importer"
require_dependency "#{Rails.root}/lib/lift_wing_api"

class RevisionFeedbackController < ApplicationController
def index
set_latest_revision_id
return if @rev_id.nil?
revision_data = RevisionScoreImporter.new.fetch_data_for_revision_id(@rev_id)
@feedback = RevisionFeedbackService.new(revision_data[:features]).feedback
revision_data = LiftWingApi.new(@wiki).get_revision_data([@rev_id])[@rev_id.to_s]
@feedback = RevisionFeedbackService.new(revision_data['features']).feedback
@user_feedback = Assignment.find(params['assignment_id']).assignment_suggestions
@rating = revision_data[:rating]
@rating = revision_data['prediction']
Comment on lines +10 to +13
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the easiest way to fix that for now. Instead of using the RevisionScoreImporter, it uses the LiftWingApi directly. This is because the RevisionScoreImporter won't return the LiftWing features or the predictions for en.wikipedia (and this controller ony works for en.wikipedia).

I can still create an issue to move this to the frontend if we think that's better.

end

private
Expand Down
13 changes: 10 additions & 3 deletions lib/revision_score_api_handler.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,16 @@ def complete_score(score)
# Fetch the value for 'deleted, or default to 'false if not present.
completed_score['deleted'] = score.fetch('deleted', false)

# Ensure 'features' is a hash.
features = score.fetch('features', {}) || {}
completed_score['features'] = features.deep_merge({ 'num_ref' => score.fetch('num_ref', nil) })
# Ensure 'features' is a hash or nil.
# For Wikidata, use the 'features' key in the score. Otherwise, use 'num_ref' key.
completed_score['features'] =
if @wiki.project == 'wikidata'
score.fetch('features', nil)
else
# If there was an error hitting the API, set features to nil. Otherwise, use the 'num_ref'.
score.fetch('num_ref').nil? ? nil : { 'num_ref' => score['num_ref'] }
end

completed_score
end

Expand Down
9 changes: 4 additions & 5 deletions spec/lib/importers/revision_score_importer_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@
later_score = later_revision.wp10.to_f
expect(early_score).to be_between(0, 100)
expect(later_score).to be_between(early_score, 100)
expect(later_revision.features['feature.wikitext.revision.external_links']).to eq(12)
expect(later_revision.features['num_ref']).to eq(13)
end
end
Expand All @@ -74,7 +73,7 @@
revision = article.revisions.first
expect(revision.deleted).to eq(true)
expect(revision.wp10).to be_nil
expect(revision.features).to eq({ 'num_ref' => nil })
expect(revision.features).to eq({})
end
end

Expand All @@ -93,7 +92,7 @@
revision = article.revisions.first
expect(revision.deleted).to eq(true)
expect(revision.wp10).to be_nil
expect(revision.features).to eq({ 'num_ref' => nil })
expect(revision.features).to eq({})
end
end

Expand Down Expand Up @@ -124,7 +123,7 @@
# no value changed for the revision
revision = Revision.find_by(mw_rev_id: 662106477)
expect(revision.wp10).to be_nil
expect(revision.features).to eq({ 'num_ref' => nil })
expect(revision.features).to eq({})
expect(revision.deleted).to eq(false)
end

Expand Down Expand Up @@ -169,7 +168,7 @@

it 'returns a hash with a predicted rating and features' do
VCR.use_cassette 'revision_scores/single_revision' do
expect(subject[:features]).to have_key('feature.wikitext.revision.wikilinks')
expect(subject[:features]).to have_key(Revision::REFERENCE_COUNT)
expect(subject[:rating]).to eq('Stub')
end
end
Expand Down
45 changes: 19 additions & 26 deletions spec/lib/revision_score_api_handler_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,15 @@
expect(subject).to be_a(Hash)
expect(subject.dig('829840090', 'wp10').to_f).to eq(62.805729915108664)
expect(subject.dig('829840090', 'features')).to be_a(Hash)
expect(subject.dig('829840090', 'features',
'feature.wikitext.revision.ref_tags')).to eq(132)
expect(subject.dig('829840090', 'features', 'num_ref')).to eq(132)
# Only num_ref feature is stored. LiftWing features are discarded.
expect(subject.dig('829840090', 'features')).to eq({ 'num_ref' => 132 })
expect(subject.dig('829840090', 'deleted')).to eq(false)
expect(subject.dig('829840090', 'prediction')).to eq('B')

expect(subject.dig('829840091', 'wp10').to_f).to eq(39.507631367268004)
expect(subject.dig('829840091', 'features')).to be_a(Hash)
expect(subject.dig('829840091', 'features',
'feature.wikitext.revision.ref_tags')).to eq(1)
expect(subject.dig('829840091', 'features', 'num_ref')).to eq(1)
# Only num_ref feature is stored. LiftWing features are discarded.
expect(subject.dig('829840091', 'features')).to eq({ 'num_ref' => 1 })
expect(subject.dig('829840091', 'deleted')).to eq(false)
expect(subject.dig('829840091', 'prediction')).to eq('C')
end
Expand All @@ -48,21 +46,16 @@
.to_raise(Errno::ETIMEDOUT)

expect(subject).to be_a(Hash)

expect(subject.dig('829840090', 'wp10').to_f).to eq(62.805729915108664)
expect(subject.dig('829840090', 'features')).to be_a(Hash)
expect(subject.dig('829840090', 'features',
'feature.wikitext.revision.ref_tags')).to eq(132)
expect(subject.dig('829840090', 'features').key?('num_ref')).to eq(true)
expect(subject.dig('829840090', 'features', 'num_ref')).to eq(nil)
expect(subject.dig('829840090')).to have_key('features')
expect(subject.dig('829840090', 'features')).to be_nil
expect(subject.dig('829840090', 'deleted')).to eq(false)
expect(subject.dig('829840090', 'prediction')).to eq('B')

expect(subject.dig('829840091', 'wp10').to_f).to eq(39.507631367268004)
expect(subject.dig('829840091', 'features')).to be_a(Hash)
expect(subject.dig('829840091', 'features',
'feature.wikitext.revision.ref_tags')).to eq(1)
expect(subject.dig('829840091', 'features').key?('num_ref')).to eq(true)
expect(subject.dig('829840091', 'features', 'num_ref')).to eq(nil)
expect(subject.dig('829840091')).to have_key('features')
expect(subject.dig('829840091', 'features')).to be_nil
expect(subject.dig('829840091', 'deleted')).to eq(false)
expect(subject.dig('829840091', 'prediction')).to eq('C')
end
Expand All @@ -76,9 +69,9 @@
.to_raise(Errno::ETIMEDOUT)
expect(subject).to be_a(Hash)
expect(subject.dig('829840090')).to eq({ 'wp10' => nil,
'features' => { 'num_ref' => nil }, 'deleted' => false, 'prediction' => nil })
'features' => nil, 'deleted' => false, 'prediction' => nil })
expect(subject.dig('829840091')).to eq({ 'wp10' => nil,
'features' => { 'num_ref' => nil }, 'deleted' => false, 'prediction' => nil })
'features' => nil, 'deleted' => false, 'prediction' => nil })
end
end

Expand All @@ -97,17 +90,17 @@
expect(subject.dig('144495297', 'features')).to be_a(Hash)
expect(subject.dig('144495297', 'features',
'feature.len(<datasource.wikidatawiki.revision.references>)')).to eq(2)
expect(subject.dig('144495297', 'features').key?('num_ref')).to eq(true)
expect(subject.dig('144495297', 'features', 'num_ref')).to eq(nil)
# 'num_ref' key doesn't exist for wikidata features
expect(subject.dig('144495297', 'features').key?('num_ref')).to eq(false)
expect(subject.dig('144495297', 'deleted')).to eq(false)
expect(subject.dig('144495297', 'prediction')).to eq('D')

expect(subject.dig('144495298', 'wp10').to_f).to eq(0)
expect(subject.dig('144495298', 'features')).to be_a(Hash)
expect(subject.dig('144495298', 'features',
'feature.len(<datasource.wikidatawiki.revision.references>)')).to eq(0)
expect(subject.dig('144495298', 'features').key?('num_ref')).to eq(true)
expect(subject.dig('144495298', 'features', 'num_ref')).to eq(nil)
# 'num_ref' key doesn't exist for wikidata features
expect(subject.dig('144495298', 'features').key?('num_ref')).to eq(false)
expect(subject.dig('144495298', 'deleted')).to eq(false)
expect(subject.dig('144495298', 'prediction')).to eq('E')
end
Expand All @@ -118,9 +111,9 @@
.to_raise(Errno::ETIMEDOUT)
expect(subject).to be_a(Hash)
expect(subject.dig('144495297')).to eq({ 'wp10' => nil,
'features' => { 'num_ref' => nil }, 'deleted' => false, 'prediction' => nil })
'features' => nil, 'deleted' => false, 'prediction' => nil })
expect(subject.dig('144495298')).to eq({ 'wp10' => nil,
'features' => { 'num_ref' => nil }, 'deleted' => false, 'prediction' => nil })
'features' => nil, 'deleted' => false, 'prediction' => nil })
end
end
end
Expand Down Expand Up @@ -148,9 +141,9 @@
.to_raise(Errno::ETIMEDOUT)
expect(subject).to be_a(Hash)
expect(subject.dig('157412237')).to eq({ 'wp10' => nil,
'features' => { 'num_ref' => nil }, 'deleted' => false, 'prediction' => nil })
'features' => nil, 'deleted' => false, 'prediction' => nil })
expect(subject.dig('157417768')).to eq({ 'wp10' => nil,
'features' => { 'num_ref' => nil }, 'deleted' => false, 'prediction' => nil })
'features' => nil, 'deleted' => false, 'prediction' => nil })
end
end
end
Expand Down
Loading