Rake Task Errors When Importing From Google Analytics (7.2)

[carrickr]

Descriptive summary

Version 7.2

We've run through the instructions page and everything is configured correctly, generators run, etc. However when I run the generator I get an undefined method each' for true:TrueClass`. I've started to dig through the user stats service to see if there might be a bug, but guidance and where the bug might lurk is helpful (or for that matter if I just did something flat out wrong).

$ RAILS_ENV=production bundle exec rake sufia:stats:user_stats

I, [2017-02-17T15:19:48.086405 #15067]  INFO -- : Sufia::UserStatImporter: Begin import of User stats.
I, [2017-02-17T15:19:48.385400 #15067]  INFO -- : ActiveFedora: loading fedora config from /var/www/nufia/releases/20170217201258/config/fedora.yml
I, [2017-02-17T15:19:48.386373 #15067]  INFO -- : ActiveFedora: loading solr config from /var/www/nufia/releases/20170217201258/config/solr.yml
I, [2017-02-17T15:19:48.431632 #15067]  INFO -- : Attempted to init base path `nuf`, but it already exists
I, [2017-02-17T15:19:49.024702 #15067]  INFO -- : Sufia::UserStatImporter: Retried FileViewStat on #<struct Struct::UserRecord id=1, user_key="d", last_stats_update=Fri, 03 Feb 2017 00:00:00 +0000> for file sf268508b too many times.
I, [2017-02-17T15:19:49.029462 #15067]  INFO -- : Sufia::UserStatImporter: Last exception undefined method `sufia__pageview' for #<Legato::Management::Profile:0x007fd3ee691fd0>
rake aborted!
NoMethodError: undefined method `each' for true:TrueClass

[carrickr]

Okay, I'm extra confused because I brought up rails console to trouble shoot and running just these two lines works in the sense it creates rows in my database, but all the view counts are zero, despite google analytics having data.

[mjgiarlo]

@carrickr Nothing jumps out at me. If you run the rake task with --trace, do you get more information or a traceback?

[carrickr]

I'm seeing this in my larger trace

** Invoke sufia:stats:user_stats (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute sufia:stats:user_stats
I, [2017-02-20T13:19:59.823185 #23529]  INFO -- : Sufia::UserStatImporter: Begin import of User stats.
I, [2017-02-20T13:20:00.128889 #23529]  INFO -- : ActiveFedora: loading fedora config from /var/www/nufia/releases/20170217201258/config/fedora.yml
I, [2017-02-20T13:20:00.129798 #23529]  INFO -- : ActiveFedora: loading solr config from /var/www/nufia/releases/20170217201258/config/solr.yml
I, [2017-02-20T13:20:00.391446 #23529]  INFO -- : Attempted to init base path `nuf`, but it already exists
I, [2017-02-20T13:20:00.937777 #23529]  INFO -- : Sufia::UserStatImporter: Retried FileViewStat on #<struct Struct::UserRecord id=15, user_key="cla876", last_stats_update=Fri, 03 Feb 2017 00:00:00 +0000> for file rn301137d too many times.
I, [2017-02-20T13:20:00.941923 #23529]  INFO -- : Sufia::UserStatImporter: Last exception undefined method `sufia__pageview' for #<Legato::Management::Profile:0x007f550fee0f00>
rake aborted!
NoMethodError: undefined method `each' for true:TrueClass
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/sufia-7.2.0/app/services/sufia/user_stat_importer.rb:120:in `tally_results'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/sufia-7.2.0/app/services/sufia/user_stat_importer.rb:53:in `block in process_files'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/sufia-7.2.0/app/services/sufia/user_stat_importer.rb:50:in `each'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/sufia-7.2.0/app/services/sufia/user_stat_importer.rb:50:in `process_files'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/sufia-7.2.0/app/services/sufia/user_stat_importer.rb:30:in `block in import'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/sufia-7.2.0/app/services/sufia/user_stat_importer.rb:23:in `each'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/sufia-7.2.0/app/services/sufia/user_stat_importer.rb:23:in `import'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/sufia-7.2.0/tasks/stats_tasks.rake:6:in `block (3 levels) in <top (required)>'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `block in execute'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `each'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `execute'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:194:in `block in invoke_with_call_chain'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:180:in `invoke'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:152:in `invoke_task'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:108:in `block (2 levels) in top_level'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:108:in `each'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:108:in `block in top_level'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:117:in `run_with_threads'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:102:in `top_level'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:80:in `block in run'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/var/www/nufia/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:77:in `run'
rake:9:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:63:in `load'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:63:in `kernel_load'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:24:in `run'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli.rb:304:in `exec'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli.rb:11:in `start'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/exe/bundle:27:in `block in <top (required)>'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/exe/bundle:19:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.3.0/bin/bundle:23:in `load'
/home/vagrant/.rbenv/versions/2.3.0/bin/bundle:23:in `<main>'
Tasks: TOP => sufia:stats:user_stats

I'm not sure if I should be focusing on that Legato exception that gets flagged as Info or further down on the actual error

[jcoyne]

I don't think you can run Legato in development mode, I think it requires eager loading. @carrickr can you try with RAILS_ENV=production?

[carrickr]

@jcoyne I am using this in the production environment with RAILS_ENV=production prior to invocation of my rake task/console. My full command for the dump above was:

RAILS_ENV=production bundle exec rake sufia:stats:user_stats --trace

[mjgiarlo]

@carrickr Yeah, I'd focus on the Last exception undefined method sufia__pageview'` bit. See the following related issue: #2380

And also check out the workaround here: #2460