ODC performance improvement plan

[Floppy]

Surveyor is becoming a liability - let's make it all better! YAY!

[pkqk]

The Surveyor gem is both not currently being maintained and the data model does not support the size of the questionnaires we are using it for very well. (eg. theodi/open-data-certificate#1142)

The gem is written using very simple ActiveRecord relations which regularly create severe n+1 query issues.

Since the app is so heavily dependent on how Surveyor operates I don't think rewriting it directly is a good solution.

We could however make reasonable improvements in several steps:

• vendor Surveyor in the the codebase so it's easier to refactor
  • we can then move our monkey patches into the actual classes
  • move the MethodModules into the actual classes as it wouldn't need to be extensible anymore
• start improving the query creation by preloads etc.
• stop storing questionnaire logic in the database and make the Survey object have a decision tree that does that
• store dependencies between questions in the Survey object as well.

This will get us part way to a more performant system and leave the option for more speed improvements.

[dicknorris]

Book to ODM > WP3 > Task 3.5

[quadrophobiac]

This issue was moved to theodi/open-data-certificate#1535