document requirements and how to run the tests

[filmaj]

It would be nice to know how to get set up in this repository, as well as how to run the tests.

My assumptions / attempts so far have included:

1. Get a virtualenv, install/update pip, setuptools and wheel as per https://grimoirelab.gitbooks.io/tutorial/content/before-you-start/installing-grimoirelab.html
2. Run python3 setup.py install
3. Run pip install -r requirements.txt - maybe? Maybe the last command implicitly installs the reqs?

But how do I run the tests? I tried python3 tests/run_tests.py, but that failed completely. I tried to run it from the tests/ directory and got farther, but it looks like I'm missing a ton of dependencies:

src/grimoirelab-mordred/tests on master [?] via grimoirelab
➔ ./run_tests.py
.EEEEEE
======================================================================
ERROR: test_task (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_task
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/Users/maj/src/grimoirelab-mordred/tests/test_task.py", line 33, in <module>
    from mordred.task import Task
  File "../mordred/task.py", line 27, in <module>
    from grimoire_elk.elk import get_ocean_backend
ImportError: cannot import name 'get_ocean_backend'


======================================================================
ERROR: test_task_collection (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_task_collection
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/Users/maj/src/grimoirelab-mordred/tests/test_task_collection.py", line 35, in <module>
    from mordred.task_collection import TaskRawDataCollection
  File "../mordred/task_collection.py", line 42, in <module>
    from grimoire_elk.elk import feed_backend
ImportError: cannot import name 'feed_backend'


======================================================================
ERROR: test_task_enrich (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_task_enrich
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/Users/maj/src/grimoirelab-mordred/tests/test_task_enrich.py", line 36, in <module>
    from mordred.task_projects import TaskProjects
  File "../mordred/task_projects.py", line 36, in <module>
    from mordred.task import Task
  File "../mordred/task.py", line 27, in <module>
    from grimoire_elk.elk import get_ocean_backend
ImportError: cannot import name 'get_ocean_backend'


======================================================================
ERROR: test_task_identities (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_task_identities
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/Users/maj/src/grimoirelab-mordred/tests/test_task_identities.py", line 26, in <module>
    import httpretty
ModuleNotFoundError: No module named 'httpretty'


======================================================================
ERROR: test_task_panels (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_task_panels
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/Users/maj/src/grimoirelab-mordred/tests/test_task_panels.py", line 33, in <module>
    from mordred.task_panels import TaskPanels
  File "../mordred/task_panels.py", line 35, in <module>
    from mordred.task import Task
  File "../mordred/task.py", line 27, in <module>
    from grimoire_elk.elk import get_ocean_backend
ImportError: cannot import name 'get_ocean_backend'


======================================================================
ERROR: test_task_projects (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_task_projects
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/Users/maj/src/grimoirelab-mordred/tests/test_task_projects.py", line 27, in <module>
    import httpretty
ModuleNotFoundError: No module named 'httpretty'


----------------------------------------------------------------------
Ran 7 tests in 1.630s

FAILED (errors=6)

If someone could help me with this, I will happily issue a pull request to update the README.md.

Thanks!

[filmaj]

I would love to contribute to this project but without being able to run the tests I do not have confidence in sending pull requests 😢

[filmaj]

@jgbarah if you could help me work through this issue, I will involve myself in this project more and file more pull requests and try to help out with issues filed.

[jgbarah]

Thanks a lot for your interest, @filmaj

I'm in the process of integrating this project in the set of tests that build_grimoirelab can run automatically, when building packages. Recently, chaoss/grimoirelab#124 was merged, which provides the basis for this testing. I already integrated sortinghat tests (see chaoss/grimoirelab-sortinghat#154), and the next one is sirmorded. So, we're almost there ;-)

Meanwhile, maybe you can have a look at the travis config file, which is what Travis uses to run the tests. In it, you can see how you need Elasticsearch running for some tests, and also some packages in specific versions. Copying from that file:

  - pip install pandas==0.18.1
  - pip install httpretty==0.8.6
  - pip install -r "requirements.txt"
  - pip install flake8
  - pip install coveralls

So, not all dependencies are in requirements.txt...

Could you try starting from here?

[filmaj]

That's great! I got pretty far to running the tests, but it looks like a dependency on sortinghat deep down the chain is still causing an error, 7 failures that all are the same issue:

======================================================================
ERROR: test_task_projects (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_task_projects
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/Users/maj/src/grimoirelab-sirmordred/tests/test_task_projects.py", line 35, in <module>
    from mordred.config import Config
  File "../mordred/config.py", line 29, in <module>
    from grimoire_elk.utils import get_connectors
  File "/Users/maj/.virtualenvs/grimoirelab/lib/python3.6/site-packages/grimoire_elk/utils.py", line 77, in <module>
    from .enriched.git import GitEnrich
  File "/Users/maj/.virtualenvs/grimoirelab/lib/python3.6/site-packages/grimoire_elk/enriched/git.py", line 37, in <module>
    from .study_ceres_aoc import areas_of_code, ESPandasConnector
  File "/Users/maj/.virtualenvs/grimoirelab/lib/python3.6/site-packages/grimoire_elk/enriched/study_ceres_aoc.py", line 28, in <module>
    from cereslib.events.events import Git, Events
  File "/Users/maj/.virtualenvs/grimoirelab/lib/python3.6/site-packages/cereslib/events/events.py", line 27, in <module>
    from grimoire_elk.enriched.sortinghat_gelk import SortingHat
  File "/Users/maj/.virtualenvs/grimoirelab/lib/python3.6/site-packages/grimoire_elk/enriched/sortinghat_gelk.py", line 32, in <module>
    from sortinghat.exceptions import AlreadyExistsError, WrappedValueError
ImportError: cannot import name 'WrappedValueError'

I took a look at the sortinghat code and it looks like that exception class was recently removed. So, no wonder 😉 . In that commit they rename the exception to InvalidValueError. The offender seems to be grimoirelab-elk, which is still using the old name for the exception. I'll send a PR there. When I fixed that in my local virtualenv, the tests ran! Yay!

I'll send a PR here for updated README instructions.

[filmaj]

The PR at chaoss/grimoirelab-elk#382 that is currently open will fix the above test failures in Mordred, FYI.