Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix integration tests (v 0.7.1) #1110

Closed
gkirgizov opened this issue Jun 8, 2023 · 5 comments · Fixed by #1112
Closed

Fix integration tests (v 0.7.1) #1110

gkirgizov opened this issue Jun 8, 2023 · 5 comments · Fixed by #1112
Assignees
Labels
help wanted Extra attention is needed invalid This doesn't seem right test The additon or modification of the unit test

Comments

@gkirgizov
Copy link
Collaborator

gkirgizov commented Jun 8, 2023

A respectful issue.

As now tests badge is on the main project page -- they must be more predictable, don't fall stochastically.
And even if they must be stochastic -- let's run them multiple times and estimate required number of times they finish with positive results.

Failing tests from the last run on v0.7.1.

=========================== short test summary info ============================
FAILED test/integration/quality/test_quality_improvement.py::test_classification_quality_improvement - assert 0.816 >= 0.823
FAILED test/integration/quality/test_quality_improvement.py::test_multiobjective_improvement - IndexError: list index out of range
FAILED test/integration/real_applications/test_examples.py::test_api_example - ValueError: Must pass 2-d input. shape=()
FAILED test/integration/real_applications/test_examples.py::test_multi_modal_example - ValueError: No data for primary node tfidf
FAILED test/integration/real_applications/test_real_cases.py::test_credit_scoring_problem - ValueError: array must not contain infs or NaNs
========== 5 failed, 213 passed, 6250 warnings in 2379.71s (0:39:39) ===========
@gkirgizov gkirgizov added help wanted Extra attention is needed invalid This doesn't seem right test The additon or modification of the unit test labels Jun 8, 2023
@IIaKyJIuH IIaKyJIuH self-assigned this Jun 9, 2023
@IIaKyJIuH
Copy link
Collaborator

IIaKyJIuH commented Aug 3, 2023

Проблема, которая препятствует разрешению, связана с GOLEM и находится в следующем методе Dispatcher'а:
image
То есть, в метод приходит один объект, в то время как ожидается список объектов.

Проблема всплыла после коммита 4c58deda3646f91ce4053b40571f2fd08a9f1bb8, до него всё норм.

Ошибка воспроизводится в FEDOT на тесте test/integration/api_params/test_main_api_params.py::test_timeout

@nicl-nno
Copy link
Collaborator

nicl-nno commented Aug 3, 2023

В тесте в конфиге указано что:

    'pop_size': 1,

Есть ли вообще смысл в такой настройке - для эволюции это достаточно вырожденный случай.
Если поставить 2, то у меня тест проходит.

@nicl-nno
Copy link
Collaborator

nicl-nno commented Aug 3, 2023

Можно в GOLEM добавить issue на корректную обработку краевых значений - но вроде это не тот сценарий, который есть смысл интеграционно тестировать в fedot-е.

@gkirgizov
Copy link
Collaborator Author

Согласен с Колей. Можно поставить 2 и закрыть вопрос. А вообще, даже при pop_size=1 должна передаваться популяция, то есть список, пусть и из 1 элемента. Почему это не так -- посмотрю

@maypink
Copy link
Collaborator

maypink commented Aug 7, 2023

@IIaKyJIuH попробуй пж прогнать с этим фиксом, у меня проходит

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed invalid This doesn't seem right test The additon or modification of the unit test
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants