Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to save new tasks #1686

Open
Didayolo opened this issue Nov 28, 2024 · 4 comments
Open

Unable to save new tasks #1686

Didayolo opened this issue Nov 28, 2024 · 4 comments
Labels
Bug Competition-specific Problem specific to a given competition or benchmark

Comments

@Didayolo
Copy link
Member

Didayolo commented Nov 28, 2024

Description

Issue related to saving new tasks in Codabench. The issue is explained step-by-step in this document:
https://docs.google.com/document/d/1-InYnhHznnU0nqGfxfmHKYfSTsLX_klB4V32Uf08SNQ/edit?usp=sharing

Error logs

django-1  | Internal Server Error: /api/competitions/3863/
django-1  | Traceback (most recent call last):
django-1  |   File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 34, in inner
django-1  |     response = get_response(request)
django-1  |   File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 115, in _get_response
django-1  |     response = self.process_exception_by_middleware(e, request)
django-1  |   File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 113, in _get_response
django-1  |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
django-1  |   File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
django-1  |     return view_func(*args, **kwargs)
django-1  |   File "/usr/local/lib/python3.9/site-packages/rest_framework/viewsets.py", line 116, in view
django-1  |     return self.dispatch(request, *args, **kwargs)
django-1  |   File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 495, in dispatch
django-1  |     response = self.handle_exception(exc)
django-1  |   File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 455, in handle_exception
django-1  |     self.raise_uncaught_exception(exc)
django-1  |   File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 492, in dispatch
django-1  |     response = handler(request, *args, **kwargs)
django-1  |   File "/usr/local/lib/python3.9/site-packages/rest_framework/mixins.py", line 84, in partial_update
django-1  |     return self.update(request, *args, **kwargs)
django-1  |   File "/app/src/apps/api/views/competitions.py", line 311, in update
django-1  |     self.perform_update(serializer)
django-1  |   File "/app/src/apps/api/views/competitions.py", line 590, in perform_update
django-1  |     self.run_new_task_submissions(instance.phases.get(pk=phase_id), new_tasks)
django-1  |   File "/app/src/apps/api/views/competitions.py", line 563, in run_new_task_submissions
django-1  |     leaderboard=submission.children.first().leaderboard,
django-1  | AttributeError: 'NoneType' object has no attribute 'leaderboard'
@Didayolo Didayolo added Bug Competition-specific Problem specific to a given competition or benchmark labels Nov 28, 2024
@shmuhammadd
Copy link

shmuhammadd commented Dec 9, 2024

@Didayolo jjust checking in on this!

We are about to start our evaluation phase and are still facing this issue on our task (https://www.codabench.org/competitions/3863/).

We tried creating a new Codabench from scratch with all the tasks we wanted, and it worked. However, we don’t want to migrate to a new Codabench as we already have many participants with many submissions (421 registered participants for our task, the largest on Codabench to date).

@Didayolo
Copy link
Member Author

@ihsaan-ullah May have a clue about this problem. Please keep us updated.

@ihsaan-ullah
Copy link
Collaborator

ihsaan-ullah commented Dec 17, 2024

Update

When you update a competition by adding a new task to a phase, the following happens:

  • we fetch all submissions that are parent submissions
  • we loop over all these submissions and we find missing tasks (missing task = all_tasks_of_this_phase - submission.children.tasks).
  • we then create a new submission which is a child submission with this new task or missing task
  • Problem occurs in the leaderboard assigning
leaderboard=submission.children.first().leaderboard

Error says AttributeError: 'NoneType' object has no attribute 'leaderboard'. children.first is None. I am checking why it is None

@ihsaan-ullah
Copy link
Collaborator

I tired to debug this problem using the following steps:

  1. Uploaded a new competition
  2. Added 2 tasks to the final phase (Dev Task and Final Task)
  3. Ran 2 submissions (one on dev task only, second on the 2 tasks)
  4. Added a new task to the phase and everything went well without any problem

Do you know how to reproduce this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Competition-specific Problem specific to a given competition or benchmark
Projects
None yet
Development

No branches or pull requests

3 participants