Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Scheduler does not pay respect to running jobs #152

Closed
peshkira opened this issue Oct 23, 2012 · 3 comments
Closed

[Core] Scheduler does not pay respect to running jobs #152

peshkira opened this issue Oct 23, 2012 · 3 comments
Assignees
Labels
Milestone

Comments

@peshkira
Copy link
Member

The scheduler does not check if a job that is scheduled is still running and just runs another job. This can result in two or more jobs that reuse the same adaptor object, which potentially messes up the state of the adaptor.

When a new job should be scheduled, check if there is still another one for the same adaptor that is still running from the last time. If there is skip this schedule and wait for the next period. Repeat this check in every period.

Here is what happens if you schedule a pronom adaptor that takes longer than a minute to get the values

15:18:13,664 INFO  [stdout] (CoreScheduler_Worker-1) 15:18:13,664 INFO  [PronomAdaptor] Getting next batch of '200' with offset '800'
15:18:13,686 INFO  [stdout] (CoreScheduler_Worker-2) 15:18:13,686 INFO  [PronomAdaptor] Getting next batch of '200' with offset '800
@ghost ghost assigned kduretec Oct 23, 2012
kduretec added a commit that referenced this issue Oct 23, 2012
@kduretec
Copy link
Contributor

@peshkira please check and close if solved

@peshkira
Copy link
Member Author

Commit 6cabb5f solves the issue (Thanks!) but has revealed another problem.
If some error occurs during data storage or somewhere else in the core, then the adaptor execution stops for some reason, but the scheduler continues to skip the new adaptor execution, because the exception event is not handled properly.

You can close this issue, but please create a new one for the other problem, or close this one, when the other problem is solved as well.

@luis100
Copy link
Member

luis100 commented Nov 5, 2012

#162

@luis100 luis100 closed this as completed Nov 5, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants