Work with request

Work with steps in request

General description

In this section you will find how to work with steps and slices inside a single request, how to deal with broken tasks and how to extend request with new data.

Main concepts

It's a four main concepts in the system: Request, Slice(Chain), Step, Task.

• Task - instance which do actual work(data transformation).
• Step - template for Task. After submitting Step parameters translated to Task or group of Tasks. Submitted Steps are not mutable, but could be cloned. Steps are hierarchical, one Step can have one ancestor and many descendants.
• Slice(Chain) - set of consecutive Steps. Slice also could store some common for these Steps data.
• Request - set of Steps and some common bookkeeping values.

Working with steps

On request page Steps are grouped by Slices:

pic1

Step status

Each Step could have one of the following states:

• Default - orange background with black border. Step will become a Task after Approval(Submitting).
• Approved(Submitted) - orange background with green border. Step was submitted for tasks creation. Step are nor mutable. If no problem occurs, link on Tasks for this Step will be placed below it.
• Waiting - orange background with blue border. Step will be approved after Task created from first Step in Slice reaches some goal(e.g. 50% done).
• Skipped - white background with black or green border. Needed for bookkeeping and input search. Step will not be used for Task creation.

Also if Slice has a Step with yellow background, it is the link to the Step from other Slice, which are parent for this Slice Steps.

pic2

Step submission

To approve and by that submit Steps user could use 'Approve' or 'Approve only steps before' button. Clicking 'Approve' command to approve all Steps in selected Slices is sent. In some cases(e.g. for MC) only first Step will be submitted, other will be changed to "Waiting" state.

pic3

Button 'Approve only steps before' sends command to submit only steps before value in the to this button list box.

pic4

If approved Step has no Tasks defined it status could be changed back to default by clicking Reject button. It could be useful if problem is spotted immediately or Tasks were not created because of any problem.

picwithreject

It's two ways to make Step skipped: 1) using pattern(link) for many slices 2) using slice edit form: picSliceExtension

picSkip

Step parameters modification

Step parameters could be changed using several ways: slice edit form, "Bulk steps modification" form, using pattern. Step is modified on client side without propagating changes to the server. When any Steps in a Slice are changed, Slice has "edit(not saved)" state. Steps could be saved by clicking on 'Save' button:

picSave

Cloning steps

To modify steps which are produced faulty tasks or to submit similar slice with some modified option 'Clone' and 'Fix' button could be used. 'Clone' creates slices which has same steps and exactly same slice parameters as selected. If some steps approved in slected slices their status are changed to Default in cloned slices. Using 'Fix' only part of steps in slice are cloned whereas last on not cloned step becomes parent for cloned slice. E.g. if slice has 4 steps and 'Fix' is applied with 'Clone from step 2', then new slice will have two steps, which were 3rd and 4th on selected slice, and second step of selected slice becomes parent of cloned slice.

picbeforeFix

picafterClone

Slices could be cloned to a new request, using 'Menu'->'Clone' option.

Hiding slices

Slices could be hide, so they are not shown on the request page. It could be useful, then slice has only failed tasks or it was created by mistake. Clicking 'Hide' button hides all selected slices. If some slice was hided by mistake it could be rescued: go to "Show xx hidden slices" link on the bottom of request page and then 'Hide' again wrongly hidden slices

Find input

Input container for a slice could be identified using tags name and job options name. Clicking on 'Find input' system searches container which matches tags from skipped steps at the beginning of the slice and job option name. If containers are found they are shown in droplist where user can choose a right one. After clicking 'Save' chosen container is set as an input for the slice.

picFindeddatasets

Retrying finished tasks

If for some reason task in finished state couldn't be retried and new task is needed to process missed files, such task could be created by clicking 'Retry' button then slice with finished task is selected.

Filtering, sorting and selecting

Selecting options in 'Sort' dropdown list slices could be rearranged by 'input dataset name', 'Job options name' or slice id order. Clicking on 'Select all' all slices which are shown will be selected/deselected depends on the value of the first shown slice. To select(or deselect) set of consequential slices user can select one slice and holding ctrl select other slice, then all slices between them will be selected.

Slice edit form

Bulk step modification form

Bulk slice modification form

Manage tasks

Open ended requests

Creating "trains"