available vs generated data

[leclairm]

No description provided.

[GeigerJ2]

Thank you for the PR, @leclairm! Looks fine to me. Just added one minor comment.

In addition, I think we should still have a way to denote if a data entry lives on the local machine, irrespective if it's available or generated. It could be via type='remote', even though, one then loses the file/dir specification for the type. However, as I mentioned in my comment in issue #28, this actually doesn't matter for AiiDA, and I'm not sure if we really use the information somewhere else in the code. I further assume generated would basically always be on the remote HPC?

As this isn't addressed in this PR, I'll work on it locally (and have already been), and will open another PR.

[leclairm]

Thank you for the PR, @leclairm! Looks fine to me. Just added one minor comment.

In addition, I think we should still have a way to denote if a data entry lives on the local machine, irrespective if it's available or generated. It could be via type='remote', even though, one then loses the file/dir specification for the type. However, as I mentioned in my comment in issue #28, this actually doesn't matter for AiiDA, and I'm not sure if we really use the information somewhere else in the code. I further assume generated would basically always be on the remote HPC?

As this isn't addressed in this PR, I'll work on it locally (and have already been), and will open another PR.

Yes, let's keep that for the separate discussion.

[leclairm]

@agoscinski you can check if everything is fine now.

[agoscinski]

Can you also adapt the test_config_small.yml? Most formatting issues should be fixed automatically with hatch fmt.

[leclairm]

Can you also adapt the test_config_small.yml? Most formatting issues should be fixed automatically with hatch fmt.

What is ruff trying to tell me with this?

src/wcflow/core.py:27:9: FBT001 Boolean-typed positional argument in function definition
src/wcflow/core.py:102:9: FBT001 Boolean-typed positional argument in function definition
src/wcflow/core.py:161:9: FBT001 Boolean-typed positional argument in function definition

[agoscinski]

You can always look up the code FBT001 online and find more explanations. Then you find such a page https://docs.astral.sh/ruff/rules/boolean-type-hint-positional-argument/
I think the arguments on the page are fair. Would make it also enforce keyword argument usage by changing to

    def __init__(
        self,
        name: str,
        type: str,  # noqa: A002
        src: str,
        lag: list[Duration],
        date: list[datetime],
        arg_option: str | None,
        *
        available: bool,
    ):

I actually thought at some point to enforce keyword argument usage to every input argument to increase readability. But that we can do another time.

[leclairm]

Can you also adapt the test_config_small.yml? Most formatting issues should be fixed automatically with hatch fmt.

done

[leclairm]

You can always look up the code FBT001 online and find more explanations. Then you find such a page https://docs.astral.sh/ruff/rules/boolean-type-hint-positional-argument/ I think the arguments on the page are fair. Would make it also enforce keyword argument usage by changing to

    def __init__(
        self,
        name: str,
        type: str,  # noqa: A002
        src: str,
        lag: list[Duration],
        date: list[datetime],
        arg_option: str | None,
        *
        available: bool,
    ):

I actually thought at some point to enforce keyword argument usage to every input argument to increase readability. But that we can do another time.

did it for avaialble

[agoscinski]

Looks good! Thank for the work. Could you squash merge with a commit message like this?

Introduce keywords that distinguish data that is available on init and not in yml file (#29)

We introduce the keywords `available` and `generated` in the yml file that
specify if the data exists already on initialization or is generated by the
workflow. Before that we needed to refer this from how the data is used in the
workflow. We decided to let the user specify this for better error control
handling. Implements suggestion from issue #28