[OPIK-218] Remove limitations in dataset items

[thiagohora]

Details

We will remove limitations in dataset items by deprecating the input, expected_output and metadata to favor a more generic structure. The new field data is a Dict[String, String] or Map<String, String>, allowing users to send any number of columns they want.

To maintain backward compatibility (until we actually remove the old fields), the data from the input, expected_output and metadata fields are automatically copied to the new input_data field. Also, a new columns column was added to the Page object to show the frontend which columns are present in the dataset.

The new input_data field has the following format:

DatasetItem model

{
    ...
    "data": {
        "{{key}}":  {}        # {} json or "" string
        ...
    }
}

Sample:

{
    ...
    "data": {
        "user_question":  "What is this model name?",
        "results": {
                 "context": {
                     "message": "Be an expert."
                 }
                 "user": {
                      "message": "What is a shooting star¿"
                 }
        },
        "model":  "Lambda 3"
    }
}

As per the page object, it will look like this:

{
    "content": [ ... ],
    "page": 1,
    "size": 10,
    "total": 10,
    "columns": [ { "name": "user_question", "type": "String" },  { "name": "results", "type": "Object" } , { "name": "model", "type": "String" } ]
}

Issues

#OPIK-218

Testing

• Automated tests for:

1. input present
2. input data present
3. input data and input not present
4. verify a copy of input, expected_output, and metadata values to data

Documentation

OPENAPI docs were updated to reflect new and deprecated columns.

[andrescrz]

There are some minor and other no blocking comments. Let's focus and discuss:

• the rollout strategy.
• how co-existence of the old and new fields should be handled.
  • considering multiple versions of the SDK released.
  • considering the frontend expectations.
• the data migration strategy.
• fern generation for polymorphic types.

[andrescrz]

As discussed and agreed this morning, this LGTM.

[andrescrz]

I believe the latest revision has two major issues that we have to review:

1. It's not returning the type of the columns, which is a requirement for FE per the ticket.
2. It's resolving the type of the fields before storing them (which might be fine for caching purposes etc.) but then it isn't doing anything with them. The type of a field is implicit any valid JSON payload, so there's no need to resolve and store this extra information unless there's a reason to.

Finally, I left some guidance to easily accommodate the next requirements as filtering.

Once we're on the same page about these suggestions, following this approach is much easier as it's just a matter or removing some of the current code, like happened with the previous revision.

[thiagohora]

inally, I left some guidance to easily accommodate the next requirements as filtering.

From what @ferc said, the column types do not need to be returned yet.

[andrescrz]

The last two issues were discussed, agreed and addressed. This is ready to go from my side.

[andrescrz]

inally, I left some guidance to easily accommodate the next requirements as filtering.

From what @ferc said, the column types do not need to be returned yet.

Agreed to move forward with at least the type in the returned payload. However, getting the type value was trivial, so the latest revision resolves it as well.