Properties with custom types inheritance fix #18052

dreambrother · 2024-03-07T13:29:32Z

Approach to deep clone Schema's introduced in #16992 breaks support for properties with custom types (added using schemaMappings or type/importMappings) when such properties are inherited using allOf. AnnotationsUtils#clone returns null for them. I've replaced AnnotationsUtils#clone with ObjectMapper's typed serialization/deserialization.

@OpenAPITools/generator-core-team

PR checklist

Read the contribution guidelines.
Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
Run the following to build the project and update samples:
```
./mvnw clean package 
./bin/generate-samples.sh ./bin/configs/*.yaml
./bin/utils/export_docs_generators.sh
```
(For Windows users, please run the script in Git BASH)
Commit all changed files.
This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
These must match the expectations made by your contribution.
You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.
File the PR against the correct branch: master (upcoming 7.1.0 minor release - breaking changes with fallbacks), 8.0.x (breaking changes without fallbacks)
If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

wing328 · 2024-03-09T09:39:11Z

thanks for the PR

cc @martin-mfg

wing328 · 2024-03-21T03:44:27Z

@martin-mfg can you please review when you've time? thank you.

martin-mfg · 2024-03-24T11:02:33Z

Hi,
the changes look good to me. 👍

just some additional results from my experiments, in case this PR is referenced again in the future:

```
  String serialized = "{\"type\" : \"custom\"}";
  new ObjectMapper().readValue(serialized, io.swagger.v3.oas.models.media.Schema.class);
```
This works. The relevant difference to what AnnotationsUtils.clone does internally is that a standard ObjectMapper is used above. In AnnotationsUtils.clone, the ObjectMapper is heavily customized.
```
  AnnotationsUtils.clone(new Schema().type("custom"), false);
```
This does not work. Which means that in general, using allOf is not necessary to trigger the issue. Using an unknown type is sufficient.
The changes in the R sample code are not related to the changed cloning logic. Updating the samples with the existing logic results in the same sample changes.

dreambrother · 2024-04-08T12:14:06Z

@wing328 I resolved conflict and removed ModelUtils.cloneSchema added here 6746be9 . It uses AnnotationUtils.clone with the same problems described in this PR

dreambrother · 2024-04-23T08:57:07Z

Any updates?

wing328 · 2024-04-23T09:01:18Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/utils/ModelUtils.java

-        // in openapi-generator, we also store type in `type` for 3.1 schema
-        // to make it backward compatible with the rest of the code base.
-        if (schema.getType() != null) {
-            clone.setType(schema.getType());


looks like this logic is completely removed, right?

SchemaUtils.cloneSchema must not lose a type info after clone (instead of AnnotationUtils), so yep, I've removed it :)

👍

FYI. we've refactored the code a bit and no longer using type to store the actual type in 3.1 spec: #18577

wing328 · 2024-04-23T09:06:31Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/utils/SchemaUtils.java

+            return schema;
+        }
+    }
+}


personally i prefer not having another class SchemaUtils just for cloneSchema?

can we merge this (or add this back) into model utils instead?

…to custom-types-inheritance

dreambrother · 2024-05-14T07:52:49Z

Hi! Please notify me before merge, I'll resolve merge conflicts.

wing328 · 2024-05-14T12:14:43Z

Can you please do so and PM me via Slack when done?

dreambrother · 2024-05-14T13:46:22Z

@wing328 sorry, I don't use slack. I've resolved merge conflicts. Had to add jackson-annotations dependency explicitly, because openapi-generator right now has different minor versions of jackson in dependencies (lower for jackson-datatype-threetenbp).

wing328 · 2024-05-15T03:49:28Z

samples/client/echo_api/r/R/data_query.R

    #' @param suffix test suffix
    #' @param text Some text containing white spaces
    #' @param date A date
    #' @param ... Other optional arguments.
    #' @export
-    initialize = function(`id` = NULL, `outcomes` = [SUCCESS, FAILURE], `suffix` = NULL, `text` = NULL, `date` = NULL, ...) {
+    initialize = function(`id` = NULL, `outcomes` = ["SUCCESS","FAILURE"], `suffix` = NULL, `text` = NULL, `date` = NULL, ...) {


one more minor request if you don't mind. can you please add a test for the following schema:

outcomes: type: array items: type: string enum: - SUCCESS - FAILURE - SKIPPED default: - SUCCESS - FAILURE

to ensure the default values are clone correctly for string enum?

sure! I've added ModelUtilsTest.testCloneArrayOfEnumsSchema

wing328 · 2024-05-15T04:00:33Z

fyi @OpenAPITools/generator-core-team

wing328 · 2024-05-15T14:03:23Z

PR merged. Thanks again for the PR.

* custom types support in inheritance fix * files changed after scripts run * remove unused method * move cloneSchema to ModelUtils * imports * changes after scripts run * test cloning array of enums schema

wing328 · 2024-06-05T09:50:45Z

[main] ERROR o.o.codegen.utils.ModelUtils - Can't clone schema class DateTimeSchema {
    class Schema {
        type: string
        format: date-time
        $ref: null
        description: null
        title: null
        multipleOf: null
        maximum: null
        exclusiveMaximum: null
        minimum: null
        exclusiveMinimum: null
        maxLength: null
        minLength: null
        pattern: null
        maxItems: null
        minItems: null
        uniqueItems: null
        maxProperties: null
        minProperties: null
        required: null
        not: null
        properties: null
        additionalProperties: null
        nullable: null
        readOnly: null
        writeOnly: null
        example: 2020-02-02T20:20:20.000222Z
        externalDocs: null
        deprecated: null
        discriminator: null
        xml: null
    }
}
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Type id handling not implemented for type java.lang.Object (by serializer of type com.fasterxml.jackson.databind.ser.impl.UnsupportedTypeSerializer) (through reference chain: io.swagger.v3.oas.models.media.DateTimeSchema["example"])
        at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
        at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1330)
        at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:414)
        at com.fasterxml.jackson.databind.JsonSerializer.serializeWithType(JsonSerializer.java:173)
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:734)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:770)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:653)
        at com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrappedSerializer.java:32)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:502)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:341)
        at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4799)
        at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:4040)
        at org.openapitools.codegen.utils.ModelUtils.cloneSchema(ModelUtils.java:2191)
        at org.openapitools.codegen.DefaultCodegen.lambda$mergeProperties$13(DefaultCodegen.java:2865)
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
        at org.openapitools.codegen.DefaultCodegen.mergeProperties(DefaultCodegen.java:2865)
        at org.openapitools.codegen.DefaultCodegen.updateModelForComposedSchema(DefaultCodegen.java:2779)
        at org.openapitools.codegen.DefaultCodegen.fromModel(DefaultCodegen.java:3120)
        at org.openapitools.codegen.languages.AbstractJavaCodegen.fromModel(AbstractJavaCodegen.java:1738)
        at org.openapitools.codegen.languages.JavaClientCodegen.fromModel(JavaClientCodegen.java:1055)
        at org.openapitools.codegen.DefaultGenerator.processModels(DefaultGenerator.java:1757)
        at org.openapitools.codegen.DefaultGenerator.generateModels(DefaultGenerator.java:525)
        at org.openapitools.codegen.DefaultGenerator.generateModels(DefaultGenerator.java:449)
        at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:1302)
        at org.openapitools.codegen.cmd.Generate.execute(Generate.java:535)
        at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32)
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66)

@dreambrother can you please take a look at the above exception when you've time?

To repeat it, please run ./bin/generate-samples.sh ./bin/configs/java-okhttp-gson.yaml

dreambrother added 2 commits March 7, 2024 16:21

custom types support in inheritance fix

8ffcac5

files changed after scripts run

8ffc4f8

martin-mfg approved these changes Mar 24, 2024

View reviewed changes

dreambrother added 2 commits April 8, 2024 15:04

resolve merge conflicts

3444c24

remove unused method

3b19669

wing328 reviewed Apr 23, 2024

View reviewed changes

wing328 added this to the 7.6.0 milestone Apr 23, 2024

wing328 added the Issue: Regression label Apr 23, 2024

wing328 reviewed Apr 23, 2024

View reviewed changes

dreambrother added 5 commits April 23, 2024 12:58

move cloneSchema to ModelUtils

64cc657

imports

ccdea62

Merge branch 'master' of github.com:OpenAPITools/openapi-generator in…

b2b0b48

…to custom-types-inheritance

changes after scripts run

4fe01bb

resolve merge conflicts

3fc798c

resolve merge conflicts

7d20daf

wing328 reviewed May 15, 2024

View reviewed changes

test cloning array of enums schema

7236a06

wing328 merged commit 70130ed into OpenAPITools:master May 15, 2024
14 checks passed

dreambrother mentioned this pull request Jun 5, 2024

clone Schema using AnnotationUtils.clone with WA #18867

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Properties with custom types inheritance fix #18052

Properties with custom types inheritance fix #18052

dreambrother commented Mar 7, 2024 •

edited

Loading

wing328 commented Mar 9, 2024

wing328 commented Mar 21, 2024

martin-mfg commented Mar 24, 2024 •

edited

Loading

dreambrother commented Apr 8, 2024

dreambrother commented Apr 23, 2024

wing328 Apr 23, 2024

dreambrother Apr 23, 2024

wing328 May 15, 2024 •

edited

Loading

wing328 Apr 23, 2024

dreambrother Apr 23, 2024

dreambrother Apr 23, 2024

dreambrother commented May 14, 2024

wing328 commented May 14, 2024

dreambrother commented May 14, 2024

wing328 May 15, 2024

dreambrother May 15, 2024

wing328 commented May 15, 2024

wing328 commented May 15, 2024

wing328 commented Jun 5, 2024

Properties with custom types inheritance fix #18052

Properties with custom types inheritance fix #18052

Conversation

dreambrother commented Mar 7, 2024 • edited Loading

PR checklist

wing328 commented Mar 9, 2024

wing328 commented Mar 21, 2024

martin-mfg commented Mar 24, 2024 • edited Loading

dreambrother commented Apr 8, 2024

dreambrother commented Apr 23, 2024

wing328 Apr 23, 2024

Choose a reason for hiding this comment

dreambrother Apr 23, 2024

Choose a reason for hiding this comment

wing328 May 15, 2024 • edited Loading

Choose a reason for hiding this comment

wing328 Apr 23, 2024

Choose a reason for hiding this comment

dreambrother Apr 23, 2024

Choose a reason for hiding this comment

dreambrother Apr 23, 2024

Choose a reason for hiding this comment

dreambrother commented May 14, 2024

wing328 commented May 14, 2024

dreambrother commented May 14, 2024

wing328 May 15, 2024

Choose a reason for hiding this comment

dreambrother May 15, 2024

Choose a reason for hiding this comment

wing328 commented May 15, 2024

wing328 commented May 15, 2024

wing328 commented Jun 5, 2024

dreambrother commented Mar 7, 2024 •

edited

Loading

martin-mfg commented Mar 24, 2024 •

edited

Loading

wing328 May 15, 2024 •

edited

Loading