Converter for polymorphic types

[zachlefevre]

I'd like to read a string from the config into a Foo.

I have a Yaml of the form

foo:
  thing: 42

and a class of the form

public class Foo<T> {
    public T t;
    public void set(T t) { this.t = t; }
    public T get() { return t; }
}

And I'd like to read the config into a Foo via an interface like so:

@StaticInitSafe
@ConfigMapping(prefix = "foo")
public interface TestConfig {
    Foo<String> thing();
}

However, I get the error Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.IllegalArgumentException: SRCFG00013: No Converter registered for class com.socrata.config.Foo [in thread "main"]

As far as I can tell implementing a Converter<Foo<T>> is not possible. Is there any way to write a Converter<Foo<T>>?
I tried with something like

public class FooConverter<T extends Converter<T>> implements Converter<Foo<T>> {
  @Override
  public Foo<T> convert(String value) {
      throw new IllegalStateException("aaah");
  }
}

But the FooConverter is never used.

In summary I'd like to have an config interface with a field that is a parameterized type where the type parameter itself implements a Converter.

Thank you,

[dmlloyd]

I'll start by saying you can probably get the effect you want by explicitly specifying the converter to use; use the @WithConverter type annotation on your property for this purpose.

Next, I'll point out that SmallRye Config does indeed support generic converters to an extent, though mostly it's done internally with certain fixed types (lists and arrays of objects, for example). So we're part way there already.

Lastly, I will address the idea of full-blown generic converters. This has been discussed several times since the inception of this project IIRC, and is something that I think we would want to do but doing it right is pretty tricky.

The implementation would probably require something like these interfaces:

public interface GenericConverter1<T, U> {
    T convert(String value, Converter<U> nestedConverter);
}

public interface GenericConverter2<T, U, V> {
    T convert(String value, Converter<U> nestedConverter1, Converter<V> nestedConverter2);
}
// that's probably as far as we need to go, but we could take it further if needed...

Then when we encounter a generic parameterized type, we can search for a converter with the correct parameters. As a trivial example, to support a converter for Map.Entry<K, V> which splits the key from the value with ;, you could then do:

public class MyEntryConverter<U, V> implements GenericConverter2<Map.Entry<U, V>, U, V> {
    public Map.Entry<T, U> convert(String value, Converter<U> nested1, Converter<V> nested2) {
        String[] items = value.split(";", 2);
        return Map.entry(nested1.convert(items[0]), nested2.convert(items[1]));
    }
}

The negatives with this design would be the ugliness of having multiple generic converter interfaces, and the increased complexity of searching for converters.

On the other hand, it would allow our special-case converters (collections, Optional, etc.) to be implemented in the same way as user converters, which is a nice cleanup IMO.

[zachlefevre]

I see. For what it's worth I'm trying to integrate smallrye config with our current projects, which are Scala projects.
Converter induction (the ability to form a Converter<F<T>> assuming the existence of a Converter<T>) would allow us to read into arbitrary higher kinded types like scala Option, Either, List, etc.

I'll give @WithConverter a shot and report back. Thanks

[dmlloyd]

I see. For what it's worth I'm trying to integrate smallrye config with our current projects, which are Scala projects. Converter induction (the ability to form a Converter<F<T>> assuming the existence of a Converter<T>) would allow us to read into arbitrary higher kinded types like scala Option, Either, List, etc.

The problem with doing this in Java is not that we can't induce the higher-kinded converter type (in fact, we can and do in several cases), it's that the Converter interface's convert method doesn't have any means to compose the corresponding converters. Its only input is a string. When the converter can produce the whole generic type from a single input in one pass, that's fine; but the composed converters like Optional<T> need to delegate to the "inner" converter in order to be applicable to any subordinate type conversion. While you could hard-code the delegate converter within your converter implementation, this would prevent users from having nested custom conversions (for example, Optional<@WithConverter(MyConv.class) Foo>).

I guess an alternative to having multiple interfaces would be to require that the converter type have a constructor which accepts one converter for each generic type, so one converter instance is constructed per use site. This approach has a complexity and memory footprint tradeoff though.

[zachlefevre]

I'll start by saying you can probably get the effect you want by explicitly specifying the converter to use; use the @WithConverter type annotation on your property for this purpose.

Actually I'm not quiet sure what you mean by this. I would understand if FooConverter were not type parameterized. But since it is:

public class FooConverter<T extends Converter<T>> implements Converter<Foo<T>> {
  @Override
  public Foo<T> convert(String value) {
      throw new IllegalStateException("aaah");
  }
}

@StaticInitSafe
@ConfigMapping(prefix = "foo")
public interface TestConfig {
    @WithConverter(FooConverter.class)
    Foo<String> thing();
}

fails for

 error: incompatible types: Class<FooConverter> cannot be converted to Class<? extends Converter<?>>

and

    @WithConverter(FooConverter<String>.class)

is not allowed for, I think, type erasure.

I'll admit to not being great with java generics and reflection. Am I doing something obviously wrong here?

Thank you

[zachlefevre]

I think you're saying I need to implement a Converter for every concrete type:

public class FooIntConverter implements Converter<Foo<Integer>> {
  @Override
  public Foo<Integer> convert(String value) {
      Foo<Integer> foo = new Foo();
      foo.set(22);
      return foo;
  }
}

and

public class FooStringConverter implements Converter<Foo<String>> {
  @Override
  public Foo<String> convert(String value) {
      Foo<String> foo = new Foo();
      foo.set(value);
      return foo;
  }
}

which does work.

[dmlloyd]

Right, it's not ideal but it does the job for limited cases.

[dmlloyd]

I'll start by saying you can probably get the effect you want by explicitly specifying the converter to use; use the @WithConverter type annotation on your property for this purpose.

Actually I'm not quiet sure what you mean by this. I would understand if FooConverter were not type parameterized. But since it is:

public class FooConverter<T extends Converter<T>> implements Converter<Foo<T>> {
  @Override
  public Foo<T> convert(String value) {
      throw new IllegalStateException("aaah");
  }
}

@StaticInitSafe
@ConfigMapping(prefix = "foo")
public interface TestConfig {
    @WithConverter(FooConverter.class)
    Foo<String> thing();
}

fails for

 error: incompatible types: Class<FooConverter> cannot be converted to Class<? extends Converter<?>>

Right, essentially you're saying that FooConverter, rather than converting any T, converts any Converter<T>. Since String is never a Converter, you have an error (though SmallRye Config does its best to make it work if it can).

    @WithConverter(FooConverter<String>.class)

is not allowed for, I think, type erasure.

Right, when you give a class constant, you can't give type arguments (in other words, every class has only one Class instance, even if it is a generic class).

[zachlefevre]

I've realized fixing this at the Converter level with your suggested interfaces would not fix the issue at hand. A converter takes a single string. There is no way to pass to the converter an entire tree. If I have something like

bar:
  name: bar
  age: 88

I cannot lift this into a Foo[Bar] even with a Converter[Bar] in scope. The Converter[Bar] could not be used as it requires I give it a single String. I could not give it the set of properties name and age.

With the suggested implementation we can only lift primitives and things which can be constructed from a single string into a Foo.

[dmlloyd]

Correct; independent keys can be placed into the same group, but they're still independent.