Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-63343] Validate element types in RobustCollectionConverter in some cases #9727

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 34 additions & 4 deletions core/src/main/java/hudson/util/RobustCollectionConverter.java
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,14 @@
import com.thoughtworks.xstream.io.HierarchicalStreamReader;
import com.thoughtworks.xstream.mapper.Mapper;
import com.thoughtworks.xstream.security.InputManipulationException;
import edu.umd.cs.findbugs.annotations.CheckForNull;
import java.lang.reflect.Type;
import java.util.Collection;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CopyOnWriteArraySet;
import java.util.logging.Logger;
import jenkins.util.xstream.CriticalXStreamException;
import org.jvnet.tiger_types.Types;

/**
* {@link CollectionConverter} that ignores {@link XStreamException}.
Expand All @@ -52,14 +55,33 @@
@SuppressWarnings({"rawtypes", "unchecked"})
public class RobustCollectionConverter extends CollectionConverter {
private final SerializableConverter sc;
/**
* When available, this field holds the declared type of the collection being (de)serialized.
* This is used during deserialization to ensure that elements have the expected type.
*/
// TODO: We only support one level of parameterization, i.e. the elements of a List<List<Integer>>
// will be treated as List<Object>, and so the inner list could have invalid non-Integer elements. It is unclear if
// we can do better without significant changes.
private final @CheckForNull Class<?> elementType;

public RobustCollectionConverter(XStream xs) {
this(xs.getMapper(), xs.getReflectionProvider());
this(xs.getMapper(), xs.getReflectionProvider(), null);
}

public RobustCollectionConverter(Mapper mapper, ReflectionProvider reflectionProvider) {
this(mapper, reflectionProvider, null);
}

public RobustCollectionConverter(Mapper mapper, ReflectionProvider reflectionProvider, Type collectionType) {
super(mapper);
sc = new SerializableConverter(mapper, reflectionProvider, new ClassLoaderReference(null));
if (collectionType != null && Collection.class.isAssignableFrom(Types.erasure(collectionType))) {
dwnusbaum marked this conversation as resolved.
Show resolved Hide resolved
var baseType = Types.getBaseClass(collectionType, Collection.class);
var typeArg = Types.getTypeArgument(baseType, 0, Object.class);
this.elementType = Types.erasure(typeArg);
} else {
this.elementType = null;
}
}

@Override
Expand All @@ -85,9 +107,17 @@ protected void populateCollection(HierarchicalStreamReader reader, Unmarshalling
reader.moveDown();
try {
Object item = readBareItem(reader, context, collection);
long nanoNow = System.nanoTime();
collection.add(item);
XStream2SecurityUtils.checkForCollectionDoSAttack(context, nanoNow);
try {
if (elementType != null) {
// When possible, disallow invalid elements.
elementType.cast(item);
}
long nanoNow = System.nanoTime();
collection.add(item);
XStream2SecurityUtils.checkForCollectionDoSAttack(context, nanoNow);
} catch (ClassCastException e) {
RobustReflectionConverter.addErrorInContext(context, e);
}
} catch (CriticalXStreamException e) {
throw e;
} catch (InputManipulationException e) {
Expand Down
4 changes: 4 additions & 0 deletions core/src/main/java/hudson/util/RobustReflectionConverter.java
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,10 @@ private boolean fieldDefinedInClass(Object result, String attrName) {

protected Object unmarshalField(final UnmarshallingContext context, final Object result, Class type, Field field) {
Converter converter = mapper.getLocalConverter(field.getDeclaringClass(), field.getName());
// TODO: Is there a better way to make something like this work? Does it break any custom converters?
if (converter == null && new RobustCollectionConverter(mapper, reflectionProvider).canConvert(type)) {
converter = new RobustCollectionConverter(mapper, reflectionProvider, field.getGenericType());
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In particular I am wondering about things like NodeList and AxisList which extend ArrayList. They might be ok because of the exact equality comparisons here, but I am not sure. I guess also that those classes don't support any of the desired recoverability features in RobustCollectionConverter, but that shouldn't be affected by my PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed that NodeList is unaffected by this change - its custom converter continues to be used and its deserialization is not robust against invalid values.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at #342, I think we only expect getLocalConverter to matter for fields that have the @XStreamConverter annotation, which seems to be more or less unused based on this search, and also none of those annotations are for collections or maps.

return context.convertAnother(result, type, converter);
}

Expand Down
50 changes: 50 additions & 0 deletions core/src/test/java/hudson/util/RobustCollectionConverterTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import jenkins.util.xstream.CriticalXStreamException;
import org.junit.Test;
Expand Down Expand Up @@ -173,4 +174,53 @@ private Set<Object> preparePayload() {
}
return set;
}

@Test
public void checkElementTypes() {
var expected = new Numbers(1, 2, 3);
var xmlContent =
"""
<hudson.util.RobustCollectionConverterTest_-Numbers>
<numbers>
<int>1</int>
<int>2</int>
<string>oops!</string>
<int>3</int>
</numbers>
</hudson.util.RobustCollectionConverterTest_-Numbers>
""";
var actual = (Numbers) new XStream2().fromXML(xmlContent);
assertEquals(expected, actual);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I guess OldDataMonitor will report the mistyped element?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not check it, but yes that is the hope.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As of 61db49c the error messages have been improved and the tests log the warnings. Here is an example of the warning for a bad list element:

Sep 18, 2024 5:09:31 PM hudson.diagnosis.OldDataMonitor report
WARNING: could not read hudson.util.RobustCollectionConverterTest$Data@2cc3ad05 (and Jenkins did not start up)
com.thoughtworks.xstream.converters.ConversionException: Invalid type for collection element
---- Debugging information ----
message             : Invalid type for collection element
cause-exception     : java.lang.ClassCastException
cause-message       : Cannot cast java.lang.String to java.lang.Integer
path                : /hudson.util.RobustCollectionConverterTest$Data/numbers/string
line number         : 5
-------------------------------
    at hudson.util.RobustCollectionConverter.populateCollection(RobustCollectionConverter.java:120)
    at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:81)
    at hudson.util.RobustCollectionConverter.unmarshal(RobustCollectionConverter.java:101)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
    at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:461)
    at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:350)
    at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:289)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
    at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1464)
    at hudson.util.XStream2.unmarshal(XStream2.java:230)
    at hudson.util.XStream2.unmarshal(XStream2.java:201)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1441)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1321)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1312)
    at hudson.util.RobustCollectionConverterTest.checkElementTypes(RobustCollectionConverterTest.java:207)
    ... surefire ...
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)
Caused by: java.lang.ClassCastException: Cannot cast java.lang.String to java.lang.Integer
    at java.base/java.lang.Class.cast(Class.java:3889)
    at hudson.util.RobustCollectionConverter.populateCollection(RobustCollectionConverter.java:114)
    ... 65 more

Here is an example for a bad map value:


WARNING: could not read hudson.util.RobustMapConverterTest$Data@2a5b3fee (and Jenkins did not start up)
com.thoughtworks.xstream.converters.ConversionException: Invalid type for map entry key/value
---- Debugging information ----
message             : Invalid type for map entry key/value
cause-exception     : java.lang.ClassCastException
cause-message       : Cannot cast java.lang.Integer to java.lang.String
path                : /hudson.util.RobustMapConverterTest$Data/map/entry[2]/int
line number         : 9
-------------------------------
    at hudson.util.RobustMapConverter.read(RobustMapConverter.java:114)
    at hudson.util.RobustMapConverter.putCurrentEntryIntoMap(RobustMapConverter.java:81)
    at com.thoughtworks.xstream.converters.collections.MapConverter.populateMap(MapConverter.java:99)
    at com.thoughtworks.xstream.converters.collections.MapConverter.populateMap(MapConverter.java:93)
    at com.thoughtworks.xstream.converters.collections.MapConverter.unmarshal(MapConverter.java:88)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
    at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:461)
    at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:350)
    at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:289)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
    at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1464)
    at hudson.util.XStream2.unmarshal(XStream2.java:230)
    at hudson.util.XStream2.unmarshal(XStream2.java:201)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1441)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1321)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1312)
    at hudson.util.RobustMapConverterTest.robustAgainstInvalidValueType(RobustMapConverterTest.java:244)
    ... surefire ...
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)
Caused by: java.lang.ClassCastException: Cannot cast java.lang.Integer to java.lang.String
    at java.base/java.lang.Class.cast(Class.java:3889)
    at hudson.util.RobustMapConverter.read(RobustMapConverter.java:110)
    ... 67 more

}

public static class Numbers {
private final List<Integer> numbers;

public Numbers(Integer... numbers) {
this.numbers = new ArrayList(Arrays.asList(numbers));
}

@Override
public int hashCode() {
int hash = 7;
hash = 37 * hash + Objects.hashCode(this.numbers);
return hash;
}

@Override
public boolean equals(Object obj) {
if (this == obj) {
return true;
} else if (obj == null || getClass() != obj.getClass()) {
return false;
}
final Numbers other = (Numbers) obj;
return Objects.equals(this.numbers, other.numbers);
}

@Override
public String toString() {
return "Numbers[" + numbers + "]";
}
}
}
Loading