Skip to content

JSON serialization improvements #31335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/fundamentals/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2314,6 +2314,8 @@ items:
href: ../standard/serialization/system-text-json/customize-properties.md
- name: Ignore properties
href: ../standard/serialization/system-text-json/ignore-properties.md
- name: Require properties
href: ../standard/serialization/system-text-json/required-properties.md
- name: Allow invalid JSON
href: ../standard/serialization/system-text-json/invalid-json.md
- name: Handle overflow JSON, use JsonElement or JsonNode
Expand All @@ -2328,6 +2330,8 @@ items:
href: ../standard/serialization/system-text-json/use-dom-utf8jsonreader-utf8jsonwriter.md
- name: Migrate from Newtonsoft.Json
href: ../standard/serialization/system-text-json/migrate-from-newtonsoft.md
- name: Visual Basic support
href: ../standard/serialization/system-text-json/visual-basic-support.md
- name: Supported collection types
href: ../standard/serialization/system-text-json/supported-collection-types.md
- name: Advanced
Expand Down
13 changes: 2 additions & 11 deletions docs/standard/serialization/binary-serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ author: "ViktorHofer"
---
# Binary serialization

Serialization can be defined as the process of storing the state of an object to a storage medium. During this process, the public and private fields of the object and the name of the class, including the assembly containing the class, are converted to a stream of bytes, which is then written to a data stream. When the object is subsequently deserialized, an exact clone of the original object is created.
Serialization is the process of storing the state of an object to a storage medium. In binary serialization, the public and private fields of the object and the name of the class, including the assembly containing the class, are converted to a stream of bytes, which is then written to a data stream. When the object is subsequently deserialized, an exact clone of the original object is created.

When implementing a serialization mechanism in an object-oriented environment, you have to make a number of tradeoffs between ease of use and flexibility. The process can be automated to a large extent, provided you are given sufficient control over the process. For example, situations may arise where simple binary serialization is not sufficient, or there might be a specific reason to decide which fields in a class need to be serialized. The following sections examine the robust serialization mechanism provided with .NET and highlight a number of important features that allow you to customize the process to meet your needs.
When implementing a serialization mechanism in an object-oriented environment, you often need to make tradeoffs between ease of use and flexibility. The process can be automated to a large extent, provided you are given sufficient control over the process. For example, situations may arise where simple binary serialization is not sufficient, or there might be a specific reason to decide which fields in a class need to be serialized. The following sections examine the robust serialization mechanism provided with .NET and highlight a number of important features that allow you to customize the process to meet your needs.

> [!NOTE]
> The state of a UTF-8 or UTF-7 encoded object is not preserved if the object is serialized and deserialized using different .NET versions.
Expand Down Expand Up @@ -344,12 +344,3 @@ Contains classes that can be used for serializing and deserializing objects.

- [XML and SOAP Serialization](xml-and-soap-serialization.md)\
Describes the XML serialization mechanism that is included with the common language runtime.

- [Security and Serialization](/previous-versions/dotnet/framework/code-access-security/security-and-serialization)\
Describes the secure coding guidelines to follow when writing code that performs serialization.

- [.NET Remoting](/previous-versions/dotnet/netframework-4.0/72x4h507(v=vs.100))\
Describes the various methods in .NET Framework for remote communications.

- [XML Web Services Created Using ASP.NET and XML Web Service Clients](/previous-versions/dotnet/netframework-4.0/7bkzywba(v=vs.100))\
Articles that describe and explain how to program XML Web services created using ASP.NET.
3 changes: 1 addition & 2 deletions docs/standard/serialization/custom-serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Custom serialization is the process of controlling the serialization and deseria
[!INCLUDE [binary-serialization-warning](../../../includes/binary-serialization-warning.md)]

> [!IMPORTANT]
> In versions previous to .NET Framework 4.0, serialization of custom user data in a partially trusted assembly was accomplished using `GetObjectData`. In .NET Framework version 4.0 - 4.8, that method is marked with the <xref:System.Security.SecurityCriticalAttribute> attribute, which prevents execution in partially trusted assemblies. To work around this condition, implement the <xref:System.Runtime.Serialization.ISafeSerializationData> interface.
> In versions previous to .NET Framework 4.0, serialization of custom user data in a partially trusted assembly was accomplished using `GetObjectData`. In .NET Framework versions 4.0 - 4.8, that method is marked with the <xref:System.Security.SecurityCriticalAttribute> attribute, which prevents execution in partially trusted assemblies. To work around this condition, implement the <xref:System.Runtime.Serialization.ISafeSerializationData> interface.

## Running custom methods during and after serialization

Expand Down Expand Up @@ -173,4 +173,3 @@ End Class

- [Binary Serialization](binary-serialization.md)
- [XML and SOAP Serialization](xml-and-soap-serialization.md)
- [Security and Serialization](/previous-versions/dotnet/framework/code-access-security/security-and-serialization)
16 changes: 8 additions & 8 deletions docs/standard/serialization/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,19 @@ Serialization is the process of converting the state of an object into a form th

.NET features the following serialization technologies:

- [Binary serialization](binary-serialization.md) preserves type fidelity, which is useful for preserving the state of an object between different invocations of an application. For example, you can share an object between different applications by serializing it to the Clipboard. You can serialize an object to a stream, to a disk, to memory, over the network, and so forth. Remoting uses serialization to pass objects "by value" from one computer or application domain to another.
- [JSON serialization](system-text-json/overview.md) serializes only public properties and does not preserve type fidelity. JSON is an open standard that is an attractive choice for sharing data across the web.

- [XML and SOAP serialization](xml-and-soap-serialization.md) serializes only public properties and fields and does not preserve type fidelity. This is useful when you want to provide or consume data without restricting the application that uses the data. Because XML is an open standard, it is an attractive choice for sharing data across the Web. SOAP is likewise an open standard, which makes it an attractive choice.
- [Binary serialization](binary-serialization.md) preserves *type fidelity*, which means that the complete state of the object is recorded and when you deserialize, an exact copy is created. This type of serialization is useful for preserving the state of an object between different invocations of an application. For example, you can share an object between different applications by serializing it to the Clipboard. You can serialize an object to a stream, to a disk, to memory, over the network, and so forth. Remoting uses serialization to pass objects "by value" from one computer or application domain to another.

- [JSON serialization](system-text-json/overview.md) serializes only public properties and does not preserve type fidelity. JSON is an open standard that is an attractive choice for sharing data across the web.
- [XML and SOAP serialization](xml-and-soap-serialization.md) serializes only `public` properties and fields and does not preserve type fidelity. This is useful when you want to provide or consume data without restricting the application that uses the data. Because XML is an open standard, it is an attractive choice for sharing data across the Web. SOAP is likewise an open standard, which makes it an attractive choice.

## Reference

<xref:System.Runtime.Serialization>
<xref:System.Text.Json>\
Contains classes that can be used to serialize objects into JSON format documents or streams.

<xref:System.Runtime.Serialization>\
Contains classes that can be used for serializing and deserializing objects.

<xref:System.Xml.Serialization>
<xref:System.Xml.Serialization>\
Contains classes that can be used to serialize objects into XML format documents or streams.

<xref:System.Text.Json>
Contains classes that can be used to serialize objects into JSON format documents or streams.
1 change: 0 additions & 1 deletion docs/standard/serialization/selective-serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,3 @@ If possible, make an object that could contain security-sensitive data nonserial

- [Binary Serialization](binary-serialization.md)
- [XML and SOAP Serialization](xml-and-soap-serialization.md)
- [Security and Serialization](/previous-versions/dotnet/framework/code-access-security/security-and-serialization)
1 change: 0 additions & 1 deletion docs/standard/serialization/serialization-guidelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,4 +141,3 @@ This article lists the guidelines to consider when designing an API to be serial
- [Binary Serialization](binary-serialization.md)
- [.NET Remoting](/previous-versions/dotnet/netframework-4.0/72x4h507(v=vs.100))
- [XML and SOAP Serialization](xml-and-soap-serialization.md)
- [Security and Serialization](/previous-versions/dotnet/framework/code-access-security/security-and-serialization)
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ You can also write custom converters to customize or extend `System.Text.Json` w

In the code you write for a custom converter, be aware of the substantial performance penalty for using new <xref:System.Text.Json.JsonSerializerOptions> instances. For more information, see [Reuse JsonSerializerOptions instances](configure-options.md#reuse-jsonserializeroptions-instances).

Visual Basic can't be used to write custom converters but can call converters that are implemented in C# libraries. For more information, see [Visual Basic support](how-to.md#visual-basic-support).
Visual Basic can't be used to write custom converters but can call converters that are implemented in C# libraries. For more information, see [Visual Basic support](visual-basic-support.md).

## Custom converter patterns

Expand Down
13 changes: 3 additions & 10 deletions docs/standard/serialization/system-text-json/how-to.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,15 +40,6 @@ The code samples in this article:
:::code language="csharp" source="snippets/system-text-json-how-to/csharp/SerializeBasic.cs" id="wf":::
:::code language="vb" source="snippets/system-text-json-how-to/vb/WeatherForecast.vb" id="WF":::

## Visual Basic support

Parts of System.Text.Json use [ref structs](../../../csharp/language-reference/builtin-types/ref-struct.md), which are not supported by Visual Basic. If you try to use System.Text.Json ref struct APIs with Visual Basic you get BC40000 compiler errors. The error message indicates that the problem is an obsolete API, but the actual issue is lack of ref struct support in the compiler. The following parts of System.Text.Json aren't usable from Visual Basic:

* The <xref:System.Text.Json.Utf8JsonReader> class. Since the <xref:System.Text.Json.Serialization.JsonConverter%601.Read%2A?displayProperty=nameWithType> method takes a `Utf8JsonReader` parameter, this limitation means you can't use Visual Basic to write custom converters. A workaround for this is to implement custom converters in a C# library assembly, and reference that assembly from your VB project. This assumes that all you do in Visual Basic is register the converters into the serializer. You can't call the `Read` methods of the converters from Visual Basic code.
* Overloads of other APIs that include a <xref:System.ReadOnlySpan%601> type. Most methods include overloads that use `String` instead of `ReadOnlySpan`.

These restrictions are in place because ref structs cannot be used safely without language support, even when just "passing data through." Subverting this error will result in Visual Basic code that can corrupt memory and should not be done.

## Namespaces

The <xref:System.Text.Json> namespace contains all the entry points and the main types. The <xref:System.Text.Json.Serialization> namespace contains attributes and APIs for advanced scenarios and customization specific to serialization and deserialization. The code examples shown in this article require `using` directives for one or both of these namespaces:
Expand Down Expand Up @@ -170,7 +161,9 @@ You can [implement custom converters](converters-how-to.md) to handle additional

## How to read JSON as .NET objects (deserialize)

A common way to deserialize JSON is to first create a class with properties and fields that represent one or more of the JSON properties. Then, to deserialize from a string or a file, call the <xref:System.Text.Json.JsonSerializer.Deserialize%2A?displayProperty=nameWithType> method. For the generic overloads, you pass the type of the class you created as the generic type parameter. For the non-generic overloads, you pass the type of the class you created as a method parameter. You can deserialize either synchronously or asynchronously. Any JSON properties that aren't represented in your class are ignored.
A common way to deserialize JSON is to first create a class with properties and fields that represent one or more of the JSON properties. Then, to deserialize from a string or a file, call the <xref:System.Text.Json.JsonSerializer.Deserialize%2A?displayProperty=nameWithType> method. For the generic overloads, you pass the type of the class you created as the generic type parameter. For the non-generic overloads, you pass the type of the class you created as a method parameter. You can deserialize either synchronously or asynchronously.

Any JSON properties that aren't represented in your class are ignored. Also, if any properties on the type are [required](required-properties.md) but not present in the JSON payload, deserialization will fail.

The following example shows how to deserialize a JSON string:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ We're investing in adding the features that have most often been requested. If y

Most of this article is about how to use the <xref:System.Text.Json.JsonSerializer> API, but it also includes guidance on how to use the <xref:System.Text.Json.JsonDocument> (which represents the Document Object Model or DOM), <xref:System.Text.Json.Utf8JsonReader>, and <xref:System.Text.Json.Utf8JsonWriter> types.

In Visual Basic, you can't use <xref:System.Text.Json.Utf8JsonReader>, which also means you can't write custom converters. Most of the workarounds presented here require that you write custom converters. You can write a custom converter in C# and register it in a Visual Basic project. For more information, see [Visual Basic support](how-to.md#visual-basic-support).
In Visual Basic, you can't use <xref:System.Text.Json.Utf8JsonReader>, which also means you can't write custom converters. Most of the workarounds presented here require that you write custom converters. You can write a custom converter in C# and register it in a Visual Basic project. For more information, see [Visual Basic support](visual-basic-support.md).

## Table of differences between Newtonsoft.Json and System.Text.Json

Expand Down
17 changes: 9 additions & 8 deletions docs/standard/serialization/system-text-json/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,17 @@ helpviewer_keywords:

# JSON serialization and deserialization (marshalling and unmarshalling) in .NET - overview

The `System.Text.Json` namespace provides functionality for serializing to and deserializing from JavaScript Object Notation (JSON).
The `System.Text.Json` namespace provides functionality for serializing to and deserializing from JavaScript Object Notation (JSON). *Serialization* is the process of converting the state of an object, that is, the values of its properties, into a form that can be stored or transmitted. The serialized form doesn't include any information about an object's associated methods. *Deserialization* reconstructs an object from the serialized form.

The library design emphasizes high performance and low memory allocation over an extensive feature set. Built-in UTF-8 support optimizes the process of reading and writing JSON text encoded as UTF-8, which is the most prevalent encoding for data on the web and files on disk.
The `System.Text.Json` library design emphasizes high performance and low memory allocation over an extensive feature set. Built-in UTF-8 support optimizes the process of reading and writing JSON text encoded as UTF-8, which is the most prevalent encoding for data on the web and files on disk.

The library also provides classes for working with an in-memory [document object model (DOM)](use-dom-utf8jsonreader-utf8jsonwriter.md#json-dom-choices). This feature enables random access to the elements in a JSON file or string.

There are some limitations on what parts of the library that you can use from Visual Basic code. For more information, see [Visual Basic support](how-to.md#visual-basic-support).

## Run-time reflection vs. compile-time source generation

By default, `System.Text.Json` uses run-time [reflection](../../../csharp/programming-guide/concepts/reflection.md) to gather the metadata it needs to access properties of objects for serialization and deserialization. As an alternative, `System.Text.Json` can use the C# [source generation](../../../csharp/roslyn-sdk/source-generators-overview.md) feature to improve performance, reduce private memory usage, and facilitate [assembly trimming](../../../core/deploying/trimming/trim-self-contained.md), which reduces app size. For more information, see [How to choose reflection or source generation in System.Text.Json](source-generation-modes.md).
There are some limitations on what parts of the library you can use from Visual Basic code. For more information, see [Visual Basic support](visual-basic-support.md).

## How to get the library

The library is built-in as part of the shared framework for .NET Core 3.0 and later versions. The source generation feature is built-in as part of the shared framework for .NET 6 and later versions. Use of source generation requires .NET 5 SDK or later.
The library is built-in as part of the shared framework for .NET Core 3.0 and later versions. The [source generation feature](source-generation-modes.md#source-generation---metadata-collection-mode) is built-in as part of the shared framework for .NET 6 and later versions. Use of source generation requires .NET 5 SDK or later.

For framework versions earlier than .NET Core 3.0, install the [System.Text.Json](https://www.nuget.org/packages/System.Text.Json) NuGet package. The package supports:

Expand All @@ -36,6 +32,10 @@ For framework versions earlier than .NET Core 3.0, install the [System.Text.Json
* .NET Core 2.1 and later
* .NET 5 and later

## Run-time reflection vs. compile-time source generation

By default, `System.Text.Json` uses [reflection](../../../csharp/programming-guide/concepts/reflection.md) to gather the metadata it needs to access properties of objects for serialization and deserialization *at run time*. As an alternative, `System.Text.Json` can use the C# [source generation](../../../csharp/roslyn-sdk/source-generators-overview.md) feature to improve performance, reduce private memory usage, and facilitate [assembly trimming](../../../core/deploying/trimming/trim-self-contained.md), which reduces app size. For more information, see [How to choose reflection or source generation in System.Text.Json](source-generation-modes.md).

## Security information

For information about security threats that were considered when designing <xref:System.Text.Json.JsonSerializer>, and how they can be mitigated, see [`System.Text.Json` Threat Model](https://github.com/dotnet/runtime/blob/main/src/libraries/System.Text.Json/docs/ThreatModel.md).
Expand Down Expand Up @@ -68,6 +68,7 @@ The `System.Text.Json` types are thread-safe, including:
* [Enable case-insensitive matching](character-casing.md)
* [Customize property names and values](customize-properties.md)
* [Ignore properties](ignore-properties.md)
* [Require properties](required-properties.md)
* [Allow invalid JSON](invalid-json.md)
* [Handle overflow JSON or use JsonElement or JsonNode](handle-overflow.md)
* [Preserve references and handle circular references](preserve-references.md)
Expand Down
Loading