Skip to content

Commit 7caa3c7

Browse files
jkoritzinskyjkoritzinsky
committed
Add design doc, update API name, remove scratch pad
1 parent b9f3597 commit 7caa3c7

File tree

3 files changed

+404
-78
lines changed

3 files changed

+404
-78
lines changed

‎docs/design/libraries/ComInterfaceGenerator/VTableStubs.md

Lines changed: 399 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,399 @@
1+
# Generating Virtual Method Table Stubs
2+
3+
As a building block for the COM interface source generator, we've decided to build a source generator that enables developers to mark that a given interface method should invoke a function pointer at a particular offset into an unmanaged virtual method table, or vtable. We've decided to build this generator as a building block for a few reasons:
4+
5+
1. As part of the migration of dotnet/runtime to use source-generated P/Invokes, we encountered a few scenarios, particularly in the networking stacks, where non-blittable delegate interop was used because the native APIs do not have static entry points. For at least one of these scenarios, MsQuic, the native library provides a table of function pointers. From our experience, this mechanism for versioning is not uncommon and we feel that supporting native libraries that use a versioning scheme similar to this model is worthwhile for us to support.
6+
2. There are native APIs that we are likely to interoperate with in the future that use native vtables but are not COM-oriented. In particular, the Java Native Interface API, which both dotnet/runtime and xamarin/java.interop interface with in various capacities, uses a vtable model to support exposing their APIs to C and C++. Additionally, its API does not conform to a COM-style IUnknown-based API.
7+
3. Some COM-style APIs have some corner cases with non-COM-style interfaces. Specifically, some corners of the DirectX APIs are still vtable-based, but do not implement IUnknown. Providing this building block will allow developers to more easily consume these APIs with similar gestures as the rest of the DirectX API surface.
8+
4. Our future COM interface source generator can build on this building block to provide sane defaults while allowing developers to use the features of this generator to override any default settings provided by the COM generator.
9+
10+
## Defined types
11+
12+
To support this generator, we will define the following APIs.
13+
14+
The `VirtualMethodIndexAttribute` can be applied to an interface method to trigger the generator. This method will provide the index into the vtable for the method, whether or not the method implicitly takes the native `this` pointer, and which marshalling directions to support. It also has many of the same members as `LibraryImportAttribute` to consistently provide the same marshalling support across source-generated marshalling.
15+
16+
```csharp
17+
namespace System.Runtime.InteropServices;
18+
19+
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
20+
public class VirtualMethodIndexAttribute : Attribute
21+
{
22+
public VirtualMethodIndexAttribute(int index)
23+
{
24+
Index = index;
25+
}
26+
27+
public int Index { get; }
28+
29+
public bool ImplicitThisParameter { get; set; } = true;
30+
31+
public CustomTypeMarshallerDirection Direction { get; set; } = CustomTypeMarshallerDirection.Ref;
32+
33+
/// <summary>
34+
/// Gets or sets how to marshal string arguments to the method.
35+
/// </summary>
36+
/// <remarks>
37+
/// If this field is set to a value other than <see cref="StringMarshalling.Custom" />,
38+
/// <see cref="StringMarshallingCustomType" /> must not be specified.
39+
/// </remarks>
40+
public StringMarshalling StringMarshalling { get; set; }
41+
42+
/// <summary>
43+
/// Gets or sets the <see cref="Type"/> used to control how string arguments to the method are marshalled.
44+
/// </summary>
45+
/// <remarks>
46+
/// If this field is specified, <see cref="StringMarshalling" /> must not be specified
47+
/// or must be set to <see cref="StringMarshalling.Custom" />.
48+
/// </remarks>
49+
public Type? StringMarshallingCustomType { get; set; }
50+
51+
/// <summary>
52+
/// Gets or sets whether the callee sets an error (SetLastError on Windows or errno
53+
/// on other platforms) before returning from the attributed method.
54+
/// </summary>
55+
public bool SetLastError { get; set; }
56+
}
57+
58+
```
59+
60+
Additionally, a new interface will be provided. This new interface will be used by the source generator to fetch the native `this` pointer and the vtable that the function pointer is stored in. This interface is designed to provide an API that various native platforms, like COM, WinRT, or Swift, could use to provide support for multiple managed interface wrappers from a single native object. In particular, this interface was designed to ensure it is possible support a managed gesture to do an unmanaged "type cast" (i.e. `QueryInterface` in the COM and WinRT worlds).
61+
62+
```csharp
63+
namespace System.Runtime.InteropServices;
64+
65+
public readonly ref struct VirtualMethodTableInfo
66+
{
67+
public VirtualMethodTableInfo(IntPtr thisPointer, ReadOnlySpan<IntPtr> virtualMethodTable)
68+
{
69+
ThisPointer = thisPointer;
70+
VirtualMethodTable = virtualMethodTable;
71+
}
72+
73+
public IntPtr ThisPointer { get; }
74+
public ReadOnlySpan<IntPtr> VirtualMethodTable { get; }
75+
76+
public void Deconstruct(out IntPtr thisPointer, out ReadOnlySpan<IntPtr> virtualMethodTable) // This method allows tuple-style `var (thisPtr, vtable) = virtualMethodTableInfo;` statements from this type.
77+
{
78+
thisPointer = ThisPointer;
79+
virtualMethodTable = VirtualMethodTable;
80+
}
81+
}
82+
83+
public interface IUnmanagedVirtualMethodTableProvider<T> where T : IEquatable<T>
84+
{
85+
VirtualMethodTableInfo GetVirtualMethodTableInfoForKey(T typeKey);
86+
}
87+
```
88+
89+
## Required API Shapes
90+
91+
In addition to the provided APIs above, users will be required to add a `readonly static` field or `get`-able property to their user-defined interface type named `TypeKey`. The type of this member will be used as the `T` in `IUnmanagedVirtualMethodTableProvider<T>` and the value will be passed to `GetVirtualMethodTableInfoForKey`. This mechanism is designed to enable each native API platform to provide their own casting key, for example `IID`s in COM, without interfering with each other or requiring using reflection-based types like `System.Type`.
92+
93+
## Example Usage
94+
95+
### Flat function table
96+
97+
In this example, the native API provides a flat table of functions based on the provided version.
98+
99+
```cpp
100+
// NativeAPI.cpp
101+
102+
struct NativeAPI
103+
{
104+
int(*getVersion)();
105+
int(*add)(int x, int y);
106+
int(*multiply)(int x, int y);
107+
};
108+
109+
namespace
110+
{
111+
int getVersion()
112+
{
113+
return 1;
114+
}
115+
int add(int x, int y)
116+
{
117+
return x + y;
118+
}
119+
int multiply(int x, int y)
120+
{
121+
return x * y;
122+
}
123+
const NativeAPI g_nativeAPI = {
124+
&getVersion,
125+
&add,
126+
&multiply
127+
};
128+
}
129+
130+
extern "C" bool GetNativeAPI(int version, NativeAPI const** ppNativeAPI)
131+
{
132+
if (version > getVersion())
133+
{
134+
*ppNativeAPI = nullptr;
135+
return false;
136+
}
137+
*ppNativeAPI = &g_nativeAPI;
138+
return true;
139+
}
140+
141+
```
142+
143+
```csharp
144+
// User-written code
145+
// NativeAPI.cs
146+
using System.Runtime.CompilerServices;
147+
using System.Runtime.InteropServices;
148+
149+
[assembly:DisableRuntimeMarshalling]
150+
151+
// Define the interface of the native API
152+
partial interface INativeAPI
153+
{
154+
// There is no concept of casting for this API, but providing a type key is still required by the generator.
155+
// Use an empty readonly record struct to provide a type that implements IEquatable<T> but contains no data.
156+
readonly static NoCasting TypeKey = default;
157+
158+
[VirtualMethodIndex(0, ImplicitThisParameter = false, Direction = CustomTypeMarshallerDirection.In)]
159+
int GetVersion();
160+
161+
[VirtualMethodIndex(1, ImplicitThisParameter = false, Direction = CustomTypeMarshallerDirection.In)]
162+
int Add(int x, int y);
163+
164+
[VirtualMethodIndex(2, ImplicitThisParameter = false, Direction = CustomTypeMarshallerDirection.In)]
165+
int Multiply(int x, int y);
166+
}
167+
168+
// Define the key for native "casting" support for our scenario
169+
readonly record struct NoCasting {}
170+
171+
// Define our runtime wrapper type for the native interface.
172+
unsafe class NativeAPI : IUnmanagedVirtualMethodTableProvider<NoCasting>, INativeAPI.Native
173+
{
174+
private CNativeAPI* _nativeAPI;
175+
176+
public NativeAPI()
177+
{
178+
if (!CNativeAPI.GetNativeAPI(1, out _nativeAPI))
179+
{
180+
throw new InvalidOperationException();
181+
}
182+
}
183+
184+
VirtualMethodTableInfo IUnmanagedVirtualMethodTableProvider<NoCasting>.GetVirtualMethodTableInfoForKey(NoCasting _)
185+
{
186+
return new(IntPtr.Zero, MemoryMarshal.Cast<CNativeAPI, IntPtr>(new ReadOnlySpan<CNativeAPI>(_nativeAPI, 1)));
187+
}
188+
}
189+
190+
unsafe partial struct CNativeAPI
191+
{
192+
IntPtr getVersion;
193+
IntPtr add;
194+
IntPtr multiply;
195+
196+
[LibraryImport(nameof(NativeAPI))]
197+
public static partial bool GetNativeAPI(int version, out CNativeAPI* ppNativeAPI);
198+
};
199+
200+
// Generated code for VirtualMethodIndex generator
201+
202+
// NativeInterfaces.g.cs
203+
partial interface INativeAPI
204+
{
205+
[DynamicInterfaceCastableImplementation]
206+
partial interface Native : INativeAPI
207+
{
208+
}
209+
}
210+
211+
// ManagedToNativeStubs.g.cs
212+
partial interface INativeAPI
213+
{
214+
unsafe partial interface Native
215+
{
216+
int INativeAPI.GetVersion()
217+
{
218+
var (_, vtable) = ((IUnmanagedVirtualMethodTableProvider<NoCasting>)this).GetVirtualMethodTableInfoForKey(INativeAPI.TypeKey);
219+
int retVal;
220+
retVal = ((delegate* unmanaged<int>)vtable[0])();
221+
return retVal;
222+
}
223+
}
224+
}
225+
partial interface INativeAPI
226+
{
227+
unsafe partial interface Native
228+
{
229+
int INativeAPI.Add(int x, int y)
230+
{
231+
var (_, vtable) = ((IUnmanagedVirtualMethodTableProvider<NoCasting>)this).GetVirtualMethodTableInfoForKey(INativeAPI.TypeKey);
232+
int retVal;
233+
retVal = ((delegate* unmanaged<int, int, int>)vtable[1])(x, y);
234+
return retVal;
235+
}
236+
}
237+
}
238+
partial interface INativeAPI
239+
{
240+
unsafe partial interface Native
241+
{
242+
int INativeAPI.Multiply(int x, int y)
243+
{
244+
var (_, vtable) = ((IUnmanagedVirtualMethodTableProvider<NoCasting>)this).GetVirtualMethodTableInfoForKey(INativeAPI.TypeKey);
245+
int retVal;
246+
retVal = ((delegate* unmanaged<int, int, int>)vtable[2])(x, y);
247+
return retVal;
248+
}
249+
}
250+
}
251+
252+
// LibraryImport-generated code omitted for brevity
253+
```
254+
255+
As this generator is primarily designed to provide building blocks for future work, it has a larger requirement on user-written code. In particular, this generator does not provide any support for authoring a runtime wrapper object that stores the native pointers for the underlying object or the virtual method table. However, this lack of support also provides significant flexibility for developers. The only requirement for the runtime wrapper object type is that it implements `IUnmanagedVirtualMethodTableProvider<T>` with a `T` matching the `TypeKey` type of the native interface.
256+
257+
The emitted interface implementation can be used in two ways:
258+
259+
1. The user's runtime wrapper object can directly implement the emitted `Native` interface. This method works for cases where all interfaces are statically known to exist (interfaces are not conditionally implemented on each object).
260+
2. The user's runtime wrapper object can implement `IDynamicInterfaceCastable` and can return the handle of `INativeAPI.Native` when user code casts the wrapper to `INativeAPI`. This style is more commonly used for COM-style APIs.
261+
262+
### COM interface
263+
264+
```cpp
265+
// C++ code
266+
struct IUnknown
267+
{
268+
virtual HRESULT QueryInterface(REFIID riid, void **ppvObject) = 0;
269+
virtual ULONG AddRef() = 0;
270+
virtual ULONG Release() = 0;
271+
};
272+
273+
```
274+
```csharp
275+
// User-defined C# code
276+
using System;
277+
using System.Runtime.InteropServices;
278+
279+
interface IUnknown
280+
{
281+
public static readonly Guid TypeKey = Guid.Parse("00000000-0000-0000-C000-000000000046");
282+
283+
[UnmanagedCallConv(CallConvs = new[] { typeof(CallConvStdcall), typeof(CallConvMemberFunction) })]
284+
[VirtualMethodIndex(0)]
285+
int QueryInterface(in Guid riid, out IntPtr ppvObject);
286+
287+
[UnmanagedCallConv(CallConvs = new[] { typeof(CallConvStdcall), typeof(CallConvMemberFunction) })]
288+
[VirtualMethodIndex(1)]
289+
uint AddRef();
290+
291+
[UnmanagedCallConv(CallConvs = new[] { typeof(CallConvStdcall), typeof(CallConvMemberFunction) })]
292+
[VirtualMethodIndex(2)]
293+
uint Release();
294+
}
295+
296+
class UnknownCOMObject : IUnmanagedVirtualMethodTableProvider<Guid>, IDynamicInterfaceCastable
297+
{
298+
private IntPtr _unknownPtr;
299+
300+
public UnknownCOMObject(IntPtr unknown)
301+
{
302+
_unknownPtr = unknown;
303+
}
304+
305+
unsafe VirtualMethodTableInfo IUnmanagedVirtualMethodTableProvider<Guid>.GetVirtualMethodTableInfoForKey(Guid iid)
306+
{
307+
if (iid == IUnknown.TypeKey)
308+
{
309+
return new VirtualMethodTableInfo(_unknownPtr, new ReadOnlySpan<IntPtr>(**(IntPtr***)_unknownPtr), 3);
310+
}
311+
return default;
312+
}
313+
314+
RuntimeTypeHandle IDynamicInterfaceCastable.GetInterfaceImplementation(RuntimeTypeHandle interfaceType)
315+
{
316+
if (Type.GetTypeFromHandle(interfaceType) == typeof(IUnknown))
317+
{
318+
return typeof(IUnknown.Native).TypeHandle;
319+
}
320+
return default;
321+
}
322+
323+
bool IDynamicInterfaceCastable.IsInterfaceImplemented(RuntimeTypeHandle interfaceType, bool throwIfNotImplemented)
324+
{
325+
return Type.GetTypeFromHandle(interfaceType) == typeof(IUnknown);
326+
}
327+
}
328+
329+
// Generated code for VirtualMethodIndex generator
330+
331+
// NativeInterfaces.g.cs
332+
partial interface IUnknown
333+
{
334+
[DynamicInterfaceCastableImplementation]
335+
partial interface Native : IUnknown
336+
{
337+
}
338+
}
339+
340+
// ManagedToNativeStubs.g.cs
341+
partial interface IUnknown
342+
{
343+
partial interface Native
344+
{
345+
int IUnknown.QueryInterface(in Guid riid, out IntPtr ppvObject)
346+
{
347+
var (thisPtr, vtable) = ((IUnmanagedVirtualMethodTableProvider<Guid>)this).GetVirtualMethodTableInfoForKey(IUnknown.TypeKey);
348+
int retVal;
349+
fixed (Guid* riid__gen_native = &riid)
350+
fixed (IntPtr* ppvObject__gen_native = &ppvObject)
351+
{
352+
retVal = ((delegate* unmanaged[Stdcall, MemberFunction]<IntPtr, Guid*, IntPtr*, int>)vtable[0])(thisPtr, riid__gen_native, ppvObject__gen_native);
353+
}
354+
return retVal;
355+
}
356+
}
357+
}
358+
partial interface IUnknown
359+
{
360+
partial interface Native
361+
{
362+
uint IUnknown.AddRef()
363+
{
364+
var (thisPtr, vtable) = ((IUnmanagedVirtualMethodTableProvider<Guid>)this).GetVirtualMethodTableInfoForKey(IUnknown.TypeKey);
365+
uint retVal;
366+
retVal = ((delegate* unmanaged[Stdcall, MemberFunction]<IntPtr, uint>)vtable[1])(thisPtr);
367+
return retVal;
368+
}
369+
}
370+
}
371+
partial interface IUnknown
372+
{
373+
partial interface Native
374+
{
375+
uint IUnknown.Release()
376+
{
377+
var (thisPtr, vtable) = ((IUnmanagedVirtualMethodTableProvider<Guid>)this).GetVirtualMethodTableInfoForKey(IUnknown.TypeKey);
378+
uint retVal;
379+
retVal = ((delegate* unmanaged[Stdcall, MemberFunction]<IntPtr, uint>)vtable[2])(thisPtr);
380+
return retVal;
381+
}
382+
}
383+
}
384+
385+
// Native-To-Managed code omitted as the design has not been finalized yet.
386+
```
387+
388+
This example shows how we can build COM support on top of the vtable stub generator. The generator will support specifying a custom calling convention using the already-existing `UnmanagedCallConvAttribute`, so it will automatically support forwarding any calling conventions we implement with our extensible calling convention support to the function pointer signature.
389+
390+
## FAQ
391+
392+
- Why emit a nested interface instead of a DIM on the existing interface?
393+
- By emitting a nested interface, we enable flexibility in the implementation of the user-defined interface without our implementations getting in the way. With the current design, a managed implementation of a given interface would require the user to implement all members. If we emitted the member implementations directly as DIM implementations, then the compiler would happily allow a developer to only override one method and leave the rest using the native implementation, which would make the development experience of a managed implementation more difficult as there would be no IDE/compiler assistance to fully implement the contract.
394+
395+
## Open Questions
396+
397+
- Should we automatically apply the `[DynamicInterfaceCastableImplementation]` attribute to the generated `Native` interface?
398+
- It is a nice convenience, but it isn't applicable in all scenarios and bloats the metadata size. Additionally, since the generated interface is `partial`, we could direct users to add it themselves to the generated interface.
399+

0 commit comments

Comments
 (0)