Call C Data Interface callbacks through any delegate type on .NET Framework #491

Describe the enhancement requested

On .NET Framework (the netstandard2.0 build), Apache.Arrow calls C Data Interface callbacks through Marshal.GetDelegateForFunctionPointer<T>, using its own internal delegate types such as CArrowArrayStreamExporter.ReleaseArrayStream. If the function pointer was created in the same process from a managed delegate, .NET Framework returns that original delegate and casts it to T. When the other side used a different delegate type for the same signature, the call fails with InvalidCastException.

This means an Arrow consumer can't use a stream, array or schema exported in the same process by another library with its own delegate types, even though the native signature is identical. For example, CArrowArrayStreamImporter.ImportArrayStream and CArrowArrayStream.Free fail on a stream exported by ADBC's CAdbcDriverExporter. The only workaround available to such a library is to find Arrow's internal delegate types by reflection and export its callbacks with them, which is what apache/arrow-adbc#4983 currently has to do.

The non-generic Marshal.GetDelegateForFunctionPointer(IntPtr, Type) returns the original delegate without the cast, and a delegate of another type can be wrapped by binding to its Invoke method. Using an internal helper like the one below at the .NET Framework call sites would let Arrow call callbacks created with any delegate type:

internal static T GetDelegateForFunctionPointer<T>(IntPtr ptr) where T : Delegate
{
    Delegate d = Marshal.GetDelegateForFunctionPointer(ptr, typeof(T));
    return d as T ?? (T)Delegate.CreateDelegate(typeof(T), d, "Invoke");
}

It would replace the generic calls in CArrowArray, CArrowArrayImporter, CArrowArrayStream, CArrowArrayStreamImporter, CArrowDeviceArrayStream, CArrowDeviceArrayStreamImporter, CArrowSchema and CArrowSchemaImporter.

• It adds no public API.
• It changes nothing on .NET 5+, which calls function pointers directly.
• It costs nothing extra for native callbacks or for callbacks Arrow created itself, where d as T succeeds; the wrapper is only created in the in-process case that fails today.

ADBC already uses this approach on its import side, and it works on net472. Once Arrow does the same, ADBC can export its callbacks with its own delegate types and drop the reflection.

This issue was drafted with AI assistance (Claude Code).