Status stack buffer overrun error with dotnet 10

[ariwan]

Dear reader,

When upgrading our .net8 projects to .net10, using the RailDriver becomes impossible, as it crashes without indicating what is going wrong, no exception is thrown and no logging is shown.
After some digging, I've found the problematic method causing it:

PIEDevice device = PIEDevice.EnumeratePIE(VendorId)
(here VendorId = 0x05F3)

By running it with the debugger, the command prompt window indicates that it exited with code -1073740791 (0xc0000409) which is supposed to be a status stack buffer overrun error.

I've also used Claude Opus 4.8 to have a look at what could be a reason this occurs. It states two possible issues at

// From the decompiled PIEDevice.cs, line 617-629
HidApiDeclarations.HidD_GetHidGuid(ref HidGuid);
IntPtr deviceInfoSet = DeviceManagementApiDeclarations.SetupDiGetClassDevs(ref HidGuid, ...);
// ...
IntPtr intPtr = Marshal.AllocHGlobal(RequiredSize); // ⚠️ Never freed = memory leak
Marshal.WriteInt32(intPtr, Environment.Is64BitProcess ? 8 : 6);
DeviceManagementApiDeclarations.SetupDiGetDeviceInterfaceDetail(..., intPtr, ...); // 💥 Crashes here
list2.Add(Marshal.PtrToStringAuto(intPtr + 4)); // Raw pointer arithmetic

with the problems being:

• Marshal.AllocHGlobal never calls Marshal.FreeHGlobal -- Memory is leaked every call, causing heap pressure
• Raw IntPtr arithmetic + SetupDiGetDeviceInterfaceDetail -- .NET 10 enforces stricter Control Flow Guard (CFG) — old-style unsafe pointer patterns can now trigger 0xc0000409

Perhaps this could be a starting point to resolve the issue.

[perpetualKid]

@ariwan would you be able to verify your solution against the current RailDriver.NET code? THere is no published package yet, but the native P/Invoke implementations had been updated recently for some improvements, including also guards and type safety. The updated code is in main branch,