Any idea why the DirectML is so buggy?

[elephantpanda]

Just wondered if anyone knew why the DirectML mode (Microsoft's own provider) seems to be causing so many problems?
I have used onnxruntime for a long time and never had any crashes or bugs.
And isn't genai just built on top of onnxruntime?

I'm just curious how the bugs could get in?

I presume genai is doing something fancy over and above what onnxruntime is capable of in order to get that extra bit of speed. Some "unsafe" DML code maybe?

If I knew what kind of hacks are being used to speed up onnxruntime, why can't these things be added as part of the onnxruntime API to be fully tested?

Sorry, I don't mean to complain about open source software. It is beyond my area of expertise certainly. I am just curious. Keep up the good work. 👍

[RyanUnderhill]

The genai library was originally designed as some utility functions that had separate optimizations if the cuda provider was used. We're working on making it more generalized to support other providers better, through a better abstraction vs the current if(device_type==Cuda/Dml) stuff. So the problem is that the Dml was tacked on to make it work, but it's not very clean yet. You can see that webgpu is also tacked on similarly, and the webgpu team didn't like how ugly it was either :)

[elephantpanda]

Thanks for the reply!