Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It's not just the API, it's the scheduler in NT itself that won't move threads from one process group of up to 64 hardware threads (on a 64bit system) to another and so it has to be manually managed by the application if you want to scale out farther than that on NT.

Given that it's a fundamental limitation of the NT scheduler (not present in Linux), it seems like it'd be on the table for "yeah, windows makes this way harder, and a lot of applications won't scale the same way on windows as their Linux versions will", rather than "oh, that just doesn't count because they aren't using it right".

EDIT: As an aside, this kind of thing is exactly why Linux doesn't provide binary compatibility on the driver level. It's easy to paint yourself into a corner by making decisions that were perfectly sane 20 years ago. Now NT has fundamental limitations, hitting even harder in kernel space where nearly every driver out there has some macros compiled in that touched these structures. It's bad enough at the syscall layer, but it's even worse when you can't change things with code that's directly modifying internal structures.

This is exactly why Linux won't provide a driver ABI, and why it's a good thing.



I forgot that the API doesn't allow thread affinity across processor group boundaries. It's been a couple of years since I last touched all of this. Revisiting it, it becomes clear that this limitation actually prevents transparent support for >64 hardware threads in the C++ STL or pthreads on Windows.


I don't think it's a fundamental binary API limitation type thing as the issue does not exist in Windows Enterprise or Server. This was covered the last time this was posted https://www.anandtech.com/show/15483/amd-threadripper-3990x-...


> the issue does not exist in Windows Enterprise or Server.

It does. There's maximum of 64 threads in a processor group. This is because the affinity mask in tons of internal data structures inside NT are pointer width (so 64 bits on a 64bit platform).

Server and Enterprise's scheduler adjustments are just around making better decisions balancing which processor group a new process is assigned to at creation time.

You can read more about processor groups and the manual work by user space needed to manage them on all flavors of windows that support them here: https://docs.microsoft.com/en-us/windows/win32/procthread/pr...


So it is, thanks for the link. Does anyone know how to access this[1] page that is referenced towards the end of that, it comes up as "Access Denied" with no hint as to what access is needed but it's referenced all over the place in these documentation pages.

[1] https://www.microsoft.com/whdc/system/Sysinternals/MoreThan6...


It's available on archive.org


link in my comment below.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: