I've had the best luck with Moonlight (OSS client) and Sunshine (OSS server replacement for GeForce Experience's "GameStream") -- very minimal latency (almost completely undetectable) from my RTX 3090 PC to my Nvidia Shield.
If your server is Linux and you have an NVIDIA card, I would also recommend applying the NVFBC consumer card restriction removal patch[1] to your driver libraries to allow you to capture directly from the GPU rather than X11/Wayland. Sunshine will automatically detect this and use it and it reduces the latency even further.
Following up here - Moonlight/Sunshine is the best remoting experience by far on a local network. Windows to Windows is virtually transparent, supported via almost all GPUs, and lagless. Linux to Windows is slightly harder to set up (The NVidia host has been the lowest-latency for me, but you really do need the NVidia driver patches).
Moonlight and Sunshine is great, but I couldn't figure out how to get HDR working on the Nvidia Shield- perhaps changing the Sunshine codec settings from the default was needed- I didn't bother fiddling with it.
I switched to using my Steam Deck. Moonlight on a docked Steam Deck with Sunshine works perfectly with HDR on an Ethernet connection. The Steam Deck should also have better controller support than the Nvidia Shield- I've been using the official Xbox wireless adapter, there's a community supported driver to use it with the Steam Deck- a video on Youtube says how to get it working.
This is my current setup as well. While on the subject of Raspberry Pi 5, if you are using RPi5 with Raspbian as a Moonlight client and want to capture window manager shortcuts like `alt+tab` but unable to do so, Wayland is the problem. I'm trying to put this knowledge out their in the hopes of a search engine indexing it.
