I recently switched to an iPod and have a Samba server that I can use to access my music library on the go if needed, via WireGuard. Honestly, I could not be more happier with it. I think there are very few advantages for technically inclined people to still pay for streaming services, and the fees are not one of them.
I am using Classic iPod using Rockbox, it is basically drag and drop without Apple's gatekeeping. I also changed battery and increase storage capacity using iFlash adapters. Although I am very happy with this setup, I recommend you check some cheap mp3 players that can be used with Rockbox. It was pain in the ass to open it, and second hand market is way too expensive.
I am using Bandcamp and Youtube to discover music. I also would like to check Musicbrainz.