Personally I like the size of the iPhone SE, which has a 4" screen. I've talked to people and seen a lot of discussion on it so I know I'm not the only one.
The screen could be a bit larger if there was less bezel though: An iPhone SE size phone in the iPhone X style could have a 5" screen without getting any larger. That would be perfect.
The Sony Xperia Z5 Compact got the screen size just about perfect for me at 4.6". If someone could build a phone with a Pixel 2 camera, a Z5C screen, the thickness of a V35 ThinQ, compatibility with Project Fi, and an audio jack, that would be my sweet spot.
For me the 5" screen of the regular Pixel (2) is perfect.