> While they sit there holding my passport hostage, scrolling through who knows what data

Legally it's their passport, not yours. A passport is the property of the US government at all times.