It's certainly convenient, but then the authorities can track where you were/where you're going. I don't know if they do the same when manually checking the passport, but I'll assume they don't?
Just about any passport issued in the past decade or so has a machine-readable text strip on the photograph page. When the passport control officers take your passport they don't just compare your passport photo to your face -- they swipe the passport.
The RFID chip just allows them to store more data and get at it faster than the older machine-readable passports. The information that your passport (and its bearer) have arrived in country X is still available to country X's immigration database the moment you're past the checkpoint.
(Other risks of RFID passports include tracking or cloning, if the owner isn't keeping it in a tinfoil wallet or if the issuing agency's private key gets leaked. See also discussions on comp.risks passim.)
I've been to Peru a few times in the past couple years. Every time I passed through the immigration counter, they swiped my passport through a reader, and up on the customs officer's screen came a list of my travels since getting my current passport. Yeah, Peru sees a lot of tourism traffic -- more than many countries, but they're also not exactly first world, and technology penetration is spotty, at best.
They definitely do; that's the whole reason why you are required to have a passport in the first place. RFID just makes the process more efficient; there's nothing on the chip that isn't contained within the combination of your passport number and their database.
