Good answer. One thing overlooked in this discussion is that, if things went this far bad (scene in conference room and embarrassing elevator conversation) then something went wrong. This implies there's room for improvement in the way the interviewee handled the situation.
2) Explain your concerns and be prepared to listen.
3) If you still want to leave, thank them, and leave.