This is a long standing source of frustration, but maybe there is something I'm missing. If i'm in the middle of debugging, and I want to exit the debugger and return to IRB or Rails Console, "quit" won't work as it will exit IRB. "finish" also seems to have the same effect as continue. Using "delete" to remove breakpoints and then trying "continue" or "finish" doesn't work.
This SO question has a few good suggestions. It deals with specifically with debugging inside of loops. One great solution is to set the break point outside the loop, then from irb set it inside the loop and clear it manually when you want to.
Basically it comes down to putting a little bit of thought into where you set your breakpoints.
Other than that there doesn't appear to be anything else you can do.