i know some times the game does kick u to the lobby when it finds a match. it happens to both players. but maybe ur right about people quitting as well. but when chatting to my friends when we both get matched, sometimes it will kick both of us to the lobby..
that wait time is an average. so if ur higher rank and waiting for a match u will most likely wait longer than the average if there arent higher rank players online, whereas a lower rank player might be matched faster. is its an average across different users, so maybe me and my buddy get matched instantly since we arranged it, but then no one else is available for u, so it will give a warped queue time