You will not be able to search for other players in the game or get the removed player back. If you ever get matched with them again, you can add them back.