UK Prime Minister Boris Johnson said that he will stay on as a caretaker Prime Minister until the Conservative party elects a new leader.