UK and EU still apart on Brexit but PM doesn’t want to walk yet – minister