You don't have to mine nonstop on a block to finish it off. You can pause, and come back later.
backport to 1.20.1
