Well, my sister got a new horse that is in her teens. No one bothered to remove those baby caps and now she has a double row of front teeth. The vet had to get creative to work on her mouth. Better to deal with those baby teeth when they are babies.
I had a horse several years ago that had to have caps removed. And then he had a tooth that was growing sideways into his mouth, as there wasn't enough room. That tooth had to be pulled and it was quite an ordeal for vet and horse. But he turned out to have a perfect mouth, so the dental work was well worth it.