I so rarely disagree with Al when it comes to driving...so in this case, I'm simply going to ask a lot of questions to get a better idea of the situation instead.
Okay, so first question: When you say it doesn't touch him at any time, does that include both standing and moving in the aforementioned uphill/downhill/on the flat?
Where are the shafts while this is happening? Are they at the point of the shoulder as normal while the traces are bulging forward or draping down?
Do you use breeching? If so, how is it adjusted? If your breeching is way too tight it can pull the cart back constantly and cause problems although you'll usually notice the shaft tips too far back when this happens.
How are your tugs riding in this situation? Do they hang straight downwards when the shaft tips are at the point of his shoulder or are they being pulled backwards? Do you use wrap straps?
The best possible thing you could do for us is to post a picture but I know that can be difficult. Since you say it's happened with three different breastcollars and no other horses I'm going to assume the problem is not with a stiff breastcollar and traces that refuse to drape/hang properly. How is your horse built through the chest/neck area? Do you put the neck strap back over the saddle or use a false martingale (the one that buckles to the breastcollar)? Tell us as much as you can!
The usual cause of this is either an extremely stiff synthetic breastcollar that sort of holds its own shape regardless of the horse or traces that are fastened much too loosely. Stand the horse on a slight uphill and adjust the traces so they are snug and taking weight when the shaft tips are at the point of the shoulder. Once they are adjusted, position the tugs ("shaft holders") so they are completely vertical or just slightly ahead of the vertical. Tighten the breeching if you have it so that while the horse is in draft you can get a couple of fingers easily between the breeching and the horse. Now walk the horse around on the flat and see if that fixed the problem. It really should, if it didn't then there's something very odd going on!
I'm assuming (always a bad idea
) that your horse is already trained to drive, i.e. knows how to pull forward into the traces and doesn't shy back from them. And that he's not prancing and sucking back prior to blowing up, since I hope you would have mentioned that.
Let us know and post pictures if you can, especially if this continues despite readjustment!