The bit is not the problem you have a back problem and need to check the pelvic girdle, he has a slight "tilt" there.
This has happened over a long time and it could well be the saddle causing it- you need the whole thing checked, not just his back.
You will find he is also out just behind the withers in the first vertebra of the back proper- if the back specialist misses this get them to check again because it will be there.
I am more concerned about the pelvic tilt and you should stop riding him until he and the saddle can be looked at by a professional and you can be given professional advice- do not even loose school, just turnout.
Good Luck!!!