I often find that the horses are quieter in the morning than they are in the evening, for whatever reason, so used to be when I was starting the big horses under saddle or in harness I'd try to have the first few rides/drives in the morning--same with the minis, they get hitched the first few times in the morning. If one tends to be a bit lazy, he'll get worked in the evening more often than not, and once they've got some experience then it doesn't matter, I work them whenever there is the time. Might be mornings some days, evenings other times, very late evenings some time. Weather is another factor--I don't function well in really hot temperatures so in summer I work mornings and late evenings. Have been known to be out there at midnight or later working horses in the big corral under the yard light!
I have friends who always want to work their horses at regular times--and I knw there were times when they'd go to a show & have to show very very late at night, and they'd complain that their horses didn't work well because they just weren't used to being worked at such a late hour. Mine never have that problem--work is work, at whatever weird, late hour 'mom' wants us to work!