That's been a concern of mine whenever we do things with the horses, or let people come and do things with our horses. It's been a long time ago now, be we used to have a dad come by with his son, and his friend with his friend's son, every other weekend -- when they had their boys. They'd come for cart rides. One boy was deaf and autistic. His dad said he never laughed so much as when he was in the cart and he would throw his arms around Eclipse when we were done. AND the "relationship" between us started with me thinking some stupid redneck was trying to make me wreck my cart when he yelled out his truck window at me LOL! It went from me wanting to basically flip the bird to a situation that routinely had me in happy tears.
My advice would be to talk to your agent about what is and is not okay to do on and off your property. I'd also recommend most people consider an "Umbrella Policy", which is a policy that picks up after the limits of your liability under homeowner's and auto leave off. In our case, there are restrictions regarding what we can and cannot do, to maintain coverage, with our horses when it comes to other people interacting with them.
Part of my career involves the insurance industry, but on the life / health insurance side (and honestly, just really because I need that licensing to do fixed / guaranteed annuities for clients it makes sense for -- I am not licensed for liability insurance or property and casualty insurance). But because part of my job is to help my clients make sure they are protected as best as possible against financial loss, I do always recommend looking into an umbrella policy. It is not expensive, and it's the kind of thing you would get from the agent who handles you home and auto coverage.
One bad thing about our world today is that you don't have to be wrong or doing anything bad to be successfully sued. You basically just have to be unlucky. So this umbrella coverage is a good thing to look into, along with being sure you are not doing anything to jeopardize the coverage you already have in place. If so, you may be able to have an attorney draft a waiver for not a lot of money, or even find one online, that would be at least some form of protection.
The situations vary from state to state, and a lot of people have different understandings of how things would really work (and the best of intentions). Lots of people will give advice with the best intentions, but without a full understanding of the rules, regulations, laws, etc. It just varies so much depending on where you live and one person may think their own situation was similar, or advice they heard and thought was good applies, but it really may not be the case. I think your first step should be a call to your P&C Insurance agent and ask her / him what you have posed here (and also, ask about that umbrella policy!).
Good luck.