When a bounty is not manually awarded, the system will automatically award it to the [earliest] highest-scored answer that was posted within the bounty period. But an automatic award is only half of the amount (which makes the only way to win bounties of odd multiple of 25 number of points).
But only answers with score +2 or higher are counted, because the rationale (I have, anyway) is that there should be at least two people thinking the answer has some qualities to it (or two more than those thinking it's a bad answer).
Ultimately, however, the bounty can be given to any arbitrary answer if the bounty owner wishes to do so, including answers which are spam or pure crap. Of course, these answers will be deleted quickly enough, making the bounty useless.
But since the owner of the bounty can choose to spend their bounty as they wish, there is no reason the system should "know what is the best answer" either.
Now, as for the moderators, they do have the power to cancel the bounty and refund it, but this is done in extreme cases only (e.g. if a users uses the bounty to prevent the question from being closed).
I do not think the moderators should, or even want, to get themselves involved in choosing a mathematically correct answer. If on the current team there are no set theorist moderator, how would any of them know what answer to award a "lost bounty" in set theory?
Would you let them delegate this result to me? Or some other highly voted user in the tags of set theory? If so, why not let me do it in the first place, why involve the moderators? Why not directly let the gold badge holders have a say about bounties?
There are three [valid], not mutually exclusive, reasons to put a bounty on a question:
Exposure. You're paying your best reputation points to make it more visible. Like a commercial.
Reward existing answers. You want to point at some answer and say, "this answer is fantastic and if I could vote it 10 times, I would. But here is a +100 bounty which is equivalent to voting it 10 times".
Attract new answers, and hopefully quality ones.
Of course, by getting more exposure, you also hope that good existing answers get more votes, and new good answers are added. This is an example of the three things working in unison.
The point is, that even if the bounty is not awarded to anyone, or awarded to "the wrong answer", the system is working fine.
So yeah, I don't think the moderators should have any say about who gets the bounty, with or without the presence of the bounty owner. It's simply out of their jurisdiction, and it should remain like that.
The only exception is when there is foul play in the reason for setting a bounty or the choice of the bounty winner (e.g. reputation laundering, and yes that happened before on the site), when I want the moderators to get involved.