A bit about time spent moderating
Jyrki mentioned
+1 to the thread idea. For example it would be nice to know: How many hours does a moderator work here, say, weekly? How many hours of moderator work is required weekly? And then do the arithmetic, and get an idea how many moderators we should have so that the workload won't be too much?
In short, there is lots of work that can be done and it is not easy to quantify. In many ways, mods are little different than high rep users: newer users look to the posts of experienced and high rep users as indicators of what is good, expected, or desired. Along these lines, mods are chosen by the community in recognition (I hope) of their behavior and good posts on the site - so we would hope that they continue doing whatever they were doing, though we don't know how much time that actually takes.
So I spend much of my time on MSE reading and browsing questions and/or answers as I always have. Sometimes, I take some time to write a comment or edit. I think I would have done these without being a 'moderator,' but I nonetheless think these are important actions for a moderator to do. This is simply part of being a part of the site and wanting it to function well, I think, and I've enjoyed the mixture of reading interesting content and providing a little guidance here and there.
Unfortunately, many things change once there is a little diamond next to your name.
Firstly, there are the flags. The order of magnitude of the number of flags that have occurred in the last 30 days is in the thousands. I maintain the view that this is good in that mods cannot be everywhere. If you see things that should not be here and that you either cannot handle or have doubts about, flagging is the way to go. There are many hundreds of questions and many more hundreds of answers each day - I do not read anywhere close to them all. Nonetheless, flags take time. Some flags are easy to handle: spam flags are usually particularly easy. But most flags are not - duplicate flags, plagiarism flags, etc. require us to read questions and/or answers and thus take time. Very often, experienced users use custom-message flags to try to say that some new user is doing something strange. These can take a long time, as we often have to read questions and/or answers and/or try to decipher intent. Finally, there are some flags that may require us to reach into the moderator toolbox and really do some detective work and perhaps consult the other mods, and these can be painful. And sometimes flags demand for messages to be written to users (mods can write private messages of sorts, for those that don't know), which take their own time.
Secondly, and deceptively unpalatable, are the subtle restrictions on actions that somehow affect everything. Mod votes are final: votes to close or delete are immediately binding. Flags to remove are immediately acted upon. Consequences for a slip of the tongue seem to be far more severe for mods. Mods, like everyone else, can get a bit hot under the collar in situations, and it is just so easy to quickly type a comment-reply and hit enter before fully thinking. Many members of this community have lashed out at mods in the past for this. At times, meta posts essentially targeting a particular moderator's actions are created; I feel obligated to respond in such cases. I try to consider everything I do to prevent any such unfortunate occurrence, and consequently everything I do is a bit slowed down. I'm much quieter on MSE now than I am in real life, and I've noticed that my comments are sort of watered down from what they were in the past.
To provide a concrete answer: it's hard to quantify. But I spend less than 30 minutes a day on average doing 'moderator-only' things, not counting the sort of regular browsing. Comparing my stats to the other mods for reference, I handle about the median number of flags and create/delete about the median number of comments; but I write fewer answers (compared to both the other mods and what I used to).
And to speak to the intent: if the mods of MSE thought we were getting swamped, I'm sure we could talk to the SE community team (who are great and provide a lot of guidance) and could get a few more mods.
A note about my response: I'd like to note that my answer may contain opinions that are not the same as the other moderators; this is merely my opinion.
2014 addendum
In response to a comment below, I'd like partially update this answer from the original answer given a year and a half ago. There are two significant changes: more posts than ever before, and the enhanced review queue.
It might be hard to believe now, but I remember when I first started paying attention to MSE, and I could (if I really wanted) read every post that occurs in a day. The number of posts is vastly larger, and no mod (nor any single person) reads them all. In fact, there are now many questions asked that none of the mods read at all -- even a large majority of questions. This isn't bad, because moderators aren't gatekeepers - they're there for exceptional cases.
So much of the generic moderating that occurs on the site is from non-diamond users (again, as it should be). Another key difference is the domain-mastery superpowers given to gold-badge users in a tag. Certain members of the community have taken significant but largely hidden responsibilities for sections of the site, perhaps especially through these golden superpowers. (We also have some core editers, and it is no secret that the perhaps single most impactful person behind the tag system of MSE is not a mod nor from the Community Team).
On the other side, the review queue is now far more active, and submits automatically generated flags to mods all the time. So the landscape of flags has changed. Altogether, far more flags occur now than 18 months ago. For no particularly good reason, the number of flags handled by each moderator is not at all equal. In general, one mod handles far more flags than the others (not me).
;-)$\endgroup$