Welcome to the site. I think your question is a fantastic one: there is a tension between two worthy goals: of having a site with the best (including quickest) possible answers, and having a site which actually develops people's answering skills.
I agree that the focus at present seems to be on the former, but I think that the latter is also important and more emphasis on that -- if possible -- would be a good thing.
One of the aspects of the site that I wish could be improved (somehow) is that there seems to be a rather clear dichotomy here between the question askers and the question answerers. This is in contrast to the site mathoverflow.net, which is for research-level questions. No research active mathematician is "too experienced" to ask research-level questions: indeed, asking such questions is part of doing research! Thus there is more of a give and take among the active users on that site: in particular I am one of the most active users on MO in terms of answering questions, but I have also asked almost 50 questions (not all of which have been answered, but most have!). I imagine that many of the people who answer my questions get a little something extra out of it. Essentially they get to show off in a positive way: I may know the answers to a lot of questions, but they know the answers to questions I don't know and can help me out as well. (I imagine this mostly because every once in a while a really eminent mathematician asks a question that I happen to know the answer to, and I really do get an emotional boost from that.)
Here on math.SE though we have people who have answered literally hundreds of questions and asked literally none, and indeed most of the high rep users have asked very few questions. I wish it was otherwise. I have asked only two questions here myself, but I'm thinking about how to improve in that regard. (I do admit that because I have a high rep on MO, there's much less chance that a borderline-too-elementary question that I ask there will get closed, so it's tempting to throw out questions to a community of mostly research-active mathematicians. But I think that there must be plenty of questions that I could ask here and get just as good an answer to, especially since there is a lot of overlap between the answerers here and the MO people.)
I think you're absolutely right that the reputation system has a way of drawing people in -- that's pretty much the genius of the stack exchange platform as I see it: sublimation of the natural urge to exhibit your intelligence and quickness into a socially beneficial outcome -- and getting them to answer questions that they could just as well leave to someone else. I personally have tried to make a point to only answer questions at the advanced undergraduate level or above. I totally agree that "How do I do this integral?" is something that an undergraduate math major is equally qualified to answer, and such a person probably has more to learn about writing and explaining mathematics to a general audience than I. (I don't always adhere to this rule: just today I wrote out a complete solution to a simple recursion problem, for the not very good reason that the way the problem was posed confused me for almost a day, and I finally wanted to get it out of my system.) Or, if a question gets a few answers but not the one I was expecting to see, I may chime in at a later point.
Anyway, I'd be delighted if we could have a larger discussion of these issues. Is there some way to leave room for people at various levels to answer questions? Is there some way to encourage PhD mathematicians to ask questions on this site? Do we dare admit that we do not know it all, not even everything relatively close to the ground? :)