An Identity Crisis

Question

I'm raising a philosophical question, here, having to do with the format and layout of M.SE versus the stated goals. Essentially, I am asking the question of whether the format of M.SE is in alignment with its stated goals. If this question sounds a lot like Marshall McLuhan, I won't deny the connection. I don't go so far as to say that the medium is the message, but I will definitely assert that the medium is an extremely important component of the message, and that certain mediums are better-suited to certain messages than others. Singing a song with wedding words to a funeral dirge tune simply doesn't work. So in McLuhan-esque fashion, does the medium of M.SE match its intended message?

For reference, the stated goals of M.SE, so far as I can make out, stem from a few disparate sources. The Tour says that M.SE is a Q&A "site for people studying math at any level and professionals in related fields." On the other hand, from various questions such as Homework questions - avoiding giving a complete solution, and FAQ for math.stackexchange, it is evident that helping students learn is clearly important to M.SE. The second thread there even says that

Providing an answer that doesn't help a student learn is not in the student's own best interest, and if a solution complete enough to be copied verbatim and handed in is given immediately, it will encourage more people to use the site as a free homework service.

1. For most of the SE sites, what are valued in an answer are completeness, good research, and length. Unfortunately, the self-flagging features of M.SE are continually flagging short answers for the review queues even if the answers are short, helpful hints provided by high-rep, reliable, proven members. I've seen many answers by 100k+ members in the Low Quality Posts review queue! In other words, the kind of answer most helpful to students is the kind of answer auto-flagged by the review queues, thus causing a significant waste of users' time.

2. However, with regard to long, complete, and researched answers, what is most helpful for students is precisely not that. What is most helpful for students is for them to present what they understand, pinpoint as accurately as they can where they're stuck, and then for someone to come along and help them get unstuck just in that one spot. That is, the student does the heavy lifting, and the helper is the spotter, to use literal weight-lifting terminology. The problem is that this process nearly always requires some give-and-take, some back-and-forth. The format of question and answer, with comments on questions and answers, is not well-suited to this, because that incredibly important information in the comments is a. not terribly visible, particularly when there are a lot of comments, and b. rarely re-incorporated back into the question or answer, because that duplicates the effort of typing up the comment in the first place.

3. With regard to students using M.SE to cheat and essentially have someone else do their homework for them: it does not appear to me that M.SE has the moderation, technical tools, etc., to curb this one. How do we know whether M.SE is not being massively misused by a large number of students to inflate their grades without gaining true understanding? Again, the format of question and answer will have a tendency to produce rep-seekers who "jump the gun" on a question and just provide a complete answer (I know I've done that!), either because they know that complete, long, well-researched answers get up-voted, or because they are highly interested in the question (obviously, the second reason is totally legit; even the first reason is legit, but it raises the question of what we want to reward). So I ask the question, "What does a site like M.SE reward?" And is that really what we want to reward? Another way of looking at it is this: does this site provide to students what will really help them, or does it attract students who simply want someone else to do their work for them?

Solutions to these problems, if they are acknowledged as problems, would not be terribly easy, I should think. No. 1 might be the easiest: simply have a function in place that makes a post less likely to show up in the Low Quality Posts review queue depending on the rep of the user, and perhaps other factors. Some feature engineering (to use ML language) might be necessary to figure out the most accurate way to trim the review queues down. Numbers 2 and 3 are more systemic issues. It wouldn't be too much of a stretch to say that Numbers 2 and 3 are kind of like saying, "The problem with M.SE is the SE part." N.B., I am not saying that the SE format is bad across the board. Stack Overflow is obviously a tremendous success, even with only a $71\%$ answer rate. I will say this: a more forum-like environment (where you just have one category: posts, and they're all on the same level) can be very effective in curbing the problems I've laid out, if you have good moderators. And maybe the solution to these problems is simply to elect way more moderators than we have now - say, ten times the number we currently have. I don't know.

What are your thoughts?

Answer 1

The question that you ask is quite broad, an comments are a bit too terse for discussion-in-detail. As such, I am going to attempt to address your points (1)-(3) in an answer.

(1) For most of the SE sites, what are valued in an answer are completeness, good research, and length. Unfortunately, the self-flagging features of M.SE are continually flagging short answers for the review queues even if the answers are short, helpful hints provided by high-rep, reliable, proven members. I've seen many answers by 100k+ members in the Low Quality Posts review queue! In other words, the kind of answer most helpful to students is the kind of answer auto-flagged by the review queues, thus causing a significant waste of users' time.

I think that it is important to make a distinction between "reputation" (i.e. the currency of social exchange; the way in which we regard people on the basis of their past actions) and "reputation points" (i.e. the imaginary internet points awarded by MSE to users for upvotes on questions and answers). To avoid confusion, I'll refer to the former as "reputation," and the latter as "XP."

It is very easy to very quickly earn a lot of XP by answering homework questions from calculus classes. If you can dash off an answer in a minute or two, you are virtually guaranteed to get at least 25 XP (an upvote from the questioner, plus a green check). It would take 13 downvotes to "cancel out" that quantity of XP. If one's goal is to earn XP, then all one has to do is spend a few hours a day answering questions that are tagged "pre-calculus" or "calculus."

On the other hand, such users are are diminishing their reputation, as well as the value of the site. Most of these kinds of questions are either duplicates of previous question, or are of very low quality, and should be closed rather than answered. It is, therefore, not unreasonable that answers to low-quality questions themselves get flagged as low quality.

As such, it should not be surprising at all that (a certain subset of) high XP users often have answers show up in the review queue. The point is to remember that XP and reputation are not the same thing, and should not be conflated. Playing Devil's Advocate, I might even argue the opposite position from that staked out in the question, i.e. that users who have a great deal of XP (especially in certain homework-prone tags) should receive extra scrutiny via the review queue.

(2) However, with regard to long, complete, and researched answers, what is most helpful for students is precisely not that. What is most helpful for students is for them to present what they understand, pinpoint as accurately as they can where they're stuck, and then for someone to come along and help them get unstuck just in that one spot. That is, the student does the heavy lifting, and the helper is the spotter, to use literal weight-lifting terminology. The problem is that this process nearly always requires some give-and-take, some back-and-forth. The format of question and answer, with comments on questions and answers, is not well-suited to this, because that incredibly important information in the comments is a. not terribly visible, particularly when there are a lot of comments, and b. rarely re-incorporated back into the question or answer, because that duplicates the effort of typing up the comment in the first place.

I think that it is, perhaps, time to revisit the homework policy. Every discussion of homework that I can find seems to date back to the very early days of MSE (for example, the guidelines for answering homework questions were written 8 years ago, and have receive no substantial revision since then). At the time, there was a (homework) tag which has since been burninated. To me, this indicates that the community was once somewhat more tolerant of homework style questions, but my feeling is that the culture here has shifted since those early Wild West days, and that Problem Statement Questions (PSQs) are much less welcome (at least by a large percentage of the folk who are active on Meta, which is, perhaps, a non-representative sample).

To stay a little more on-topic, I understand that the mission of MSE is to build a repository of high-quality questions with thorough, complete, well-researched answers. In some sense, the goal of MSE is to be an über-FAQ for mathematics. The goal is not to help students do their homework—there are other places that can do that (Yahoo! Answers, Quora, your local tutoring center, etc.). If students are helped by the answers here, great! But I think that the needs of students are somewhat orthogonal to the goals of MSE. I don't think that the desires of students should be given very much consideration when thinking about what the policy of MSE should be regarding PSQs.

(3) With regard to students using M.SE to cheat and essentially have someone else do their homework for them: it does not appear to me that M.SE has the moderation, technical tools, etc., to curb this one. How do we know whether M.SE is not being massively misused by a large number of students to inflate their grades without gaining true understanding? Again, the format of question and answer will have a tendency to produce rep-seekers who "jump the gun" on a question and just provide a complete answer (I know I've done that!), either because they know that complete, long, well-researched answers get up-voted, or because they are highly interested in the question (obviously, the second reason is totally legit; even the first reason is legit, but it raises the question of what we want to reward). So I ask the question, "What does a site like M.SE reward?" And is that really what we want to reward? Another way of looking at it is this: does this site provide to students what will really help them, or does it attract students who simply want someone else to do their work for them?

First off, I refuse to adjudicate the issue of who is genuinely interested in understanding the answer to a question, and who is attempting to cheat—not my circus, not my monkeys; this is the purview of the academic integrity office (or whatever) at the student's institution.

As I see it, the real issue is that homework-style PSQs are not good questions. Indeed, I would argue that they aren't questions at all; they are problems or exercises in mathematics, rather than being questions about mathematics. This is true whether or not an asker is attempting to cheat. Hence PSQs should be rapidly closed not because someone might be trying to cheat, but because they are off-topic for MSE.

---

Addressing the question more generally, I (and, if recent activity on Meta is any indication, many others) agree that there are problems with MSE. Personally, I think that these are cultural problems and not technical problems. Students have been trained to believe that MSE is a place where they can get their homework questions answered, which leads to the proliferation of such question on the site. People who answer those questions encourage the behaviour, and we end up with a vicious feedback.

Part of the solution is to aggressively close low-quality PSQs. I also think that it is reasonable to encourage people to downvote answers to PSQs, which should (hopefully) send the message that one should not answer such questions, but instead use the comment box to ask for further context or otherwise explain how the question itself could be improved. Again, this is a cultural problem, not a technical problem.

Answer 2

Referencing @XanderHenderson's excellent answer, how about the following proposal:

1. Keep the existing voting incentives for lower-rep users. We don't want to encourage serial-downvoters.
2. Once user rep has gotten past some suitable number, maybe 1k, use +1 rep for any two votes, up or down. The reason not to have +1 for every vote is that we probably don't want to make the "Epic" and "Legendary" badges that easy to get.
3. Once user rep has gotten past some higher suitable number, maybe 10k, use +1 rep for 5 votes, up or down. (This aspect optional; maybe we keep #2 and don't reduce rep points for voting. We do already have a 200-vote limit for a day. We want high-rep users to vote.)

The idea here is to get experienced users to vote. This would mitigate our current need for more moderators, by essentially motivating everyone to moderate a lot more. We could even play around with the idea of subtracting from a user's rep a small amount (say, 1-5 points) for not voting in, say, a one-week period.

Now the disadvantage of this scheme (as opposed to, say, simply electing more moderators) is that it would probably require coding changes, or at least configuration changes; the distinct advantage is that we can essentially gain an enormous number of moderators. Also, we are tapping into what is supposed to be essentially the main moderation mechanism of all SE sites: voting.