Enforcement of Quality Standards

Question

Math SE has a long-standing policy regarding what constitutes a Good Question. The policy is intended to ensure that the collection of questions and answers here is of high quality, is searchable, and is generally of use as a long-lasting reference.

Historically, we have tried to deal with low-quality questions by closing them and encouraging the asker to add appropriate details and context. Unfortunately, it often takes longer to close a question than it does for one or more users to rapidly post an answer. In addition to making it more difficult to clean up low-quality questions, these kinds of answers reward users for their low-quality contributions, and encourage further low-quality contributions in the future. While these answers might help the individual asker, they are not good for the site as a whole.

Strictly speaking, such answers are already in violation of Math SE's policies regarding quality, but enforcement of this policy by the moderators has been relatively lax—we typically prefer to remain more hands off, and let the community moderate itself. However, this problem has grown, and it seems that the laissez-faire approach is not working.

In order to address this dynamic, we will be taking a more active role in moderating the actions of users who post a large number of moderate- to low-quality answers in response to low-quality and/or duplicate questions. Our goal is to do this in a fair and transparent manner. If we find that a user has a tendency to make low-quality contributions, i.e. if we discover that a user answers a lot of "Problem Statement Questions" (PSQs) with hastily written answers, we will warn that user via private message before we take any further action.

We recognize that this looks like a shift in policy, and that some folk may be a little worried. Let me again emphasize that it has always been against Math SE policy to answer low-quality questions, and that our increased enforcement is not a change in policy, only a change in procedure. Let me also reassure all of you that we are not looking to punish anybody, and that we have no intention of sanctioning anyone without first trying to discuss the issue with them.

We would like to ask that the community help us to spread the word regarding our stepped-up enforcement of this long-standing policy. We would also like to ask that the community help us to maintain quality standards. You can help by taking the following concrete actions:

Encourage New Users to Improve Their Posts

If you encounter a low-quality question, don't answer it immediately. Instead, direct the user to the meta post How to Ask a Good Question. Help them to understand what the standards are on this site, and consider offering advice on how they might improve their question. Make it clear that there are expectations for participation in this community, but always remain polite.

Search for Duplicates

Many of the low-quality questions which appear on this site have been answered before, or are a minor variation of some question which has been answered before. Take a few minutes to search for potential duplicate questions and, if you find a good candidate, flag the question as a duplicate. If your search fails but it seems likely that the question is a duplicate then please delay answering until others have had the chance to search. This will help to organize the database, and should help the asker find the answer they need. If you find the answers to the duplicate target to be lacking, please feel free to add a new answer to the older question, rather than answering the newer duplicate.

In addition to the built-in searchbar (which has some more powerful options), you may also want to consider using the advice given in the question How to search on this site?. The math-aware search engine approach0 may also be useful.

Spread the Word

If you encounter users answering low-quality questions, please don't argue or fight with them. Politely direct them to this post, and move on. Only a small fraction of the user-base actively engages with meta, and so it is quite likely that many of these users don't even know that they are violating site policy, or that there has been an on-going debate regarding those policies.

Keep interactions Civil

Please keep all interactions on Math SE civil.

When engaging with users (old or new) who have posted (or answered) a low-quality question, remember to remain polite and courteous. Please do not issue unpleasant orders or leave judgemental comments: we will not tolerate rudeness, even when in the context of a honest attempt to improve the quality of the site. Please do not engage in long back-and-forths with other users. If a gentle, polite comment does not change someone's mind, please leave a flag and walk away.

Raise Flags

If you encounter a user who consistently answers low-quality (or duplicate) questions over a significant period of time, and polite references to this post do not seem to be helping, please raise a flag on one of their answers. Use the "in need of moderator intervention" option and let us know what is happening. Again, please do not argue with them about the quality of their contributions, nor downvote their answers in a targeted manner. Raise a flag, and move on.

Answer 1

Since this policy was announced, we have witnessed a shift in attitude from the moderators. No longer are users trusted to make their own decisions on what can be posted and what cannot, given their experience on this site. And no longer does SE respect and value its members, but rather uses intimidation and retalitatory measures such as timed suspensions to keep its users in check.

The effects of this policy have been unprecedented. Take for instance the day I received an email from the moderators, in my personal email, which has never happened in my 5-plus years on this site. Here is what I was sent:

"We have noticed that a significant number of your posts are answers in reply to "Problem Statement Questions." Which ones? Only the ones after the policy was announced, or including those before? Furthermore, the part about duplicate questions was not relevant to me, as I have used Approach0 to search for duplicate questions before answering, and reminded other users that the site exists for the past year (or more). The email was vague and sounded like it was copy-and-pasted (yes, I know that how to ask a good question exists), with a refusal to engage in how I could improve my posts or why their judgement of my posts should supersede mine. In short, it demonstrated a lack of care and sincerity, a top-down approach which stands at odds with Stack Exchange's founding philosophy of community moderation and decision-making through meta sites.

I cannot help but be reminded of the events of 2019, which led to moderators site-wide resigning over SE's failure to communicate. Despite their apologies, Stack Exchange still has not learned their lesson.

The effects of this policy have reverberated throughout the site. For example, during this time, the accounts of several high-ranking users have since been placed in timed suspension. But by arbitarily suspending people for their personal standards of when or when not to answer, our site will ultimately stagnate without anyone to do the meta tasks. This site is nothing without its users. Let me repeat: this site is nothing without the users who have voluntarily contributed their time and effort into helping this site grow, and have created a repository of high-quality mathematics and a place for users to expand and share their mathematical knowledge.

Contrary to what is believed, there is no single site policy with unified consensus. There have been influential policies such as the aforementioned how to ask a good question, but this was never intended to be black-and-white. These 'policies' are in fact suggestions which have helped users navigate the site and its unwritten rules, which have benefited many users, including myself. Even the seemingly obvious Is it appropriate to close every question that doesn't show efforts and/or lacks of context? is more nuanced than you think, not least because each question needs to be interpreted: is one sentence of context enough? There are still questions that have definitions and references attached being closed, not because those users who closed the question are idiots and did not read how to ask a good question, but because the question could have still been lacking in some way according to their subjective judgement. Once you open your eyes to the ethical conundrums in closing/reopening/suspending users, you start to see that these powers cannot be taken for granted, but rather should be vigorously debated and held to account. And once we fall into the mindset of "this question is bad, so it must be closed" without stopping to think about why, we risk a decline into a less fair and less open society.

My issue is not directly with the policy itself. Of course, Stack Exchange should have some kind of quality standard so that we don't become like the (now-defunct) Yahoo Answers. Yet we have indirectly created a culture of fear and self-censorship by not allowing users to judge if their own posts meet the standards, and by presuming users guilty before being innocent. In a time where democracy and freedom of speech is in decline around the world, this does not deserve to happen.

This policy is an indication that this site does not fully trust its users. And without its users, there will be no one to contribute. There will be no one to moderate, like we saw in 2019, without any real incentives for contributing, save for the gamification that includes badges, privileges and the like(s). This site is turning back on its original ideals and morphing into something that no one could have ever predicted.

I do not expect that this policy will be reversed, nor that you will suddenly change tack. This policy has created an imagined reality where the approach that has persisted for many years is now 'laissez-faire', which implies that the new normal is one where users are treated with suspicion and hostility. And dissent will always lurk under the surface, but once you start provoking users with this policy, and then double down with even more timed suspensions, there will be no way out of this mess. This is how this site will die.

This site hasn't been fine for a long time and we shouldn't deny it. Welcome to the beginning of the end.

Answer 2

The real problem with so-called low quality questions and answers (which are often duplicates) is that the incentive system is set up to encourage it. If a user has several tens or hundreds of thousands of reputation points, then they are obviously a highly accomplished mathematical professional. They don't care about getting more reputation points, so they can be exclusively high-minded and focus on the perceived quality of questions and answers, and gripe when those don't meet high standards. But users of lesser accomplishment get a sense of contributing when they see an answer and get points for helping an OP with an immediate issue, as opposed to directing that OP to a more general previous post, which effort gets them no recognition. If you want the less mathematically accomplished user (say, users with less than 10k reputation points) to do the work to search for previous posts and duplicates, then reward them with reputation points when they link to a previous post which the OP agrees helps with or answers the posed question. I realize this creates a need for another layer of accounting, but if a user can get nothing for doing fifteen minutes of work looking through previous posts, but might get ten or twenty five points for presenting an answer which the OP finds helpful or accepts, the incentives are heavily slanted toward skipping the work of searching and simply answering the question.

Answer 3

A week has passed since this post, and I'd like to leave some useful pointers for everyone, which I think will help everyone continue to have a positive experience in the site, and keep the community healthy.

1. If you already commented a post regarding the EoQS and the behavior did not change, do not rinse and repeat. Please leave further interaction to the moderators. We know that seeing a user going against site policy may be frustrating, but we insist you do not escalate the situation, and refer to the post above regarding a better alternative action. If you feel like the pointers above may not be clear enough, feel free to ask for clarification, but when in doubt, err on the side of caution.

2. It goes without saying that one should not systematically go through a users posts to comment on their answers to low quality questions. One flag and one comment is enough: give users time to respond to those comments, and give us time to reach out to them.

3. Seeing changes to the site will take a while (perhaps some months). There are many flags and a few mods. The new enforcement only started a while ago, so please be patient. We are definitely looking at all the flags, and we appreciate all of you pointing to the posts. I hope that in the future there will be a visible improvement to the experience of everyone in the site.

4. Regarding the above, you may always consult your own user flags on the link: https://math.stackexchange.com/users/flag-summary/current. This may help you keep track of your activity regarding this new EoQS and give you a more concrete picture of how you are contributing to the site. For example, you can check whether your flag has been accepted or not, or whether it yet pending. You may also find this in your profile in the "Impact" box, as the image below shows:

Wishing everyone a pleasant experience in math.SE,

Pedro

Answer 4

My concern, while related to another answer, is the timing of such things. I have seen many times in the past on others', and again today even on an answer I gave, a near-rabid rapid descent of the vultures on putatively bad questions.

But where does this put users who don't have time to just check their inbox every five seconds? In this particular case, it was an absolutely brand-new user, and within one hour the question was closed, even though mathematically it was a very reasonable one and I didn't see any obvious dups (though as an occasional user I did not use EVERY one of the search tools above).

If the people wishing to close it had really cared about the user learning math (what I thought we were here for), they would have looked up what an NFA was and given suggestions for how to improve the question. As I saw on another meta post somewhere (and I'm sorry I couldn't find it now, it was a good one), if you don't know what the tag is about, maybe you should wait to let the hammer fall.

In any case, it seems to me that there should be some additional guidance in this question above for how long to go before following through on close/delete votes. I understand there should be a way to remove truly bad questions quickly, but the editors here are meanwhile nearly as zealous as Wikipedia ones - and that's not necessarily a good thing.

Answer 5

Well, it's been over a year. How is the EoQS going. Really. I'd say personally that it is going badly and is in need of a rethink.

I did express my opinions about the ludicrous EoQS here with a little bit of satire and humor in one of the answers What does "low quality contributions" mean and who decides this? [+33 upvotes to -12 downvotes as of 6/6 16:48 EDT]. But maybe it is worth restating in another way. The EoQS seems to misjudge the cause of the problem and maybe even the problem itself. I'll elaborate.

To quote the OP: "Math SE has a long-standing policy regarding what constitutes a Good Question. The policy is intended to ensure that the collection of questions and answers here is of high quality, is searchable, and is generally of use as a long-lasting reference."

Well, that sounds fine and grand, and there are indeed some great questions and answers here. But in reality, that is NOT quite how this site works. In actuality, we get a lot of elementary homework-style questions from students who are not yet proficient in the basics yet but seem to be putting in the effort. Many such students who have asked 30,40,50 such questions in a semester. In reality, almost each one of these questions are 'low-quality' as per the paragraph I quoted in italics above, and cannot be 'improved' into a so-called high-quality question, because it has already been asked before on here time and time again. And yet, the general consensus on here is that these types of questions are still to be respected. And indeed they are respected--most of these questions draw a thread of comments where the question is answered somewhere in that thread of comments [and do not get downvoted or voted to close]. And so the students asking these questions keep coming back.

That these homework questions get respect is not the issue with me, I get that this is the consensus of the forum. The issue I have is this: EoQS as enforced is pretty inconsistent. On the one hand, EoQS as enforced says those rather pedantic homework questions that are all basically duplicates can stay. But then EoQS as enforced has to make scapegoats out of someone, and it seems to be those who answer the next-level-up questions, as if they are driving the problem [whatever the problem may be exactly I'm still not sure and I don't think anyone else is either]. It all seems so arbitrarily and capricious.

I've seen a few other posters who have made all sorts of brilliant contributions put into the corner by EoQS, for varying lengths of time. Some effectively banned. Examples have been cited in Meta time and time again. It is truly MSE's loss as far as I am concerned, as THESE are the posters who can answer the questions that stump everyone else. I'd say these posters rare and valuable. Why do you think these posters were put in the corner? I have 700+ answers accumulated over 4 years. Yes some of my answers were shorter and quicker. Enough of my answers though, are novellas to some questions on here that seemed pretty challenging. I myself was given a slap on the wrist from EoQS about a month ago, and in so the moderators actually even pulled as evidence answers that I had deleted myself beforehand. Also pulled as evidence was a thread where the person asking the question mutilated their question and deleted their post. Kinda felt like they were really digging.

Meanwhile, while the homework questions that are all duplicates get to stay, it is the 'sign-post' type questions in each field that have nontrivial proofs that highlight important techniques are marked as duplicates. [An example of such a question is proving that $\gcd(n,2^{2^n}+1)=1$.] Such a shame, as a well-written proof for such a question come up with from scratch would probably be good for both the person asking AND the person answering.

If you still support EoQS, I'd ask you how well you think it is actually working. Is all this effective for MSE? What was even the problem again? What we seem to have now is, instead of a repository of high-quality questions with high-quality well-written answers, you have mixed in a bunch of lower-quality questions with the answers hidden in a stream of comments. It seems to be happening even more now than before EoQS came to be. You still have a bunch of high-quality questions with some very clever, well-written answers, but that is IN SPITE of EoQS, not because of it.

Anyway Moderation, can we rethink EoQS?

ETA I think most of us, myself included, recognize the really low-quality question when we see them, and want to see those closed/not answered. The lowest quality questions aren't getting answered by anyone. They were ALREADY being shut down.

Answer 6

Ideally, the EoQS gets a revamp with a proper manifesto/playbook with a more rigorous delineation such that the community's interpretation and upholding of it will become more homogeneous and consistent and feel less capricious.

Theatre productions undergo a preview period (the public performances that precede the official opening), during which tweaks and major adjustments are continually being made, before being frozen on opening night. This is not just to ensure the most effective show possible, but also so that the cast and crew can settle into, and evolve, their roles throughout the show's run without their creative environment feeling erratic and unconducive.

Any future rewrite of the EoQS will simiarly benefit from a preview (intensive feedback-loop) period.

Answer 7

Instances such as this one illustrate a thorny problem with judging what is a "low quality" question. Answers provided to that question were criticized as duplications, and the question itself was closed as a duplicate. If one goes to the duplicate link, there is in fact a method provided to address the kind of question in the closed post. However, one of the closing moderators then adds comments to the closed question that specifically flesh out how the method in the referenced post should be applied to the specifics of the question in the closed post. In so doing, the commenter essentially covers the same specifics as are in the answers he himself criticized as duplicative.

If there is a method to solve a posed question in an older post, but even the moderator pointing to that method effectively concedes by commenting that simply reading the referenced method is potentially insufficient to readily answer the specifically posed question, where does that leave the common user in his decision process concerning whether previous posts render the present question duplicative, and collaterally the quality of questions?