30
$\begingroup$

In Paramanand's questionnaire response, he suggests that the effects of the EoQS should be studied. This made me realise that I don't really know anything past the "theory" of the EoQS since its announcement (28th April 2021), and I guess this is because I haven't been actively trying to enforce it (I spend my 'moderation time' more on editing and reviews), I don't frequent CURED, and I have not (yet?) been banned by it. Really, my main exposure to it is via others' meta posts. I only sometimes notice EoQS offenders "in the wild". I also suspect certain high-rep users no longer post, or even have an account.

There are the posts 2020: a year in moderation that give summary statistics on moderator actions (community moderation and otherwise) Is it possible to similarly get a rough idea on what and how much has happened since the EoQS began to be enforced?

Some ideas off-the-cuff:

  • number of comments (on main) referencing the EoQS

  • number of answers/questions deleted/closed due to enforcing the EoQS.

  • number of suspensions due to enforcing the EoQS (possibly split into low/high rep, and duration of suspension)

  • number of users requesting to close their account citing disagreement with the EoQS

  • Do high rep users post answers less frequently?

  • Is there some sort of metric to see how much time the EoQS is taking out of moderators? (This would also be nice for the upcoming new mods to know what they're getting into.)

  • Are users that receive multiple EoQS comments downvoted more?

And of course, the effect of the EoQS. Any way to quantify the following questions?

  • Do we have significantly less low quality Q&A pairs in the system than we would have without the EoQS?
  • Do we have significantly more good Q&As in the system than we would have without the EoQS? (i.e. did the low quality questions get improved into good ones)
  • Has the situation with duplicates improved?

Of course, the easier to obtain the data, the better; the mods are busy enough as is (hence the election). But I suspect many of these are not so easy with merely SEDE because low quality answers (with said comments) are more likely to be deleted (and hence deleted off SEDE IIRC).

One rough measure I suppose would be the difference between the 2020 and 2021 "a year in moderation" numbers, when 2021's comes out, but there are many confounding variables.

$\endgroup$
76
  • 106
    $\begingroup$ I've counted at least 6 suspensions of high-rep users (who shall remain nameless) for "low quality contributions", all of whom have only ever output quality work, and endeavoured to help the users here as much as possible. Further, I've encountered one such user confess to being afraid to answer a question, one that I believe was adequate if not outstanding, for fear of being suspended again. Perhaps some will see this as a victory, but as I said in the EoQS announcement, this is a dark period in MSE's history. $\endgroup$
    – Theo Bendit
    Nov 11, 2021 at 8:30
  • 7
    $\begingroup$ @lhf I don't think "early days" is to the point. The "don't post answers in comments" is rather a global SE thing, which was faced with some skepticism. $\endgroup$
    – quid Mod
    Nov 11, 2021 at 13:29
  • 12
    $\begingroup$ A problem with this request is that a lot is subjective; see @TheoBendit comment. Moreover as regards, closure deletion and improvement, there is no direct link as EoQS is concerned. What did change is that the creators of the deleted content are now held responsible. For years they could just keep posting and posting while others kept deleting and deleting. That's specifically regarding the second point. $\endgroup$
    – quid Mod
    Nov 11, 2021 at 13:35
  • 11
    $\begingroup$ @TheoBendit: I can fully understand the sentiments behind your comments. I have deep respect for some of the people who were suspended due to EoQS primarily because of their mathematical knowledge and polite interactions with me. And believe me this is not about victory or losing some battle. $\endgroup$
    – Paramanand Singh Mod
    Nov 11, 2021 at 14:32
  • 11
    $\begingroup$ What you might have counted, High Rep or not, might merely indicate how so many "High Rep" users continue to accumulate rep, by answering "do my work for me" questions. I'd argue, if that's how they continued to work, it was a well deserved action taken by mods. They would not have gotten suspended, if they took comments from users seriously, read the oft-linked EoQS, and take it seriously. Suspension occurred only after a moderator warning, on top of all the "heads up". It's been a long-standing myth on this site that "very high rep" correlates with "users to respect." $\endgroup$
    – amWhy
    Nov 11, 2021 at 16:55
  • 13
    $\begingroup$ Assuming that something like a three strikes rule has been in use, I'm a bit surprised by the relatively high number of EoQS related suspensions. To get such a suspension one needs to ignore two stern warnings from the mods. That. Requires either gall or stubborness (combined with having ignored meta for a number of years). Then again, EoQS definitely brought a change of the site culture. So perhaps inertia is the more likely explanation. $\endgroup$
    – Jyrki Lahtonen
    Nov 11, 2021 at 18:02
  • 19
    $\begingroup$ Personally, what I really want to see is the thing that would likely be hardest to measure: the effects of our policies on PSQs. EoQS is, ultimately, a tool in service of the long term goal to combat the ever increasing flood of PSQ spam, and improving question quality in general. What we want is a costs benefits analysis to see if it's helping us achieve this goal efficiently, but it's difficult to do a costs benefits analysis without a good way to measure the benefits, and without that, all we can hope to see is downside. $\endgroup$
    – Alexander Gruber Mod
    Nov 11, 2021 at 19:53
  • 38
    $\begingroup$ @amWhy "Also, as any researcher will acknowledge, asking your questions only six months after introduction is far too premature, given six months is only 1/20 th time length of time many bad habits evolved." Sorry, I think this is completely unfounded. "mid year evaluation" gives so many results on google. And many interventional studies (published) are conducted on the scale of weeks (e.g. training camp studies), so there's no backing from "research". What I will give you though is perhaps we should have it named "2021: a half-year of EoQS"! $\endgroup$
    – Calvin Khor
    Nov 12, 2021 at 1:15
  • 10
    $\begingroup$ @TheoBendit If you are so convinced that EoQS is bad, why are you not running in this election, rallying for support? In the previous election there was one candidate openly critical about most forms of quality control. They did not make it into top three IIRC. May be the voters are actually in favor of quality control measures, and your opinion is less popular than you think? Only one way to find out! $\endgroup$
    – Jyrki Lahtonen
    Nov 13, 2021 at 15:40
  • 39
    $\begingroup$ @Jyrki I'm not sure if you're trying to encourage me to run, or trying to talk me out of my views with an ad populum. I do not express my opinions because they are or are not popular, and my reasons for not running are my own business. One reason I'm comfortable sharing is that I don't believe that I would fit well into the moderation team such as it is. Perhaps, Jyrki, you might want to observe the popularity of your own position outside the CRUDE/meta echo chamber, and observe how people are actually using the site? $\endgroup$
    – Theo Bendit
    Nov 13, 2021 at 21:57
  • 17
    $\begingroup$ @quid The message was mostly in response to Jyrki, interrogating the popularity of my opinions. I believe that the opinions on meta are not as representative of the community as a whole as we take them to be, and this seemed to be a core assumption in Jyrki's line of reasoning. But, I'm pretty sure this isn't what you wanted to discuss, so more to follow... $\endgroup$
    – Theo Bendit
    Nov 14, 2021 at 1:52
  • 12
    $\begingroup$ More generally (and controversially), I'm increasingly concerned that much of our moderation system is entirely at odds with how almost everyone uses our site. The site's ultimate purpose has to be the dissemination of mathematical knowledge (and hopefully skills too); a repository of Q&As that is unread and inaccessible is wasted effort. Not only does it take specialised skills and third party tools in order to search this site, but this is just not how our users tend to use the site. We're supposed to be for students of all levels, but we seem mostly suited to postgraduate level and above. $\endgroup$
    – Theo Bendit
    Nov 14, 2021 at 1:53
  • 32
    $\begingroup$ I take the role of moderators (diamond or no) quite literally: they are here to moderate the community. This means, let the community largely do its thing, and aim to curb self-destructive patterns. I am worried that we have spilled over from "moderation", and now we are in "control" territory. We are trying to mould the community into an ideal. I buy that, if the community conformed to this ideal (we all used MathJax, searched long and hard for existing answers, etc), then there would be no problem, but if the community doesn't conform, then we are proceeding from a false assumption. $\endgroup$
    – Theo Bendit
    Nov 14, 2021 at 1:53
  • 27
    $\begingroup$ Believe it or not, but I generally support us encouraging users to conform to our quality standards, both askers and answerers. They're not my weapon of choice, but down-votes are a sensible tool here. What I oppose is the enforcement through things like bans. When we start punishing people en masse for the sharing of good mathematical knowledge, this is where our community becomes self-destructive. This is where we need a little moderation in our moderation. $\endgroup$
    – Theo Bendit
    Nov 14, 2021 at 1:53
  • 8
    $\begingroup$ @Theo "but down-votes are a sensible tool here" no they often are not because they have too little impact and (thus) little to no effect. There are users that get hundreds of downvotes and over all this feedback does not change their conduct at all. There are users that ostentatiously act counter to the QS, some even explicitly so. This is unacceptable and ought to have actual consequences. Otherwise we better drop the QS. $\endgroup$
    – quid Mod
    Nov 14, 2021 at 16:54

2 Answers 2

Reset to default
11
$\begingroup$

There needs to be an analysis of whether this year's EQOS changes have improved the site or damaged it, and to what extent. If a large part of the community identified a problem and felt something should be done, have the changes been successful or is this more an example of this is something, so we must do this?

  1. The key question is whether fewer bad questions and more good questions are being asked. My impression is that the truly awful questions (such "see photo link" and nothing more) are still coming in at the same rate.
  2. Are valuable answers to interesting questions being closed or deleted, when they would have survived before? A recent example is this which has been closed, re-opened and closed again following a campaign in the CURED chatroom, with seven upvotes and seven downvotes and three sophisticated answers.
  3. Does the new approach encourage types of questions which would previously been closed? A typical example would be questions of the "I know my detailed approach gives a wrong answer because I know the right answer, but why?" type which previously closed as a duplicate of an earlier question showing a solution that works, but now survives because it is the opposite style to no-context questions that are now seen as particularly egregious.
  4. Has the sense of community and mutual trust been damaged? One group of intensive users feel that a substantial number of other long-standing users are deliberately ignoring the new approach. Some of these other individuals feel they are being targeted by organised vigilantes determined to get them suspended from the site.

My view is that the answer to 4 is clearly yes, there has been damage and mutual trust is lower than it was a year ago. That might possibly be worthwhile in an ends justify the means sense if the answers to 1, 2 and 3 showed that the site had become better as a result as a repository of useful questions and answers. And that is why the analysis needs to be done.

$\endgroup$
16
  • 4
    $\begingroup$ As I commented under the question, closure/deletion are not really related to EoQS. The question you linked in 2. is a PSQ and will be closed even before EoQS. $\endgroup$
    – Arctic Char
    Nov 12, 2021 at 22:41
  • 1
    $\begingroup$ As a frequent close-voter I completely agree with 1. But again, it seems to me that EoQS is not directly related to that (though it seems that the ultimate goal is to decrease low quality questions). 4. is also true.... $\endgroup$
    – Arctic Char
    Nov 12, 2021 at 22:45
  • 35
    $\begingroup$ @ArcticChar I doubt it would have been closed a second time after it had good answers. It may not have been closed at all: one of my old low-effort/high-vote answers looks like a response to something which now looks similar to a "problem statement question" with 78 upvotes and 0 downvotes $\endgroup$
    – Henry
    Nov 12, 2021 at 22:53
  • 6
    $\begingroup$ It's not 2013 now. That PSQ is discouraged was discussed back in 13/14. $\endgroup$
    – Arctic Char
    Nov 12, 2021 at 22:58
  • 7
    $\begingroup$ I appreciate you raising your concerns. A big part of the problem has always been that the users getting targeted by EoQS flags and suspensions never seem to have read the related meta discussion. Further comments. About your point 1. It took several years for the site to develop a reputation as a do-my-homework mill. Putting a few answerers on a shorter leash is not going to magically erase that reputation in any short period of time. $\endgroup$
    – Jyrki Lahtonen
    Nov 13, 2021 at 4:00
  • 6
    $\begingroup$ 2. I cannot comment on the particular question for lack of familiarity with the topic. At a glance it looks like a totally unmotivated query with the potential to be a masked homework assignment. There is probably more to it, and we probably need to discuss further how to improve the appearance of such questions, and to what extent questions can be judged by their appearance alone. That will not be easy. $\endgroup$
    – Jyrki Lahtonen
    Nov 13, 2021 at 4:06
  • 3
    $\begingroup$ 3. Sounds like you are discussing about solution verifications and such. Yes, those are problematic. I think that discussion is currently going on elsewhere. It has even made to the list of the questions that the nominees in the current election are asked to address. I think that an even wider discussion is called for. $\endgroup$
    – Jyrki Lahtonen
    Nov 13, 2021 at 4:09
  • 8
    $\begingroup$ 4. I mostly disagree with this sentiment. The policy against PSQs cannot really be called the new approach given that the problem was identified ages ago, and the policies were discussed extensively, leading to a compromise rule on PSQs (that, by design, makes everybody unhappy). Any mutual trust was also erased a long time ago by the block of users who chose to deliberately ignore the rule while the other block was glenching their fists in disgust. Making the homework answering machines accountable is necessary for the lost trust to be rebuilt. $\endgroup$
    – Jyrki Lahtonen
    Nov 13, 2021 at 4:14
  • 5
    $\begingroup$ So from that point of view, mutual trust is now improving, because the other block can no longer ignore the rules. My fists have been relaxing a bit, and I have stopped casting EoQS downvotes (which was one of the rules/recommendations listed in the post where the moderators described the EoQS-policy). $\endgroup$
    – Jyrki Lahtonen
    Nov 13, 2021 at 4:22
  • 5
    $\begingroup$ Anyway, to me EoQS is chiefly targeting exactly the high-rep PSQ mass-answerers. Others most likely want to see a wider scope. Which is intellectually honest when the goal is to get rid of the homework-mill reputation. $\endgroup$
    – Jyrki Lahtonen
    Nov 13, 2021 at 4:40
  • 4
    $\begingroup$ @JyrkiLahtonen: thanks a lot for your responses. I really never understood why most of the users who faced the EoQS sanctions never participated in meta discussions and elaborated their opinions. $\endgroup$
    – Paramanand Singh Mod
    Nov 13, 2021 at 6:17
  • 2
    $\begingroup$ @Henry What do you think about Joonas Ilmavirta's analogy? I suspect many affected users might share the feeling. That's fine, I suppose. But in a comment under a follow up I tried to explain how that attitude has lead to this schism. $\endgroup$
    – Jyrki Lahtonen
    Nov 13, 2021 at 7:17
  • 1
    $\begingroup$ Interesting questions/answers can always survive, one can always use their own context and their own attempts to write a new question, or use the context rewrite guidelines, or engage in context addition with OP. I wrote an answer on this here so the question in point number 2 should not survive (as someone with a background in probability) but should instead have been taken up as an improvement project. $\endgroup$
    – Sarvesh Ravichandran Iyer
    Nov 13, 2021 at 9:32
  • 5
    $\begingroup$ "I've counted at least 6 suspensions of high-rep users [...] all of whom have only ever output quality work, and endeavoured to help the users here as much as possible." To get a suspension, they would have to be squandering their gifts on low quality posts. Is there some shortage of good questions to answer that I've not yet heard about? It looks to me like there are plenty such questions to go around. Wouldn't it be better if they were spending their time on posts that clear the quality bar? $\endgroup$
    – rschwieb
    Nov 18, 2021 at 19:46
  • 7
    $\begingroup$ Why do bad posts deserve a great answer before their quality is rectified? Sometimes it doesn't take much at all to help make a bad question acceptable, and if the user learns how to ask a better question, another victory has been achieved. Then there's no problem with an answer being contributed. Why is so much value being placed on the capability to answer a bad question before it has a chance to improve? $\endgroup$
    – rschwieb
    Nov 18, 2021 at 19:53
6
$\begingroup$

I'm not involved in the site much anymore and don't know much about CRUDE or EoQS. I do know that the very low signal-to-noise ratio (with most questions being of the form "do this homework problem for me," and most effort going to answering these questions while more specialized, research-level questions get Tumbleweeds) is the main reason I stopped being as active here.

If certain users (a) saw that there was a very clear community consensus to stop writing "sophisticated" answers to low-quality questions; (b) decided, for whatever reason, to thumb their nose at this consensus; (c) stubbornly persisted despite multiple warnings from moderators; and finally (d) got banned, then in my opinion, good riddance. Whether these users were high-rep "vested contributors" or not.

$\endgroup$
5
  • 8
    $\begingroup$ $(a)$ has has never been true - and likely never will be w/o banning everyone whose viewpoint differs from $(a)$. So we've always strived for compromises - with varying degrees of success. $\endgroup$
    – Bill Dubuque
    Nov 21, 2021 at 23:13
  • 5
    $\begingroup$ +1 It is irrelevant whether there is a consensus or not. The banned users ignore the compromise carefully designed to leave everybody unhappy. For me the nail in the coffin is the duplication. We have users who think that acing a Calculus 101 exam a hundred times (or some other exam at that level) makes them a hundred times more valuable as contributors. $\endgroup$
    – Jyrki Lahtonen
    Nov 22, 2021 at 5:16
  • 5
    $\begingroup$ I know that the EoQS sanctions have also hit users capable of producing math well above what is typically seen in PSQs. What I cannot understand, why they want to keep their posting volume at higher level? When gifted with the ability to answer more difficult questions, skip the banal ones, please. $\endgroup$
    – Jyrki Lahtonen
    Nov 22, 2021 at 9:05
  • 1
    $\begingroup$ @JyrkiLahtonen even difficult questions can be PSQs too. so not a good idea to answer a qn just because it is difficult. $\endgroup$
    – cineel
    Jan 24, 2022 at 19:41
  • 1
    $\begingroup$ @cineel True. But that is unrelated to my gripe of capable users spewing out tons of banalities. A question can be banal even if it is not a PSQ. I would let competent newbies answer the simple questions for otherwise it is difficult for them collect rep points. As a veteran I step in only if the earlier answerers make a muck of it. $\endgroup$
    – Jyrki Lahtonen
    Jan 27, 2022 at 10:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .