For at least the third time, I was invited to approve or reject a proposed edit that turned out to be only a test. Why must real editors repeatedly take time-wasting tests which appear to be randomly generated to catch out automatic approvers of proposed edits? If a user has a fair amount of reputation, and no record of her editing being disputed, why may she not be given the benefit of the doubt as to her editing ability, until evidence arises to indicate otherwise? If tests must be given, they would be more illuminating if they proposed superficially plausible, but actually wrong, edits of poorly presented, but basically sensible, questions. The present tests take the form of nonsensical edits to incomprehensible questions. Having taken (and passed) these stupid tests twice, I would have regarded the third one as an insult had I not recognized it as the work of a robot.

  • $\begingroup$ One thing I'm curious about: what are the consequences, if any, of failing these (extremely irritating) tests? $\endgroup$ – user64687 Feb 28 '14 at 10:36
  • 1
    $\begingroup$ @Asal: You can't apply to an Ivy League college; and if you're a member (student or faculty) then you get thrown out immediately. $\endgroup$ – Asaf Karagila Feb 28 '14 at 10:51
  • 2
    $\begingroup$ @AsalBeagDubh Repeated failures, especially in short succession can lead to your review privileges being suspended for a week or longer, depending on the frequency of the failures. See the faq on meta.SO. Basic information and some other links can also be found in review-audit tag-wiki. $\endgroup$ – Martin Sleziak Feb 28 '14 at 10:52
  • 3
    $\begingroup$ @MartinSleziak: thanks for that; I was too lazy to search myself. To be honest, that doesn't seem like much of a deterrent; who cares about having review privileges? (It's not as if we're talking about something that really counts, like points or badges!) $\endgroup$ – user64687 Feb 28 '14 at 10:55
  • 1
    $\begingroup$ @AsalBeagDubh: There are badges available for completing review tasks. $\endgroup$ – user642796 Feb 28 '14 at 13:04
  • $\begingroup$ @ArthurFischer: true. I should think more carefully about my sarcastic asides in future! $\endgroup$ – user64687 Feb 28 '14 at 13:09
  • 1
    $\begingroup$ What do you mean by a "real editor"? $\endgroup$ – user642796 Feb 28 '14 at 13:26
  • $\begingroup$ I'm a little ignorant about the details of the generation of reviews, but I know that at least the other queues do get material from actual posts, and many are so plausible/ambiguous that they do fool even good editors. $\endgroup$ – rschwieb Feb 28 '14 at 13:31
  • $\begingroup$ @rschwieb: For suggested edit reviews, it's mainly including nonsense into an existing post. Here's an example. (See also MSO for more details.) $\endgroup$ – user642796 Feb 28 '14 at 13:40
  • $\begingroup$ @ArthurFischer Some of them were obviously that way, but I didn't know if all of them were that way. So most of them are this type of material? $\endgroup$ – rschwieb Feb 28 '14 at 14:10
  • $\begingroup$ @rschwieb: According to the MSO link I provided, the suggested edit review audits are randomly generated using actual posts as seeds to setting up Markov chains producing "phrases" that likely use words appropriate for the site (so mathematical words are more likely to show up here than on arqade), but are almost certainly complete nonsense at somewhat closer inspection. $\endgroup$ – user642796 Feb 28 '14 at 15:18
  • $\begingroup$ @ArthurFischer Wow, that's really surprising that Markov chains are producing examples this unconvincing. I guess given the variety of content on the site, though, it isn't probably that surprising. $\endgroup$ – rschwieb Feb 28 '14 at 15:26
  • $\begingroup$ @ArthurFischer: I used "real editor" in contradistinction to "automatic approvers of proposed edits". The latter include people who mindlessly approve edits and computers set up to do so. A "real editor" might also be called "a human being who edits in good faith". $\endgroup$ – John Bentin Feb 28 '14 at 17:48

What are the audits all about? See maybe this.

Are the audits always perfect? See maybe this.

Do we need these audits? See maybe this.

  • $\begingroup$ Thanks, Thomas. I now see that this issue has been raised before (although nothing appears to have been done about it). $\endgroup$ – John Bentin Feb 28 '14 at 13:08
  • 2
    $\begingroup$ Dear @JohnBentin : Do you have suggestions about what should be done? Regards $\endgroup$ – rschwieb Feb 28 '14 at 13:30
  • 1
    $\begingroup$ @rschwieb: I would get rid of these tests. Instead, invited assessors of proposed edits should be people with no record of rejected edits and a tally of accepted edits. I haven't thought about what the threshold count for the latter should be. $\endgroup$ – John Bentin Feb 28 '14 at 17:30
  • 1
    $\begingroup$ @JohnBentin Hmm, I guess you would have to be willing to change the "no rejected edits" to something like "a very low percentage of rejected edits," because many competent editors have at least one rejected edit. Sometimes this is due to an automatic process that rejects an edit (and not because of some problem with the edit.) I'd also guess that the subset of this set actually willing to carry out assessments would be very small and probably not representative. $\endgroup$ – rschwieb Mar 3 '14 at 13:50
  • $\begingroup$ @rschwieb: Yes to your first point. Regarding your last point, I would have thought that quite a high proportion of those who have taken the initiative to edit posts would respond to invitations to assess edits, if they are not discouraged by dysfunctional features of the system. $\endgroup$ – John Bentin Mar 3 '14 at 18:46
  • $\begingroup$ @JohnBentin You're right, I'm sure there would be some :) Putting a real person in charge of audits would also raise the potential for personal conflict a little higher. $\endgroup$ – rschwieb Mar 3 '14 at 18:58
  • $\begingroup$ @rschweib: I don't see the need for audits at all. Proven good editors should be trusted to assess aspiring editors. The OP or anyone else is free to object if an an editor goes wrong. $\endgroup$ – John Bentin Mar 3 '14 at 20:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .