13
$\begingroup$

I noticed two answers that arguably should have been deleted from the review queue (Exhibits A and B; see also Exhibit C given by dustin in the comments, and there are probably many others as well).

However both reviews were "completed" once two users clicked "Looks OK", this despite the fact that each of them has garnered several deletion recommendations.

I understand that two people having "approved" the post is often a signal that the post is okay; but in both these cases, several others have suggested that the posts are to be deleted (and at least in one case, it's not an answer at all, it's a separate question).

Could it be possible to accommodate this by having a "score" assigned in the review process, and not a "race" between the two type of votes? So if two users vote to delete, it will require four users to approve the post, and if one user has approved the post it will take six users to vote for deletion instead?

Of course this is not without incident, what if a post is being alternately approved and voted for deletion so the score fluctuates between 0 and 1? Well, we add a cut off, and after ten users have reviewed the post without a decision, there is an automatic moderator flag with a link to the review included.

$\endgroup$
28
  • $\begingroup$ The first post you linked to, reminds me of an email I once received, from someone who obviously bothered to check my papers, asking whether it was possible for the usual ordering of the reals to be a projective well-ordering. He even cited on sci.math a couple of my papers as evidence for this. $\endgroup$
    – Andrés E. Caicedo
    Mar 6, 2015 at 17:48
  • 1
    $\begingroup$ Moderators eventually get to make a final call, when the votes on the review queue are split. I cannot speak for other mods, but it would help my decision making immensely, if the users who have earned the privilege to vote to delete would simply do so. Or, at least write a comment like Asaf did here. Remember that as a rule moderators are not expected to rule on the correctness of an answer. And are not always able to (ok, I could do Exhibit A). We do know whose opinion to trust on many an occasion though. $\endgroup$
    – Jyrki Lahtonen
    Mar 6, 2015 at 17:57
  • $\begingroup$ @Jyrki: To be fair, on at least one of these I only wrote a comment after I noticed the review was completed. $\endgroup$
    – Asaf Karagila Mod
    Mar 6, 2015 at 18:12
  • 1
    $\begingroup$ Still, I appreciate it when you try to make our job easier. $\endgroup$
    – Jyrki Lahtonen
    Mar 6, 2015 at 18:13
  • 1
    $\begingroup$ here is a post I flagged which was disputed and then later deleted so it must have received two okays. If you want to add it to other two. $\endgroup$
    – dustin
    Mar 6, 2015 at 19:52
  • $\begingroup$ @dustin I tried to find a link to the review of that post, but could not -- probably because the post is now deleted. FWIW, there was at least one Recommend Deletion action and one Delete vote (mine). The story ended with a mod deletion, but some of the intermediate chapters are missing from it. $\endgroup$
    – user147263
    Mar 6, 2015 at 22:24
  • $\begingroup$ @Woodface the two reviewers who said it is okay, on that post, were asleep at the wheel. No biggie but I was just floored to see that flag was disputed. $\endgroup$
    – dustin
    Mar 6, 2015 at 22:27
  • $\begingroup$ One backup option: on occasions like these you can place a delete request in the RUOD room. One does not have to chat there... $\endgroup$
    – user147263
    Mar 6, 2015 at 22:28
  • $\begingroup$ @Woodface I didn't think it would of survived the queue but I will remember that next time. $\endgroup$
    – dustin
    Mar 6, 2015 at 22:36
  • 2
    $\begingroup$ @Woodface, dustin: Here is a link to the review. $\endgroup$
    – Asaf Karagila Mod
    Mar 6, 2015 at 22:38
  • $\begingroup$ @AsafKaragila impressive detective work. $\endgroup$
    – dustin
    Mar 6, 2015 at 23:16
  • 1
    $\begingroup$ @AsafKaragila This item could be your Exhibit C, or even A -- as it's so obviously a non-answer that one had to be asleep (as dustin said) to let it pass. $\endgroup$
    – user147263
    Mar 6, 2015 at 23:41
  • $\begingroup$ @Woodface: The set of exhibits is not an ordered set, and the letters are assigned arbitrarily using the axiom of choice. $\endgroup$
    – Asaf Karagila Mod
    Mar 7, 2015 at 12:29
  • $\begingroup$ @Woodface A bad answer survives the low quality review again. $\endgroup$
    – dustin
    Mar 10, 2015 at 5:37
  • $\begingroup$ @dustin: That review is incredible. $\endgroup$
    – Asaf Karagila Mod
    Mar 10, 2015 at 5:40

2 Answers 2

Reset to default
10
$\begingroup$

Completed via the method outlined in pizza's answer: Low Quality review tasks now require at minimum two Looks OK reviews to complete.

To illustrate why this is needed, here are the "post mortem" stats for all posts reviewed via the Low Quality queue:

Reviews Review Result      Final outcome                                   
------- ------------------ ----------------------------------------------- 
   938  Close              878 closed, 775 deleted (45 author-deleted)     
    41  Recommend Close    20 closed, 16 deleted (1 author-deleted)        
  1636  Delete             0 closed, 1627 deleted (5 author-deleted)       
 32940  Looks Good         3463 closed, 7731 deleted (1808 author-deleted) 
  1408  Edit               313 closed, 473 deleted (58 author-deleted)     
  4863  Recommend Deletion 0 closed, 4787 deleted (38 author-deleted)

Reviews completed as "Looks Good" and "Edit" both have a pretty high error rate; Edit is actually over 50%, however edits complete only a few % of all review tasks while over 3/4 of all reviews end as Looks OK - so adding a bit of additional oversight there should help to improve the accuracy.

$\endgroup$
2
  • 3
    $\begingroup$ I wonder, do you plan to update this answer as "pizza" changes his user name? $\endgroup$
    – Asaf Karagila Mod
    Apr 1, 2015 at 19:15
  • 2
    $\begingroup$ No I do not. I do reserve the right to just start referring to him as "Slim" for convenience though. $\endgroup$
    – Shog9
    Apr 1, 2015 at 19:18
11
$\begingroup$

I agree with the premise: too many junk answers drop out of view because the review completes too soon for enough deletion votes to accumulate.

Low Quality review implementation involves the site-configurable parameter ReviewsRequired which is set to $1$ here, compared to $2$ on Stack Overflow. An answer cannot escape the LQ queue alive unless it's either edited or gets $$ \text{ReviewsRequired}+\text{NumberOfFlags} \tag{*} $$ "Looks OK" votes.

The Number Of Flags is the total of VLQ and NAA flags raised against the answer. Typically it is $1$, so, $(*)$ evaluates to $1+1=2$.

Proposal

We should ask SE to raise ReviewsRequired to $2$ on this site. This will require a flagged post to get at least $3$ Looks OK votes, giving more opportunities for $20$K users to take a close look at it.

Supporting Evidence

This site has the fastest-moving LQ review queue across Stack Exchange, by far: median handling time is 6 minutes. This indicates we can afford a more thorough review process and still complete it within reasonable time.

$\endgroup$
7
  • $\begingroup$ Do you know what happens if a sub-20k like me votes Recommend Deletion? $\endgroup$
    – Lord_Farin
    Mar 6, 2015 at 19:54
  • $\begingroup$ @Lord_Farin: Some people make like 5k a month, so if you shut your yappers on meta, and start posting answers in two months maybe we can trust you to delete more answers! :-) $\endgroup$
    – Asaf Karagila Mod
    Mar 6, 2015 at 19:56
  • $\begingroup$ @Lord_Farin It's described in the implementation guide I linked. If the number of RecDel+Del votes reaches 6, the answer gets deleted. (This deletion is a bit softer than 3 Del votes deletion, giving the OP an option to self-undelete.) In fact, nearly all deletions from review are due to reviewers under 20K. The delete votes of 20K users are kind of wasted there. $\endgroup$
    – user147263
    Mar 6, 2015 at 19:58
  • $\begingroup$ @Asaf Alas, most of my meta-browsing is on my phone nowadays, hardly a nice answer writing device :(. There's also the issue of some prolific contributors in my favourite tags - you know who ;). $\endgroup$
    – Lord_Farin
    Mar 6, 2015 at 19:59
  • $\begingroup$ @Lord_Farin: I write answers from my phone. That is not a valid excuse! :-) $\endgroup$
    – Asaf Karagila Mod
    Mar 6, 2015 at 20:04
  • $\begingroup$ By the way, to that last link to meta.SE, once again we prove to be the best site on the network in so many aspects! What does that * mean by the site's name? $\endgroup$
    – Asaf Karagila Mod
    Mar 6, 2015 at 22:09
  • $\begingroup$ @Asaf not sure if it is still relevant: the star indicates those where the reviews are hidden from mods for 1h (as opposed to the 15min standard) $\endgroup$
    – quid Mod
    Apr 1, 2015 at 19:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .