21
$\begingroup$

This morning I was greeted with this: enter image description here

It's (I think) the third day in a row where the number of close review tasks I can do exceeds my daily limit of 20... at 9AM. MSE has grown a lot since the beginning, and I read today that there were 600 new questions every day. Obviously not all of them are bad, but even if 10% (and I think I'm very generous here) don't meet the quality standards, that's 60 questions a day, or 300 close votes needed to be cast. And since not all questions are reviewed the day they are asked, the problem only compounds itself.

I don't directly have access to the review stats, but as far as I can see it's always the same small number of people who are effectively closing the questions. The review limit of 20 doesn't make a lot of sense, especially since the number of close votes per day was bumped to 50 for users with enough rep. I know there are workarounds (the bookmarklet "Skip and open in a new tab" is very useful), but there shouldn't be a need for a workaround. What can be done about this?

$\endgroup$
12
  • $\begingroup$ Note per user only the $60$ votes are necessary for consideration and we have a lot more than fifteen reviewers to cast the appropriate close votes ;) Also, the first close vote most of the time is not cast from review queue (only if the low-quality auto-review kicks in I guess). I don't really see the problem. $\endgroup$
    – AlexR
    Oct 6, 2014 at 7:58
  • $\begingroup$ @AlexR That number of 300 votes (or 60/user) is a big underestimation. I highly doubt that 90% of new questions meet quality standards; and as I said, old, bad questions sometimes go through and need to be reviewed later. $\endgroup$
    – Najib Idrissi
    Oct 6, 2014 at 8:13
  • $\begingroup$ And if I look at the review history, I don't count "a lot more" than 15 reviewers during the last two days. Maybe 25-30 different people (and some of them are people who refuse to close most questions, but that's another discussion). $\endgroup$
    – Najib Idrissi
    Oct 6, 2014 at 8:16
  • $\begingroup$ Sadly, I have to agree on that. I think the reasioning behind this is that you are much faster in casting votes from the review queue than browsing questions and casting them there. Looking at todays review stats we have a much smaller queue at the moment. Maybe if you visit at peak hours of certain countries the number of trivial questions peaks as well. I think the review limit might as well be removed (or removed at a milestone of, say, 5k rep) $\endgroup$
    – AlexR
    Oct 6, 2014 at 8:17
  • 5
    $\begingroup$ There are other methods for reviewing. For example, look around the front page and check out questions that seem problematic. You have 50 closing votes. Use them wisely. $\endgroup$
    – Asaf Karagila Mod
    Oct 6, 2014 at 9:01
  • $\begingroup$ You do realize StackOverflow had a growing close queue, and had gone from around 50k to 80k before TPTB made the partly cosmetic change of just showing the number queued for the tier with the highest number of votes so far. This is a hard problem. $\endgroup$
    – djechlin
    Oct 13, 2014 at 6:48
  • $\begingroup$ See also this newer related question. $\endgroup$
    – Bill Dubuque
    Jan 1, 2015 at 19:07
  • $\begingroup$ Related: Moderators have unlimited reviews (corrected by 147263), and their reviews clear a question from the queue. I.e., if they select "Close" it's closed, and if they select "Leave Open" it leaves the queue without effect on the number of close votes. This seems relevant because I recently noticed a moderator using 30 close queue reviews in a day, perhaps to help alleviate the perceived problem of a large number of questions in the queue. $\endgroup$
    – Jonas Meyer
    Jan 1, 2015 at 19:40
  • 1
    $\begingroup$ Ah, the good old days when 26 seemed like a lot. $\endgroup$
    – user147263
    Oct 10, 2015 at 4:55
  • $\begingroup$ @MiceElf I hope it gets better once the "start of classes" burst is done. $\endgroup$
    – Najib Idrissi
    Oct 10, 2015 at 7:41
  • $\begingroup$ I now regularly see more than 100. $\endgroup$
    – JRN
    Oct 10, 2015 at 9:59
  • $\begingroup$ @JoelReyesNoche Look at the chart Mice Elf posted below, it has only been under 100 four times in the past ten days (and it was above 95 anyway). $\endgroup$
    – Najib Idrissi
    Oct 10, 2015 at 10:41

2 Answers 2

Reset to default
16
$\begingroup$

The fact that a single person cannot review everything in the queue is not bad in itself. The growth of the site should be matched by the growth of the number of review-capable users. With more users involved in reviewing, the outcomes will (on average) better represent the opinions of a wider subset of the site population, which I consider a better situation than having a dozen slightly maniacal reviewers handling everything.

What can be done? Perhaps we could expand the rudimentary Review FAQ with an explanation of the importance of review, and the role that audits play in it. Some users may be avoiding it because of the audits... it would help to realize that the "Stop, look and listen" refers to "watch out when crossing a street", it's not a form of "you are under arrest".

Back in the days when the Close Review queue on Stack Overflow had over $70000$ questions, Anna Lear reported on the results of SE Community Team deliberations:

We have considered raising the number of reviews available to people [...] However, we want more people doing fewer reviews a day to spread the work around rather than just overload the already highly engaged reviewers to ensure that nobody burns out and that as many people get involved as possible.

New (October 2015) tools for tracking and reducing the queue.

  • Close queue tracker, a plot of its size against the time. Nowadays, it is rare for the queue to be under 100 items.

  • Enhanced Close Review, a userscript that enables more actions from the queue: (1) downvote; (2) one-click vote to close that does not count toward the review limit.

An older tool is still available: New questions with a downvote or closevote, a bookmarklet which lists newest questions with at least one closevote or downvote. (The format is upvotes : downvotes : closevotes : title). Due to same-origin policy, the bookmarklet works only when you are already on the site.

$\endgroup$
4
  • 8
    $\begingroup$ "The growth of the site should be matched by the growth of the number of review-capable users." That's only true in an ideal world. What percentage of new users stay on the website 1/ long enough to get review privileges 2/ actually use these privileges? My impression is that this number is very small. The review limit is a problem with the closing process; another problem is my (perhaps flawed) perception that not enough users are part of this process. $\endgroup$
    – Najib Idrissi
    Oct 6, 2014 at 13:28
  • $\begingroup$ To clarify, I'm not asking to be able to review every single item every day. I'm saying that the close review queue having 20+ items early in the morning several days in a row is the sign of a problem. Upping review limits would, at least, alleviate some symptoms (but probably not solve the underlying problem). $\endgroup$
    – Najib Idrissi
    Oct 6, 2014 at 13:32
  • $\begingroup$ Stats are available on StackExchange. Mathematics has 148k users and 638 questions per day. In contrast, Super User has 683k users and 200 questions per day. $\endgroup$
    – Casebash
    Oct 29, 2014 at 12:01
  • 1
    $\begingroup$ @Casebash Though these "user" numbers are largely useless, the majority of them being inactive accounts. Older sites accumulate more of those, but this does not say much about their current state. $\endgroup$
    – user147263
    Oct 29, 2014 at 12:02
-2
$\begingroup$

What exactly is the point of having a different cap for the number of reviews you can do, and the number of close votes you can cast? If the "right" behavior is to review 19 items on the queue, then skip-and-new-tab the rest of the items on the queue, what is this accomplishing?

$\endgroup$
2
  • 4
    $\begingroup$ Some of the close votes may be cast from outside a review, as you encounter questions naturally. "Skip and new tab" is not an intended (or "approved") behaviour; this is something that a few slightly maniacal reviewers may be doing on their own. $\endgroup$
    – user147263
    Jan 1, 2015 at 17:52
  • 1
    $\begingroup$ If god forbid, you ran into a question you think is worth keeping open (or you might have edited it, to make it worthy of keeping around), that's a review right there and without a closing vote. Sure, it might be an argument for "more reviews than votes", but it's an arguments for inequality in the amount anyway. $\endgroup$
    – Asaf Karagila Mod
    Jan 1, 2015 at 17:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .