11
$\begingroup$

I could be remembering this totally wrong, but I thought there was a time when there were stacks of close votes, first posts and late answers awaiting review. Then one day, when I noticed the review tool changed and a few new badges appearing, all the counts dropped to zero within a few days, where they seem to remain, with minor perturbations. (As an aside: is it just me or did the question count jump by 40k in just two or three months?)

I like the auto-queue format, but it makes me wonder about the quality of the review that's going on. It looks like limits were imposed to discourage too much streak reviewing, but now it looks like hardly any users are able to perform more than a handful of any given review type per day anyhow.

How can we adjust the system to balance rapid review with quality consideration?

EDIT: Here is one suggestion: How about bumping up the number of reviews needed to confirm an edit slightly? (I'm not even sure what it is now, nor am I sure what the average lifespan of a pending edit is.)

$\endgroup$
10
  • 1
    $\begingroup$ The question count by month can be found here. The visible speed of review depends on the number of users doing it more than on the time an individual user spends pondering each item. So I do not take the perceived quickness as evidence of rash decisions. Is there any other evidence? $\endgroup$
    – user53153
    Dec 31, 2012 at 17:08
  • $\begingroup$ @PavelM Thanks for the link! I don't have any hard data, just the anecdotal evidence I have seen where a majority of reviewers make an obvoiusly bad call on a review. I would expect that if everyone is spending sufficient time, these bad reviews would be more rare than that. $\endgroup$
    – rschwieb
    Dec 31, 2012 at 18:02
  • 2
    $\begingroup$ It would be nice if some meta-SE guru gave us the current status of review queue policies. I understand they've been tweaked a few times, and I'm not sure which of the meta threads have current information. $\endgroup$
    – user53153
    Dec 31, 2012 at 20:45
  • 9
    $\begingroup$ It's a common fallacy that two people glancing at an item will have better judgement than a single person feeling solely responsible for the outcome. $\endgroup$
    – Phira
    Jan 2, 2013 at 0:15
  • 1
    $\begingroup$ @Phira It's like in statistics: if you want a better estimate, you can increase your sample size. I think calling it a fallacy is unfounded (although if your point is that a sample size of two is too small, then I can agree.) While these calls are very subjective measurements, it seems likely that having more people looking at it would generate better readings. $\endgroup$
    – rschwieb
    Jan 2, 2013 at 14:29
  • 1
    $\begingroup$ Condorcet's Jury Theorem, perhaps? $\endgroup$
    – Charles
    Jan 2, 2013 at 23:33
  • 6
    $\begingroup$ I just found that StackOverflow is watching the watchmen by automatically planting low-quality items in the review queue. Reviewers can get suspended from reviewing if they do not pass the test. What is the world coming to... $\endgroup$
    – user53153
    Jan 3, 2013 at 4:07
  • $\begingroup$ No, please don't increase the number of reviews needed to confirm an edit even further -- I already think it's a bad idea that it requires more than one. $\endgroup$
    – joriki
    Jan 7, 2013 at 10:03
  • $\begingroup$ @joriki The opinion is a little useful but I would also like to hear your reason! Thanks... $\endgroup$
    – rschwieb
    Jan 7, 2013 at 14:19
  • 3
    $\begingroup$ Since this came up again, let me explain my last comment: Several reviewers are NOT an independent larger sample because they KNOW that there is someone else to pick up their slack. $\endgroup$
    – Phira
    Aug 17, 2013 at 21:30

1 Answer 1

Reset to default
15
$\begingroup$

Apparently "YES!"

Just today a relatively new member came up with the bright idea of removing the "soft-question" tag from a few dozen "ancient" questions. In some cases the edit was ok, in some it was questionable, in yet others it was IMHO wrong. I approved some of them early on, but then I caught on that this is what the entire review queue was all about.

Two requests:

  1. To new members: Don't do this! It floods the front page for no observable improvement to the quality of the material on the site. B for effort, but such an edit is just "too minor".
  2. To reputable members: Do check what the review is all about, please. Reject minor edits apparently done for the sole purpose of somebody wanting to earn an edit-related badge.
$\endgroup$
5
  • 6
    $\begingroup$ The irony of me bumping and old thread in this way did not escape me. I just didn't want to post a duplicate, and also had to blow off some esteem. $\endgroup$
    – Jyrki Lahtonen
    Aug 11, 2013 at 11:18
  • 9
    $\begingroup$ I think the tool made it possible for a lot of people to become hard working sloppy reviewers. Just a day or two ago, I was the only dissenting vote on an edit which attempted to convert a question statement from degrees to radians. There was no reason to do that, and it was both potentially confusing to the op and unnecessary $\endgroup$
    – rschwieb
    Aug 11, 2013 at 12:16
  • 2
    $\begingroup$ Another incident today: a minor edit changing $\vec{x_0}$ to $\vec{x}_0$. That might have been ok, but it was forcing a stylistic preference of the editor over that of the author. Who knows? May be somewhere, the latter means the zeroth component of a vector!! I had just clicked "reject", when I received the message that it has already been approved, by 2 votes to 1. I think that it should take at least 3 ayes to cancel one nay, as reviewers usually don't reject without a good reason, but accept-automatons abound. $\endgroup$
    – Jyrki Lahtonen
    Aug 12, 2013 at 18:15
  • 1
    $\begingroup$ Maybe the best solution is to make editing absolutely thankless: no rep, no badges! Heck: maybe we can even make it cost rep to improve posts :) $\endgroup$
    – rschwieb
    Aug 12, 2013 at 18:30
  • 3
    $\begingroup$ There's a definite advantage to allowing edits from almost anyone and should be encouraged imo. There are probably other ways to deter 'serial editing' without affecting the vast majority or well-meaning edits. $\endgroup$
    – Dan Rust
    Aug 13, 2013 at 1:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .