Review queue tests in need of revision?

Question

I was going through the review queue earlier when I was asked to review this question, which, in my opinion, is poorly written and most of its content is an image.

Although, judging by this thread, there does not seem to be clear consensus on the use of images as a form of asking questions (indeed, it may make sense in some cases), this question is poorly written and it is very difficult to edit.

That said, I decided to write a comment, but I got the message "This is an audit." I then voted to close it, but it turns out that for some reason I fail to understand, this has received 6 positive votes.

Apparently 6 users judged that this is an interesting question. Yet, I feel these tests don't make sense. I am very careful with my reviews, still I have got about 5 "STOP. Look and Listen(?)" just this week. There have been similar concerns on this topic (see here for example).

This appears to happen mostly in the queue of low quality posts. My first concern is that there are posts which are badly written, but can be improved. In such cases I would be more inclined to write a comment, than recommend deletion, but at the same time this may be a test. I feel that this setup doesn't allow the reviewers to be more lenient or more strict than (or have a different rationale from) the script that generates the audits.

My suggestion would be to limit the authority of this robo-auditor and rely more on people's opinions. In order to prevent users from blindly accepting or rejecting, there could be a timer that the reviewer spends some minimum time on each item.

Answer 1

Bad robo audits can be frustrating, but they don't really do much, apart from make you wish the robo auditor had a neck you could squeeze. The criteria for how it chooses the "right" choice is explained here and is clearly prone to occasional errors. (If it weren't, why would we need reviewers?) It is possible to get an automatic review ban for failing audits, but only after you fail a lot of them in a short span of time. Realistically, auto review bans are issued rarely enough that you would have to be very unlucky to get one by mistake.

Otherwise, review ban failures are only kept track of as a statistic for diamond mods to see. While I can't speak for other SE sites, the math mods never review ban on audit failures alone. In particular, while failing an unusually high proportion of audits may raise some eyebrows, we will always look at the reviews, and won't ever hold an "unjust" audit failure against you.

That said, I know what it's like to get tilted at that smug robo auditor telling you're wrong when you're not. My best advice is to keep in mind that it's just a script (reading how it works actually helps) and that when a human mod sees the review, you'll be vindicated. If you are careful with reviews, failed audits should not yield any consequences.