# Examples of poor review audits

Since threads about review audits seem to be turning up fairly frequently now, it seemed like a good idea to open a thread to collect examples of those audits where people failed but feel like they should not have.

This serves both the purpose of getting at least a small idea of the scale of this (ie, how often does it happen), and also means that we will potentially get information enough to suggest concrete changes to the algorithms used for picking the audit cases (if this seems necessary).

• board of shame is a better name for this question – Norbert Sep 11 '13 at 14:52
• And for venting! It's frustrating to spend time thinking about a post, only to be told that I've chosen the wrong answer. – Douglas S. Stones Sep 11 '13 at 16:50
• @DouglasS.Stones that too, without each person needing to vent having to open a new thread for it (assuming they actually see this one). – Tobias Kildetoft Sep 11 '13 at 17:52
• I seem to recall that it is possible to remove audits from the audit system -- this post could be useful for that. But I can't find the source. Is there anyone who can verify this? – Lord_Farin Sep 12 '13 at 14:05
• I get quite a few, but they are glaringly obvious and quick to dismiss. – copper.hat Oct 27 '13 at 20:38
• math.stackexchange.com/questions/535020/… is too broad for sure and to some extent lacks context. – dfeuer Nov 3 '13 at 2:53
• It appears that the audits have now been redesigned - clicking close on an audit wasn't an automatic pass for me just now, but rather gave me a chance to click an option first. – user61527 May 22 '14 at 5:46
• While the thread is obviously useful as a way to vent frustration, seeing it bumped repeatedly gets old. I think it would be better implemented as a chat room, similar to Whining Room on Meta. Or just add the (whining) tag. – user147263 Jul 30 '14 at 17:46
• @900sit-upsaday I agree. It was originally supposed to provide examples that could show how it might be improved, but it seems clear that this will not happen. And if you think it is annoying to see it bumped, consider getting a notification each time making you think "ahh, someone is speaking to me" only to see that it is just another answer to this question. – Tobias Kildetoft Jul 30 '14 at 17:58
• @TobiasKildetoft You can ask SE to be disassociated from this thing; I'm sure that seeing the situation they'll honor the request at once. – user147263 Jul 30 '14 at 18:00
• Should we add the tag (big-list) to this question ? – GNUSupporter 8964民主女神 地下教會 Dec 12 '17 at 17:17

Although this thread seems fairly quiet now, I have to report this one:

Review Close Votes - Math.SE

Although the Question has problems, these are not currently of the unclear what you are asking variety. No doubt the Question was properly closed and downvoted on that account, but subsequently it was edited in a way that makes the problem posed clear and precise.

I bring it up, not to argue for reopening, but to point out that the criteria for Review Audits do not seem to take into account post closure edits, other than to presumably require that the item remains closed currently.

• Almost the same happened to me. The question had problems, probably leading to a close, but was market as "unclear what you are asking". Nevertheless, the question was so, so clear that somebody commented the answer. I suppose I had to vote for the formatting issues without considering anything else... – Rafa Budría Jun 3 '17 at 12:43
• @RafaBudría: If it's simply formatting, I often try to fix them and reopen, though in some cases the edit preview flummoxes me and I fail. More dubious are cases that might be closed for lack of context. Sometimes, if I think the Question is of genuine interest, I vote to reopen despite that lack. – hardmath Jun 3 '17 at 14:23

Just got this one:

https://math.stackexchange.com/review/close/532401

The question shows no effort, starts out with "community, I need to solve the following problem" and then ends with (bold face) "I try several ways, and the hint given below but I have no luck to overcome this problem, I hope someone knows how to do it. It will be really appreciated if someone knows the actual proof, I'd like to learn how did they arrived that such a series converges. Thanks, and I will give some of my reputation for the actual proof. Thanks to all the community" Give me a break.

• Did you happen to notice who last edited it? – Daniel Fischer Nov 27 '15 at 19:21
• Meh. The annoying bold stuff could be edited out. And the question does have some redeeming qualities to offset the perceived lack of effort. – user147263 Nov 27 '15 at 20:46
• See also: meta.math.stackexchange.com/a/22071 – user147263 Nov 27 '15 at 21:03
• @DanielFischer Ha, no I did not. I don't think that makes any difference. – 6005 Nov 30 '15 at 18:51
• Doesn't make a difference, I just found it amusing. – Daniel Fischer Nov 30 '15 at 18:55

https://math.stackexchange.com/review/first-posts/1068861

The question is about the boundedness of manifolds embedded in Euclidean space, with closed curves being the motivating example. The answer reads

How is your definition of closed curve? If it just a continuous image of $S^1$ then it is compact, so bounded in $\mathbb{R}^n$

This would be appropriate for a comment (specifically, it appears that the answer is asking for clarification about what is meant by a "closed curve", then addresses only that part of the question).

• However it does assume a choice of definition and answer, in admittedly terse fashion favored by some, the part of the Question raised in the title. Of course it is far from a true first post, which is irksome. – hardmath Sep 30 '18 at 7:02
• @hardmath I am not trying to argue that the answer is terrible as an answer (it is pretty borderline if you ask me, and my reaction in the review queue was that it should have been a comment, not an answer, hence I flagged it as such, but I agree that it is a close call). My complaint is that it is terrible as a review audit. Review audits are supposed to be unambiguous, and are meant to ensure that people are actually reading what they review and not just click-click-clicking. – Xander Henderson Sep 30 '18 at 11:55

I've commented on an answer to a question about division by zero from a person in the 9th grade saying that the answer may be hard to understand by the OP and I failed because I should have not commented.

I suggest that this specific answer is removed from the test list.

I just encountered this queston in the Reopen Queue. I voted to leave closed, as it needs details; as it stands it's just a problem statement. Unfortunately I failed the audit, and I was supposed to vote to reopen. I disagree strongly enough to post this here.

I failed this audit today:

Is $$x_1^2+x_2^2+\ldots+x_{2020}^2=-1$$ solvable in $$\mathbb{Q}(\sqrt[3]{2}\zeta_3)$$?

This post is a standard example of problem-statement questions with no context such as what the OP has tried and where the question comes from, yet it received 5 up votes and was put into review audit. I have cast a closure vote on it.

https://math.stackexchange.com/review/close/1408914

The question is long but, at the end of the day, it is nothing more than a statement of a result which the asker wants proved. There is no additional context. This is even pointed out in the comments:

It would be helpful if you could share some of your thoughts on the problem: Where did you encounter the problem; do we have a reason to suspect that this problem has a nice solution? What have you tried? How far did you get before you got stuck? Are there any results/techniques that you suspect would be applicable here? — Omnomnomnom

I passed this one (the alleged closure reason was "too broad", which it's not, and the score was artificially $0$), but Monoids with left common multiples actually does lack context. What is the source of the OP's intuition (is there a special case where this holds)? What did they try?

• I think 'So this is a result which I think is true but have yet to find a quick proof for.' constitutes enough context. It may not be question of the year, but it's far from a PSQ. – Dan Rust Sep 17 '13 at 10:06
• @DanielRust, I don't see how stamping words like that onto the end of a PSQ make it anything more than a PSQ. – dfeuer Sep 17 '13 at 19:35
• I've had a few now with alleged closure reasons which don't fit but which should be closed for lacking context. Most recently this one. – Peter Taylor Sep 24 '13 at 18:57
• @Peter: Fortunately, André got there first. – Brian M. Scott Dec 1 '13 at 5:00

I got this one: Maximal value of dimension. I voted to close because of "his question is missing context or other details".

https://math.stackexchange.com/review/close/929400

The system claimed it's marked with unclear what you're asking, but I voted to close this question for off topic due to the question writer's lack of response to another user's question "What did you try? ..."

https://math.stackexchange.com/review/first-posts/929848

It came up as a first post and I intended to close it, either as unclear what you're asking, or as off-topic.

The problem itself is terribly worded, as most answers discuss. The question also shows no thoughts or effort whatsoever.

Turns out it has 24 upvotes.

• The question looks fine to me. "The problem itself is terribly worded" is the reason why the question was asked in the first place, so I don't understand this complaint. "The question also shows no thoughts or effort whatsoever." Effort and thoughts are not a requirement for questions here. We sometimes ask this for problems that look like homework, but from the first sentence of the question, it is clear that this is not the case. – Trevor Gunn Dec 28 '17 at 1:02
• @TrevorGunn Ok, perhaps the wording is not the issue here. But I disagree about lack of context. I thought that every question lacking at least minimal effort should be closed as off-topic and I have been reviewing accordingly. People downvote and request OP's thoughts on every such question. Could you please link to a source for your claim: "Effort and thoughts are not a requirement for questions here." I'm aware that it is sometimes a grey area, of course. If I'm wrong, I'm happy to be convinced otherwise. – mechanodroid Dec 28 '17 at 1:10
• See the discussion here. You'll also note that a lot of discussion re: effort is about homework exercises (which this question is not). You should also notice how many well received questions there are with no effort to solve them by the asker (here's one of David Speyer's). For the question in the review, I'm at a loss for what sort of effort could even be provided to a question of "this doesn't make sense, what does it mean?" – Trevor Gunn Dec 28 '17 at 1:34
• If you're still not convinced, consider opening a new question on meta so the discussion gets more attention. I would also be interested in seeing what other people think. – Trevor Gunn Dec 28 '17 at 1:34
• @TrevorGunn Ok, thank you. I'm actually a bit relieved because I found the "show your effort" policy a bit too strict myself, but I thought that is the way things are done here. – mechanodroid Dec 28 '17 at 19:57

https://math.stackexchange.com/review/close/959283

I failed this Review Audit for the Close queue, but curiously it seems a different user is shown passing it! However I'm using it to call attention to the mechanism of changing the imputed Close Reason.

The Question had been closed as unclear what you are asking but shows in review as (potentially) too broad. While one might agree that asking something which is essentially an immediate application of the defintion (Why is the sequence of Fibonacci numbers countable?) leaves Readers wondering what the problem is really asking, it is clearly something that can be answered (and was) in brief but detailed fashion.

I just wanted to point out the strange alteration of the actual close reason. It seems to me many "false negative" reviews use primarily opinion based as the proposed close reason, which I suspect is one of the least actually used reasons, and "too broad" strikes me as a less rare but still infrequent case.

https://math.stackexchange.com/review/close/1140429

This question, while it is nicely typeset and all, asks two questions at once. Therefore it should be closed as "too broad". ("Avoid asking multiple questions at once.")

The question is asking for references. A reference is given, with enough context to know that the answerer has not read the book themselves, but found the table of contents useful. This is not a great answer, but is certainly an answer to the question, and not poor enough (in my opinion), to be considered a "low quality answer". Frankly, the question is the problem, not the answer.

Latest dubious audit question: https://math.stackexchange.com/review/close/202870

Yesterday I failed this audit in the following way : The answer was presented as a First Post. I spent some time to understand the chain of ideas and conclude that it is a mathematically correct answer. Although I estimated (and still do) that it deserves more explanations. I don't believe it would be accepted either as an answer in a school test not in a peer-review article. As it was presented as a First Post, it is logical to react and ask for improvement. Therefore I wrote a comment and got the "STOP ! Look and listen" message before I could select to keep the answer.

• While I disagree with your assessment of the answer, it is definitely problematic that merely writing a comment is considered an incorrect action. – Eric Wofsey Nov 6 '15 at 17:36
• @EricWofsey. The other comments were not available during the audit. Some of them tend to agree that the answer could have been more detailed. – Tom-Tom Nov 7 '15 at 21:13

Question link: Proving $x^2+x+1\gt0$

Screen capture:

When I got to this question, I actually opened up the post to see the score and the close votes of the post (I almost always do that when reviewing, unless the post is really bad and obviously deserve to be closed).

However, although this question is well-received, I think that this question is pretty much opinion-based, as it asks for as many ways to prove $x^2+x+1>0$ as possible.

I decided to vote to close as "primarily opinion-based" and of course I failed the audit, luckily I hadn't failed any other audits in the last 30 days.

At this point, I'm not really sure if there is a "close vote guidelines" about this type of questions that ask "as many ways as you can think of", or maybe I'm a bit too strict.

• I don't see how a request for a list of different ways of proving a given result can be interpreted as "primarily opinion-based". – Gerry Myerson Jul 20 '18 at 0:30
• I probably think so too, after seeing the highest-rated question here: "Visually stunning math concepts which are easy to explain". – user061703 Jul 20 '18 at 1:28

https://math.stackexchange.com/review/close/1201934

I voted to close the question as being "Too Broad." In general, I think that questions beginning "What kinds of mathematics can be done with [insert topic here]?" are not really good for this site (I'm curious---is there a consensus on this?). Even if the community ultimately feels that this question is appropriate, I don't think that it is a good audit question.

I failed an audit regarding an answer to the question "Is there a name for categories whose objects are sets?" because I thought that the answer was short and I commented asking to expand it a bit.

In my case, the comment was not negative and it was within the usual scope of the comments: to ask for more information.

Furthermore, I based my reasoning on the fact that the question was present for almost a month, and in these cases I think that it is even more important that the answer should be more detailed to benefit other users besides the original poster.

• I'm not sure that this is a problem with the audit item, so much as it is a problem with the audit system, which considers commenting to be bad form. Frankly, I think that commenting should always be considered a reasonable action from the point of view of auditing. :\ – Xander Henderson Sep 30 '19 at 20:13

This question appeared in the low quality review queue, and I failed the audit because I voted to close the question. I was instructed "You should have said "looks okay"."

The question is not okay. It includes no context, no considerations from the OP, nothing, just a question asking others to give them the desired example/counterexample.

Another gem. The question was whether to close it as unclear, and the answer was clearly no — the OP is explicitly asking how to construct such a completion. This was so obvious that I foolishly didn’t bother to check the original. It turns out that the line How might I go about this? was added by someone else, and the original question was genuinely unclear.

https://math.stackexchange.com/review/close/231217

This is a badly-written question, no doubt about it, and it deserves its downvotes. But I disagree that this question should be closed (except perhaps as a duplicate), and strongly disagree that the close is so clear-cut that it should be included as a review audit. After all, an identical question got 20 upvotes on Mathoverflow...

• Link to MO question? – Did Jun 24 '14 at 17:28
• – user7530 Jun 24 '14 at 19:46
• The questions are hardly identical: the MO version provides two specific suggestions, the MSE none. – Did Jun 24 '14 at 20:13
• @Did So asking, "I've heard 2 and 5 are prime numbers. What are more examples of prime numbers?" is hardly identical to "What are some examples of prime numbers"? The former shows more conscientiousness and effort, perhaps, but beyond that I'm afraid I don't agree. – user7530 Jun 24 '14 at 20:33
• I think the question on MSE is bad because it's not at all clear why the asker believes that there is a connection between pi and primes (I think there is, but need help with proof), nor even at all what they mean by "direct link." In the MO question, it was fairly clear what was being asked, IMHO. – user61527 Jun 25 '14 at 2:06

https://math.stackexchange.com/review/close/1242652

I actually think that this question is interesting, but it suffers a bit from being overly broad. I voted to keep it, but can see the other side of the argument, as well. I don't think that it is a good audit question, as there is a reasonable argument which could be made in favor of closure.

I failed the audit in the "Close" review queue when asked about this question post.

After the revisions, it already demonstrates the asker's effort. It should no longer be closed actually, but I'm not confident in the re-opening process so posting here anyway.

• No one else voted to reopen and there's at least one person disagreeing here on this meta via downvote, so it seems that I'm misunderstanding something. Perhaps people think the math is just fundamentally wrong so it should remain closed, but I was trying to say that IF the close reason was "needs details or clarity" then it was already fixed via edit. Getting the basic math wrong (misunderstanding the definition of a limit) might deserve some downvotes as well as removal from the system, but I don't see how closing as "needs details or clarity" is the proper course. – Lee David Chung Lin Feb 24 at 21:26