The problem with proof verification

Question

What I am going to bring up is not a new problem. It was hashed again and again and again. It was brought up as far as 2013, and repeatedly almost every year. See 1, 2, 3, 4, 5, 6, 7, and 8.

Some of these questions discuss whether or not it makes sense, or what to do when the supposed proof tries to prove a famous open problem, or how to even answer such question, or how to make them more attractive.

These are all valid concerns. But most of these were asked quite some time ago. And the growth of the site makes it harder to fix any future damages, so I think we need to talk about this again. In this question, I am not offering any solutions. I just want to highlight the problems and bring this back to the table, and engage with a discussion on what are possible solutions in the answers below.

---

I had recently asked a user who had been posting a lot of fairly elaborated proofs (sometimes in response to suggestions on previous questions). With a little discussion in the comments, this is what I managed to extract as far as pros to proof verification questions:

1. Not everyone in this site is a student, not everyone has a teacher, assistant, or a grader to check their work and make remarks. Many people here are hobbyists or self-educating themselves, and they need help from others to make sure that they are not making mistakes in their work.

2. Supposedly, there is a contribution of an elaborate proof to the corpus of proofs on the site. This helps future visitors, since they can find this proof and learn from it, and learn from its mistakes if those are pointed out in the answers.

3. In one instance, the proof seemed fine, and even one person said it's fine, but then someone else pointed out a subtle mistake. Which requires closer attention to details. This is something you might not be able to do yourself if you're asking these sort of verification related questions, and this is how we truly learn mathematics.

On the contra side:

1. Future visitors will also want their proofs to be verified. They will simply post their proof, and this will not be a duplicate, because the question is not about the problem, but rather about the solution, which is likely to be somewhat different than the previous instances. This leads to an inflation of questions which are asking essentially the same, but are not "formally duplicates".

2. It might be that someone looks at the proof says "Yeah, that looks good", but they glanced over the details. In that case, especially if this user is more established, any mistake might be overlooked by future readers. (The issue here is that finding a mistake is a $\Sigma_1$-type problem: until you found it, you don't know if it's there.)

3. If you are looking to contribute an elaborate proof, then you're looking to contribute an answer to this website. Not a question. It is far more reasonable to post the suggested proof as an answer to a question about the statement you're proving (or post a question and an answer if there is no such question). Of course the problem is that not everyone bothers to read the answers, you cannot award people a bounty for proof reading your answers, and so there is no "points incentive" for people to engage with your answer. This is why sometimes a user would post an answer to their question after receiving a partial or broad-strokes answer, and a few days later move this to a proof verification question, where they can incentivize more people to look at it.

---

What do you think we should be doing with proof verification?

The issues here are that the site is growing fast in users, and this means that it grows even faster with more questions. If we decide that we enable a proof verification culture, this might cause an even more serious problem in the future in the form of repeated questions with the same proofs.

For example "Posts containing 'algebraic numbers is:q'" show $9$ questions, out of the first $10$, asking to verify the proof that the algebraic numbers are countable.

This will continue to blow up, rapidly. Whereas the problem with "homework mill questions" (for lack of a better term) can be solved by dedicated people who point out duplicates and close those questions where the OP is truly disinterested in providing any contribution, proof verification are hardly like that. The OP usually shows their work (often to a degree one might call excessive), and seem to be willing to contribute to the site their version of the solution.

Answer 1

I do not at all agree with rschwieb's statement that "the way we get feedback for solutions is to post them as a solution, and then people have at them". People who have a very weak foundation in mathematics should not be dumping their so-called 'proof attempt' of any mathematical problem onto an existing thread asking about that problem!

Proof-verification questions exist to serve the needs of students who need guidance in writing proofs, perhaps because they were not taught basic first-order logic, and thus cannot quite put together a logical argument from their ideas.

I think that proof-verification questions can be closed as duplicate if roughly the same approach had been asked about before. In my opinion, it is the responsibility of the asker to do sufficient research to determine whether his/her approach has been tried before, and only post a question if he/she cannot find it (at least on Math SE). As amWhy commented, proof-verification questions of very common problems have been asked so often that it is almost surely a duplicate of a prior question that shows likely every way to proceed.

I also think that a good proof-verification question will avoid the problem of insisting "But I want my proof evaluated..." as follows. After a bit of research one should be able to (or learn to be able to) identify in what key ways one's proof attempt is different compared to existing proofs, and thus ask for verification about those specific new parts. If one cannot identify any part of the proof that is different, then there is no new question to be answered.

This criterion for proof-verification questions would largely solve the key issues in Asaf's post:

• A proof-verification question will not just be a dump of a proof and a request to check it. Rather, it would specifically identify something that distinguishes it from other known proofs (unless it is a new problem, in which case the issue of duplication does not arise). That makes it not only a contribution to the site but also a self-classifying one; future readers can quickly classify the myriad of proof-verification questions about the same theorem by their distinguishing features.

• Mistakes are also more easily identified; if the asker takes effort to highlight how his/her proof differs from the known ones, experts can much more quickly tell whether the difference is inessential or not.

• Future visitors will also have an easier time searching for their proof attempt, if past proof-verification questions each highlight the part the asker is most concerned about. After all, it is likely that short wrong proofs by sincere students are wrong in similar ways, and most proofs on Math SE are short.

• Inflation of questions that are essentially the same will also be curbed, not only because highlighting specific points of uncertainty in a proof makes it more likely to be a duplicate, but also that it is easier for experts to find duplicates.

Answer 2

I thought my take on the situation existed somewhere, but I think it must only be in comments, or possibly in deleted comments. But thanks for the chance to get it down in a solution!

Answers should almost always be posted as answers.

If the question exists elsewhere on the site, and you have an answer for it, it should go on that question. if the question doesn't exist, you should write it up and self-answer it.

This is the best fit for our format.

The primary goal is to get feedback about your solution. And the way we get feedback for solutions is to post them as a solution, and then people have at them.

I imagine there are a couple reasons some users might shy away from this suggestion.

1. "I didn't know you could self-answer/that it was allowed." Maybe they worry that it'll look like they don't actually have a question, or just plain don't know that there's no block against it.

2. "I don't know if it's right, so I can't post it as a solution." (aka "afraid of getting it wrong") . Maybe putting the answer with the question affords some comfort for this. Of course, we shouldn't post things we know are wrong, but there's nothing wrong with putting forth a solution we think is best. That is, essentially, what all solutions are, with varying levels of success. If this is not already the default, readers should already know that every answer is potentially wrong, and that it requires evaluation on the viewer's part. There is no guarantee a solution is correct on this site.

3. "My question won't get any attention if I put the answer as a solution." This ought not to be true, and I sure hope it isn't. I'm pretty sure that new answers to old questions get attention bumps. (And we might consider having a mild attention bump for new self-answered questions. Mild enough that it hopefully can't be abused. But let's not discuss that particular idea in this thread.)

Does that mean all types of "check" questions should be like that?

No, not necessarily. There are still species of "check" type questions that are actually questions. For example

1. "I came up with this proof <...> but I also know that <...> and that's a contradiction. Where am I going wrong? . ("I have reasonable cause to believe that there's a problem and I need help finding it.") (Example where that's well done)

2. "in this proof of mine <...> I think I should be able to say that , but I'm not sure if I'm justified in doing that step. Am I? ("I have trouble validating this step.")

A common thread for these exceptions is that there is a concrete direction for the question, rather than an open-ended "Are all the steps right?"

Asaf's "pro points"

To address the points about the contras:

I think 1+2 are both handled equally well by solutions-as-solutions. I don't see why solutions-in-questions handles the situation any better. If people are worried their proposed solution will go unnoticed, it should be a fine thing to point out in the question that a proposed solution follows.

3 seems like a clear case of one of the exceptions I listed above: i.e., there is actually a question ('It doesn't work for this reason... how can I fix this gap?')

---

I hope I'm making myself clear that I am not taking a very hard line. I am not trying to obliterate anyone's options. I'm pretty sure that given any such question I can reason why it should be in one of the buckets above, without alienating anyone.

And by going this extra mile and organizing things so, I think we get a leaner body of questions with a fuller body of answers which is easier to learn from. Moreover, the behavior I'm suggesting has a few healthy aspects in it that are good to encourage, namely:

1. Thinking harder about how to formulate your question (rather than just "here's some stuff I wrote, amirite?")
2. Not being afraid to put your work out there as a solution
3. Being aware of already existing questions and answers on the site

I also feel like I am forgetting a couple of other points I had previously made too, but hopefully they come back to me in ensuing discussion.

---

Update:

Until now, the existence of a user who asks too many proof verification questions was hypothetical, but recently I stumbled across a real user in the wild. I won't name the user directly, but I'd like to share some data from SEDE queries that may (unavoidably) be enough to identify them.

The user has 700+ questions. I didn't wade through them all, but I looked back until 2017 since the posts seemed to have a regular repetitive structure ("Here is Prob. blah of book blah is this all correct?") There are ~275 since 2017, and at least 170 begin that way. Before then there appears to have been a 2-3 year inactivity period: I didn't do any analysis on those. Some of the earliest posts are what I'd call good quality, so they're not all bad.

This is admittedly an outlier, and I suppose a couple hundred posts is a drop in the bucket, but knowing it is actually happening is a little harrowing.

---

Another user, distinct from the one above, has come to my attention. There are 194 questions asked between May 2021 and October 2022, seemingly (as discerned from a sample) someone posting their solutions to standard books by Munkres, by Rudin, and Hoffman. As of the writing of this post, 90 were unanswered. Probably some of the single-answer ones are just "yeah that's right" too.