# Under what criteria should we remove questions from the HNQ list?

Since the Hot Network Questions list was revamped 6 months ago, moderators have had the power to remove questions from the list. As Catija, the community manager for the SE network says,

I recommend each site have a meta discussion with guidance for moderators about when - if ever - a question should be removed.

I have noticed our moderation team have been exercising this power (specifically, it always seems to be Asaf Karagila). Most of the choices I agree with, but some I find questionable. So, I think we should outline some guidelines for a question to be removed from the HNQ list.

Specifically, I would like to discuss the following points:

1. Should an HNQ be removed if the question is not interesting, but it has lead to interesting answers?

2. Should an HNQ be removed if it is an interesting question, but has not generated interesting answers?

3. Is a lack of context grounds for removing otherwise interesting questions/answers from the HNQ?

4. Should we adopt a policy of removing all homework questions from the HNQ?

5. Given that controversy (which was the cause for the HNQ revamp) is an extremely rare issue for mathematics questions, should we be removing questions from the HNQ at all?

6. (Added, from discussion below my answer) Should there be any guidelines at all for removing questions from the HNQ?

7. (Another added question, after discussing with quid) Should we, as a community, care about HNQs at all?

• I'd love to write something, but I am hosting a workshop that starts this week, so I will only join in by the end of the week, or even after the weekend. – Asaf Karagila Sep 9 '19 at 7:45
• @AsafKaragila Ah, what terrible timing. :-) – Theo Bendit Sep 9 '19 at 7:46
• Let me just add a quick thing. When this feature was into the site, there was some discussion amongst the moderators. We had some agreement that it is better to err on the side of removal, since the natural state of a question is not on HNQ anyway. And in general HNQ questions should be good representation of content we want to encourage. So context, presentation, etc. I personally also remove lazy titles as well, since they should be improved and almost always include $\rm\LaTeX$ that would disqualify them anyway. – Asaf Karagila Sep 9 '19 at 7:49
• In favour of 4. on condition of lack of sufficient context (relates to 3.) Removing all HW questions may be too extreme as quite a few of them are well-written with attempts by asker etc. And maybe 4. should be generalised to include contest problems. – TheSimpliFire Sep 9 '19 at 9:35
• In favour of 3. In a way, lack of context are grounds as a question is knocked off the HNQ once closed (presumably off-topic in this case). "Interesting" is another matter, and has caused considerable controversy (example). IMO interesting LQ HW/contest problems should be removed; the asker doesn't deserve the upvotes/attention their post gets for poorly asking something not of their own creation (relates to 4.) The controversy lies in interesting apparently self-motivated questions without evidence of actual motivation from the asker. – TheSimpliFire Sep 9 '19 at 9:44
• I am not sure whether the claim that "controversy is an extremely rare issue" for questions on this site. There was definitely a fair share of close/reopen, detele/undelete wars, including some questions that were in the HNQ list. (It all probably depends on what kind of controversy you have in mind.) – Martin Sleziak Sep 9 '19 at 11:03
• I suppose that for the purposes of this discussion it might be useful to check which questions have been removed from the HNQ. Here is a SEDE query which lists such removals: data.stackexchange.com/math/query/1013246/… – Martin Sleziak Sep 9 '19 at 11:19
• @MartinSleziak Just to (hopefully) clarify: due to the fact that Theo mentions that "controversy (...) was the cause for the HNQ revamp", I think he doesn't mean controversy in a broad sense of the word, but controversial posts in the sense of involving a political/psychological/social disrespectful/improper/sensitive aspect. This is indeed rare in MathSE. (Although it has happened before.) – Aloizio Macedo Sep 9 '19 at 16:33
• Theo, just to point out, I did find the time today... – Asaf Karagila Sep 9 '19 at 18:34
• @AsafKaragila Sorry, I was sleeping! I have replied in the form of an answer. – Theo Bendit Sep 9 '19 at 23:23
• To confirm, I am indeed talking about the kind of controversy that Aloizio Macedo mentions. More broadly, I'm talking about controversy that affects people outside this community; things that would draw members of the general public to click on a question more out of outrage (or the expectation thereof) than interest in the mathematical content. – Theo Bendit Sep 9 '19 at 23:27
• I have posted one SEDE query related to HNQs in the above comment. You can find a few more queries related to this topic in this chatroom. Feel free to use the room also for any related discussion which seems more suitable for chat than for comments. – Martin Sleziak Sep 10 '19 at 1:26
• – Jack Sep 12 '19 at 13:33
• According to the query in @Martin 's prior comment, about 97.4% of the 500 questions removed by mods in the last 5 months were executed by Asaf. I don't think it is good that a single user execises so much control over what posts we expose to the global SE community. Many posts that would do a good job of educating global readers are wrongly being blocked – Gone Oct 31 '19 at 17:30

Okay. I have a few minutes to spare. So I can write an answer to explain my approach to this, which is mainly on the table, seeing how I account for the majority of uses of this feature across the entire network (to the best of my knowledge).

I don't like clickbait. Or at least in most cases, I have a strong dislike to clickbait (I will admit to having written clickbaity titles before). Why is this relevant?

Well, seeing how any $$\rm\LaTeX$$ in the title of a question immediately excludes it from the HNQ, it means that in order to get a question on HNQ to begin with, it needs to have a title that has no $$\rm\LaTeX$$ in it.

Why is that an issue? Well, because it's a mathematics Q&A, and a lot of the questions should have titles that involve mathematical symbols. This means that a lot of the time, questions that end up on the HNQ list have one of three types of titles:

1. Lazy titles (which may or may not stem from laziness), for example "Simple integral question" or "Help with this proof".

These titles are terrible. They tell you nothing about the question, and force you to click on the question just to get an idea of what the subject might even be. In 99.9999% of cases, such a question can be given a much better, clearer, and informative title. And in 99.9999% of those cases, such title would involve $$\rm\LaTeX$$.

Why do I remove these from the HNQ? Well, because sometimes I just don't have the time to edit the title. Sometimes I do both. Sometimes I would still remove it, even if someone just edited the title. Why? Because we had users unroll titles in the past, just so they can have an answer on a question appearing on the HNQ list. And that, to be honest, sickens me.

2. Genuine titles, for example "Are there any new advances in computer-assisted algorithms for experimentally verifying Riemann Hypothesis?".

This title is great. It's informative, it tells you what's up, where it is up, and why it is up. Great.

In some cases, however, I would still remove such a question from the HNQ list. Why? Because maybe it's badly phrased. Maybe its body is just "My question is in the title (more characters)". Maybe something in the question just felt... lazy. Or maybe I acted too rashly.

3. Genuine clickbait titles. I don't have to provide examples of these, but this is somehow in the middle between the previous two categories, and it is often removed for the same reason as lazy titles, even if it's not entirely a lazy title.

Now. What about content? Well. When this feature was introduced, there was a fairly unanimous agreement between the moderators that questions on the HNQ should reflect content as we want it, rather as picked by an algorithm (which is designed, to the best of my knowledge, around how many answers were given, how quickly, and how many votes those answers received).

That means that lacking context, or lacking some crucial information, or just asking to verify a calculation. Those are all things we don't want to show the SE network as "good content".

It is unfortunate, yes. Most of the good content tend to have good titles, and those tend to have mathematical text in them, for a good reason. And that means that a lot of good content is missed out by the HNQ out of technical necessity.

But that is no excuse to fill it in with mediocre-at-best questions.

1. Maybe, depends on the question and depends on the answers.

3. Usually yes, but this is more of a case-by-case basis.

4. Not because they are "homework", no. If they are lacking context, or somehow very uninteresting ("please verify my proof that sine integrates to cosine"), they should be removed. If it is a well-written homework question, and it has nice answers, there's no reason to exclude it just because it started from a homework assignment or a textbook.

5. It is bad enough that some people regard this site as a homework mill, and as a place with terrible questions. I don't see why we shouldn't remove questions that act to support this belief. If it was up to me to promote questions that I think are good, it would be my pleasure to suggest a handful per day. But it's not, and I can't.

Let me finish by saying that the natural state of a question is that it is not on the HNQ list. Most questions will never get there. So it is an extra privilege to be on the HNQ list. But it is not a privilege you get by having actual good content, rather it is a privilege you get due to hitting certain points for the algorithm. So it is not something you "work for", and if it is, then you're using the site wrong, to be honest.

This, to me, at least, is a justification to remove any and all questions from the HNQ list, which I would gladly do. But I understand that people would be happy to see mathematical questions across the network. I just try to make sure those are not homework-mill, low-effort, low-interest kind of questions.

• "Well, because it's a mathematics Q&A, and a lot of the questions should have titles that involve mathematical symbols." — as a data point, looking at the top questions on mathoverflow this week, there are more TeX-free titles than titles using TeX. High points: "The Planck constant for Mathematicians", "Is the union of a chain of elementary embeddings elementary?", "Do Sobolev spaces contain nowhere differentiable functions?", "How rare are unholey permutations?" are all above 10 upvotes. I see no +10 questions on the week's top list that use TeX in their titles. – Steven Stadnicki Sep 12 '19 at 16:24
• Broadening that to the month's top list barely changes this result; of the most highly upvoted questions on mathoverflow, the research mathematics site, the overwhelming leaning is towards titles with no TeX in them whatsoever. I feel like this may be an unconscious bias coming to play. – Steven Stadnicki Sep 12 '19 at 16:25
• I know what you mean, and I think that the culture of questions on MO is very different than here, for a good reason too. I also think that some of those highly upvoted questions are there because they had no LaTeX in the title and made it to the HNQ list. So you have shown correlation, yes, but it's more complicated than just that. – Asaf Karagila Sep 12 '19 at 18:12
• @StevenStadnicki As another (more quantitative) data point, about 22% of the latest 100 Mathoverflow questions have MathJax in their title. – Gone Apr 13 at 0:20

Thank you for replying Asaf. This is my response to your post. There's plenty of stuff here that I agree with to some extent. I do have some concerns though.

Firstly, I would have liked it if the community was privy to the discussion between the moderators about when to remove questions from the HNQ. This could easily have been a meta post a long time ago. We are discussing it now, but I get the impression that we are discussing a policy decision that was made in the past, rather than being a part of formulating policy.

Secondly, I'm broadly worried that we are being too prescriptive in determining what's "hot" and what isn't. I see the HNQ as a way of sowing greater interest in mathematics amongst the SE network, and various members of the wider public. What our more learned members might see as an utterly mundane "low-interest" question, might be still of interest to people for whom the subject never quite clicked (example). For that reason, I can't commit to this idea that we should be constantly fighting the algorithm, which I do think is largely competent at sniffing out general appeal.

I'm almost always in favour of mathematical outreach (though my introverted self finds it personally difficult). Mathematical anxiety and mental blocks are, in my opinion, one of the most serious issues facing our discipline today. The HNQ is a good way of showing people, who don't consider themselves interested in maths, tidbits of maths that they are more likely to find interesting. It's an ongoing debate against all of the common negative perceptions of mathematics (e.g. it's irrelevant, boring, just algorithms, too hard for most people to get, etc).

Basically, I think I would prefer (though this is not a firm opinion yet) for there to be less interference in the HNQ. If a title is bad, for whatever reason, then edit it, and see if the question stays in the HNQ. If that means adding in Latex, then so be it; make sure the question fits into MSE first, and let the HNQ algorithm do its thing.

Primarily, I want to encourage clear questions with well-written answers, and leave questions of interest up to the algorithm. If a question lacks context, then close it for that reason, but I don't think it's necessary to hold HNQs to a higher standard, in the hope that all the context will absorb in the minds of people asking questions to get help with their homework. If for no other reason, I think it'd simply be ineffective. As an example, the question that prompted me to launch this discussion. It provides adequate (but not exemplary) context, in my opinion.

I also don't generally feel comfortable with reasoning based on perceptions about the asker. This includes your disgust at people who title their questions to be clickbait, as well as SimpliFire's comments about people "deserving" of HNQ status. (Maths people being judgemental is another negative perception of mathematics that I prefer to combat, even if there is a grain of truth in it.) Again, I think, edit the title. If they start an edit war, then remove them from the HNQ and maybe talk to them about SE etiquette.

As I said, it's not really a firm opinion yet, but my impression of the policy used so far is that it's too prescriptive and smacks of judgement. I don't really see people "working for the HNQ" as a really big problem, and while context-free questions are a problem, if we keep our usual regimen of closing them (and writing more informative titles), the HNQ should contribute to this problem very little.

• @quid Are you talking about my "Firstly..." paragraph? I don't think it was a big deal, but I do think a discussion with the community would have been the better move. Put another way, for however long the HNQ list was implemented before 6 months ago, having no ability to remove HNQs wasn't a big deal for MSE, so we could have not removed questions until the community reached a decision. I don't have major complaints about the questions removed, and if the policy continues as is, I wouldn't have major objections, but I do think the community should have been part of the discussion. – Theo Bendit Sep 9 '19 at 23:33
• @quid Point taken. On the topic of your point, do you recall the outrage over the autocratic decision-making made by SE staff on the topic of HNQs? It probably would have helped if the staff had consulted the community before implementing changes... – Theo Bendit Sep 9 '19 at 23:44
• Why should I have to justify that? Are we thinking of implementing a five hour time limit for all maths questions in the HNQ? If not, then this line of inquiry seems disingenuous. I thought the HNQ algorithm would replace it once another question become more "hot". Since specific action was taken with this question by the mods, to remove it before another question supplanted it, I don't think it's unfair to want to talk about their reasons for doing so. And while guidelines will always have room for interpretation, that doesn't mean people in power get to make all decisions for themselves. – Theo Bendit Sep 9 '19 at 23:51
• @quid Right, but that's not what I'm arguing. I've never expressed opposition to a blanket policy, one that mods are free to interpret as they see fit in a case-by-case basis. I just wanted the community to help shape this policy, not just for the mods to decide for themselves. If the community gets literally no say in this, then why even have a discussion in the first place? Why not every mod for themselves? We're a community, and your reluctance to even acknowledge the value of discussion with the community frankly baffles me. – Theo Bendit Sep 9 '19 at 23:55
• @quid So, I have, in the past, encountered a mathematician who doesn't like latin squares, and calls it "toy mathematics". Hypothetically, if a fellow mod decided to start removing good, interesting, well-written questions about latin squares simply because they didn't believe anyone should find such things interesting, would you stand by their practices? – Theo Bendit Sep 10 '19 at 0:02
• @quid Right, good to know. I already explained my interest in the HNQ in my answer above. I don't care much at all about any individual questions in the HNQ; I used the question I answered as an example, but I'm fine with it being off the HNQ. The fact is, I have seen a handful of similar examples that I don't agree with (none that I have such a convenient link to share), and I wanted to discuss policy, not the specific implementation. I really don't appreciate you trying to shoehorn an ad hominem in here. I have plenty of reputation, and I will delete my answer on request. – Theo Bendit Sep 10 '19 at 0:14
• @quid I have never made an attack on your character, and I would appreciate if you, a moderator no less, would extend me the same courtesy when engaging in discussion. – Theo Bendit Sep 10 '19 at 0:16
• @quid Sure, why not? Plenty of people who peel away from mathematics learn about groups, and struggle with proving them. It was a fun little exercise, and I think worthy of interested people seeing and trying for themselves. But again, I feel like you are trying to get me to justify one small example, which is not (and never has been) the point. – Theo Bendit Sep 10 '19 at 0:22
• @quid Let me refer you back to my previous comment. The flag was to bring to the mods' attention, for them to act as they see fit. Up until then, I had not noticed a question removed from the HNQ; I wasn't even sure if that was something we were doing. There's nothing wrong with mods acting at their discretion (obviously, this needs to happen a lot), but formulating guiding policy with the community is almost always a positive thing. – Theo Bendit Sep 10 '19 at 0:37
• No @quid, that's not why I started the debate. I deliberately did not share my opinions initially, to encourage other viewpoints to surface first. I replied after Asaf's comment, which I did so as not to be ungrateful for his time (which is in short supply at the moment). I tried not to be leading on any of my questions, except 5, and only because I think most people would answer yes without thinking about the wider context. – Theo Bendit Sep 10 '19 at 0:45
• @quid So, my bias is that I wanted to participate in the moderation of this community? At Stack Exchange, we believe moderation starts with the community itself. The point of electing people to positions of power is not so that they can then ignore everyone and just do whatever they want! – Theo Bendit Sep 10 '19 at 1:17
• @quid Yes, actually, I concede that this would have been a good opening question. I actually thought that there weren't guidelines, and I was reasonably OK with how the HNQ was treated before. It was only when Asaf mentioned that there was a discussion amongst mods about when to use this power that I became a little disappointed. I couldn't see why such discussions couldn't have happened on meta instead, and I still don't. If the mods just acted of their own accord, independently, I would still want to discuss this, but I wouldn't have felt disappointed. – Theo Bendit Sep 10 '19 at 1:27
• I did not yet catch up on everything that was said but one point (maybe already made): the hotness formula is for all I know quite sensitive to multiple answers and answers arriving quickly. Thus it favors a the type of questions that tend to get multiple answers quickly, which is also a source of bias and arguably towards the elementary part of the spectrum. I'll collect my thoughts and try to formulate an answer later but wanted to interject this. – quid Sep 10 '19 at 16:05
• @JyrkiLahtonen (Rant incoming.) That is precisely the attitude that I'm opposed to, not just here, but in the wider mathematical community. Even without explicitly calling them dimwits, I don't think we gain anything good by trying to convince people that this site (or maths in general) isn't for them. It's incredibly elitist, needlessly personal, and just harmful to mathematics in general. Answerers are providing a service. It's OK for that service to not include providing ready-made solutions to homework. But so many people here don't just enforce policy, they actively profile the askers... – Theo Bendit Sep 11 '19 at 6:01
• @JyrkiLahtonen They search for clues in the question to support their prejudices, and then deny them service based on personal judgements about their character. This is exactly the behaviour that makes maths anxiety worse, where a bunch of really intelligent people interact with you for a tiny amount of time, then dismiss you and any future you would have with mathematics! We are a big site for mathematics help (I'm guessing the biggest?). We help with maths at all levels. If we can't act with courtesy and respect to the wider community, then god help us all! – Theo Bendit Sep 11 '19 at 6:01