Hot Network Questions. How are they selected?

Question

Now that we get to see the SE-network wide list of "hot" questions, I am just shaking my head in disbelief. At the time I am writing this, the two hot questions from Math.SE are titled (get a barf-bag, quick)

• https://math.stackexchange.com/q/599520/8348
• $1/i=i$. I must be wrong but why?

Who gets to select these questions? How? Irrespective of how this is done, this is ridiculous, as neither question has any even remotely serious content (the latter one is more or less a common fake-proof).

My proposal:

The representatives of Math.SE on this list should be based only on the votes of people who are active on Math.SE. Not just all voting members (suspecting/pointing finger at SOers, who get the right to vote from association bonus alone).

The exact rep limit (if any) is open to debate, may be 1000? Probably shouldn't put the bar too high, for that would introduce different kind of problems. But something that ensures a valued history of contributions on this site - not elsewhere on the SE network.

Answer 1

I've criticized the hot questions algorithm quite heavily on MSO in the past, it does favor certain types of questions without a good reason, and it has a strong self-reinforcing effect where questions that manage to get into the hot questions list get disproportionally more votes, which keeps them there longer.

But the algorithm is not the only thing responsible for the result, it has to work with the information it can get about the questions. It can't actually judge the quality of questions, the only measure it has available is the score of the question and its answers. Voting generally favors questions that are accessible to a wide audience, very specialized question tend to get rather low scores. The algorithm will prefer questions that appeal to a wider audience due to that, but that isn't necessarily a bad thing for the hot questions list. The votes from users that arrive at the question from the hot questions list will exaggerate this effect, but they are not the primary cause of it.

One aspect particular to MSE is that this site is one of the very few sites that still allows certain kinds of questions (the type often tagged as soft-question or big-list) as community wiki question. If this kind of question is so ridiculous or embarrassing, why are they still allowed? If ridiculous questions are not getting closed, but instead upvoted, the problem is not the selection algorithm for the hot questions list, but that those questions are upvoted and left open.

I've proposed a similar change and a few others in the following feature request on MSO: Better criteria for the hot questions list

There are several aspect that could be improved in my opinion:

Questions can stay in the list for too long, making the list a bit too static for regular SE users. I've previously posted a feature request about this, and I still think that no question should be able to stay in the hot questions list for a whole week.

The current criteria are prone to select problematic questions. They value a high number of answers, which you find for example in list questions. So questions like this example from Math.SE have it easy to get into the list, while this type of question is regarded as problematic by most of the SE network and explicitly disallowed on many sites.

Being in the list of hot questions is self-perpetuating. Questions on the list get significantly more exposure, which means more votes, which then improves the position in the list of hot questions. The votes from outside users are also problematic as they represent mostly the popular appeal of the posts, not necessarily a judgement of the quality by an expert.

I've some ideas on what could be changed:

Exclude community wiki questions from the list. Those are usually big list-style questions that aren't a good example for high-quality content across the network.

Put less value on a high number of answers. The current method values (as far as I understand it) a high number of answers and a high total score of all answers. This preferentially puts more subjective or list-type questions to the top. A good question that got one high-quality answer that is highly upvoted shouldn't be at a disadvantage compared to a popular question that gets lots of opinions. Maybe counting only the first two or three answers and their combined score would be enough.

Weight external votes differently from internal votes. External votes from users that discover the question via the hot questions list shouldn't be counted in full, they mostly represent the additional exposure and the mass-appeal of the question, not necessarily the quality. But they also shouldn't be disregarded completely, a question that doesn't get any votes from the external users is probably too specialized to be of interest to a wide audience.

Normalize votes for each site. The voting behaviour varies a lot between different sites. Currently sites that have an above average number of votes are favoured compared to other sites. Normalizing the votes on a per-site base would put a stronger emphasis on the number of votes a specific questions gets compared to other questions on the same sites. It would prevent certain sites from being overrepresented due to their general voting behaviour.

Answer 2

There is at least something that can be done to reduce the degree of embarrassment that "hot" questions bring to Math.SE: improve their titles. The titles are what ~3 million daily visitors to SE actually see; relatively few will click through to the question (although the absolute number of visits may still be high). Misspelled words, poor grammar, multiple question signs, lack of actual information in the title... these things tell the rest of network and its visitors that Math.SE does not care about the quality of its content. Current example:

How to show some function is constant ??

Yes, removing one ? and the space between text and ? looks like a small edit; but when the text gets shown to millions of people, maybe the fix isn't so minor after all. No other SE site contributes similarly embarrassing titles to Hot questions. (Arqade is often on the quirky side, but this is different.)

Proposal: whenever you see a less-than-stellar title in the sidebar (and have 2000 points in the bank), just go ahead and edit it. If applicable, make it more specific: this will reduce the number of passers-by who click the title just to find out there's nothing for them there. On the scale of Math.SE traffic, one edit bump is nothing; especially since hot questions get bumped by answers anyway.

Answer 3

These lists are automatically generated by a formula as described here.

According to this accepted MSO answer, in the past there was no intent to change the formula to account for the reputation of voters.¹ And who's to say that "active" math.SE users are not upvoting our poetry question? Five answers were given by $\geq 1000$ reputation users. Note, also, that the current formula does account for the number of answers given, and many "serious" mathematical questions get only very few answers.

¹_{It's an old answer, but maybe MadScientist's feature request will yield a new answer.}

Answer 4

I could say a lot about this, but I'll do my best to be as brief as possible. The hot questions list is something that I've been collecting data on for a few months now. Barring a few strange cases, I have a formula which seems to be approximately correct for most questions. Note that the formula is different from the one Arthur Fischer's answer references; that formula is known to be out of date. However, I think the most important thing is not that, but the qualitative description in MadScientist's answer.

---

I'm significantly more radical when it comes to this than the OP's proposal. I personally think that the hot questions list should be removed from the network entirely (though I'm not formally proposing this here). IMO it does very little good for the purpose of answering questions. Though it does sometimes communicate interesting questions, I think that's more the job for other sites e.g. social networking like Reddit. What it does is bring in a lot of non-experts to vote on questions and answers. The sorts of things that make it on the list are typically the most basic questions allowed by the site. It also creates a positive feedback loop which disproportionately favors the interests of SO and essentially lets SO take over smaller communities on individual questions. This really distorts the reputation awarded from these questions as well. I've played around with answering some hot questions both on this account and anonymously, and the results were quite startling (see e.g. this answer of mine which is my top answer in the network despite being a joke). I can not think of any case in my memory where a hot question got a much better answer than the existing ones because it was hot. I can however find numerous cases of mediocre answers getting a lot of upvotes solely because the question was hot.

However, it seems that the MSO crowd for some reason really likes this list. I can't see why, since to me it's basically like a social networking site like reddit and has almost nothing to do with generating good questions and/or answers. But I'm aware that's not a fight I'm going to win. I think a case can be made that this is not doing us much good as a community, and in many cases it's actively harming this site by distorting reputation and emphasizing mediocre/basic content. This to me seems like a problem. While it seems extreme, I think we should at least consider whether we want MSE questions to remain on the hot questions list at all. I'm not formally making this proposal yet though.

---

You might argue that this is really a non-issue. Sure, some questions get hot, and the voting on those can somewhat disrupt the reputation, but it's a pretty small effect, and the rep cap is there for the worst cases. I've actually personally found it to be rather exploitable, answering only hot questions on anonymous accounts quickly got me much more rep than I could by answering the questions I'd normally see. I do still sympathize with this viewpoint though. In this case, the main thing that we should do as a community is make sure that those questions that do get hot are at least edited to be well-written and good content. Any particularly bad questions that happen to be on the list should be edited if they are improvable, or downvoted and/or closed if not.

I will note that advanced questions are quite rare on the hot questions list. I've only seen a few questions which made it anywhere on the list that were fairly advanced. For example, How did we know to invent homological algebra? and categorical interpretation of quantification reached around #25 on the list for a few hours before falling back off. These just don't do very well in the long run, because the SO crowd doesn't appreciate them as much and so the positive feedback effect is much smaller. That's the highest I've seen any questions reach which was at a higher level than basic calculus/linear algebra/number theory.

---

The suggestion of the OP to ignore votes cast by users with very low participation here might fix some of the problem of being dominated by external voters, but it's likely programatically unfeasible. The hot questions formula has to be calculated many times per question, which rules out anything but the simplest possible formulas. Querying exactly which users has upvoted the question and checking each of their user profiles would be a much more computationally expensive operation than just looking at the vote total. I've seen much less expensive suggestions be shut down as not feasible on MSO, so I'm almost 100% certain this is out of the question. One might be able to come up with subtle ways to try to do something similar without requiring so many expensive database queries, but at the end of the day I just don't see a lot of marginal benefit of doing this just so that we can have MSE appear on the hot questions list but only a little bit of the time.

I am sure some reading this will think "why not just do the simplest thing and penalize Math questions some?". IMO This will just cause more problems along the lines of those mentioned in the OP. The problem of math questions on the hot questions list is (at least) 2-fold. Questions from Math make it onto the hot questions list too frequently, to the point that even bad ones can make it on if they quickly get answered and voted on. That could be fixed by penalizing math questions so that only the really good ones make it. However, the second problem is that the questions which do make it to the top and stay there are more controlled by the masses and less by the experts here at Math SE. Applying a penalty would just make that worse, since the few questions that would make it to the top of the list would be the ones that can bring in lots of votes from outside Math SE's regular voter base.

---

For reference, in case anyone wants some statistics, I've compiled a list of all the questions which I know of from the past few months which topped the hot questions list. There's no data on how long they topped it, and I did not put ones which made the list but not the top of the list. The full hot questions list has 100 questions at any given time, and most of the bottom ones are only hot for a few minutes, so that data would not be very useful and would not be systematic. By contrast, only looking at the top questions, I can be fairly sure that I've not missed very many important ones over the span of this time period since most of them remain hot for quite a while and I've collected data on this roughly every few hours.

I don't intend to keep updating this list, so keep in mind that it only goes from the dates July 28-December 21, 2013. I'm primarily including this list so that anyone who wants to can look at the statistical distribution of these "worst offender" questions. Also, apologies that this list is so long, but I could not find a better method for listing it.

• Can the golden ratio accurately be expressed in terms of e and $\pi$
• $2+2 = 5$? error in proof
• Is $83^{27} +1 $ a prime number?
• Prove that if $AB$ is invertible then $B$ is invertible.
• Why does factoring eliminate a hole in the limit?
• What if $\pi$ was an algebraic number? (significance of algebraic numbers)
• Prove that $\gcd(M, N)\times \mbox{lcm}(M, N) = M \times N$.
• math fallacy problem: $-1= (-1)^3 = (-1)^{6/2} = \sqrt{(-1)^6}= 1$?
• How to prove $n!>(\frac{n}{e})^{n}$
• Finding $\sum\limits_{n=2}^\infty \ln\left(1-\frac{1}{n^2}\right)$
• Is there another simpler method to solve this elementary school math problem?
• Slice of pizza with no crust
• Why do you add +1 in counting test questions?
• Why is compactness so important?
• Why does my calculator show $2^{-329} = 0?$
• Is there a simple, constructive, 1-1 mapping between the reals and the irrationals?
• Surprising identities / equations
• Why is there no "remainder" in multiplication
• Riddle: 1 question to know if the number is 1, 2 or 3
• What is the correct spelling of Paul Erdős's name?
• Fastest way to check if $x^y > y^x$?
• Solve $4^{9x-4} = 3^{9x-4}$
• Is math built on assumptions?
• 31,331,3331, 33331,333331,3333331,33333331 are prime
• What is the simplest proof of the pythagorean theorem you know?
• How can I prove that one of $n$, $n+2$, and $n+4$ must be divisible by three, for any $n\in\mathbb{N}$
• What is the antiderivative of $e^{-x^2}$
• Why does $(3\sqrt3)^2 = 27$?
• Are there any open mathematical puzzles?
• Is there any simple method to calculate $\sqrt x$ without using logarithm
• How To Present Algebraic Topology To Non-Mathematicians?
• Can the inverse of a function be the same as the original function?
• Rational + irrational = always irrational?
• Is the number $333{,}333{,}333{,}333{,}333{,}333{,}333{,}333{,}334$ a perfect square?
• Why don't I get $e$ when I solve $\lim_{n\to \infty}(1 + \frac{1}{n})^n$?
• If $f(x) = \cos x$, explain, without taking the derivative, how you would find the $f^{(99)}(x)$?
• Is 10 closer to infinity than 1?
• How to put 9 pigs into 4 pens so that there are an odd number of pigs in each pen?
• What are some examples of notation that really improved mathematics?

• Is $A + A^{-1} always invertible?

• which is larger number? $\sqrt{7}-\sqrt{6}$ or $\sqrt{6}-\sqrt{5}$

• Is $\tan\theta\cos\theta=\sin\theta$ an identity?
• Why can't you square both sides of an equation?
• Prove if $56x = 65y$ then $x + y$ is divisible by $11$
• Does L'Hôpital's work the other way?
• Is the number 100k+11 ever a square?
• The Intuition behind l'Hopitals Rule
• Integration of $\sqrt{1+u^2}\,du$
• Prove that of any 52 integers, two can always be found such that the difference of their squares is divisible by 100.
• Are most matrices invertible?
• What is the best way to think about partial derivatives?
• Monty hall problem extended.
• Infiniteness of non-twin primes.