In Mathematics Stack Exchange, has there been an increase (since the end of March) in the frequency of questions about probability and statistics (say, as a fraction of the weekly total of questions asked)?

I ask because I hardly visited MSE from April to July, and when I returned in August I seemed to notice such an increase. I'm just idly curious about it, and I'm sorry if it's too trivial a question to ask in Meta. That I can only find one appropriate tag is not a good sign! (I couldn't post without adding 'discussion'.)

  • How did you notice? If it is just an impression by looking at the page it might be that it is the "interesting" tab that shows you more such question. Also do you mean in absolute or in relative terms? – quid Aug 28 at 15:37
  • 2
    A natural tool to check something like this is Data Explorer. I have tried to create a very rudimentary query in the Data Explorer chatroom. I hope somebody with better knowledge of SQL will suggest something which works better. – Martin Sleziak Aug 28 at 15:51
  • @quid It was by viewing the "Interesting" tab of the main page, which I look at several times, each day that I visit MSE. Just now, there were 96 questions listed there. I would informally classify 9 of these as being about probability or statistics. On checking, I find that these are precisely the questions tagged with one or more of: 'normal-distribution', 'probability', 'probability-distributions', 'probability-theory', 'random-matrices', 'random-variables', 'statistics', 'stochastic-calculus', 'stochastic-processes'. My impression is that 9/96 would have been high in March, but is low now. – Calum Gilhooley Aug 28 at 16:08
  • At about 21:30 BST today, 19 of the 96 questions on the "Interesting" tab were about probability or statistics (20, if you count one on economics). – Calum Gilhooley Sep 7 at 20:41

Thanks to Martin Sleziak and rene (neither of whom should be blamed for the [mis]use I have made of all their hard work) in the Data Explorer (SEDE) chat room, a preliminary SQL query using only the tags 'probability' and 'statistics' suggests (even if only to me!) an increase in the frequency of statistical questions in MSE from October 2015, and another (smaller, and less evident) since October 2017.

Percentage of questions in given tags per month

There is admittedly no evidence of an increase from April 2018. On reviewing my personal records, I see that I was actually almost inactive on MSE from early December last year - not just from April this year - so it is likely that if there has been a real increase in the frequency of probability/statistics questions since around October last year, then that is what I was noticing.

Following Martin's suggestion, now that the list of relevant tags seems to be settling into a steady state, I will soon start work on winnowing it down into a short list of 'big' tags, and rerun the query. (Then it will probably be time to do some proper statistical testing.)


I cut the eventual list of 89 tags down to 25, and reran the query, with this result:

bayesian,correlation,covariance,ergodic-theory,estimation-theory,information-theory,machine-learning,markov-chains,maximum-likelihood,mean-square-error,normal-distribution,poisson-distribution,probability,probability-distributions,probability-theory,random,random-variables,regression,sampling,statistical-inference,statistics,stochastic-calculus,stochastic-processes,time-series,variance

I think it is fair to say that the change since around October 2017 is now more marked; but there is of course no substitute for a proper statistical test. (First I'll look into how to merge the monthly totals into quarterly ones.)


This is the list I've been building up over the last few days (no need to read it, it's just for reference):


This is a list of all the relevant tags that I happen to have noticed in questions that have been posted [in the last 21 hours] on the "Interesting" tab of the main page of MSE. Eventually I intend to produce a table like the one Martin Sleziak produced for the 'statistics' tag, but including all questions with one of the listed tags (when the list seems sufficiently complete):

  • bayes-theorem
  • bayesian
  • chi-squared
  • conditional-probability
  • correlation
  • estimation-theory
  • exponential-distribution
  • markov-chains
  • markov-process
  • maximum-likelihood
  • normal-distribution
  • parameter-estimation
  • percentile
  • probability
  • probability-distributions
  • probability-limit-theorems
  • probability-theory
  • random-matrices
  • random-variables
  • random-walk
  • sampling
  • statistical-inference
  • statistics
  • stochastic-calculus
  • stochastic-processes
  • stopping-times
  • variance

Incidentally, would it be a good idea to create tag synonyms for one or both of 'markov-process' and 'stochastic-processes', for the sake of uniformity and predictability?

[See comments for an expansion of what I meant. But it was a passing remark, not important.]


I'm continuing to build up a list of relevant tags. To the above list of 27 that I observed (randomly!) in the first 21 hours, I add these 22, all of which I've seen in use in the last 22 hours:

  • algorithmic-randomness (*)
  • binomial-distribution
  • confidence-interval
  • density-function
  • distribution-tails
  • descriptive-statistics
  • estimation
  • expected-value
  • gambling
  • independence
  • law-of-large-numbers
  • log-likelihood
  • martingales
  • mean-square-error
  • order-statistics
  • poisson-distribution
  • poisson-process
  • probabilistic-method (**)
  • sde
  • standard-deviation
  • stochastic-integrals
  • uniform-distribution

(*) Although in practice the use of this tag fits my informal criteria, in theory it is a separate topic, and it isn't used much, so I omit it.

(**) It also seems appropriate to omit this.

Clearly the list isn't settling down yet.


In the last 22 hours, I have noticed 15 more tags that should be added to the previous list of 47:

  • conditional-expectation
  • covariance
  • dice
  • hypothesis-testing
  • levy-processes
  • linear-regression
  • machine-learning
  • means
  • mixing
  • moment-generating-functions
  • regression
  • reliability
  • stationary-processes
  • stochastic-analysis
  • time-series

I rested from my labours yesterday, and probably missed some relevant tags because of this, but even so, the list seems to be settling down. I have only noticed 13 more tags in the last 79 hours:

  • balls-in-bins
  • card-games
  • characteristic-function
  • central-limit-theorem
  • coupling
  • entropy
  • ergodic-theory
  • geometric-probability
  • information-theory
  • median
  • noise
  • poker
  • random

In the last 143 hours (!), there were only 14 more tags to add to the existing list of 75:

  • bivariate-distributions
  • brownian-motion
  • concentration-of-measure
  • data-analysis
  • empirical-processes
  • fisher-information
  • hidden-markov-models
  • large-deviation-theory
  • logistic-regression
  • malliavin-calculus
  • polya-urn-model
  • regression-analysis
  • stochastic-approximation
  • uniform-integrability

I decided enough was enough. By an informal but laborious process (it could probably have been automated using SEDE, but that might have been even more laborious), I cut the list of 89 down to 25, which seemed to cover pretty much everything:

  • bayesian
  • correlation
  • covariance
  • ergodic-theory
  • estimation-theory
  • information-theory
  • machine-learning
  • markov-chains
  • maximum-likelihood
  • mean-square-error
  • normal-distribution
  • poisson-distribution
  • probability
  • probability-distributions
  • probability-theory
  • random
  • random-variables
  • regression
  • sampling
  • statistical-inference
  • statistics
  • stochastic-calculus
  • stochastic-processes
  • time-series
  • variance
  • 1
    The tags 'markov-process' and 'stochastic-processes' are certainly not synonymous. – Did Aug 29 at 15:28
  • 2
    @Did Of course not. I was remarking on the use of the singular form on one case and the plural form in the other, and suggesting creating synonyms so that either the singular or the plural form could be used in each case. – Calum Gilhooley Aug 29 at 15:44
  • Hmmm... I understand your suggestion then. – Did Aug 29 at 16:04
  • Is that a statistically significant increase? ;-) – Adrian Keister Sep 4 at 13:54
  • 1
    @AdrianKeister I don't have enough confidence to say. :) – Calum Gilhooley Sep 4 at 14:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .