32
$\begingroup$

I recently came across the notion of a scale free network, see Scale Free Network. These networks tend to have power-law distributions. I was wondering if reputation in SE was distributed like this, but can't find the overall stats for reputation anywhere. I see things like "top 4% this year" on people's pages, so I'm guessing the stats exist somewhere, but are they accessible by all?


Update (1): Thanks to the commenters, it looks like we have a power law indeed. The shape seems to match the cumulative distribution function as found on Wikipedia: Cumulative Power-Law Distribution. Now the question is, what is the exponent?


Update (2): Due to a thorough answer below by apnorton, we have our best-fit exponent! It's −1.701. This is very interesting. It would imply that there is no well defined mean or variance for reputation*, because according to the Wikipedia page for power laws, the mean is only defined for exponents whose absolute value is greater than $2$ and variance is only defined if the same is greater than $3$. This makes sense when one considers the harmonic series divergence.

*Of course, there will be a mean and variance because the sample is finite, but in the infinite limit, these values will not converge if the power law indeed holds.

$\endgroup$
  • 7
    $\begingroup$ A bit dated by now, but likely still relevant blog.stackexchange.com/2011/07/power-laws $\endgroup$ – quid Aug 27 '15 at 18:44
  • $\begingroup$ Have you clicked on the "top x% this year"? $\endgroup$ – Asaf Karagila Aug 27 '15 at 21:09
  • 7
    $\begingroup$ I think that if rep on m.se doesn't follow a power-law distribution, the moderators should fiddle with it a little bit so it does. Oh, and Benford's Law, too. $\endgroup$ – Gerry Myerson Aug 27 '15 at 23:13
  • 1
    $\begingroup$ @Gerry: What about Sylvester's Law? $\endgroup$ – Asaf Karagila Aug 28 '15 at 0:03
  • 3
    $\begingroup$ Data.SE query. This plots Users vs. Reputation, instead of the other way around, but it still is kinda cool. $\endgroup$ – apnorton Aug 28 '15 at 2:17
  • 1
    $\begingroup$ @apnorton: We really want the derivative of this function, though. :-) $\endgroup$ – Asaf Karagila Aug 28 '15 at 7:46
  • $\begingroup$ @apnorton- nice. Asaf Karagila is right, this is the cumulative function. Still, it looks like we have a power law indeed. With a pretty high exponent it seems. Because from the Wikipedia page for the Pareto distribution we get a similar image for the cumulative function, which I've added to the question. $\endgroup$ – Colm Bhandal Aug 28 '15 at 12:04
  • $\begingroup$ @quid, nice article. It seems power laws are bound to crop up in many places on a big site like this. $\endgroup$ – Colm Bhandal Aug 28 '15 at 12:12
  • 1
    $\begingroup$ Yes. A thing that might disturb it a bit is the rep-cap though. It might not be strong enough here, but on SO were more users max-out on a regular basis it should be noticeable. $\endgroup$ – quid Aug 28 '15 at 12:17
  • $\begingroup$ Interesting I wasn't aware of a rep-cap. Yes I'd imagine that would "stunt" the distribution a bit. Although maybe it would still look like a power law with the entire tail "bundled" onto the point for max-rep $\endgroup$ – Colm Bhandal Aug 28 '15 at 12:19
  • 2
    $\begingroup$ @AsafKaragila I'm not 100% sure how to do that in a raw SQL Query. When I get back from class, I'll use Excel (or maybe something nicer) to find the discrete derivative. $\endgroup$ – apnorton Aug 28 '15 at 12:51
37
$\begingroup$

As mentioned in comments, I made a Data.SE query that gave the cumulative distribution of users by reputation. However, we're interested in the distribution (not cumulative). So, I wrote a quick program in Python (or, at least, it would have been quick if I knew how to use the scipy toolkit before this) to compute the "derivative" of this CDF. The interesting plots are below:

Distribution

Distribution

The most glaring oddity here is the two "starting descents." After a little thought, this is an expected result of the association bonus. I've truncated the tail of the plot (only going to reputation = 1000) because it is similar and not very illuminating.

Log-Log plot

Log-Log Distribution

This is the same as above, but with the axes using a log-log scale. This is nearly linear, which suggests a power law distribution. It's a bit "top heavy" towards the end--if I'm interpreting this correctly, that means users are "slowing down"/"clumping" in their reputation as they reach higher and higher reputation values. I'm not exactly sure, but this could be related to the 200 rep cap (if users can't gain rep freely, we'd expect a "clump" of users with approximately the same reputation that slowly moves further to the right and down).

I'm not really familiar with this type of math, so if I've made a mistake, please point it out. :) Also, I have yet to figure out how to make a best fit line (and, hence, the exponent) using the scipy toolkit... I'll see about doing so after I complete some homework. The Python 3 program I wrote to generate these plots can be found here.

Update

So, I finally got around to doing a best-fit line. This was done by performing linear regression the log-log data (e.g. I found a linear relationship between the log of R and the log of U). I ignored the datapoints below the 101 rep-value cutoff (as mentioned in the comments, including it would introduce unnecessary error). The slope of this line was $-1.701$ and the intercept was $12.707$. Since I've been using $U$ to designate the cumulative users, we have:

$$\frac{dU}{dR} \approx \exp(12.707)\cdot R^{-1.701}$$

Thus, we know our exponent.

With best fit

$\endgroup$
  • 6
    $\begingroup$ Kudos for the hard work! $\endgroup$ – Asaf Karagila Aug 28 '15 at 23:15
  • $\begingroup$ Fascinating. I wasn't expecting such a thorough answer! $\endgroup$ – Colm Bhandal Aug 29 '15 at 16:16
  • 5
    $\begingroup$ This might be an obvious comment, but I'd imagine that for the best fit line of the log/log plot it would be best to ignore the first segment (corresponding to rep 1-101) because the association bonus messes it up, as you correctly pointed out. $\endgroup$ – Colm Bhandal Aug 30 '15 at 20:52
  • $\begingroup$ Amazing. Great work. $\endgroup$ – Colm Bhandal Sep 8 '15 at 12:40
6
$\begingroup$

Estimating the PDF

By taking the derivative, you are estimating the pdf from the cdf. A more straightforward estimate of the pdf is to slightly alter the query to make a count by reputation.

Like the cdf query you have, this data needs to be smoothed (specifically, binned) before we can estimate the distribution. A telltale sign that the data is not smoothed is that the pdf estimate in your plots are not one-to-one. That is, there are multiple scatter plot points for a given value of $\log(R)$. This means we aren't really plotting the pdf and that it doesn't make sense to fit this data to infer whether we see a power law.

What we are seeing in both of the unbinned dataset (yours above and mine below) are reputation values in that appear much more rare than they are. For instance, only one user has a reputation of 729, but that doesn't mean that a 729 rep is a 1/283,128 event (there are 283,128 users) . In fact a reputation of 729 is much closer to being a 1/280 event.

Binned Data

I took a crack at binning the data and then regressing with a linear and quadratic fit (in log-base-10 scale). A linear fit implies a power law distribution, I am not sure what a quadratic fit implies other than a pdf of the form $f(x)=b e^{-a \log^2(x)}$. For comparison, I also plotted the raw reputation vs user count in blue, which is the un-binned data that is similar to the data you are plotting.

Mathematics Data

Stack Overflow Data

I repeated this for Stack Overflow user data with this query. SO has 1.5 orders of magnitude more users, so the trend is a bit more clear.

Stack Overflow Data

Code

The python code to make these plots in a Jupyter Notebook is:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import statsmodels.api as sm
%matplotlib inline

def plot_hist(df,alpha,c,label, ax=None, limits = [-1,7,-8,1], trends=False):

    ax = df.plot(x='log_reputation',
                 y=['log_user_proportion'],
                 kind='scatter',
                 alpha=alpha,
                 ax=ax,
                 color=c,edgecolor=c,
                label=label,
                figsize=(15.0, 10.0))
    if trends:
        X = df.log_reputation
        X = sm.add_constant(X)
        model = sm.OLS(df.log_user_proportion, X)
        results = model.fit()
        df['linear_trend'] = results.fittedvalues
        plt.text(df.log_reputation.min(),df.linear_trend.max(),
                 '{log_reputation:10.4f}x +{const:10.4f}'.format(**results.params)+
                '\nr**2 = {}'.format(results.rsquared))

        X = np.column_stack((df.log_reputation, df.log_reputation**2))
        X = sm.add_constant(X)
        model = sm.OLS(df.log_user_proportion, X)
        results = model.fit()
        df['quad_trend'] = results.fittedvalues

        plt.text(3,-2,
                 '{x2:10.4f}x**2 +{x1:10.4f}x +{const:10.4f}'.format(**results.params)+
                '\nr**2 = {}'.format(results.rsquared))
        ax = df.plot(x='log_reputation',
                     y=['linear_trend','quad_trend'],
                     kind='line',
                     alpha=1,
                     ax=ax)
    plt.axis(limits)
    plt.grid(True)
    return ax


def process_df(url):
    df = pd.read_csv(url,names=['Reputation','UserCount'],dtype=float,skiprows=1)
    df['user_proportion'] = df.UserCount/sum(df.UserCount)
    df['log_user_proportion'] = np.log10(df.user_proportion)
    df['log_reputation'] = np.log10(df.Reputation)
    ax = plot_hist(df,.1,c='b',label='un-binned')


    cut_points = np.power(10,np.arange(0,8,.1))
    bins = pd.cut(df.Reputation,cut_points,right=False)
    df = df.groupby(bins).user_proportion.sum().reset_index()
    df['cut_points'] = cut_points[:-1]
    df.dropna(inplace=True)
    df['log_reputation'] = np.log10(df.cut_points)
    df['log_user_proportion'] = np.log10(df.user_proportion)
    plot_hist(df,1,c='m',ax=ax,label='binned',trends=True)
    plt.legend()
    return df

df = process_df('http://data.stackexchange.com/math/csv/715124')
plt.title('Mathematics')
df2 = process_df('https://data.stackexchange.com/stackoverflow/csv/715082')
plt.title('Stack Overflow')
$\endgroup$
  • $\begingroup$ Nice! I especially like the extension to StackOverflow. :) $\endgroup$ – apnorton Nov 6 '16 at 3:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .