This data has been a bit longer time coming than first expected... but the wait is over and I have some interesting results for you... but not a lot of answers.
Based on the question and comments here I wanted to be careful about what information I looked at for a few reasons...
- The summer already has lower visits and there's an increase in September, so looking at this year alone might make it look like there was more activity after the introduction of the theme due to school starting in late August/September.
- The number of questions/day may not necessarily be a good indicator.
To address the first point, we'll compare the most recent year to the prior one for the same period and the second by looking at additional stats, like the number of visits from users >1k rep.
Through querying the internal SEDE, we see the following info:
For the period from
- 4,502 questions / week
- 5,336 answers / week
- 4,281 visits from users with at least 1,000 rep (as of
After the change we saw
For the period from
- 4,142 questions / week
- 4,607 answers / week
- 4,670 visits from users with at least 1,000 rep (as of
So, there is a decrease in the number of questions and answers dropped in a proportional number - the answers to question ratio before was 1.19 and after the change it was 1.11 but the raw numbers there are a loss of 360 questions and 729 answers... which is about double (more on this after the break).
Despite the slight slump in answers - which could be due to other factors like if more questions are being closed/deleted before they're answered - the number of visits from higher rep users increased, which is great.
And, if you like something a bit more visual, here's a chart of the posts created for the weeks before and after the change. You can see that it's on the expected start-of-school year incline but there's no obvious point where the post volume changed dramatically.
So, all-in-all, my first instinct is that the change of layout has had little to no impact on your site health.
I wanted to go looking for an explanation for the difference in the questions vs the answers so I decided to increase the spread of my search a bit... and I started digging and saw that there is actually a big drop in questions asked on a monthly basis and it looks like it starts in August.
Those numbers in the last hump look remarkably lower than the prior one and that startled me. I wanted to get a comparison, so I pulled the numbers into a spreadsheet and saw that there were between 1800 and 5800 fewer posts over the course of the months. This looks horrible!
I went searching for an explanation there were a lot of little things that changed on the network around the same time - the ability of low-rep users to flag comments on their posts (August 10th), the new contributor icon (August 17th), Google changed their algorithm a bit but when I looked around the network at our other big sites... there wasn't any similar drop off in participation and yet, all of those sites had also had the same changes.
So, I went back to the numbers... and I realized that I'd missed the difference between July 2018 and 2017 - it's pretty big at around 3400 posts short... and that extends all the way back to about January 2018, long before any of the changes we made. Here's a year of numbers for your reference:
2017 2018 Diff %
Apr 46255 44226 -2029 -4.39%
May 45244 42959 -2285 -5.05%
Jun 37462 33393 -4069 -10.86%
Jul 33824 30348 -3476 -10.28%
Aug 34413 32583 -1830 -5.32%
Sep 39430 36112 -3318 -8.41%
Oct 46771 42422 -4349 -9.30%
Nov 46767 41013 -5754 -12.30%
Dec 41827 36284 -5543 -13.25%
Jan 43364 37499 -5865 -13.53%
Feb 40867 35162 -5705 -13.96%
Mar 44432 39164 -5268 -11.86%
The numbers are certainly more consistently down than they were before August but the June and July numbers indicate that there was already something causing a downward trend in posts here on Math that doesn't seem to be reflected to this degree on the other sites I looked at (Super User, Ask Ubuntu, Server Fault, Unix & Linux, Cross Validated, and Physics).
Wrap up / TL;DR:
So, I think it's safe to say that the design didn't cause the change but there's definitely fewer posts being created - fewer people both asking questions and answering them over the last year but the returning users over 1000 rep are more likely to be around.