Here are a few of my own thoughts on this issue:
I think the reason is that Math SE is fundamentally a question-and answer-site (tour). While learning and problem-solving are promoted here, questions created solely to encourage problem-solving are not on-topic here.
In these types of sites, there would be a lot of discussion, and lots of different ideas would emerge as people gradually discover the solution. In other words, there would be a lot of chit-chat as expected from a forum-like site. As a consequence, this would disrupt the flow of 'ask questions, get answers' with no distractions, which would not be a good fit on an SE site.
In the comments, it has been mentioned that there have been previous proposals to create an exclusive site for problem-solving on Area 51. However, a rigorous process is needed to ensure that the site will be useful as its own site, as well be able to sustain a community around the site. A future 'challenge-problems' site is too close to Math SE to be its own site, and seeing that questions focusing on alternative answers to a question already are on-topic on this site, there really should be no need for a separate site.
So where should challenge-type questions be asked? From my personal opinion, I think Brilliant would be a much better site to ask. The difference with Brilliant is that it is fundamentally a site about improving problem-solving.
As such, there are many features such as the daily questions, and the community tab where there are hundreds of user-submitted questions created every day. There is also a discussion page where users can post an interesting questions and users can discuss freely. The site would be more suited for a 'challenge-type' question because it is not so focused on asking and answering.
All in all, Math SE is primarily a question-and-answer site, and although there will always be a need for problem-solving, I believe this site just isn't the place to ask these types of questions.