I want to place a bounty on this question, but I'm not sure if MSE is the best place for it. In general, where is the best place to ask that type of question. It seems that neural networks are generally more on-topic elsewhere, but from the literature involved and the nature of the question, this particular question seems more purely mathematical than anything else.
Theoretical Comp-sci is for research level questions, try Computational Science, Computer Science. Perhaps Cross Validated or DSP too.
I've seen a chapter about neural networks in a book I have. And when I watched some lectures on machine learning, it had a little to do with statistics (and it appears that statistics students have optional lectures on machine learning), hence the suggestions. Try to ask it in their chat or meta before.