# What, exactly, is (estimation-theory)?

Disclaimer: I have no idea what estimation theory is about. But looking at I cannot help but think that either the tag means something extremely broad (so broad that I hesitate to think that there is a coherent underlying theory) or that some users are using the tag for things other than "estimation theory".

Perhaps related is the fact that we separately have the tags

Question: What are they and how are they related?

(For example: I have some vague idea about what approximation theory is. Some of the questions currently tagged under that tag have absolutely no relation to the description on Wikipedia. I suspect we either have to dump a bunch into or create some new tags. Absent a good tag wiki, I think quite a few users just tag both (approximation) and (approximation-theory) when they don't know any better.)

• I estimate that it is something approximately related to a theory. – Asaf Karagila Jul 20 '12 at 15:45
• Do mods have possibility to mass retag all questions from one tag to another, without causing bumping? (As opposed to making synonym.) – Martin Sleziak Jul 22 '12 at 6:33
• I've created some tag-wiki for approximation-theory based on Wikipedia article. I hope someone with more knowledge about this area will improve the tag wiki. – Martin Sleziak Jul 22 '12 at 10:05
• @Martin: yes, mods can effectively rename tags by (ab)using the tag-merge feature (which also allows us to merge one tag into another without creating a synonym). So your proposal below is technically feasible. – Willie Wong Jul 22 '12 at 14:22
• Tag-wiki for estimation-theory was created by Ilmari Karonen. – Martin Sleziak Jul 25 '12 at 8:29

Estimation Theory is a branch of statistics.
Approximation Theory is a branch of real analysis.
I guess (estimation) and (approximation) are used in less precise senses. Perhaps for estimates and approximations other than those specific ones in the first two.

• are you guessing that, or is that your... estimation? :-) – Asaf Karagila Jul 21 '12 at 13:56
• @AsafKaragila - you are so witty :) – Belgi Jul 21 '12 at 21:48
• This is the correct answer. – zyx Jul 24 '12 at 3:51

Estimation theory is the study of inverse problems arising from signal processing and is different from approximation theory. Broadly, given some random noise $\eta$ and observed signal $y$, one wishes to "solve" the equation $$y=A(x)+\eta$$ for $x$ ("estimate" $x$). The main complication is that for many problems of practical interest, $A$ may not be 1-1, may not be onto, or if it is 1-1 and onto it may not have a continuous inverse. Dealing with these issues for $A$'s related to signal processing is what estimation theory is concerned with.

Mathematically, the two main methods of approaching the problem are

1. Deterministic: find $x$ that minimizes $||y-A(x)||$ in some suitable norm, where the norm may be related to the noise statistics, and
2. Probabilistic: Start with prior probability measures for $x$, $\eta$, and $y$, and then find an a posteriori probability probability measure for $x$ given $y$, such that all of the probability measures are related through some suitable infinite dimensional generalization of Bayes' theorem.
• I went and added a tag wiki for estimation-theory, since it didn't have one yet. Since I don't really know anything about the topic, I just quoted the Wikipedia article (with proper attribution, of course), but feel free to improve or rewrite it. – Ilmari Karonen Jul 22 '12 at 18:29
• I think the wikipedia summary is really fair. I tend to think of it more in terms solving an inverse problem, whereas statisticians think of it more in terms of finding unbiased estimators for a conditional random variable. The way it's written now leaves open both interpretations. – Nick Alger Jul 23 '12 at 1:10
• @NickAlger I would also add that under 'deterministic', there are also cases where you know 'truth', and so you want to minimize $||y - \hat{y}||$ where $y$ is truth, and $\hat{y}$ is your estimate. This is usually done for estimation of the matrix $\bf A$ via a training sequence. – Spacey Jul 23 '12 at 16:57
• Yeah definitely, though in that case I would consider the entries of $A$ to be parameters in a larger problem $y=B(A,x)+\eta$, $B(A,x):=Ax$. – Nick Alger Jul 24 '12 at 2:09
• The signal-processing meaning of "estimation theory" is a special case of the meaning in statistics. – zyx Jul 24 '12 at 3:51
• The inverse problem perspective is more robust than you think. Given a statistical estimation problem, you can construct an inverse problem via the radon-nikodym derivative of the posterior with respect to the prior. To each statistical estimator, there corresponds an optimization problem. The signal processing usage, though, is more specialized. – Nick Alger Jul 24 '12 at 11:23
• There is a lot of overlap between signal processing and statistics, but it is incorrect to say that estimation theory "is" the signal-processing case (where, I think, the term was simply borrowed from older and more general use in statistics). If the word "signal" is taken to mean arbitrary data then the two fields cover the same ground but I don't think this is what people ordinarily mean by the words. – zyx Jul 24 '12 at 21:11
• Ok, fair enough. I would agree with that. – Nick Alger Jul 25 '12 at 1:07

Estimation theory is a part of statistics.It comes under statistical Inference. A sample from the distribution of a population is useful in making inferences about the population . The process of going from the known sample to the unknown population has been called statistical inference. Two important problems of Statistical inferences are (1)Estimation and (2)Testing of Hypothesis.

ESTIMATION: Some feature of the population in which an investigator is interested may be unknown to him and he may want to make a guess about this feature on the basis of a random sample drawn from the population. This type of problem is called the problem of estimation. That's all I know about Estimation.

Perhaps this could help in deciding what to do with tag. We will see whether there are many questions which need retagging. (BTW sorry for making 3 posts in the same question, I hope this thread is not too messy now.)

I've put here list of suggested retags of questions in this tag. The list is ordered from the oldest questions. I've included only those questions where I trusted my judgement enough to make the call. The question is CW, feel free to edit it.

The list of the questions which are, at the moment, tagged can also be found here.

In case you make some of the suggested retags, please mark this by striking the corresponding line out. (It's good to keep the question in the list, so that we know how many of the original list of 30 questions have been dealt with.)

Questions that should remain tagged

Questions that should be retagged

Questions that should be retagged

Other retags