# Is tag overlap actually a problem? If so, precisely how costly is the problem relative to the cost of fixing it?

This is a sincere question. Maybe it's obviously a bad question for some reason that's not obvious to me, but from my understanding anyway it is sincere.

One might wish that the tags applied to a given question are somewhat orthogonal from each other, because it would be a more concise way to indicate all of the different sub-topics to which a question might be relevant.

But in practice, the usefulness of tags is mostly for searching, either browsing by topic or tracking down a particular item under a particular tag you can recall (sometimes from a while ago).

In this sense, tags act a little more like a frame than a basis.

Future people might want to be able to search along lots of tag lines more than wanting a question's tags to totally avoid overlap. I'm not saying that's definitely true; just that it might be true about what future people find useful and if so there would be a benefit to being able to search for something under e.g. the "elementary-set-theory" tag and the "set-theory" tag, even though seeing both on the same question now might look kind of dumb at first glance.

Of course there could be costs to tags as well. Tags presumably take up space in some database somewhere, and maybe some things the site must do to ensure performance don't scale especially well in the number of distinct tags (so consolidating highly overlapping tags then has a material benefit).

Of course there is also a cost to fixing and maintaining this. For example, a lot of time is sunk (in terms of aggregate person-minutes) into writing, reading, commenting, editing, etc., for meta posts, like this one, that work to codify exact policies surrounding all the little things that can happen when maintaining tag overlap.

Do we really know that the total up-front cost of a meta post like that will ever really be recouped in the future? How do we really know that the cost of allowing overlap of "elementary-set-theory" and "set-theory" was material enough to warrant work to edit it away (twice) and use comments to convince that tagger of the rationale, and codify what happened in meta?

I'm not trying to claim anything definitive one way or the other. But from where I am sitting, I think the site would be more useful to me if more tag overlap was allowed, even to the point where posts end up having 10+ tags, many of which are redundant (because who knows which one(s) I (or future folks) will want/need when searching later), and that the community spent roughly zero total brain cycles of current-day effort worrying about this problem. I would immediately revise my opinion if I learned there was a serious data infrastructure reason for why overlapping tags are bad, but I suspect this is very unlikely to be a meaningful bottleneck.

• Well, one of the issues with what you propose is that at most five tags can be added to a question. – Lord_Farin Jan 8 '15 at 21:28
• One possible problem could be the tag badges. If the tags overlap significantly, with some of them at generally different levels, then this will devalue the tag badges. For example it is not entirely clear that I should have a number-theory badge. I'm sure there are others who collected most of their NT votes from questions that really should have been under elementary NT instead. The fuzzier the scope of the tag, the less meaningful the badge is. Not a very serious problem, of course. – Jyrki Lahtonen Jan 8 '15 at 21:36
• @JyrkiLahtonen I am curious how valuable users find the badges to be. I have never found them useful and have trouble imagining reasons why they would be useful. I get that they help identify users who have contributed a lot to certain tags, I'm just not sure I ever need to know that another user has met whatever criteria there are for a badge. Usually I only care if they can answer my question, ask a good question, or make useful comments, and badges don't guarantee that (nor does absence of badge give a high probability of the opposite). – ely Jan 8 '15 at 23:17
• @Lord_Farin I didn't mean for my 10+ tag comment to be taken as a literal suggestion. It was a bit exaggerated for the sake of the point. I do wish more tags were allowed, but even with just 5 tags I'm not sure that overlap is a material issue in all but the fewest cases, which are not worth the effort to track down. – ely Jan 8 '15 at 23:18
• Considering how a gold badge allows you to have a binding vote for duplicates? Quite valuable. – Asaf Karagila Jan 8 '15 at 23:44
• The value there is for the privilege not the badge. The fact that it is the badge which confers the privilege is just a matter of arbitrary convention, and it's not clear to me that granting that privilege in a climate of many overlapping badges would be different in a meaningful way than granting it when there are few overlapping badges. – ely Jan 8 '15 at 23:47
• Arbitrary convention? Are you serious? What isn't arbitrary convention? Or even "half-based in reason" convention (as the one suggesting separation of high and low level tags for various reasons)? – Asaf Karagila Jan 9 '15 at 1:33
• Tag badges allow you to place someone. For example, I read this comment yesterday, and then stalked the poster. They have a silver abstract algebra tag, and a silver commutative algebra tag. So I can tell that they are a relatively serious mathematician without having to trawl through their posts. – user1729 Jan 9 '15 at 9:03
• I disagree with the idea that someone's badges on an SE site are a useful indicator of their skill. You definitely have to do more digging to see why and exactly what kinds of posts/answers were so up-voted. – ely Jan 9 '15 at 11:53
• Not skill - seriousness. I believe that it can be a useful indicator, although not always. For example, a gold badge in the calculus tag doesn't mean much! But when the tags get more serious, so do those following them. – user1729 Jan 9 '15 at 12:57
• That is a good point -- it does at least indicate persistence. But it's still quite easy to troll the new questions list for the same kinds of simple questions that are repeated often. By way of a programming example: you could quickly get badges for the 'python' tag on SO just by trolling for the daily recurrence of questions about simple list or dict manipulation, list comprehensions, etc. You could do that for a year, get lots of python rep, and still not know nearly enough about python to be trusted with privileges. I feel that this badge system "noise" outweighs its "signal". – ely Jan 9 '15 at 13:01
• Especially as the sites grow larger, and you start to really feel the pain of the exponential drop off of reputation. SO is much larger than Math.SE, and I wonder if SO had instituted more future-proof policies about tags and things back when it was the size of Math.SE if it wouldn't have allowed behaviors to scale better. – ely Jan 9 '15 at 13:03
• @prpl.mnky.dshwshr: I don't know what the equivalent tags in a programming SE would be, but here there are tags were getting even a bronze badge takes quite a bit of commitment. The badges on freshman/sophomore level tags (calculus, elementary-number-theory, linear-algebra), not forgetting about high school math, are not to be taken seriously. – Jyrki Lahtonen Jan 9 '15 at 13:55
• Troll the [set-theory] tag all you want. It won't be easy to reach a silver badge. Those are not "simple enough" questions (and those that are, will be retagged to [elementary-set-theory]) so without actually knowing set theory, you're unlikely to garner any badges any time soon like that. Mathematics and programming are two different things and comparing them is like comparing Apple and Orange. One is a hardware and computer company, and another is a telecommunications company. Incomparable. – Asaf Karagila Jan 9 '15 at 14:05
• @AsafKaragila I would also just like to say that I find it a bit offensive the comment about this "coming from the mind of a programmer." I consider myself a statistician foremost, and have worked hard to become good at some types of programming as employment has required. All my degrees are in math. Of the SE sites, though, I find the format most useful for programming question, and just personally not as useful for stats or math (for me... I am sure it is a great format for many others). – ely Jan 9 '15 at 17:32

## 2 Answers

There is already a lot of tag overlap. For example, almost every question tagged (homotopy-theory) could be tagged (algebraic-topology). This doesn't mean the tags are useless and this is not the problem.

The problem arises when a tag is so vast and covers so many things that it becomes useless. This usually happens when the tag is the name of a whole field of mathematics that students start learning about from very early on. Asaf gave the example of set theory, there is also "algebra": "Find the solutions to $x^2=4$" is as much algebra as "Why can we use an acyclic resolution instead of a projective resolution to compute $\operatorname{Tor}$ and $\operatorname{Ext}$?". But obviously there's a huge difference between the two.

Now what are tags for? Let's look at what you can actually do with tags: you can search questions with a specific tag, you can add favorite tags (and questions will be highlighted), you can ignore tags you have no interest in... If a tag is so huge that half of all mathematics ever could be included in it, there isn't any point in doing any of that. Then the tag becomes useless. There is also the golden duplicate hammer that Asaf already addressed.

Let's take the set-theory tag for example. Imagine every question tagged elementary-set-theory was to be also tagged set-theory. First, I can guarantee that people would misuse the system and mistag questions. Second, people interested in elementary questions would favorite/search the elementary tag only, while people not interested in elementary questions would ignore it and favorite the big tag. So essentially you're left with two tags: $\text{(elementary-set-theory)}$ and $\text{(set-theory)} \setminus \text{(elementary-set-theory)}$. You've gained nothing compared to the present situation, except that now every question about elementary set theory has two tags instead of one.

To reiterate: the problem is not tag overlap. The problem is when a tag could potentially mean vastly different things and there's a need to categorize better. In the end what you suggest would result in essentially the same thing, except with more overhead (more tags on questions) and "logical tags" that are intersections / complements of over tags.

• So it seems the problem is more about tag vagueness than tag overlap. Tag overlap is only incidental as a side effect of tags that are grossly far ranging (thus overlapping with a lot of others). I'm curious whether "elementary-set-theory" vs. "set-theory" is an example of this or not. – ely Jan 9 '15 at 11:55
• I also disagree about your search example. If I am just coming to this site, I may not even know that "elementary-set-theory" is an option, and thus might try to search by just "set-theory" (which, to me, seems like it should encompass all of set theory, even the elementary kind). Likewise, what passes for "elementary" for someone might not be elementary for another. A 3rd-year grad student might think Borel-Cantelli lemma is elementary set theory, whereas an undergrad chemist needing to fulfill a course requirement might not. Tag overlap helps both in that case. – ely Jan 9 '15 at 11:58
• @prpl.mnky.dshwshr: Luckily, when newcomers use [set-theory] incorrectly, there are people to remove the tag and possibly replace it with something more suitable. Not to mention that when you type "set" into the tags, the first option is the elementary set theory tag. So unless you do this sort of "type blindly fast and hit return", you're bound to learn about this tag. – Asaf Karagila Jan 9 '15 at 15:18
• I wouldn't say they are using it incorrectly, as much as communicating what the user needs for the tag consist of. I also wouldn't characterize the action of editing that info out, to replace it with standards decided elsewhere, as lucky or fortunate or even useful. It's not clear to me that, in the long run, experienced users performing those kinds of tag edits wouldn't actually be hurting the community without realizing it. And at the very least, there are dozens of other site moderation & maintenance issues that would have been a more productive counterfactual use of that person's time... – ely Jan 9 '15 at 15:42
• @prpl.mnky.dshwshr: If someone never saw a chef's knife before, and they pick it up and use it to hammer a nail, will you say that they are using the knife wrong? – Asaf Karagila Jan 9 '15 at 16:41
• That's a specious analogy because you are presuming the conclusion you already want. Of course "using a tool wrong" means you are using a tool wrong. But it's not "wrong" to use the elementary-set-theory tag in some certain way just because some other people think there's a different tag for what you are trying to express. Instead, it might be illuminating that the preconceived opinions about what tags are good for and the distinction held by pre-existing community members about the difference between some particular tags are ill-fitted to the needs of some users. – ely Jan 9 '15 at 16:51
• I really don't see how it is illuminating to merge the set theory tags back together. You know, if you look at it from a certain angle, everything is set theory and logic. Why not put those tags on everything? – Asaf Karagila Jan 9 '15 at 17:08
• There is a big difference between a tag that appears very often versus a tag that appears so often that its occurrence no longer meaningfully distinguishes the post. You are conflating the two and your spot analogies are sweeping under the rug the possibility that a middle ground exists. I know your last example is hyperbole, but I would be more supportive of something that erred on the side of applying tags too often and with too much overlap than a policy of expending effort to remove overlap according to the present-day community subjective preferences. – ely Jan 9 '15 at 17:18
• But I do feel it's disingenuous to make an example like applying set theory to everything because, hey, isn't all of math just set theory? That's a disingenuous straw-man. I'm advocating for using tags in a way that does two things: (1) meaningfully signals that the post belongs in some sub-collection of posts according to its taxonomic properties within the site, and (2) allows an extremely wide, flexible, and forgiving degree of searchability so that partial matches of topic, similar categories, etc., often help you get where you want to go. And to minimize community effort to make it happen – ely Jan 9 '15 at 17:21

There are two main reasons, at least how I see it:

1. It helps separate noise from signal. Questions about iterated forcings and large cardinals are set theory, as is the question "Why is $A\cap\varnothing=\varnothing$?" but there is a clear difference between two types of questions. It's true, if you're a chemist, both questions are equally calculus, but we're not chemists. And the users who take their time to answer these questions, especially the advanced ones, are usually set theorists who mind the distinction.

If we want to preserve some reasonable number of experts in set theory, we need to cater to those who don't want to answer $A\cap\varnothing=\varnothing$ type of questions, and we need to allow them simple tools to browse things they are interested in. For example, separate the elementary set theory from the set theory.

This comes in addition to the five tags limit, which may as well be arbitrary, but it exists and it is beyond our control. Questions like "Why does forcing $\kappa$ random reals, when $\kappa$ is measurable gives us an extension of the Lebesgue measure to a total measure?" should be tagged with [forcing], [set-theory], [measure-theory], [large-cardinals] and possibly [intuition]. Those are five tags right there. But it can also be argued to fit into [probability-theory] or [lebesgue-measure] or many many other tags. So why clutter when we can separate?

2. Having a gold badge gives a binding vote for closing and reopening duplicates. I do expect people who are aware of this to exercise their power, but this is not always the case. If I was feasible, I'd have asked for a non-reputation (or rather, a real world reputation) based system for closure privileges which include tags. Knowing a whole lot about analysis doesn't mean you know anything about algebra, and knowing a lot about model theory doesn't mean that you know anything about how the axiom of choice is applied to calculus.

Unfortunately, we can't have a few key experts controlling closure and reopening votes (to some degree, anyway), because it's way too much work. But the system still recognizes people with a gold tag badge as those sufficiently familiar with both the system and the topic, that they are allowed to have a binding vote. Let's try not to introduce clutter to this system, which has issues as it is (not once I've had to reprimand users that mindlessly used their binding vote and clicked "reopen" to undo a closure that was very justified).