19
$\begingroup$

I see that many new users, who have no idea about MathJax, ask poorly formatted questions and somebody has to come along and edit them. Would you be interested to automate this? Will Math SE incorporate this bot provided it works well?

$\endgroup$
  • 9
    $\begingroup$ Can we get unedited, poorly formatted questions in bulk, to learn from it? $\endgroup$ – Priyatham Sep 25 '16 at 23:04
  • 4
    $\begingroup$ This is perhaps one of the coolest things I have heard in a long time. Priyatham is right; we will need a huge training set if this is to work well. $\endgroup$ – Valborg Sep 26 '16 at 0:11
  • 42
    $\begingroup$ It would have to be a pretty clever bot to do a good enough job to avoid creating more problems than it solves. For example should we translate "cost" to $\cos(t)$ or leave it as text? Should "5\$" be as written or "\$5\$" (i.e. $5$)? What about fractions where the brackets are unclear? Is $1/2x = \frac{1}{2}x$ or $\frac{1}{2x}$? These are so many cases like this where it's very hard to determine algoritmically what to do. Not to speak about the cases where we humans have a hard time interpreting what OP has written. Looks like a very hard problem, but good luck if you are going to make it. $\endgroup$ – Winther Sep 26 '16 at 2:21
  • 5
    $\begingroup$ I'd be interesting seeing that it actually works. You can always set the bot to suggest edits under your username, or preferably a separate account (with the permission of the SE admins, of course). This way every such proposed edit can be vetted by two human editors (or the OP). In that case you can also use the rejected edits as a way to train the bot better. $\endgroup$ – Asaf Karagila Sep 26 '16 at 2:52
  • 9
    $\begingroup$ I'm against this. this is equivalent to creating an AI to recognize math formulas and I don't believe the research of AI has advanced that much. If the original question is bad, it already have a lot of noise. we don't want another source of noise to further damage the question. $\endgroup$ – achille hui Sep 26 '16 at 3:09
  • 1
    $\begingroup$ @achille hui: First step towards Skynet, you say? $\endgroup$ – Asaf Karagila Sep 26 '16 at 3:34
  • 2
    $\begingroup$ You all raise valid concerns, I think it is definitely worth a shot. As Asaf said, we should vet it until it gets really good. Achille are you aware of LSTMs? Don't you think they are powerful enough for this problem? $\endgroup$ – Priyatham Sep 26 '16 at 5:24
  • 21
    $\begingroup$ I'm interested in flying self-driving cars too, but it seems pointless to hold a poll about whether people are interested in something before being confident that you can actually do it. Do you at least have a prototype, a proof-of-concept, something? $\endgroup$ – Najib Idrissi Sep 26 '16 at 8:04
  • 2
    $\begingroup$ The last time some process directed to helping users with improving their posts was automated, it caused mixed reactions. Some users praised the automated system, some users were rather critical. $\endgroup$ – Martin Sleziak Sep 26 '16 at 8:24
  • 2
    $\begingroup$ About the training set. If you want to use neural networks for this (you mentioned LSTM in a comment), wouldn't you need not only the poorly formatted posts but also their version with MathJax added? $\endgroup$ – Martin Sleziak Sep 26 '16 at 8:49
  • 2
    $\begingroup$ @NajibIdrissi, I asked if you are interested because, unless you are interested (the mods, mainly) I have no hope of collecting a dataset. No I don't have a prototype yet. MartinSleziak yes, I would need both. Where and how can I get it? $\endgroup$ – Priyatham Sep 26 '16 at 12:04
  • 2
    $\begingroup$ "I asked if you are interested because, unless you are interested (the mods, mainly) I have no hope of collecting a dataset" The relevant data is public. If anything Stack Exchange staff could help; but per site mods cannot do much anything specifically. $\endgroup$ – quid Sep 26 '16 at 13:12
  • 2
    $\begingroup$ @Hurkyl the point is really what does "works well" mean. Who will be opposed to it if it truly worked well? Very few I'd assume. I think the question as written is not phrased well. I am also against attempts to auto-insert some dollars here and there even if this worked reasonably well. Even manual insertions of that form sometimes do more harm than good, and if harm is done it's more of an issue than many autoinserted dollar do good. Because really if it already says x^2 + y^2 = 2^n we could leave it as is almost just as well. $\endgroup$ – quid Sep 26 '16 at 13:53
  • 2
    $\begingroup$ I have doubts as to the usefulness of such a bot. Would it help with some of the other problems from which poor questions seem to suffer, such as vague titles, generally poor formatting (walls-of-text and the like), and the substitution of images of equations for actually typeset maths? If we need to edit such things manually anyway, then we can edit the math as well. However, I would be very interested to see a program which, at the click of a button, can take what I'm about to post (either as a question or as an answer) and add in/check all the TeX formatting automatically. (continued...) $\endgroup$ – Will R Sep 27 '16 at 5:30
  • 7
    $\begingroup$ If you can make a program that puts in the TeX formatting in the answer box, then that can be used by questioners, thereby eliminating the need for such a "patrol" bot. So, in all, I don't see why a bot going around automatically editing questions would be necessary; why not just a button which does the same thing on command? $\endgroup$ – Will R Sep 27 '16 at 5:38
32
$\begingroup$

I think this would be a very difficult task. In the unambiguous cases, adding math formatting doesn't add much value to the post. Added value begins where the math formatting clarifies math which is hard to read in ASCII representation, and that usually requires some decisions:

  • Whether 1/2a is $\frac12a$ or $\frac1{2a}$
  • Whether a lone a or I is a variable name or part of a sentence
  • Whether Ax is $A$ times $x$ or $A_x$
  • Whether to represent a * as $*$, $^*$, $\cdot$, $\times$ or invisible times in a given situation
  • Whether < and > mean $<$ and $>$ or rather $\langle$ and $\rangle$
  • Whether (x,y) was meant as $\begin{pmatrix}x\\y\end{pmatrix}$
  • Whether ncr was meant as $_nC_r$ or $n*c*r$
  • Whether x^y+z was meant as $x^y+z$ or $x^{y+z}$

(Feel free to edit this post to add other ambiguous notation to this list.)

You could give up in all these situations, and decide to play it safe by not editing the post at all. But in that case, the situations where adding markup would be most useful would be missed. And I think you'd end up skipping the vast majority of posts.

Or you could have your bot make the above decisions without really understanding the mathematical intention (which is often hard to understand even for human editors, so I just assume your bot has no chance of actually understanding that). In which case, chances are that you'd make the wrong decisions more often than not. In which case the poster may get downvoted because your bot made the question incomprehensible, and future editors will have to look not only at the latest text but also at its history in order to fix the problems right.

To phrase this differently: yes, if your bot were working really well, i.e. so well that it avoids all the problems I outlined above, then I'd be very interested in this, and I'd like to see it in action. In this case I'd also encourage you to publish a couple of papers on how you did all the things I consider difficult. If on the other hand the purpose of your question is figuring out whether this project is worth your time, then I'd say no, because I honestly don't believe the problems to be reasonably solvable in the near future.

Are you aware of LSTMs? Don't you think they are powerful enough for this problem?

I doubt it. As outlined above, correcting math formatting requires an understanding of the question which goes beyond mere syntax but well into the mathematical semantics. So you'd have to actually understand the question at a semantic level.

I think that if your code were able to understand math this well, then applying that technology to a sufficient corpus of math literature would almost enable the bot to not only correct the formatting but also provide answers with references from literature. Sounds extremely unlikely to me.

Can we get unedited, poorly formatted questions in bulk, to learn from it?

SEDE has all the posts. Or if you want the data offline, grab the data dump from either the torrent or the HTTP download. With the right queries, it should be possible to even obtain edits which look like they were mostly adding TeX markup, so these would provide you with a before-after kind of information. Something like this query of mine. This should be more valuable than mere unedited posts, since it also tells you what to aim for, and will give you a glimpse of what kind of edits our users do perform.

$\endgroup$
  • 1
    $\begingroup$ On your first example, a concern on mine is that 1/2 a, which I'd understand as one-half times a would be rendered by a bot as $1/2 a$ (simply inserting dollars) which I would take as $(2a)^{-1}$. This is also an issue with manual MathJax-ing when not done carefully. $\endgroup$ – quid Sep 27 '16 at 8:55
  • 1
    $\begingroup$ Thank you very much for this answer (especially links to the data). I am downloading it rn. The examples that you give of the nuances of math ar valid, I am gonna experiment and definitely come back on what I found. $\endgroup$ – Priyatham Sep 27 '16 at 13:21
  • 1
    $\begingroup$ In case of 1/2a the bot should add a comment saying something like "Could you disambiguate 1/2a into (1/2)a or 1/(2a)?" and do the work only if no such problems exist (it might be a small fraction, but still could help offload some part of the work). $\endgroup$ – dtldarek Sep 30 '16 at 18:48
  • 1
    $\begingroup$ @dtldarek I hope that "it might be a small fraction" was an intentional pun. :-) $\endgroup$ – LSpice Sep 30 '16 at 18:49
  • 1
    $\begingroup$ How about $\sqrt{2a}$: we have the variants $\sqrt2 a$, $\sqrt(2a)$, $sqrt 2a$, $sqrta2$, $\sqrta2$, so the bot would have to distinguish the correct spacing (quite a difficult task I imagine without understanding the full context of the question being asked); mathematical operators, can it tell the difference between the (possible) typo \sqrta and convert to \sqrt as well as then correctly reformatting the a variable, and so on. $\endgroup$ – Daniel Buck Sep 30 '16 at 21:19
  • $\begingroup$ Most things can be corrected by simply asking the users to use parentheses to clarify expression or using unique and unambiguous functional expressions for not elementar notation like binomial() for the binomial coefficienct. And context marks like <math></math> could simply added by an initiated user. This would suffice for most situations and require less work than tex'ing everything manualy. On the other hand the question is whether this side is suited for people not able to tex. $\endgroup$ – HopefullyHelpful Oct 1 '16 at 22:26
  • 1
    $\begingroup$ @HopefullyHelpful: Personally I consider <math>a</math> to be far more work than $a$. I guess the whole point this question here is trying to address is when people don't know or care about math formatting. I'd assume that any novice user who spent 5 minutes on the TeX formatting faq should be able to mark up their question at least as well as an automatic bot would do it. So asking people to mark up math in something not TeX will probably gain exactly nothing: those who don't already use TeX won't care about other requests either. $\endgroup$ – MvG Oct 2 '16 at 23:24
  • 2
    $\begingroup$ @DanielBuck I'm sure you've also seen $√2a$, in which it doesn't occur to the OP that maybe there is some kind of way to type a square root symbol using characters available from the usual QWERTY keyboard, nor do they wonder why the overbar doesn't extend anywhere. $\endgroup$ – David R. Oct 4 '16 at 21:24
  • $\begingroup$ And when C, Q, R, Z would be set in bold or blackboard bold. Which brings me to the point that the bot can't read the OP's mind on stylistic preferences (though to be fair, neither can I; in such situations I tend to impose my preference for plain bold). $\endgroup$ – David R. Oct 4 '16 at 21:30
  • $\begingroup$ @David R Indeed, and while this may, or may not, be obvious from the context of the question, i.e., if $√2a$ came at the end of a mathematical sentence we might as humans immediately read this as $\sqrt{2a}$ and edit accordingly, I can't see how a bot would react without understanding the whole question (a terrifying thought in itself). $\endgroup$ – Daniel Buck Oct 5 '16 at 9:07
  • $\begingroup$ @Priyatham Something like s1.daumcdn.net/editor/fp/service_nc/pencil/… would be easier to make and would promote to format the maths. I think that people don't format because they don't know the code for different expression and also it is too cumbersome. Something like this would solve that to a great extent. $\endgroup$ – A---B Oct 6 '16 at 19:55
  • 2
    $\begingroup$ Even worse variation of your Ax example: Ai(x) could be $A\cdot i(x)$, $A_i(x)$, or the Airy function, $\operatorname{Ai}(x)$. $\endgroup$ – celtschk Oct 8 '16 at 10:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .