33
$\begingroup$

I suspect that this MSE question (Screenshot) and an MO question by the same account (Screenshot) have been artificially generated. But I'm not sure.

For a machine, those would be nice works of association, but that is not unheard of, as there are known paper generators such as SCIgen. The occasional typos work against a clear diagnosis.

Now I wouldn't mind if an AI posed a clear and interesting question. But those are not clear (to me), and if those questions are indeed machine-generated, this means that someone only needs to turn a knob to increase MSE's junk inflow far beyond manageability.

We may need increased machine assistance. Are there tools in place or in development that could automatically send posts like those to a review queue?

Update: Another post (#2810926, now deleted) (Screenshot) that might have been machine-generated has been identified in the comments. This time it is an answer.

Until a bot attack or a publication in the spirit of SCIgen occurs, we might remain in the dark about whether those posts have been machine-generated or not.

Currently, what matters are the characteristics that help those posts pass existing automated detectors and superficial human reviews.

Characteristics that should be machine-testable:

  • No block-displayed math, very few inline math snippets
  • Frequent references to, or quotes with superficially relevant keywords from Wikipedia / ArXiv / other sites known to host math content
  • No other hyperlinks
  • Generous use of inline markup (bold or italic portions)
  • Many blocks (paragraphs/quotations/list items), but all short

This all serves to distract from the post's thin content while creating an appearance of thorough composition. I'd like to see some scoring mechanism work against such pretension.

Further characteristics, perhaps not machine-testable, but annoying:

  • In questions: Typos and poor grammar except when reciting
  • In answers: Telegram style only
  • No complete thought in the non-recited text portions
  • Following the references yields no relevance.

I find this use of fake references particularly frustrating because it multiplies the waste of every dedicated reviewer's time and effort.

Can we get the existing detectors sharpened, or new detectors introduced, to take into account at least the machine-testable features listed above?

Update: I suppose that there is some scoring tool in place already, perhaps a linear classifier. It would apply a number of metrics, dot-multiply the resulting vector with a vector of weights, and obtain a score. Interestingly, adding more metrics often helps to better separate good examples from bad ones. Therefore, this is mostly a matter of adding new metrics and refining existing ones. I'd propose to:

  • Count links to Wikipedia and ArXiv separately from other hyperlinks
  • Measure density of block-displayed math
  • Measure markup density outside math, blockquotes, and code sections

and append those to the other existing metrics. Then recompute the optimal separating hyperplane (i.e. weights and threshold) from training data. Make sure to include gibberish posts and cross-validate. I'd be interested in the results.

$\endgroup$
  • 37
    $\begingroup$ You want a machine to fight machine generated questions? Maybe in the end they'll become friends and collude against us! $\endgroup$ – quid Jun 5 '18 at 22:59
  • 4
    $\begingroup$ @quid Whatever machine algorithm was used to generate the linked posts seems to do a better job at generating meaningless only if read questions than does the algorithm SE uses for generating fake suggested edit review audits! $\endgroup$ – Namaste Jun 5 '18 at 23:01
  • 3
    $\begingroup$ More seriously, I am not really convinced it's machine generated to begin with. There are some automatic checks in place to detect problematic content. In part as external extensions, like "smoke detector." $\endgroup$ – quid Jun 5 '18 at 23:01
  • $\begingroup$ @amWhy I am not sure it's machine generated. $\endgroup$ – quid Jun 5 '18 at 23:02
  • 2
    $\begingroup$ @ccorn My impression is that certain portions have a little more conceptual coherence than I would expect a computer to generate - I feel it may just be somebody who got the trollish urge to post gibberish questions. $\endgroup$ – Sir Jective Jun 6 '18 at 3:20
  • 2
    $\begingroup$ @MonstrousMoonshiner we have another James Joyce type: math.stackexchange.com/questions/2809643/… $\endgroup$ – Will Jagy Jun 6 '18 at 3:33
  • 1
    $\begingroup$ @WillJagy: But there is a recognizable question behind. $\endgroup$ – ccorn Jun 6 '18 at 3:35
  • 2
    $\begingroup$ CC, well, yes, did Bourbaki invent fractions. It is the comment below that raises it to literature; two comments, in fact. Priceless. $\endgroup$ – Will Jagy Jun 6 '18 at 3:37
  • 3
    $\begingroup$ Well, let's hope I'm wrong. Or better, let's fear I'm not, and get better tools. $\endgroup$ – ccorn Jun 6 '18 at 3:39
  • 1
    $\begingroup$ You perhaps should link to this meta thread on the MO thread. $\endgroup$ – Asaf Karagila Jun 6 '18 at 7:10
  • 1
    $\begingroup$ @ccorn i.stack.imgur.com/dovKo.png $\endgroup$ – Surb Jun 6 '18 at 11:50
  • 11
    $\begingroup$ No need to invoke a machine to explain this. A few drinks and/or not a native speaker of English seems to me sufficient explanation. $\endgroup$ – GEdgar Jun 6 '18 at 12:49
  • 3
    $\begingroup$ Why does no one in this entire thread give any reasons why they think the linked posts are machine-generated? Are you guys machines yourselves? $\endgroup$ – Vincent Jun 9 '18 at 20:50
  • 2
    $\begingroup$ @Vincent: Because nothing could count as proof anyway: Humans can act like machines as long as it is not too much work. Let us focus on the interesting possibilities, that is, those with potential for significant damage. Those are machine-generated low-quality posts. Accordingly, speculating that I were a machine is useless because I can create quality posts. $\endgroup$ – ccorn Jun 11 '18 at 1:20
  • 1
    $\begingroup$ @numbermaniac: Thanks, I have updated the question accordingly. $\endgroup$ – ccorn Jun 14 '18 at 10:14
14
$\begingroup$

Not to put too fine a point on it, but your question seems to be mostly about over-zealously imagining complicated technical solutions to a problem that doesn't exist and has simpler solutions even if it did.

First, all of those seem quite plausibly written by a human when you combine a non-native English speaker with someone who doesn't understand what they're talking about. But let's say they aren't. So? They were hardly massive disruptions to the site. Community moderation readily handled them.

But maybe, as you suggest, someone will flood the site with such questions. Why would they want to do that? Sure, some people do have some inscrutable desire to vandalize or produce meaningless, non-commercial spam, but they don't tend to spend months building and refining natural language generators. If they did, the user or IP would simply be banned. This will happen automatically if the user/IP produces a large number of low-quality posts. If they went even further and used a botnet to get around that, CAPTCHAs would simply be enabled. At any rate, if I wanted to flood the site with hard-to-detect-but-ultimately-low-value questions, I'd just randomly copy-paste existing questions, maybe with some minor changes or perhaps I'd mix questions together. This would not require any natural language processing sophistication.

Summary: There is no evidence that this is happening currently or that anyone would want to do this, nor is there any evidence that the existing moderation would be unable to deal with this if it did happen.

$\endgroup$
  • 3
    $\begingroup$ I think that this is an acceptable answer. But let me note that skepticism of the form Why would they want to do that? can always be answered with "because they can." Like for SCIgen. $\endgroup$ – ccorn Jun 13 '18 at 5:39
  • $\begingroup$ I completely understand why you might want to make a natural language generator and see if it can fool people. I don't understand why you would you would then want to attempt to disrupt StackExchange with it. The creators of SCIgen didn't follow its creation with flooding Elsevier journals with generated papers to attempt to destroy their viability. $\endgroup$ – Derek Elkins Jun 13 '18 at 5:50
  • 2
    $\begingroup$ True for SCIgen. (I added that as an afterthought, but that admittedly does not match the disruption scenario.). But then again: I don't understand why you would... is reassuring, but not a convincing argument by itself. Fortunately, it is not needed; the other parts of your post demonstrate quite well that even if there were a motive for disruption, there would be more likely channels for that, and high-bandwidth posting would have its own characteristics. $\endgroup$ – ccorn Jun 13 '18 at 6:06
  • 2
    $\begingroup$ I don't agree with the sentiment that you and others appear to imply here - "Why speculate about what if ...". Most of the progress made in mathematics and other sciences is the result of what was initially idle speculation - the usefulness only became apparent later, in many cases. Curiosity doesn't need any other reason. $\endgroup$ – j4nd3r53n Jun 15 '18 at 10:22
3
$\begingroup$

the MSE question seems more free verse; it would improve with the Latex verse environment. The two lines that begin with lower case letters are supposed to be indented a good deal, two to two-and-a-half words.

$$\begin{array}{l}\text{Like a skein of loose silk blown against a wall}\cr \text{She walks by the railing of a path in Kensington Gardens,}\cr \text{And she is dying piece-meal}\cr \text{ of a sort of emotional anemia.}\cr \text{}\cr \text{And round about there is a rabble}\cr \text{Of the filthy, sturdy, unkillable infants of the very poor.}\cr \text{They shall inherit the earth.}\cr \text{In her is the end of breeding.}\cr \text{Her boredom is exquisite and excessive.}\cr \text{}\cr \text{She would like some one to speak to her,}\cr \text{And is almost afraid that I}\cr \text{ will commit that indiscretion.}\end{array}$$

The question still there on MO is much closer to the end of Ulysses by James Joyce, https://en.wikipedia.org/wiki/Molly_Bloom#Soliloquy

mouth and it was leapyear like now yes 16 years ago my God after that long kiss I near lost my breath yes he said I was a flower of the mountain yes so we are flowers all a womans body yes that was one true thing he said in his life and the sun shines for you today yes that was why I liked him because I saw he understood or felt what a woman is and I knew I could always get round him and I gave him all the pleasure I could leading him on till he asked me to say yes and I wouldnt answer first only looked out over the sea and the sky I was thinking of so many things he didnt know of Mulvey and Mr Stanhope and Hester and father and old captain Groves and the sailors playing all birds fly and I say stoop and washing up dishes they called it on the pier and the sentry in front of the governors house with the thing round his white helmet poor devil half roasted and the Spanish girls laughing in their shawls and their tall combs and the auctions in the morning the Greeks and the jews and the Arabs and the devil knows who else from all the ends of Europe and Duke street and the fowl market all clucking outside Larby Sharons and the poor donkeys slipping half asleep and the vague fellows in the cloaks asleep in the shade on the steps and the big wheels of the carts of the bulls and the old castle thousands of years old yes and those handsome Moors all in white and turbans like kings asking you to sit down in their little bit of a shop and Ronda with the old windows of the posadas 2 glancing eyes a lattice hid for her lover to kiss the iron and the wineshops half open at night and the castanets and the night we missed the boat at Algeciras the watchman going about serene with his lamp and O that awful deepdown torrent O and the sea the sea crimson sometimes like fire and the glorious sunsets and the figtrees in the Alameda gardens yes and all the queer little streets and the pink and blue and yellow houses and the rosegardens and the jessamine and geraniums and cactuses and Gibraltar as a girl where I was a Flower of the mountain yes when I put the rose in my hair like the Andalusian girls used or shall I wear a red yes and how he kissed me under the Moorish wall and I thought well as well him as another and then I asked him with my eyes to ask again yes and then he asked me would I yes to say yes my mountain flower and first I put my arms around him yes and drew him down to me so he could feel my breasts all perfume yes and his heart was going like mad and yes I said yes I will Yes.

$\endgroup$
  • 1
    $\begingroup$ I am actually a bit concerned. If the MSE+MO posts in question are artificial, then SE moderation as of now cannot be effective against a flood of those. $\endgroup$ – ccorn Jun 6 '18 at 2:46
  • 6
    $\begingroup$ @ccorn I understand. We are mostly supposed to avoid speculating about the users, but this person strikes me as a combination of an urge to impress with very little actual awareness of people around him. You may have noticed that we have users who post several items per day that nobody ever reads. That strikes me as sad; your guy is a little further off the deep end. By the way, we have had machine-like submissions from time to time, people who put in a ton of effort to glue together article excerpts. $\endgroup$ – Will Jagy Jun 6 '18 at 2:55
  • 8
    $\begingroup$ Um, what are the literature excerpts in there about? The questions don't seem similar to them... $\endgroup$ – EKons Jun 8 '18 at 11:20
  • 2
    $\begingroup$ @ΈρικΚωνσταντόπουλος I think a machine might have crafted this answer. It fits precisely the structure of the original posts in question. $\endgroup$ – Alex Clark Jun 12 '18 at 1:02
  • $\begingroup$ That wall of text looks nothing like either of the posts in question. What the heck is this answer? $\endgroup$ – numbermaniac Jun 14 '18 at 8:48
  • 1
    $\begingroup$ I always appreciate the opportunity to read some Joyce. $\endgroup$ – user98602 Jun 16 '18 at 10:43
1
$\begingroup$

What about this recent answer? I was tempted to delete it as a simple lunatic answer, but the inclusion of a "special" link made me thinking of this thread here in meta...

$\endgroup$
  • $\begingroup$ "provisorical"? $\endgroup$ – Gerry Myerson Jul 5 '18 at 9:43
  • $\begingroup$ Hmmm - simply a guessed translation from the german. Perhaps a terrible one? Well - just now I found "provisorily" in leo.org ... thanks for the hint $\endgroup$ – Gottfried Helms Jul 5 '18 at 10:46
  • $\begingroup$ Human, I guess. The connections do not seem to be invented on the spot. There's more of the same where the link points to. Reminds me of some praise-all-day-long culture. The style is consistent, though not stringent. As to the MSE answer, well, it does not answer the question. $\endgroup$ – ccorn Jul 5 '18 at 12:21
  • $\begingroup$ Thanks for the example anyway. The spectrum of posts attracted (but not accepted) by MSE is amazing. $\endgroup$ – ccorn Jul 5 '18 at 12:39
0
$\begingroup$

Why would one think first of machine-generated content instead of the much simpler hypothesis of a non-native English speaker with poor mastership of both English and mathematics (Occam's razor, right?). I've seen many posts like this since joining MSE, when AI was still not a fashion. Keep in mind that, for example, "Chinese has no grammar" - so for an untrained Chinese native speaker using articles, prepositions, the plural and other features of Indo-European languages must be terribily difficult. I remember the much simpler case a Danish guy who was unable to understand and use the French subjunctive because this mood is absent from Danish.

$\endgroup$
  • 1
    $\begingroup$ Occam's razor actually led me to the AI hypothesis because I would not think it likely that someone deliberately posted stuff that could only accidentally, if ever, contain sense. Since then I have seen some more examples. For some users that seems to be a modus operandi indeed. Well, now I'd say, I wish they were bots. At least that would be interesting. $\endgroup$ – ccorn Jun 13 '18 at 6:21
  • 2
    $\begingroup$ As to poor mastership: I have met people who could barely speak a language I understood, and yet they managed to convey sincerety and meaning. If they want to be understood, that is possible. Therefore, when I see a post that does not seem to even make the attempt of getting understood, I conclude that there is no wish of that behind it. $\endgroup$ – ccorn Jun 13 '18 at 6:30
  • 1
    $\begingroup$ As a Chinese, I just want to point out that Chinese does have grammar. We have articles, prepositions. Plural too! Just not in conjugation, but some indicator words. $\endgroup$ – Brian Cheung Jun 16 '18 at 14:47
  • $\begingroup$ @BrianCheung: I know this, but for a native speaker of some Indo-European language, this is how Chinese is perceived - as having no grammar. This is also why I used quotation marks. $\endgroup$ – Alex M. Jun 16 '18 at 14:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .