Are the machines upon us?

Question

I suspect that this MSE question (Screenshot) and an MO question by the same account (Screenshot) have been artificially generated. But I'm not sure.

For a machine, those would be nice works of association, but that is not unheard of, as there are known paper generators such as SCIgen. The occasional typos work against a clear diagnosis.

Now I wouldn't mind if an AI posed a clear and interesting question. But those are not clear (to me), and if those questions are indeed machine-generated, this means that someone only needs to turn a knob to increase MSE's junk inflow far beyond manageability.

We may need increased machine assistance. Are there tools in place or in development that could automatically send posts like those to a review queue?

Update: Another post (#2810926, now deleted) (Screenshot) that might have been machine-generated has been identified in the comments. This time it is an answer.

Until a bot attack or a publication in the spirit of SCIgen occurs, we might remain in the dark about whether those posts have been machine-generated or not.

Currently, what matters are the characteristics that help those posts pass existing automated detectors and superficial human reviews.

Characteristics that should be machine-testable:

• No block-displayed math, very few inline math snippets
• Frequent references to, or quotes with superficially relevant keywords from Wikipedia / ArXiv / other sites known to host math content
• No other hyperlinks
• Generous use of inline markup (bold or italic portions)
• Many blocks (paragraphs/quotations/list items), but all short

This all serves to distract from the post's thin content while creating an appearance of thorough composition. I'd like to see some scoring mechanism work against such pretension.

Further characteristics, perhaps not machine-testable, but annoying:

• In questions: Typos and poor grammar except when reciting
• In answers: Telegram style only
• No complete thought in the non-recited text portions
• Following the references yields no relevance.

I find this use of fake references particularly frustrating because it multiplies the waste of every dedicated reviewer's time and effort.

Can we get the existing detectors sharpened, or new detectors introduced, to take into account at least the machine-testable features listed above?

Update: I suppose that there is some scoring tool in place already, perhaps a linear classifier. It would apply a number of metrics, dot-multiply the resulting vector with a vector of weights, and obtain a score. Interestingly, adding more metrics often helps to better separate good examples from bad ones. Therefore, this is mostly a matter of adding new metrics and refining existing ones. I'd propose to:

• Count links to Wikipedia and ArXiv separately from other hyperlinks
• Measure density of block-displayed math
• Measure markup density outside math, blockquotes, and code sections

and append those to the other existing metrics. Then recompute the optimal separating hyperplane (i.e. weights and threshold) from training data. Make sure to include gibberish posts and cross-validate. I'd be interested in the results.

Answer 1

Not to put too fine a point on it, but your question seems to be mostly about over-zealously imagining complicated technical solutions to a problem that doesn't exist and has simpler solutions even if it did.

First, all of those seem quite plausibly written by a human when you combine a non-native English speaker with someone who doesn't understand what they're talking about. But let's say they aren't. So? They were hardly massive disruptions to the site. Community moderation readily handled them.

But maybe, as you suggest, someone will flood the site with such questions. Why would they want to do that? Sure, some people do have some inscrutable desire to vandalize or produce meaningless, non-commercial spam, but they don't tend to spend months building and refining natural language generators. If they did, the user or IP would simply be banned. This will happen automatically if the user/IP produces a large number of low-quality posts. If they went even further and used a botnet to get around that, CAPTCHAs would simply be enabled. At any rate, if I wanted to flood the site with hard-to-detect-but-ultimately-low-value questions, I'd just randomly copy-paste existing questions, maybe with some minor changes or perhaps I'd mix questions together. This would not require any natural language processing sophistication.

Summary: There is no evidence that this is happening currently or that anyone would want to do this, nor is there any evidence that the existing moderation would be unable to deal with this if it did happen.

Answer 2

the MSE question seems more free verse; it would improve with the Latex verse environment. The two lines that begin with lower case letters are supposed to be indented a good deal, two to two-and-a-half words.

$$\begin{array}{l}\text{Like a skein of loose silk blown against a wall}\cr \text{She walks by the railing of a path in Kensington Gardens,}\cr \text{And she is dying piece-meal}\cr \text{ of a sort of emotional anemia.}\cr \text{}\cr \text{And round about there is a rabble}\cr \text{Of the filthy, sturdy, unkillable infants of the very poor.}\cr \text{They shall inherit the earth.}\cr \text{In her is the end of breeding.}\cr \text{Her boredom is exquisite and excessive.}\cr \text{}\cr \text{She would like some one to speak to her,}\cr \text{And is almost afraid that I}\cr \text{ will commit that indiscretion.}\end{array}$$

The question still there on MO is much closer to the end of Ulysses by James Joyce, https://en.wikipedia.org/wiki/Molly_Bloom#Soliloquy

mouth and it was leapyear like now yes 16 years ago my God after that long kiss I near lost my breath yes he said I was a flower of the mountain yes so we are flowers all a womans body yes that was one true thing he said in his life and the sun shines for you today yes that was why I liked him because I saw he understood or felt what a woman is and I knew I could always get round him and I gave him all the pleasure I could leading him on till he asked me to say yes and I wouldnt answer first only looked out over the sea and the sky I was thinking of so many things he didnt know of Mulvey and Mr Stanhope and Hester and father and old captain Groves and the sailors playing all birds fly and I say stoop and washing up dishes they called it on the pier and the sentry in front of the governors house with the thing round his white helmet poor devil half roasted and the Spanish girls laughing in their shawls and their tall combs and the auctions in the morning the Greeks and the jews and the Arabs and the devil knows who else from all the ends of Europe and Duke street and the fowl market all clucking outside Larby Sharons and the poor donkeys slipping half asleep and the vague fellows in the cloaks asleep in the shade on the steps and the big wheels of the carts of the bulls and the old castle thousands of years old yes and those handsome Moors all in white and turbans like kings asking you to sit down in their little bit of a shop and Ronda with the old windows of the posadas 2 glancing eyes a lattice hid for her lover to kiss the iron and the wineshops half open at night and the castanets and the night we missed the boat at Algeciras the watchman going about serene with his lamp and O that awful deepdown torrent O and the sea the sea crimson sometimes like fire and the glorious sunsets and the figtrees in the Alameda gardens yes and all the queer little streets and the pink and blue and yellow houses and the rosegardens and the jessamine and geraniums and cactuses and Gibraltar as a girl where I was a Flower of the mountain yes when I put the rose in my hair like the Andalusian girls used or shall I wear a red yes and how he kissed me under the Moorish wall and I thought well as well him as another and then I asked him with my eyes to ask again yes and then he asked me would I yes to say yes my mountain flower and first I put my arms around him yes and drew him down to me so he could feel my breasts all perfume yes and his heart was going like mad and yes I said yes I will Yes.

Answer 3

What about this recent answer? I was tempted to delete it as a simple lunatic answer, but the inclusion of a "special" link made me thinking of this thread here in meta...

Answer 4

Why would one think first of machine-generated content instead of the much simpler hypothesis of a non-native English speaker with poor mastership of both English and mathematics (Occam's razor, right?). I've seen many posts like this since joining MSE, when AI was still not a fashion. Keep in mind that, for example, "Chinese has no grammar" - so for an untrained Chinese native speaker using articles, prepositions, the plural and other features of Indo-European languages must be terribily difficult. I remember the much simpler case a Danish guy who was unable to understand and use the French subjunctive because this mood is absent from Danish.