6
$\begingroup$

In reference to this discussion on title ban for HNQ

An empirical approach would be far better than what we're doing right now imo. More specifically, a keyword should be banned for HNQ if the ratio of poor to good questions with the keyword is high. Given a large number of questions with a particular keyword it isn't that hard to reach some consensus on the bad-to-good ratio.

The problem with asking a small bunch of meta users about this is:

  1. They don't have the patience or incentive to browse through tens or hundreds of questions to arrive at a bad-to-good ratio, they're going to be biased and use a smaller pool of data - the questions which they happen to remember or have googled.

  2. The problem mentioned in 1 gets compounded since most users are infact probably look at overlapping data pools to arrive at their decision - googling a keyword turns up similar answers for everyone, remembering recent questions brings up common memories of the recent questions, and looking for high-voted questions again means wasted effort as everyone is looking at the same data to reach their decision.

  3. Many poor questions are deleted and hence don't show up in searches. This significantly biases the data you have access to, unless you're a mod or frequent visitor of the delete queue.

What this site instead needs IMHO is a vote counter that can only be used by high rep users, since that's what you guys seem to be aiming for. You don't trust "average Joe"-driven voting to decide what is worthy of HNQ, you wish to give more weight to knowledgeable persons. Which is okay I guess, that's for another debate.

Since a coding solution is off the table, what would work is if the mods provided access to a decent-sized corpus of questions including deleted ones, and then distribute the task of filtering good and poor questions among high rep users. Or if that's too much effort, perhaps the mods themselves could do a bit of such filtering, arrive at good-to-bad ratios and take the decision themselves. I don't see how your pool of meta users with biased data is going to do a better job except when it comes to the most obvious ban worthy keywords (good-to-bad ratio nearing zero).

$\endgroup$
17
  • 4
    $\begingroup$ math.meta.stackexchange.com/questions/31517/… $\endgroup$ – Asaf Karagila Mod Apr 17 '20 at 20:43
  • $\begingroup$ "What this site instead needs IMHO is a vote counter that can only be used by high rep users" ? $\endgroup$ – user486983 Apr 17 '20 at 20:56
  • $\begingroup$ @AsafKaragila Thanks that does help. A collaborative effort to fliter them would be nice? I mean my points still hold, many people are not gonna bother looking at the data. $\endgroup$ – ghosts_in_the_code Apr 17 '20 at 20:58
  • 3
    $\begingroup$ @Magic_Isa I mean you guys want a metric for a good question that is not its vote count, if vote count was enough you'd simply let questions with high vote count reach HNQ. Instead the high rep users claim that poor questions often have high vote counts, and they alone get to decide what makes a question "poor" - which basically means they don't trust the coded democratic mechanism to decide quality. And I agree tbh, democracy is not my cup of tea. So yes, if votes from high-rep users were recorded separately on day-to-day use we would arguably have a better metric for quality and wouldnt .... $\endgroup$ – ghosts_in_the_code Apr 17 '20 at 21:01
  • $\begingroup$ @Magic_Isa .... have to rely on meta discussion to reach consensus among more involved and higher rep users over what makes a question "quality" $\endgroup$ – ghosts_in_the_code Apr 17 '20 at 21:02
  • $\begingroup$ What do you consider a good question? Is that solely based on score or would answercount, acceptance contribute as well? Other factors? I'm asking to have a decent go at empirical $\endgroup$ – rene Apr 20 '20 at 6:42
  • $\begingroup$ @rene I don't think any existing metric would do the job since we've already identified that many users vote up poor questions. Similarly I'm sure they answer poor questions, or comment on them etc. Even if we can finally reach a completely automated way or deciding what makes a question poor - we'd still a large pool of questions to be manually filtered by experienced users in order to come up with or verify an automated rule $\endgroup$ – ghosts_in_the_code Apr 20 '20 at 6:57
  • $\begingroup$ @rene Looking for specific words is the only obvious way to filter - again this requires some filtered data to identify good words to filter out. An ML-based method that reads the entire text would probably fare even better, though that's out of the scope here I guess $\endgroup$ – ghosts_in_the_code Apr 20 '20 at 7:00
  • $\begingroup$ An ML needs a model too, it doesn't ML itself. If you, as an intelligent species, can't define what "good" words are some artificial buzz-word is not going to cut it. $\endgroup$ – rene Apr 20 '20 at 7:42
  • $\begingroup$ @rene We can identify good questions to feed as input and the ML model will identify combinations / relations of words in them that strongly correlate with good questions. ML is not an artificial buzzword oof $\endgroup$ – ghosts_in_the_code Apr 20 '20 at 7:44
  • $\begingroup$ Okay, what are the criteria "We" are using to identify good questions? $\endgroup$ – rene Apr 20 '20 at 8:04
  • $\begingroup$ If I get time I will see if there are ways to filter with a words list by using the SEDE (I suspect yes). The deleted part is interesting, I suspect that SEDE can pick those up and their titles—I think it would be safe to say those where bad though. $\endgroup$ – user29123 Apr 20 '20 at 14:08
  • 1
    $\begingroup$ @PaulPlummer here is a query for words in titles. Feel free to join SEDE chat if you need extra help. $\endgroup$ – rene Apr 20 '20 at 14:38
  • 3
    $\begingroup$ Ilmari Karonen posted some SEDE queries and graphs in an answer to another question: Autofilters for Hot Network Questions $\endgroup$ – Martin Sleziak Apr 21 '20 at 9:55
  • 2
    $\begingroup$ Even if 90% of questions with the word "limit" in the title are bad, that doesn't mean a filter for questions with the word "limit" in the title is useful - if almost all bad questions with limit in the title are downvoted immediately anyway, they won't be on HNQ, and filtering for "limit" will just prevent the 10% good questions from ending up there. Filtering makes sense if a disproportionate number of questions with "limit" in the title among those which are popular enough for HNQ are actually bad. $\endgroup$ – Misha Lavrov May 1 '20 at 18:11

You must log in to answer this question.

Browse other questions tagged .