Suggestions for future improvements to our forum software

Did you try it in quotes?

https://forum.quartertothree.com/search?q=%22trails%20in%20the%20sky%22

The highlighting is definitely wonky (it should only highlight the phrase when quotes are used), but I think this works, and should work.

It gets rid of the Skyrim and No Man’s Sky threads, but doesn’t make enough of a difference thanks to the priority given to recent drive-by mentions over older dedicated threads. Again, it’s not that the obviously correct thread results don’t show up at all, it’s just that they’re a ways down in the weeds, beaten out by threads that shouldn’t even be close to the same relevance to the search term.

And honestly, if the correct threads aren’t surfaced in the 5 results visible in the popup box, I’m much more likely to just pop over to google rather than futzing with advanced search or different search strings to find them. Witness:

Interestingly, I noticed that all the Google results for Qt3 threads claim that they have only 20 posts and even fewer authors. They’re obviously crawling the whole thread given that they do a much better job surfacing relevant threads than Discourse search does, but I wonder if there’s something you can adjust to present accurate stats there.

To chime in, @wumpus, my almost exclusive use case for search is finding whole threads about a topic, not a specific post with an almost unique search term buried in it. So the default behavior of, say, searching for Skyrim Mod Sales and getting a clickable link to some seemingly random spot in the thread about paid mods for Skyrim (let’s just say, for shits and giggles, that it–seemingly–randomly surfaces, oh, the 127th post out of the, say 600 or so posts in this thread I’ve never actually read before, and yes, those numbers are pulled out of my ass cuz I’m on my phone and lazy) now just drops me, context free, into the middle of a related thread, probably breaking my Last Read position in said thread in the process, when my ideal behavior would have been to click it and be taken to my first unread post (in this theoretical, the very first).

Maintaining the current specific post function you described above is certainly okay for other uses, but I join the others for search functionality that better serves thread-based searching (surfacing threads completely about the topic searched for and then linking you to the top or first unread post) would be enormously useful.

Sure, a solution was already posted for that. If you want only topic title matches, use

in:first

thus don’t search for

random

search for

in:first random

If you forget the words, just visit the full search page at https://forum.quartertothree.com/search and use the UI there which will insert the right words for you.

(We really need to add an in:title clause because “in the first post” doesn’t really capture the intent of searching by title only.)

The older the topic, the more likely this is to happen due to recency bias. So if you know you are searching for a topic from years (or even a decade) ago, you’ll need to change up your strategy. Regardless, use in:first when you search, since that’s what you seem to want, and in:title doesn’t quite exist yet.

Incidentally this is just one of many reasons why having decade (or even years) long running megatopics is a bad idea.

So then, in summary, what you want is

in:first "trails in the sky"

However this doesn’t seem to work, which I think is a bug.

Note that if people give their topic crazy super k00l l33t clever titles that don’t include the game in the title, you’re better off with the in:first assuming they aren’t so spectacularly clever that the entire first post never mentions the correct game name, either…

Sure… That’s why I complained about the recency bias in the first place. But it’s not a law of nature; you ultimately determine the weight of the recency bias in the search algorithm, right? Did you deliberately tune it to drown out all other measures of relevance? Are the sorts of results I saw for KoDP and TitS working as intended, matching your view of what should be expected for those search terms? If so, then carry on, I suppose. But if not, then I would suggest making some algorithmic weighting tweaks as future improvements to our forum software. It’s only a relatively minor inconvenience for me to go to Google for useful search results, but you have a vested interest in improving Discourse for all its users.

And I agree with retiring the truly huge mega topics, but I’m not sure that applies here. Would that 500 post KoDP conversation have benefited from being spread across a dozen threads as new people picked it up over the years, unable to easily refer to previous impressions? It’s all discussion on one particular topic.

Nah, what I want is sensible default search behavior that weights results closer to how I (and Google) perceive their relevance.

And I don’t think that not getting results there is actually a bug. I picked that thread because the original poster infamously omitted the game title from his gushing recommendation. So it serves as an example of a thread about a subject that’s not actually mentioned in the first post (unless you want extra credit by pulling the words out of the steam url). Similar, if less dramatic, examples occur when a thread starts out discussing some topic but after a few posts goes off on a lengthy tangent that ends up outweighing the original discussion. Limiting the search scope by title or first post can’t catch these, but a more sophisticated search algorithm could recognize that more that 10% of the posts in the thread contain the search term, so it is a major topic of discussion, and should be weighted accordingly.

You have to compare the number of employees and market cap of Google to Discourse. This is like looking at a toddler playing basketball and demanding to know why they can’t start as a forward for the New York Knicks in the NBA next Tuesday. Search is not an easy problem. Raise your hand if you regularly use Bing? Yahoo search? Duck Duck Go? Search seems pretty easy, anyone should be able to do it, that’s why there are so many fabulously popular alternatives to Google today, yes?

The simplest workable solution is to

  • require that topics have reasonable names that more or less match the games they are tied to: a topic titled “shit bonerz” that is about Bioshock Infinity is going to be a bad time for everyone, for eternity.

  • when searching, because you personally don’t care about post hits at all, indicate that you only care about results that are either matching the first post, or the topic title, with in:first.

In the meantime there are a few things for us to check, fix the quoted search highlighting, and we need to add in:title. But if there is no basic courtesy around topic titles (and/or first post) being recognizably related to the game they are about … you’re gonna have a bad time.

My suggestion was just to adjust the weight placed on three conceptually simple metrics (match recency, match in title vs. body, and number of matches relative to the total length of the thread), not to recreate the entire search algorithm. Agree with the other points though, thanks.

Bug: sometimes when backing out of a thread to return to Latest View (and presumably others), the forum software / browser go crazy and rapidly scroll down through dozens of posts, leaving you far below where you originally were.

Replication: Android 7 with latest Chrome. As you scroll down the Latest view, like the rest of Discourse, it begins loading the next chunk of topics, briefly showing you a spinning wheel beneath the lowest already-loaded content as the new stuff gets fetched. If you click into a topic while this loading is occurring, and then back out to Latest again, sometimes the forum will seemingly “continue” the extra content loading, but in overdrive, loading dozens of older posts and rapidly auto scrolling the view to reach the bottom of the newly loaded content.

Hey it’s not our fault you gave us a wickedly fast, powerful forum search feature that’s actually useful. Now we’re simply demanding that you make it perfect. :)

Aha good repro @eviltrout can you check this on your end?

This is better now

Nice, thanks. Looks like the partial-word matches (‘king’ and ‘looking’) is fixed.

I’m still seeing the quoted search highlight the first partial match in a post rather than the best match in the same post, though. See below, where the quote highlights ‘of’, rather than the full ‘King of dragon pass’ further down in that same post:

Maybe is time for forum software to recognize the existence of megathreads. Defined as a single thread to talk about a certain topic, making it exclusive. ((Is hard to define megathreads, but everyone knows them)). Possibly labeling a thread megathread should be a moderator action, not op action.

As of a few days ago, or something, it looks like the blockquote button now treats blank carriage returns differently. They’re ignored, now, and I’m guessing it was part of a bug fix that had unintended side effects.

So if you copy and paste a WaPo article, for instance, the button will put the text in quotes but will disregard the blank lines that appear between the paragraphs. Repo: copy and paste pretty much any article/blog post.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo.

At vero eos et accusamus et iusto odio dignissimos ducimus qui blanditiis praesentium voluptatum deleniti atque corrupti quos dolores et quas molestias excepturi sint occaecati cupiditate non provident, similique sunt in culpa qui officia deserunt mollitia animi, id est laborum et dolorum fuga.

Workaround: Insert right angle bracket (or arrow bracket) into the blank lines.

Yeah that’s a bug too, I will get that fixed. Edit: now fixed, as far as pressing the “format as a blockquote” button in the editor goes.

Great! That was fast.

You probably know this and may not care at this point, but the jump-to-post indicator goes a little wonky when you scroll to the bottom of a topic on a 10.5” iPad Pro in iOS 11 beta 4.