If you click the Reply button within a given post, it will automatically tie your post to theirs with the visual indicator you mentioned.. AFAIK, the first reply you quote from in a thread (if you are quoting from multiple) will also get that tag. Of course, quote boxes also include a little expando to see the full post you're quoting from, if you prefer that.
If you just click the Reply button at the bottom of the entire thread, it won't tie your post to anyone else's, and AFAIK, there's no way to edit that metadata in once you've made your post.
So, if someone doesn't use the Reply button in a post they're replying to, or quote from it, there's no way to definitively know who they are replying to without asking or hoping for exceedingly obvious context.
Similar but interesting tidbit: you may notice that posts that get replied to a lot will have a little "X Replies" tracker at the bottom of them. This works all the time, unless specifically the only reply to them is the very next post, in which case I guess that Discourse assumes you should be able to infer it is a reply, and hides the "X Replies" tracker. If there's any gap, or more than one reply, it will show up, though.