The new and unread topics are shown as counts in the category list, like so:
Click those words to be taken to the new topic, or unread topic, lists for that category respectively.
Just visit the topic list you are interested in and look for the red "last visit" line. Everything below that was present the last time you visited. Everything above that line was created since your last visit, so by definition you have not seen it.
This is something we explicitly added based on feedback here (not by specific request, but by interpretaton of feedback like yours) and I think it was a solid idea; happy to have added it.
If you want the system to explicitly track and "dismiss" every single topic as unread, well, Discourse begs to differ with vBulletin on default forcing people to suffer under the cognitive load of "hey you haven't 'read' every topic since you last visited, so you must click a button now because fuck you that's why".
That said, you can kinda mimic this by setting "tracking" on all categories. Visit the category, then press the notification button at upper right to set it. Once you do that, you will be forced to dismiss all new replies because you've told Discourse yes, I am super interested in every single topic in this category.
If that's a deal breaker for you, c'est la vie, de gustibus non est disputandum and all that. For the record, precious few people complain that we aren't nagging them enough, as they already have enough nagging in their lives, thankyouverymuch.