If you go to the Games Store on the console, there’s a list of all released Xbox One X games. Third one down, after New Games and Coming Soon. It’s up-to-date.
You can also go to your game library and use the filters to show which games you already own have had Xbox One X enhancements added.
For 4K specifically, it’s a little more roundabout. If you go to New Games, Most Popular, etc, you can filter the lists, and 4K is one of the options.