I assume you will be going to Nexus for your mods so my comments below will be based upon that assumption.
First off, go to this site:
This person put together modding guides for both FO:NV and Fallout 4 and it gives a good idea on how to get started using MO2. I would use his most of his suggestions down through gameplay, especially those around Bug Fixes, Utilities, Tweaks and UI. Note that he has very limted gameplay suggestions because he avoids any mods that put a strain on the game or are somewhat buggy. I usually start with the parts I mention above as a basic foundation, and then add whatever I am interested in with respect to stuff like new weapons/armor, quests, locations, etc - stuff that he pretty much avoids for the most part.
Next for any mod you are interested in (we’ll get to that below), READ the description. You’ll find there any mod requirements, sometimes they will mention conflicts or issues. In addition, you should READ the comments section for the first 3 or 4 pages - oftentimes there you will find issues/conflicts that users have identified - and sometimes fixes.
To get started selecting and downloading mods, go to the Nexus, select Fallout 4, then select Mods, then select categories, then select Models & Textures. Then I would sort by either Unique Downloads or Endorsements. By using that ranking you will get the best of those mods that improve graphics. From there you can select those that interest you.
For content, do the same - select the content you are interested in (be that weapons, armor, quests, additional locations, etc) rank them and see what interests you.
As far as playing smoothly together, that’s going to take more work. Graphic upgrades generally won’t cause problems but content can. As noted above, reading the description and comments can be very helpful.
But no matter what you do, it seems issues always arise - I don’t think I have a modded play through where a conflict/crash has not arisen. There are no guarantees that any group of mods you select will always play together. For example in a current playthrough, all my mods worked fine - until I got to the alley you take to go to Diamond city right past the battle with the Super Mutants. It crashes every time. I stopped right before Christmas but the only way to figure out what is going on is to disable all my mods, then activate maybe ten at a time, load the save, and keep doing that until it crashes again. Than I’ll have to look at the group of ten that caused the crash, load them one or two at a time, until I identify that mod that is causing the crash, and then delete it.
This is just a quick overview, hope it is helpful.