Fighters don’t count toward the limit but show up in the fleet mouseover display, so some of the really big ones look bigger than they are.
It’s a good idea to have multiple fleets, or at least a roster of ships you can sub in. I usually leave most of my capital ships at one of my colonies unless I’m planning on getting into a fight directly, and I have a bunch of exploration/salvage ships I plug in for expeditions. The big thing for really long-range exploration is to have a few large cargo ships and tankers on hand for supplies and fuel.