Now that you mention it, I do remember them saying that in DS9.
I do wonder though if the concept that Starfleet was civilian not military is something they retrofitted into the canon in the TNG era (with their Galaxy cruisers complete with families on board) that was not originally meant to be in the TOS show.
In TOS the Enterprise was a Heavy Cruiser, which kind of sounds like a warship. In addition to the Heavy Cruisers, TOS Starfleet also had Dreadnoughts. We don’t actually see any but they are mentioned in several of the original TOS movies they also showed up in their triple-nacelle glory in the Starfleet Technical Manual.
Also consider the names of the TNG Galaxy-class Cruisers vs. their TOS counterparts.
Galaxy-class Cruisers: Challenger, Enterprise, Galaxy, Odyssey, Venture, Yamato also mentioned Gandhi, Madison, Magellan. With the exception of Yamato (which was explicitly named after the IJN Yamato) these are all great exploration ship names.
DC Fontana writer for TOS and Roddenberry’s secretary suggested using “famous fighting ships of the past” for ship names and that’s what happened. TOS Constituition-class Cruisers include USS Hood (named after British battlecruiser named after British Admiral), USS Lexington (named after US Aircraft Carriers which were named after a famous battle), USS Potemkin (the name of a Russian Battleship named after a Russian military leader), USS Constellation (could be a decent exploration ship name, but in this case named for famous US warship), USS Defiant (a fine warship name, but a rather poor name for a science ship named after British warship HMS Defiant), USS Exeter (named after British warship HMS Exeter). I did find two Constitution-class Cruisers not named for warships — USS Korolev (named for a Russian Scientist) and USS Ahwahnee (named for a Native American Indian tribe).
I do see how having Starfleet non-military does fit into a Roddenberry view of our future. But having grown up with episodes like Balance of Terror where Kirk goes toe-to-toe with the Romulans with a plot taken directly from a WW2 movie “The Enemy Below” along with a half-dozen episodes of steely-eyed Kirk facing off against the Klingons, Starfleet always felt like the Navy. In retrospect though, I suppose these are less than 1/4 of the TOS episodes, although they are some of my favorites.
Perhaps a better fit would be the Coast Guard. The Coast Guard personnel are under military discipline, but their purpose isn’t primarily military, even though they will serve in that capacity when needed.