Ah, if it’s all in GPM, then I don’t have a great suggestion, as you’d probably want to keep it as the default. But basically, you’d pull up an audio app (e.g., play music) in the app settings and look around for an 'app defaults" option. I’m on a Samsung phone, so I think it “looks” different for me than it might for you on your phone, apologies.
Actually, I should try some other players, even if it means I have to store all my music locally.
I looked in Google Assistant settings, and it will allow me to switch from GPM to YouTube Music, Spotify or Pandora, but not VLC. Trying cloud player next.
Looking at a few reddit threads, sounds like you can get it to play local music if you explicitly call out the music app, eg “Play Never Mind in [Music Player]”. Though one thing specifically said VLC doesn’t work with that. Also you may have to tell it “driving mode” first.
Cloud Player streams your music from Google Drive or OneDrive. But I don’t have enough space on either service for my music library. So scratch that.
OTOH, Poweramp can play local music. But when I ask Google Assistant to “play hello nasty in poweramp”, it chooses a random song in the album instead of the first track, then plays random songs from my entire library afterward. It doesn’t just play tracks from just that album, or starting from the beginning of the album.
Why is this so hard?
From the thread I posted it said cloud player could play local music as well. I wonder if they removed that functionality? Seems unlikely.