Generally, the garbage collector will only thrash when you allocate too LITTLE memory, as it’s garbage collection gets more aggressive as the used memory approaches the cap.
While it’s possible that there is some issue where you allocate too much, I’ve never seen such a thing, and I run JVM’s allocating upwards of 20 gigs.
If the garbage collector is thrashing, you should see a CPU spike, because the garbage collector will use all the processing time trying to free memory. Is that what you are seeing?
How much CPU usage are you seeing, what did you set the Max memory to, and what are you seeing it actually allocating?