There are flattening additives for LPU and conventional enamels. These are
basically fine granular materials which stand proud when the paint dries.

As gelcoat doesn't shrink all that much in he curing. i speculate that adding
such a material to gelcoat would not do the job.

Perhaps a brisk buffing with a coarse rubbing compound or a fine Scotchbrite?

bruce wrote:

What can one add to the gelcoat/flowcoat so that once cured the
surface is more like a matt finish instead of the normal gloss finish?
I have seen this finish on a number of new boats in the cockpit and
deck areas as a way of reducing glare from the sun.