Steve is right. Pogies are OK Till it is time to swim.
I sea kayak almost exclusively: So swimming, while it happens, is not
common.
You likely have to loose your gloves to get a spray deck back on anyway
and pogies actually hold the paddle so you don't have to grip it as
much when you paddle.
I like the pogies but have a zip lock ( sandwich bag.) for dry gloves
when I get out.It is under the front hatch.
I still preffer pogies
Alex