Some generators have a rpm governor that controls the output voltage.
Sounds like yours is one of those and it's sticking open too far. When
you add a load the engine throttle gets opened further to keep the rpm
constant. When the load is removed the engine throttle is not being
backed off thus the increase in speed and voltage.