-f --storable-state
Generate a lexer which can store its inner state. This is useful in push-model lexers which are stopped by an outer program when there is not enough input, and then resumed when more input becomes available. In this mode users should additionally define YYGETSTATE () and YYSETSTATE (state) macros and variables yych, yyaccept and the state as part of the lexer state.
[EDIT] : A slightly better explanation is found here:
I think re2c may have at least parts of what you need:
From https://github.com/skvadrik/re2c/blob/master/re2c/doc/manual...
-f --storable-state Generate a lexer which can store its inner state. This is useful in push-model lexers which are stopped by an outer program when there is not enough input, and then resumed when more input becomes available. In this mode users should additionally define YYGETSTATE () and YYSETSTATE (state) macros and variables yych, yyaccept and the state as part of the lexer state.
[EDIT] : A slightly better explanation is found here:
http://re2c.org/manual/features/state/state.html
Code example here: http://re2c.org/examples/example_18.html