Serialize Lark.grammar (fixes issue #1472)

[NasalDaemon]

Fixes #1472

[erezsh]

How much does this change add to the cache size? (in rough terms)

[NasalDaemon]

How much does this change add to the cache size? (in rough terms)

In my scenario with a file.lark of 4kb:

cache without grammar: 63,994 bytes
cache with grammar: 115,970 bytes

making an increase of ~52kb

As a quick and dirty performance test, I ran my script which uses lark to parse a file and produce a cpp (with jinja) multiple times under different regimes.

Without caching: 440ms
With caching, but using Lark.grammer = load_grammar(...): 370ms
With caching+serializing grammar: 320ms

This shows that there is indeed a noticeable speed benefit even if the cache has become larger, despite the overhead of jinja and the rest of my script.

[erezsh]

How does serializing the grammar make it run faster? It only adds operations, and doesn't remove any.

[NasalDaemon]

How does serializing the grammar make it run faster? It only adds operations, and doesn't remove any.

According to the above tests, deserialising the grammar from the cache is faster than reproducing it from the source .lark file each time.

[erezsh]

I think you're mistaken.

[NasalDaemon]

I think you're mistaken.

If you would like to avoid the cost of de/serialising the grammar by default, I can make its serialisation optional instead, so you must specify cacheWithGrammar=True.

[erezsh]

I prefer cache_grammar=False.

Also consider adding text about this in the reconstructor error.