Singleton parser - how much the cost is when using multiple parse

[nguyen-vo]

I am reading the GUIDE document and found that it recommends creating a single instance of the parser to improve the performance https://chevrotain.io/docs/guide/performance.html#using-a-singleton-parser.

How much is the cost of initiating multiple parsers? I am looking to provide an interpreter that will accept a small program via an endpoint, parse the program, execute it, and return the result. Since the parser takes in an input and uses that until it finishes parsing there are no other options besides creating a new instance of the parse per request.

[msujew]

Hey @nguyen-vo,

Since the parser takes in an input and uses that until it finishes parsing there are no other options besides creating a new instance of the parse per request.

I'm a bit confused. I don't really see why this is necessary. I've built a lot of interpreters in the past that don't require a new parser instance for each parse.

How much is the cost of initiating multiple parsers?

I don't have any numbers, since it depends on the complexity of your parser rules. Generally however, you can expect that the parser initialization is orders of magnitude slower than parsing (depending on the size of your input).

[nguyen-vo]

Hi @msujew
I am planning to put my interpreter as an API service. So in case of the parser instance receives 2 concurrent requests with 2 different inputs wouldn't that causing a conflicts for the parser during parsing? As per the document setting the input properties of the parser reset the parser state.

I expect the request's input is relatively small at most 50 line of codes

[msujew]

So in case of the parser instance receives 2 concurrent requests with 2 different inputs wouldn't that causing a conflicts for the parser during parsing?

That's not possible in JavaScript, as the Chevrotain parser is synchronous, meaning it blocks the event loop while parsing any input. You cannot create concurrent parsers with Chevrotain, unless you create a worker thread (which in turn needs actually a new parser instance, as worker threads cannot share memory i.e. chevrotain parser objects).

If you need concurrency, I would recommend creating a thread pool the size of your CPU cores and instantiate a parser in each of them. That way you get the benefit of currency without requiring to recreate the parser on each parse.

[nguyen-vo]

Thank you! The event loop blocking is what I am looking for.

I was unclear how the parser will behave if setting the parser's input while it is parsing the previous input.

Thanks for answering my question!

[hydra1983]

My performance tests

nx run @dg/lcdp-formula:perf                                                                                                                                                                       ─╯

> nx run @dg/lcdp-formula:perf

● Validation Warning:

  Unknown option "coverageReporters" with value [] was found.
  This is probably a typing mistake. Fixing it will remove this message.

  Configuration Documentation:
  https://jestjs.io/docs/configuration

● Validation Warning:

  Unknown option "reporters" with value ["default", "jest-bench/reporter"] was found.
  This is probably a typing mistake. Fixing it will remove this message.

  Configuration Documentation:
  https://jestjs.io/docs/configuration

 PASS   @dg/lcdp-formula  packages/formula/perf/formula.chevrotain.bench.ts (14.063 s)
  chevrotainCalculator - singleton-parser
    ✓ chevrotainCalculator.calculate (5939 ms)
  chevrotainCalculator - multi-parser
    ✓ chevrotainCalculator.calculate (5797 ms)

Benchmarks:
  chevrotainCalculator - singleton-parser
    chevrotainCalculator.calculate    0.678 ms ±  3.17 %  (76 runs sampled)
  chevrotainCalculator - multi-parser
    chevrotainCalculator.calculate    2.68 ms ±  3.65 %  (84 runs sampled)
Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        14.11 s, estimated 15 s
Ran all test suites.

 ———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target perf for project @dg/lcdp-formula (15s)

setup tests

import { benchmarkSuite } from 'jest-bench';
import { FormulaCalculator, FormulaCalculatorImpl } from '../src';
import { ChevrotainFormulaCalculator } from '../src/lib/calculator/chevrotain/formula.calculator';

let chevrotainCalculator: FormulaCalculator;
const formula = {
  expression: '1',
};

benchmarkSuite('chevrotainCalculator - singleton-parser', {
  setupSuite() {
    process.env.CHEVROTAIN_SINGLETON_PARSER = 'true';
    chevrotainCalculator = new FormulaCalculatorImpl(undefined, undefined, new ChevrotainFormulaCalculator());
  },

  ['chevrotainCalculator.calculate']: () => {
    chevrotainCalculator.calculate(formula);
  },
});

benchmarkSuite('chevrotainCalculator - multi-parser', {
  setupSuite() {
    process.env.CHEVROTAIN_SINGLETON_PARSER = 'false';
    chevrotainCalculator = new FormulaCalculatorImpl(undefined, undefined, new ChevrotainFormulaCalculator());
  },

  ['chevrotainCalculator.calculate']: () => {
    chevrotainCalculator.calculate(formula);
  },
});

create parser

const maxLookahead = 2;
const traceInitPerf = false;
const skipValidations = true;

const createLexer = () =>
  new Lexer(allTokens, {
    ensureOptimizations: true,
    traceInitPerf,
    skipValidations,
  });

const createDefaultParser = () =>
  new FormulaParser({
    maxLookahead,
    traceInitPerf,
    skipValidations,
  });

const createLocationTrackingParser = () =>
  new FormulaParser({
    nodeLocationTracking: 'onlyOffset',
    maxLookahead,
    traceInitPerf,
    skipValidations,
  });

const defaultParser = createDefaultParser();

const locationTrackingParser = createLocationTrackingParser();

export function createParser(
  formula: string,
  config?: {
    nodeLocationTracking: boolean;
  },
): FormulaParser {
  const lexer = createLexer();
  if (lexer.lexerDefinitionErrors.length > 0) {
    throw FormulaError.create({
      code: FormulaErrorCodes.INCORRECT_INPUT,
      message: `Lexing errors encountered in formula '${formula}': ${lexer.lexerDefinitionErrors
        .map((e) => e.message)
        .join('\n')}`,
    });
  }

  const lexingResult = lexer.tokenize(formula);
  if (lexingResult.errors.length > 0) {
    throw FormulaError.create({
      code: FormulaErrorCodes.INCORRECT_INPUT,
      message: `Lexing errors encountered in formula '${formula}': ${lexingResult.errors
        .map((e) => e.message)
        .join('\n')}`,
    });
  }

  const singletonParser = (process.env.CHEVROTAIN_SINGLETON_PARSER ?? 'true').toLowerCase() === 'true';
  const parser = singletonParser
    ? config?.nodeLocationTracking === true
      ? locationTrackingParser
      : defaultParser
    : config?.nodeLocationTracking === true
    ? createLocationTrackingParser()
    : createDefaultParser();
  parser.reset();
  parser.input = lexingResult.tokens;

  const originalFormulaMethod = parser.formula;
  parser.formula = () => {
    const result = originalFormulaMethod.apply(parser);

    if (parser.errors.length > 0) {
      throw FormulaError.create({
        code: FormulaErrorCodes.INCORRECT_INPUT,
        message: `Parsing errors encountered in formula '${formula}': ${parser.errors
          .map((e) => e.message)
          .join('\n')}`,
        cause: parser.errors[0],
      });
    }

    return result;
  };

  return parser;
}

See #2083 for more details