% in inline scripts

[jmaibaum]

Hi,

I'm new to SILE and not yet sure if I got it's "TeXlike" syntax right, but I could not find anything about this in SILE's official documentation. I was expecting the following MWE to work:

\begin{document}
  \script{SILE.typesetter:typeset(tostring(5 % 3))}
\end{document}

i.e. typesetting a "2". Yet SILE (current git master) fails with:

Error detected:
["\begin{document}\n\sc..."]:3: parse error, } expected
\end{document}
^

Obviously because the % is parsed as a comment in TeXLike mode. During ordinary text input you can escape it with a backslash in order to typeset a %. I've also tried that in the MWE above, but SILE then fails with ! [string "SILE.typesetter:typeset(tostring(5 "]:1: ')' expected near <eof> at test.sil l.2, col.1.

I've read in #92 that "TeXlike-mode is just for toy examples and tests" and so this might be considered a wont-fix, but then it might be helpful to mention in the official docs that in TeXlike mode all special characters (\, {, }, and %) cannot be used in inline scripts? I'd love to be able to use % in inline scripts as you could then use string.format() inline as well.

But, perhaps I've just missed some relevant section in the documentation and there is already some special syntax for this?

[alerque]

No you're not missing something. The parser is looking for TeX like syntax features first before it parses what is inside or outside a script. This is actually pretty frustrating and i wish we had a better solution too.

I'll just throw in the other major one is curly braces. These are pretty basic to Lua syntax but cannot be used in inline scripts. As a result almost all scripts have to be included from a separate file. It's almost silly to even support the inline syntax without being able to use basic Lua syntax, so I would like to see this fixed at some point.

[jmaibaum]

If the parser is checking for TeXlike syntax first, wouldn't it be possible to hack in = escape-with-backslash solution, as if it was parsing ordinary text? Obviously not a pretty solution, but it might work until a proper fix has been worked out?

[alerque]

Just an extra note to mention this is also going to be a problem for other future features such as Lilypond support (#435). This really needs a proper fix so that a tag can wrap some other language and pass it's contents through to some other parser.

[alerque]

Progress. I have reworked the LPEG grammar SILE uses to parse "TeX like" input to pass through % characters inside script environments to Lua. I'm going to work on it a bit more to handle the command form as well as handling { and } with similar dexterity.

[alerque]

New problem. SILE's parsing is now smarter than my vim syntax highlighting.

Vim still thinks the modulus operator is the start of a comment.

[alerque]

Status report: as of last night this works for the command as well as environment form of \script.

[alerque]

Status report: as of now the environment form of script can now pass through braces to Lua. The command form will be a little trickier as it will need to count opening and closing braces rather than just skipping all of them to the end \end{script} marker.

[alerque]

Status report: braces work in the command form \script{} and are passed through to Lua. The catch is that there has to be a matching number of braces regardless of the Lua context. This will cover most basic use cases where braces are used as part of Lua syntax (where they are required to be matched pairs to evaluate anyway). The edge cases it won't cover are where braces might be inside strings and happen to be unbalanced.

For that case I'll need to work on the escaping syntax. However at least basic Lua functionality is now possible.