Recursion error in ST4152+ - switch to the Jinja2 package

[deathaxe]

For those who have issues with this syntax after upgrading to ST 4186...

see: https://forum.sublimetext.com/t/syntax-highlight-error-after-update-build-4186/74811/4

Jinja2 package covers all of this package in an even more accurate way for all ST 4152+. It supports C/CPP/CSS/HTML/JS/JSON/Text/XML/YAML Jinja2 templates extending ST's bundled syntaxes.

[willstott101]

Thanks for sharing all of this context. Jinja2 definitely looks like a reasonable alternative, and I'm not interested in replicating all of that work.

Why does this package exist?

I originally wrote this syntax to overcome a number of shortcomings in Djaneiro (including squ1b3r/Djaneiro#102 which still seems relevant even after they've massively refactored their syntax definitions to be more similar to Jinja2).

I also wanted to have a package which didn't install weird snippet-related syntaxes like Python (Django). Call me crazy but I just want my highlighting packages to add highlighting 🤷

Is there anything good here still?

I do have a handful of opinions on the details of this highlighting, which could lead me to prefer using XML mode of django-sublime-syntax over Jinja2 (in situations where the loss of Javascript/CSS highlighting wasn't too painful).

Sea of White

In my definition I was careful to avoid too much white. HTML/XML with a reasonable amount of actual text content can quite quickly turn into a sea of white. If the entirety of a templated sea of text is white I frankly don't see much point in the syntax definition at all... hence why I chose to abuse constant.character.escape scopes for the markup.

Django sublime syntax:

Djaneiro (string?! that's a variable):

Jinja2 (nothing to see here):

Punctuation or Escapes

If we weren't already inside XML/HTML which heavily makes use of punctuation.definition perhaps that'd be a more semantic scope than constant.character.escape. Here you can see a lot of punctuation syntax with very different meanings all with same colors side by side:

Keywords or Function Calls?

I also find that considering the use of tags as function calls, along with named arguments looks clearer to me.

Django sublime syntax:

Jinja2 (keywords):

However looking more closely I am very inconsistent around what should/shouldn't be a keyword - as there are definitely some even in my function call paradigm.

Where to go next

Purely out of interest and for the satisfaction I'd like to make some improvements to django-sublime-syntax's bugs (notably around keywords, operators, and more complex multi-parameter tags). In doing so I may investigate what differences there are between the Jinja2 and Django implementations in this area - where lists of named operators may differ that could affect highlighting.

I would be interested to hear your opinions on these differences I have highlighted between the Jinja2 package and mine, both the highlighting differences and the attempt to avoid any and all snippets and completions.

I would imagine there is a road forward towards encouraging the use of Jinja2 over django-sublime-syntax, but I don't know if that is via Package Control messages or a package control rename - rename doesn't seem like the right fit.

[deathaxe]

Call me crazy but I just want my highlighting packages to add highlighting

That's a very perfect request and expectation. It's better for end users to install 4 of 10 packages to fit their neeeds, than having to struggle with how to remove unwanted ones from a monolith.

Jinja2 just provides some snippets which make working with templates easier, but even that could be provided via separate package. Especially snippets and their often very short triggers are subject of a very personal tastes.

Sea of White
...
I chose to abuse constant.character.escape scopes for the markup.

I fully second your arguments about sane/useful highlighting being a goal, but finally syntax definitions just assign scopes to tokens. Highlighting is the job of color schemes.

Only if (all) syntax definitions provide (same) syntactically or semantically correct scope names it will be possible to define (simple) color schemes which work for all languages with nearly same quality. If syntaxes choose arbitrary scopes to work well for certain existing color schemes and color schemes try to fix it by language-specific highlighting rules, chaos is perfect. That's what happened in the past very often, ending up color schemes providing good results only for a small subset of languages.

Jinja2 follows ST's scope naming guidelines and scopes

• {{ and }} with punctuation.section.embedded / punctuation.section.interpolation.
• {{ variable }} with variable.other

Mariana assigning white to all of them is it's design decision not to blame syntax definitions for.

Packages can (but shouldn't) provide color scheme patches. MarkdownEditing does it to add some missing rules to Mariana and Monokai for better UX.

Highlighting can always be tweaked by end users, using UI: Customize Color Scheme.

{
	"rules": [

		{
			"name": "Templates",
			"scope": "punctuation.section.embedded, punctuation.section.interpolation",
			"foreground": "var(blue)"
		}
	]
}

Punctuation or Escapes

...Here you can see a lot of punctuation syntax with very different meanings all with same colors...

That's certainly true from highlighting point of view, but otherwise everything already said applies.

Jinja and in the meanwhile most other templating languages scope embedding/interpolation punctuation punctuation.section while tag punctuation and string quotes use punctuation.definition.

It requires color schemes to at least distinguish both or more ideally provide more fine gained highlighting rules.

It still doesn't justify syntaxes to apply totally wrong constant.character.escape scopes. A \n is an escaped character, but not {{ or {%.

Keywords or Function Calls?

There's no perfect answer which fits all syntaxes, especially those which provide all functionality by sorts of functions. Following technical specification of languages too strictly, may however quickly end up in all words/tokens having the same color. It's somewhat the same like with the punctuations.

That's why tokens, which denote control flow or operators are/can likely be scoped keyword even if they are technically function calls, to distinguish them from normal function/filter calls.

It helps to make template code look more like other source code (python, javascript, ...).

Where to go next

The main reason for this issue was learning about multiple syntax packages existing with nearly same goal, mainly triggered by user reports about them no longer working due to default syntax changes and many 3rd-party packages having been abandoned.

There's no reason to not continue this package. I just wondered whether it is still actively maintained and/or worth the effort to maintain 2 or more syntax definitions with basically the same domain.

A possible argument would be someone looking for Django syntax, probably wouldn't be aware of Jinja2 package doing the job as well, or maybe there are or will be some conflicting differences between Django Template language and Jinja2.

We also have a Twig syntax, which is 95% Jinja2 with some PHP related tweaks.

I recently also learned about Nunjucks - Jinja for JavaScript.