Atom package: Swallow tokenize.TokenError for syntax highlighting purposes

[jmanuel1]

Test case: [

Expected: Syntax highlighting completes (although no style is attached to [)

Actual: This error in the console:

child_process.js:304 Uncaught (in promise) Error: Command failed: C:\Users\jamai\Documents\projects\concat\env\Scripts\python.exe -m concat --tokenize
Traceback (most recent call last):
  File "c:\users\jamai\appdata\local\programs\python\python37-32\Lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\jamai\appdata\local\programs\python\python37-32\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\jamai\Documents\projects\concat\env\lib\site-packages\concat-0.1.2.dev1-py3.7.egg\concat\__main__.py", line 67, in <module>
    tokens = concat.lex.tokenize(code, should_preserve_comments=True)
  File "C:\Users\jamai\Documents\projects\concat\env\lib\site-packages\concat-0.1.2.dev1-py3.7.egg\concat\lex.py", line 39, in tokenize
    token = lexer.token()
  File "C:\Users\jamai\Documents\projects\concat\env\lib\site-packages\concat-0.1.2.dev1-py3.7.egg\concat\lex.py", line 68, in token
    return next(self._concat_token_iterator, None)
  File "C:\Users\jamai\Documents\projects\concat\env\lib\site-packages\concat-0.1.2.dev1-py3.7.egg\concat\lex.py", line 79, in _tokens
    for token_ in self.tokens:
  File "C:\Users\jamai\Documents\projects\concat\env\lib\tokenize.py", line 579, in _tokenize
    raise TokenError("EOF in multi-line statement", (lnum, 0))
tokenize.TokenError: ('EOF in multi-line statement', (2, 0))

    at ChildProcess.exithandler (child_process.js:304)
    at ChildProcess.emit (events.js:223)
    at maybeClose (internal/child_process.js:1021)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283)
exithandler	@	child_process.js:304
emit	@	events.js:223
maybeClose	@	internal/child_process.js:1021
ChildProcess._handle.onexit	@	internal/child_process.js:283
Promise.then (async)		
step	@	C:\Users\jamai\Docum…antic-grammar.ts:22
(anonymous)	@	C:\Users\jamai\Docum…antic-grammar.ts:23
__awaiter	@	C:\Users\jamai\Docum…antic-grammar.ts:19
module.exports.JavaScriptSemanticGrammar.markTokensForChange	@	C:\Users\jamai\Docum…ntic-grammar.ts:234
module.exports.JavaScriptSemanticGrammar.markTokens	@	C:\Users\jamai\Docum…ntic-grammar.ts:227
(anonymous)	@	C:\Users\jamai\Docum…ntic-grammar.ts:209
simpleDispatch	@	<embedded>:11
emit	@	<embedded>:11
emitDidStopChangingEvent	@	<embedded>:11
later	@	<embedded>:14
setTimeout (async)		
(anonymous)	@	<embedded>:14
emitDidChangeTextEvent	@	<embedded>:11
undo	@	<embedded>:11
(anonymous)	@	<embedded>:11
mergeSelections	@	<embedded>:11
avoidMergingSelections	@	<embedded>:11
undo	@	<embedded>:11
textEditor.undo	@	C:\Users\jamai\.atom…gle-selection.js:81
core:undo	@	<embedded>:11
n.<computed>	@	<embedded>:11
handleCommandEvent	@	<embedded>:11
KeymapManager.dispatchCommandEvent	@	<embedded>:11
KeymapManager.handleKeyboardEvent	@	<embedded>:11
handleDocumentKeyEvent	@	<embedded>:11

In this case, the issue is the unclosed bracket.