Repository navigation
python3: a # in an f-string breaks syntax highlighting #225
Description
See the code below. Since my issue exists on github, I can simply demonstrate my exact issue with the example code block below. Basically, when a # is used within an f-string, then it appears to break the syntax highlighting. It looks like the "#" in the f-string gets interpreted as the start of a comment, even though this is correctly handled for non-f-srings. The # in the format string is used to get the alternate form for the hex conversion. Here is an example of what the "#" does in a format string as well as demonstrate the bug:
>>> # syntax highlighting works here...
>>> i = 5
>>> '{:#04x}'.format(i) # <-- not using an f-string is fine
'0x05'
>>> f'{i:04x}'
'0005'
>>> f'{i:#04x}' # <---- this line breaks the syntax highlighting
'0x05'
>>> f'{i:04x}' # <-- syntax highlighting is broken!
'0005'
>>> '{:#04x}'.format(i) # <-- syntax highlighting is broken!
'0x05'
>>> Here is a screenshot to demonstrate the issue (for future reference) since, after this bug is fixed, the code above will presumably have the correct highlighting:
Edit: corrected the 4th output from 0x05 to 0005
Activity
Github doesn't seem to use MP for highlighter tokenization
elprans commentedon Nov 12, 2020
MagicPython highlights this snippet just fine:
The problem is that Github is now using something else for Python highlighting :-/ (I opened a PR to remove the outdated statement from the README).
Spectre5 commentedon Nov 12, 2020
ah, bummer. I couldn't find anywhere to post an issue for github itself, so I thought I tracked down that this library is what they use, but apparently not! Back to searching for what they use then...
Github doesn't seem to use MP for highlighter tokenization
Spectre5 commentedon Nov 12, 2020
Huh, do you have any idea what library they do use, then? Per this github documentation link, they use linguist, but linguist indicates that they use MagicPython in their grammar index.
Spectre5 commentedon Nov 12, 2020
Could they just be using an old version, or are you sure they don't use MagicPython anymore?
elprans commentedon Nov 12, 2020
Linguist is definitely using a fairly recent version of MagicPython. You might try asking on the community forum at https://github.community/
verhovsky commentedon Sep 19, 2021
For reference: https://github.community/t/what-library-does-github-use-for-python-syntax-highlighting/145558
No one responded
Edit: turns out Linguist uses tree-sitter for syntax highlighting of some languages, including Python.
Github doesn't seem to use MP for highlighter tokenization