feat: initial release and djc_py code block with DjangoComponentsPythonLexer

[JuroOravec]

Part of django-components/django-components#966.

This package that allows us to write djc_py code blocks like so:

\```djc_py
class table(Component):
    template = """
        ...
    """
\```

Using djc_py, the inlined HTML / CSS / JS will be properly highlighted:

[JuroOravec]

The project skeleton is based on django-components.

[JuroOravec]

As mentioned here - Pygments has an (IMO) strange design, where they keep all the Lexers as part of the codebase and part of the package, and don't provide some official way to add own Lexers. So we have to do so here.

Thus, when one imports pygments_djc, our Lexer will be added.

[JuroOravec]

This configures which code blocks will the syntax highlight work with.

[JuroOravec]

This is the actual business logic.

The way Pygments Lexers work is that each Lexer defines a list of states. And for each state, it defines various regex patterns that the program will try to match against the text at current cursor. And then the program reads the text, and at each step it:

1. Finds which state it's currently in
2. Fidns all regex patterns to test in current state
3. Tries the pattern one after another, until it finds match.
4. When it finds match, there is some "action" associated with the match - e.g. that if we match token """, then this token should be marked as Literal.Quote.Triple, or something like that. This is how the Lexers match pieces of text with different colors, for example.
5. Lastly, a match may also specify the next state to transition to. If so, the state changes. Either way, the program finds the end of the regex match, and continues comparing regexes on the remainder of the text.

[JuroOravec]

Here you can see that the returned value is actually a tuple of 3:

• The pattern to capture
• The token to highlight
• The next state (if any)

[JuroOravec]

This is where we define the Lexer states, and the regex patterns in each state. template_string and similar are our custom states, which mark when we're inside inlined template / js / css.

[EmilStenstrom]

Looks great from a quick read-through. Feels safe since it's only used in the docs.

[JuroOravec]

Thanks, I've just released this https://pypi.org/project/pygments-djc/ 🎉

Again I've used my PyPI account, and I've saved the API access token to repository secrets.