Potential pyright --createstub bug

[MarkBekooy]

I have the following piece of code, for which Pylance suggests the following return type: Any | float. This seems accurate to me.

Now when I run the command pyright --createstub on the directory that contains this piece of code, the following stub file is created:

"""
This type stub file was generated by pyright.
"""

class Account:
    def __init__(self, account_id, balance) -> None:
        ...
    


class Bank:
    def __init__(self) -> None:
        ...
    
    def create_account(self, account_id, initial_balance): # -> None:
        ...
    
    def get_account_balance(self, account_id): # -> float:
        ...
    
    def perform_transaction(self, sender_id, receiver_id, amount): # -> bool:
        ...

Here for the get_account_balance function, the return type is to be determined to be float instead of Any | float

Is this a bug? Or is this the expected functionality? Although that would seem a bit strange to me considering Pylance uses Pyright

[erictraut]

No, this isn't a bug.

If you do not provide a return type annotation for a function or method, pyright will infer its type from the function's body — specifically, from the return statements in the body. Pylance is simply displaying the return type inferred by pyright. In this case, the inferred return type is Any | float based on the two return statements in the function.

The --createstub option never emits inferred return type annotations directly. It simply adds a comment that represent a "guess" as to the return type. Since the purpose of type stubs is to eliminate ambiguities about types in your library, it's typically very bad for Any to appear in a type stub. For this reason, the "guess" that --createstub elides any Unknown type from an inferred return type. Unknown is the name that pyright gives to an "implied" Any.

I'm curious why you're creating a stub library for this code. Stubs are used as stand-ins for distributed libraries that lack type information, but this doesn't look like a library. If you're the author of this code, you should add type annotations directly to the file rather than creating a stub.

[MarkBekooy]

Hi Eric,

I'm sorry for not getting back to you sooner. I read your response a while back but forgot to answer it and today I found something that also made me think about this, so I revisited this discussion.

I created an algorithm that tries different combinations of ML-determined type annotations in order to fill in more type slots in the code. To decrease the search space, I use Pyright to fill in "Pyright inferred type annotations" in some slots, since I know those types to certainly be correct then. (E.g. using --createstub to create stub files and merge those types back into the Python code, before running the ML model on it.) I also assumed that the commented return type would be more accurate than just a "guess", so I use those type annotations as well for the return types, although they do indeed sometimes give me some problems as they are not always fully accurate.

I'm indeed the author of this code, but my algorithm should in essence work on any Python project that it is given. Therefore, I create stubs using Pyright, which are then used to add more annotations in the code. It can therefore also work on libraries for which I'm not the author.

[MarkBekooy]

I found something in Pyright's --createstub that gave me an issue as I use the return "guesses" as return types, which normally works rather well. However, I found a situation where the return type was cut off early, thus giving me problems.

The original code:

The created stub file:

As can be seen, the return comment ends with an ellipsis, thus not exactly matching the return type of the original code and this breaks my libcst parser, when I have this return comment as an actual return type annotation.

Why? Well, the code that libcst now tries to parse is:

from typing import Literal
import braintree
from braintree.resource import Resource

class UnknownPaymentMethod(Resource):
    def image_url(self) -> Literal['https://assets.braintreegateway.com/payment_method…']:
        return "https://assets.braintreegateway.com/payment_method_logo/unknown.png"

and this Literal return type annotation is in the incorrect encoding it seems like.
(I'll take a look at removing this incorrect encoding, but do you maybe know the character limit at which the string gets cut off?)

[MarkBekooy]

Okay, I didn't parse the return type annotation in UTF-8 encoding, which broke the ellipsis character. Now I do parse it correctly and remove it from the string, but still, the return type doesn't fully match the return statement of the original code. However, I do understand that you want to omit a part of a very long string

[erictraut]

There needs to be some limit placed on the length of strings inferred as literals. After all, it's possible in Python code to create a string literal that is megabytes in length. I suppose we could infer this as a regular str or a LiteralString rather than a precise string literal with truncated text.

[erictraut]

I've created this issue to track this.

[erictraut]

I've changed the logic that prints inferred return types in the stub generator. It now always attempts to use valid Python type syntax for all inferred return type comments. For example, rather than emitting # (int) -> str, it will emit # Callable[[int], str]. Also, it will never emit truncated literal str or bytes types. If the literal value is too long (more than 50 characters), it will revert to LiteralString or bytes.

[MarkBekooy]

Nice change! Much appreciated 😄

[MarkBekooy]

I've found another curious aspect of --createstub that I just came across and wanted to let you know about.

On the Pillow ImageColor.py file, the return type comment is the following:

I'm not exactly sure what <subclass of str and tuple> would entail in this case?

[erictraut]

This is expected behavior.

This type is synthesized when performing type narrowing when isinstance is used and the two types have no apparent overlap.

class A: ...
class B: ...

def func(x: A):
    if isinstance(x, B):
        reveal_type(x) # <Subclass of A and B>

Both mypy and pyright use this technique in their type narrowing logic.

If and when the type system is extended to include "intersection types", this will no longer be necessary.

[MarkBekooy]

Thanks for explaining! I've learned something new today

[MarkBekooy]

I ran --createstub on test_sanitizer.py from html5lib.

One of the functions there is the following:

@pytest.mark.parametrize("expected, input",
                         (pytest.param(expected, input, id=id)
                          for id, expected, input in param_sanitizer()))
def test_sanitizer(expected, input):
    parsed = parseFragment(expected)
    expected = serialize(parsed,
                         omit_optional_tags=False,
                         use_trailing_solidus=True,
                         space_before_trailing_solidus=False,
                         quote_attr_values="always",
                         quote_char='"',
                         alphabetical_attributes=True)
    assert expected == sanitize_html(input)

And the stub file now has:

@pytest.mark.parametrize("expected, input", (pytest.param(expected, input, id=id) for (id, expected, input) in param_sanitizer()}))
def test_sanitizer(expected, input): # -> None:
    ...

However, now suddenly an excessive } character appeared in the @pytest.mark.parametrize decorator (see the second to last character) and thus Python complained about parsing the file because of a syntax error

[erictraut]

Thanks, the extraneous } character will be fixed in the next release.

List comprehensions (and more generally, decorators) should not be used within a stub file, so this output is something you should clean up manually.

[MarkBekooy]

I ran --createstub on test_migration_archive_files.py from paperless-ngx.

This created a nice stub file, but there was an error when parsing it with libcst (which basically always succeeds).
In the generated stub file, there is the following code:

@override_settings(FILENAME_FORMAT="")
class TestMigrateArchiveFilesErrors(DirectoriesMixin, TestMigrations):
    migrate_from = ...
    migrate_to = ...
    auto_migrate = ...
    def test_archive_missing(self): # -> None:
        ...
    
    def test_parser_missing(self): # -> None:
        ...
    
    @mock.patch(<Expression>)
    def test_parser_error(self, m): # -> None:
        ...
    
    @mock.patch(<Expression>)
    def test_parser_no_archive(self, m): # -> None:
        ...

However, when parsing this with libcst it fails because @mock.patch(<Expression>) is present.

I understand that Pyright now adds <Expression> here, but that unfortunately makes it incorrect Python syntax according to libcst, so could it possibly be parsed differently?

[erictraut]

Pyright's "--createstub" feature is intended to give you a "first draft" of a type stub. You will almost always need to edit the resulting stub to make it into a workable stub. For example, decorators shouldn't be included in a type stub, so those should be deleted in your final stub.

That said, it's straightforward to fix the <Expression> issue you're seeing. Pyright is not properly handling an f-string. This will be fixed in the next release.

[MarkBekooy]

Thanks for your reply Eric! Indeed fixing the problem was quite easy, because I simply removed the decorator.

You mention that the --createstub feature is intended to give you a "first draft" of a type stub and also that decorators shouldn't be included in a type stub. Therefore, I was wondering why these decorators are still included in Pyright's generated stub files? Wouldn't it be better if Pyright filters out these decorators when generating the stub?

[erictraut]

The decorators are left in because some decorators modify the signature of the method they are decorating, and it's important when hand-editing a stub to have the context about what the original code did.

Also, in many cases, pyright (and other type checkers) will be able to handle the decorator in the stub even though it's recommended that they be removed and any of their side effects directly incorporated into the final signature.