[python_ast] Make the iter_mut functions public

[ndmitchell]

Summary

I needed to use these, so expose them. I am manipulating an AST to mutate it, and can't get at the inner strings here.

Test Plan

CI

[MichaReiser]

The fact that this method isn't public seems intentional to me because mutating can result in an inconsistent to_str result.

[ndmitchell]

For my particular usage, I actually only want to mutate the ranges. Thoughts on how to do that in a safe way?

[ndmitchell]

In fact, if I had Parser::new_starts_at exposed that would suit my needs even better. I have a string, but I know it starts halfway through a file, and what to parse that.

[ndmitchell]

And I guess this only applies to the StringLiteral one - since the others don't have a cached to_str, they could be made public?

[MichaReiser]

The parser exposes a parse_expression_range method. I rather not expose the parser struct

[ndmitchell]

(in all cases we'd be delighted to supply them as a PR, once we figure out the design)

[dhruvmanila]

For (1), we do use parse_expression_range, you might be interested in going through https://github.com/astral-sh/ruff/blob/5118166d21784e7c78e38ea42919ba50bb2a5142/crates/ruff_python_parser/src/typing.rs which parses string type annotations. Ruff still has a bug for complex type annotations (like implicitly concatenated strings) #10586.

For (2), we have the Transformer trait which will visit the nodes to allow mutation.

Is this useful?

[MichaReiser]

We are also looking into how to solve 1 for our type checker

[ndmitchell]

@dhruvmanila - yep, we have the identical problem, but haven't added the "simple literal" special case. We probably should. But then still end up with the same issue. If you had the parse_expression_at, then the special case becomes non-interesting and you can simplify the code.

The Transformer trait is super interesting. It's a shame that the self isn't mut - that makes accumulating in the visitor much harder. It's a shame there is no lifetime constraint, e.g. if you are walking an &'a Stmt, you have the guarantee that all the statements you bump into have the same lifetime 'a (I note SourceOrderVisitor has this). Can't you use this to screw with the literals in a StringLiteral, thus breaking the to_str cache? With those adjustments, I think we'd implement our visitors on top of your Transformer trait.

[ndmitchell]

Alternatively, if you added parse_expression_in_string_literal(x: &StringLiteral) -> Expr then that would be exactly the primitive we both need, we could delete the slightly dubious parse_expression_range (is there another good reason to have this?) and you could do very clever things to deal with things like escapes only once.

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.