[Feat] Support lazy imports #3764
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
Bobholamovic
commented
Apr 3, 2025
| - id: check-custom | ||
| name: Check Custom | ||
| entry: python .precommit/check_custom.py | ||
| - id: check-license-headers |
There was a problem hiding this comment.
将原本的自定义检查按照功能拆分成证书检查和导入语句检查
|
|
||
|
|
||
| _initialize() | ||
|
|
||
| __version__ = version.get_pdx_version() | ||
| for mod in _SPECIAL_MODS: |
There was a problem hiding this comment.
此处添加断言，确保import paddlex时没有代码将paddle等误载入了。我们保证这一点；如果用户遇到这个问题，说明我们的程序存在bug，用户可以向我们反馈。
| @@ -12,11 +12,11 @@ | |||
| # See the License for the specific language governing permissions and | |||
| # limitations under the License. | |||
|
|
|||
| import lazy_paddle as paddle | |||
There was a problem hiding this comment.
不再特殊处理paddle，而是和其他依赖一样用统一的方式管理。
| is_onnx_model_available = ( | ||
| is_onnx_model_available or "paddle" in model_paths | ||
| ) | ||
| else: | ||
| logging.debug( | ||
| "Paddle2ONNX is not available. Automatic model conversion will not be performed." | ||
| "The Paddle2ONNX plugin is not available. Automatic model conversion will not be performed." |
There was a problem hiding this comment.
体现产品层面的“插件”概念，避免抽象泄漏
| from .deps import DEP_SPECS | ||
|
|
||
| # TODO: Precompute or cache the constraints | ||
| with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False) as f: |
There was a problem hiding this comment.
由于pip目前并不会充分考虑已经安装的库，这里以PaddleX的直接依赖作为constraints，防止安装过程中PaddleX的直接依赖被变更（这是危险的）
| cons_files = [cons_file] | ||
| else: | ||
| cons_files = [] | ||
| # HACK: Avoid installing OpenCV variants unexpectedly |
There was a problem hiding this comment.
双重保险，万一套件的依赖有变更，而paddlex没有及时感知，这里也能拒绝安装错误的库。
| continue | ||
| else: | ||
| lines.append(line_s) | ||
| elif req.name in ( |
There was a problem hiding this comment.
防止安装不同变体的opencv。这种做法的缺点是：
- 需要手工维护直接和间接依赖其他变体opencv的库。好在目前这部分还可控且未来套件依赖大概率也不会有太大变更。
- 安装完成后可能显示部分套件核心库缺少依赖（一些opencv变体）。这个理论上不影响使用，如果需要消除的话可以考虑修改套件核心库的metadata等，但比较麻烦，这里没有实现。
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
本PR为PaddleX增加一系列utility函数以实现规范化、大面积应用的延迟导入，并新增静态检查脚本以发现问题。