Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dev #21

Closed
wants to merge 106 commits into from
Closed

Dev #21

wants to merge 106 commits into from

Conversation

YdrMaster
Copy link
Collaborator

No description provided.

@YdrMaster
build: update deps
a7bdde3 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
todo(llama): 初步实现基于 gguf 的 llama 加载和推理
2e6dc5f 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
test: 推理单测添加计时
2341bc9 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat(common): 支持解析和合并分片模型文件
8018d04 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
refactor(llama): 简化模型解析
93ad8e8 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
refactor(llama-nv): 从 gguf 为 nv 加载模型
ced0bb0 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster YdrMaster added enhancement New feature or request in progress Still working on it labels Aug 22, 2024
@YdrMaster YdrMaster self-assigned this Aug 22, 2024
@YdrMaster
refactor(causal-lm): render 存储 bos/eos，不依赖外部提供
1973925 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat(causal-lm): 识别是否需要替换空格，恢复添加和替换空格机制
d68a0eb 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat: 重新实现 llama 模型结构
ae6d015 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat: 支持从 LlamaMeta 生成常用张量布局，整理代码
6462640 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
todo: 初步实现 cpu 推理
c6ac5a4 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat: 添加张量格式化功能
55c8997 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Sep 28, 2024
View reviewed changes
tensor/src/lib.rs Fixed Show fixed Hide fixed
@YdrMaster
fix: 实现 cpu 推理
93f3b81 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
build: 梳理模块间依赖关系
d8f2ca8 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
style: 简化 LlamaWorker 模板，将 launch 标记为 &mut self 以约束唯一性
1054333 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
refactor: 简化张量对外的接口
050c008 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Oct 10, 2024
View reviewed changes
models/llama/common-cpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/common-cpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/common-cpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/common-cpu/src/lib.rs Fixed Show fixed Hide fixed
@YdrMaster
feat: 为 llama 模型结构实装 all reduce
cd61335 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Oct 10, 2024
View reviewed changes
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
models/llama/nvidia-gpu/src/lib.rs Fixed Show fixed Hide fixed
@YdrMaster
todo: 创建 llama-nv
d6b8fcc 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
refactor(llama-cpu): 剥离和优化 cpu“单体”推理实现
91a3f7b 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
perf(nv): 加速大模型加载
04bd8a1 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
YdrMaster and others added 9 commits December 20, 2024 11:29
@YdrMaster
Merge commit '04bd8a1b79b0f00c4ee10f8ead056301f6e4e1a1' into gpt2
060f156
@YdrMaster
style: code cleanup
dc987d9 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
fix(tokenizer): 修改空格替换机制，支持 gpt
96fe9c7 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
style(gpt2): 整理 Cargo.toml
512d235 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@onenewcode
refactor(gpt2): 重构mlp和修改部分读取参数的方法
edefc42
@onenewcode
Merge branch 'InfiniTensor:dev' into dev
ca3fec5
@YdrMaster
Merge pull request #28 from onenewcode/dev
8b270ed 
refactor(gpt2): 重构mlp和修改部分读取参数的方法
@YdrMaster
style(gpt2): cleanup code
a68552c 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
fix(nv): 解决多用户同时使用的冲突问题、tokenizer 修改的错误、支持 nv 单卡选卡号
d09838b 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Dec 24, 2024
View reviewed changes
models/clip/common-cpu/src/lib.rs Fixed Show fixed Hide fixed
models/clip/common-cpu/src/lib.rs Fixed Show fixed Hide fixed
models/clip/common-cpu/src/lib.rs Fixed Show fixed Hide fixed
@YdrMaster YdrMaster force-pushed the dev branch 2 times, most recently from d7cd3d6 to cc627dc Compare December 26, 2024 03:34
@YdrMaster
style(gpt2): 整理代码，与 clip 对齐
430044c 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat(llama): 支持加载 MOE llama
69bc397 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
refactor(llama): 拆散 mlp，使用移除 mlp 的算子库，准备实现 moe
8286ecb 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat(llama): 完成 MOE gete inp 层
6d3cf32 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat(llama): 初步实现 llama-moe
e14c588 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
style(llama): 整理和优化 MOE 模型结构
2896798 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat(llama): 实现支持 MOE 的分布式切分及线程间张量并行
e984da5 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
feat(llama): 完成 llama MOE nv 分布式推理
4247ad5 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
fix(llama-cl): fix for opencl
4a19e67 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster
docs: update README.md
b912f45 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
@YdrMaster YdrMaster closed this Dec 30, 2024
@YdrMaster YdrMaster deleted the dev branch December 30, 2024 03:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@YdrMaster YdrMaster

Labels
enhancement New feature or request in progress Still working on it
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@YdrMaster @pwhMass @CearX @onenewcode