Skip to content

Commit

Permalink
fix silu commet
Browse files Browse the repository at this point in the history
  • Loading branch information
PanZezhong1725 committed Aug 2, 2024
1 parent 3166e0f commit 2346a65
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
请在`src/operators.rs`中实现SiLU算子,其公式为:

$$
y=silu(y) × x
y=silu(x) × y
$$

其中
Expand Down
2 changes: 1 addition & 1 deletion src/operators.rs
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ pub fn rms_norm(y: &mut Tensor<f32>, x: &Tensor<f32>, w: &Tensor<f32>, epsilon:
todo!("实现 rms_norm,计算前做一些必要的检查会帮助你后续调试")
}

// y = sigmoid(y) * y * x
// y = sigmoid(x) * x * y
// hint: this is an element-wise operation
pub fn silu(y: &mut Tensor<f32>, x: &Tensor<f32>) {
// let len = y.size();
Expand Down

0 comments on commit 2346a65

Please sign in to comment.