robust scanner fusion question

[Goghzki]

hi~, I think the robust_scanner_fusion_layer.py is not the original method in is's parper https://arxiv.org/pdf/2007.07542.pdf
In git:
output = self.linear_layer(fusion_input)
output = self.glu_layer(output)
In parper :
->
output1 = self.linear_layer(fusion_input)
output1 = torch.sigmoid(output1)
output2 = self.linear_layer(fusion_input)
output = torch.mul(output1 , output2 )

Actually In my sentence level dataset, I found many alignment drift case, which should be fixed by robust_scanner_fusion_layer.
I don't know is this right?

[innerlee]

@cuhk-hbsun Pytorch has an implementation of glu, maybe we can use it.

[Goghzki]

mabey it is different?
torch split the input in to two parts a and b, and use sigmoid function only to b(not both) , then mul a and sigmoid b