1.首先我们将测试阶段的BN层(一般称为frozen BN)等效替换为一个1x1卷积层 2.将卷积层与归一化层融合 pytorch-BN融合 importtorchimporttorchvisiondeffuse(conv,bn):fused=torch.nn.Conv2d(conv.in_channels,conv.out_channels,kernel_size=conv.kernel_size,stride=conv.stride,padding=conv.padding,bias=True)# ...
BN层的融合公式如下: X b n = X ∗ s W σ +ϵ+ s ( b c o n v − m ) σ +ϵ+ b b n X_{bn} = X \frac{sW}{\sqrt{\sigma + \epsilon}} + \frac{s(b_{conv} - m)}{\sqrt{\sigma + \epsilon}} + b_{bn}Xbn =X∗σ+ϵsW +σ+ϵs(bconv −m) +bbn...