1.Multi-Dconv Head Transposed Attention 论文:https://arxiv.org/abs/2111.09881 1.1 MDTA Transformer中计算量主要来自于注意力计算部分,为了降低计算量,作者构建了MDTA,不在像素维度计算 attention,而是在通道维度计算。过程很简单,先用 point-wise conv 和 dconv 预处理,在通道维计算 atteniton,如下图所示。
- 为促进多尺度全局局部表达学习,Restormer改进了Transformer块(通过门控Dconv网络和多Dconv头部注意力转置模块)。下图分别为Gated-Dconv feed-forward network (GDFN)和multi-Dconv head transposed attention (MDTA) 在这里插入图片描述 在这里插入图片描述 2.3 INN ①介绍:INN是标准化流模型(一种生成模型)的...
Furthermore, this paper integrates a RefConv and Multi-Dconv Head Transposed Attention (RMA) feature pyramid structure into the HRNet model, augmenting the model’s capacity for semantic recognition and expression at various levels. Last, the incorporation of the Dilated Efficient Multi-Scale ...