Pixel-Wise Contrastive Loss 基于上述分析,文章提出了pixel-wise contrastive distillation(PCD),让student在像素级别上向老师学习。蒸馏的loss如下所示: 其中\ell表示contrastive loss,下文会有更详细的形式。s_i和t_i分别表示student和teacher的feature maps的第i个像素(文章假设了student和teacher的输出feature maps具有...
设计额外的loss ,只对\theta_{rpl}进行训练 (经过实验\alpha=0.05) 其中对于inlier的pixel: 其中对于OoD的pixel: 此loss的优势: 1. 没有超参数; 2. Inlier pixel数量 >> OoD pixel数量, 这种loss由于其他的loss比如hinge loss,对OoD pixel有更好的优化和收敛效果. Context-robust Contrastive Learning(CoroCL)...
Inspired by metric learning, we construct the pixel-level pairwise samples and propose a new self-supervised contrastive loss based on them, which makes full use of the class activation maps to reduce the intra-class difference and increase the inter-class difference; we also propose a novel ...
between predictions and ground-true objects: (1) cross-entropy loss between the predicted and ground-true classes; (2) focal loss [80] between the predicted center heatmaps and ground-true center heatmaps; (3) size loss computed by the L1 loss between predicted and ground true size....
Specifically, they trained a network twice by pretraining a model based on label-based contrastive learning [62] first, and then fine-tuning the model with cross-entropy loss. Unlike the method described in [61], the proposed method does not require any pretraining. 2.2. Regularization ...
Inspired by metric learning, we construct the pixel-level pairwise samples and propose a new self-supervised contrastive loss based on them, which makes full use of the class activation maps to reduce the intra-class difference and increase the inter-class difference; we also propose a novel ...
contrastive cycle-consistency loss on the level of pixels. Fi- nally, [56] performs image-to-image translation for UDA in frequency space rather than pixel space using a Fourier transform. Beyond cycle-consistency, [12] enforces cross-domain consistent predic...
to obtain final global negative embeddings, and finding local hard negative embeddings using the global and local similarity maps, and randomly sampling final local negative embeddings from the local hard negative embeddings; and minimizing a final info noise contrastive estimation (InfoNCE) loss.KE...
to obtain final global negative embeddings, and finding local hard negative embeddings using the global and local similarity maps, and randomly sampling final local negative embeddings from the local hard negative embeddings; and minimizing a final info noise contrastive estimation (InfoNCE) loss.YAN...
to obtain final global negative embeddings, and finding local hard negative embeddings using the global and local similarity maps, and randomly sampling final local negative embeddings from the local hard negative embeddings; and minimizing a final info noise contrastive estimation (InfoNCE) loss....