关键词: 自蒸馏学习, 并行结构网络, 多尺度池化金字塔, 结构化相似性, 目标分割

Abstract:

The knowledge distillation can effectively transfer the representation ability of a teacher network to a student network, and improve the performance of the network without changing the network structure. Therefore, it is of great significance to construct a self-distillation learning model in the backbone network of the HRNet (High-Resolution Net)with an excellent performance in the object segmentation tasks. Aiming to the challenge that parallel integration architecture of deep and shallow information in HRNet makes direct distillation difficult to achieve, a structured self-distillation learning framework based on multi-scale pooling pyramid is proposed in this paper. Firstly, the multiscale pooling pyramid feature modules are introduced into the branch structure in the HRNet to improve knowledge representation and learning ability. Secondly, the top-down and consistency distillation modes are constructed. Meanwhile the cross entropy loss, KL (Kullback-Leibler)divergence loss and structural similarity loss are combined for the self-distillation learning framework. The experiments on four segmentation datasets including saliency and camouflaged objects demonstrate that the proposed model improves the performance of the object segmentation of the network without increasing resource costs.

Key words: knowledge distillation, parallel network, multi-scale pooling pyramid, structural similarity, object segmentation