attention.md

Attention

SENet

Model	Size	Augmentation	Epoch	Top 1	Top 5	Time	Code	Log
ResNet_vd-50	160	Standard	120	78.93	94.48	208.1	23	1
ResNet_vd-50	224	Standard	120	78.86	94.28	377.2	101	1 2 3

• Squeeze-and-Excitation Networks

ECA-Net

Model	Size	Augmentation	Epoch	Top 1	Top 5	Time	Code	Log
ResNet_vd-50	160	Standard	120	79.24	94.67	203.1	24	1
RegNetY-1.6GF (C)	160	Standard	100	77.27	93.46	160.9	35	1
RegNetY-1.6GF (F)	160	Standard	100	77.55	93.83	159.7	38	1

• ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
• ECA-Net seems to not work well in RegNet, no matter put it at the middle conv (C) or at the final of block (F).

Reference

Model	Size	Augmentation	Epoch	Top 1	Top 5	Time	Code	Log
ResNet_vd-50	160	Standard	120	78.54	94.33	192.1	50	1 2 3
RegNetY-1.6GF	160	Standard	100	77.86	94.01	161.8	32	1