REsponsible & Accountable Learning (REAL)
@ University of California, Santa Cruz

Leaderboard

Rank Method Accuracy Method and Paper
1 CORES* 95.25±0.09 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
2 Divide-Mix 95.01±0.71 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
3 ELR+ 94.83±0.10 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
4 PES(semi) 94.66±0.18 Understanding and Improving Early Stopping for Learning with Noisy Labels.  (Code)
5 ELR 92.38±0.64 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
6 CAL 91.97±0.32 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
7 Negative-LS 91.97±0.46 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
8 F-div 91.64±0.34 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
9 Positive-LS 91.57±0.07 Does Label Smoothing Mitigate Label Noise?
10 JoCoR 91.44±0.05 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
11 CORES 91.23±0.11 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
12 Co-Teaching 91.20±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
13 Peer Loss 90.75±0.25 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
14 Co-Teaching+ 90.61±0.22 How does Disagreement Help Generalization against Label Corruption?  (Code)
15 VolMinNet 89.70±0.21 Provably end-to-end label-noise learning without anchor points.  (Code)
16 T-Revision 88.52±0.17 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
17 Forward-T 88.24±0.22 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
18 Backward-T 88.13±0.29 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
19 GCE 87.85±0.70 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
20 CE 87.77±0.38
Rank Method Accuracy Method and Paper
1 Divide-Mix 95.16±0.19 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
2 PES(semi) 95.06±0.15 Understanding and Improving Early Stopping for Learning with Noisy Labels.  (Code)
3 CORES* 94.45±0.14 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
4 ELR+ 94.43±0.41 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
5 ELR 91.46±0.38 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
6 CAL 90.93±0.31 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
7 Co-Teaching 90.33±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
8 JoCoR 90.30±0.20 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
9 Negative-LS 90.29±0.32 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
10 Positive-LS 89.80±0.28 Does Label Smoothing Mitigate Label Noise?
11 Co-Teaching+ 89.70±0.27 How does Disagreement Help Generalization against Label Corruption?  (Code)
12 F-div 89.70±0.40 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
13 CORES 89.66±0.32 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
14 Peer Loss 89.06±0.11 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
15 T-Revision 88.33±0.32 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
16 VolMinNet 88.30±0.12 Provably end-to-end label-noise learning without anchor points.  (Code)
17 GCE 87.61±0.28 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
18 Backward-T 87.14±0.34 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
19 Forward-T 86.88±0.50 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
20 CE 85.02±0.65
Rank Method Accuracy Method and Paper
1 Divide-Mix 95.23±0.07 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
2 PES(semi) 95.19±0.23 Understanding and Improving Early Stopping for Learning with Noisy Labels.  (Code)
3 CORES* 94.88±0.31 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
4 ELR+ 94.20±0.24 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
5 ELR 91.61±0.16 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
6 CAL 90.75±0.30 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
7 Negative-LS 90.37±0.12 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
8 Co-Teaching 90.30±0.17 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
9 JoCoR 90.21±0.19 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
10 CORES 89.91±0.45 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
11 F-div 89.79±0.12 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
12 Co-Teaching+ 89.47±0.18 How does Disagreement Help Generalization against Label Corruption?  (Code)
13 Positive-LS 89.35±0.33 Does Label Smoothing Mitigate Label Noise?
14 Peer Loss 88.76±0.19 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
15 VolMinNet 88.27±0.09 Provably end-to-end label-noise learning without anchor points.  (Code)
16 T-Revision 87.71±1.02 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
17 GCE 87.70±0.56 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
18 CE 86.46±1.79
19 Backward-T 86.28±0.80 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
20 Forward-T 86.14±0.24 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
Rank Method Accuracy Method and Paper
1 PES(semi) 95.22±0.13 Understanding and Improving Early Stopping for Learning with Noisy Labels.  (Code)
2 Divide-Mix 95.21±0.14 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
3 CORES* 94.74±0.03 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
4 ELR+ 94.34±0.22 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
5 ELR 91.41±0.44 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
6 CAL 90.74±0.24 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
7 Co-Teaching 90.15±0.18 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
8 Negative-LS 90.13±0.19 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
9 JoCoR 90.11±0.21 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
10 Positive-LS 89.82±0.14 Does Label Smoothing Mitigate Label Noise?
11 CORES 89.79±0.50 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
12 F-div 89.55±0.49 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
13 Co-Teaching+ 89.54±0.22 How does Disagreement Help Generalization against Label Corruption?  (Code)
14 Peer Loss 88.57±0.09 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
15 VolMinNet 88.19±0.41 Provably end-to-end label-noise learning without anchor points.  (Code)
16 T-Revision 87.79±0.67 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
17 GCE 87.58±0.29 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
18 Forward-T 87.04±0.35 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
19 Backward-T 86.86±0.41 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
20 CE 85.16±0.61
Rank Method Accuracy Method and Paper
1 PES(semi) 92.68±0.22 Understanding and Improving Early Stopping for Learning with Noisy Labels.  (Code)
2 Divide-Mix 92.56±0.42 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
3 CORES* 91.66±0.09 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
4 ELR+ 91.09±1.60 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
5 CAL 85.36±0.16 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
6 Co-Teaching 83.83±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
7 CORES 83.60±0.53 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
8 ELR 83.58±1.13 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
9 JoCoR 83.37±0.30 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
10 Co-Teaching+ 83.26±0.17 How does Disagreement Help Generalization against Label Corruption?  (Code)
11 Negative-LS 82.99±0.36 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
12 Positive-LS 82.76±0.53 Does Label Smoothing Mitigate Label Noise?
13 F-div 82.53±0.52 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
14 Peer Loss 82.53±0.52 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
15 GCE 80.66±0.35 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
16 VolMinNet 80.53±0.20 Provably end-to-end label-noise learning without anchor points.  (Code)
17 T-Revision 80.48±1.20 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
18 Forward-T 79.79±0.46 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
19 CE 77.69±1.55
20 Backward-T 77.61±1.05 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
Rank Method Accuracy Method and Paper
1 Divide-Mix 71.13±0.48 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
2 PES(semi) 70.36±0.33 Understanding and Improving Early Stopping for Learning with Noisy Labels.  (Code)
3 ELR+ 66.72±0.07 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
4 CAL 61.73±0.42 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
5 CORES 61.15±0.73 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
6 Co-Teaching 60.37±0.27 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
7 JoCoR 59.97±0.24 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
8 ELR 58.94±0.92 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
9 Negative-LS 58.59±0.98 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
10 Co-Teaching+ 57.88±0.24 How does Disagreement Help Generalization against Label Corruption?  (Code)
11 VolMinNet 57.80±0.31 Provably end-to-end label-noise learning without anchor points.  (Code)
12 Peer Loss 57.59±0.61 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
13 Backward-T 57.14±0.92 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
14 F-div 57.10±0.65 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
15 Forward-T 57.01±1.03 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
16 GCE 56.73±0.30 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
17 Positive-LS 55.84±0.48 Does Label Smoothing Mitigate Label Noise?
18 CORES* 55.72±0.42 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
19 CE 55.50±0.66
20 T-Revision 51.55±0.31 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)

Contact

Please contact us via {yangliu, jiahengwei, zhaoweizhu}@ucsc.edu, if you want to add your methods onto the leaderboard.