Datasets with biases and other related work

[boknilev]

A running issue to keep track of other tasks and datasets with biases, which may be amenable to a similar methodology.

• Stance Detection Benchmark: How Robust Is Your Stance Detection?

Other relevant papers:

• Adversarial Filters of Dataset Biases - theoretical grounding of adversarial data filtering, experiments on real and synthetic datasets.
• HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference
• Syntactic Data Augmentation Increases Robustness to Inference Heuristics
• Towards Robustifying NLI Models Against Lexical Dataset Biases
• Modular Representation Underlies Systematic Generalization in Neural Natural Language Inference Models
• Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance
• An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
• Towards Debiasing NLU Models from Unknown Biases
• An Empirical Study on Model-agnostic Debiasing Strategies for Robust Natural Language Inference
• Gone at Last: Removing the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training
• Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models
• Modeling the Second Player in Distributionally Robust Optimization
• Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers

[boknilev]

Other related work:

• Predictive Biases in Natural Language Processing Models: A Conceptual
  Framework and Overview
• Undersensitivity in Neural Reading Comprehension
• Counterfactual Samples Synthesizing for Robust Visual Question Answering
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision
• Benchmarking Robustness of Machine Reading Comprehension Models
• Shortcut Learning in Deep Neural Networks
• Language (Technology) is Power: A Critical Survey of "Bias" in NLP
• An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
• Intrinsic Bias Metrics Do Not Correlate with Application Bias
• Challenges in Automated Debiasing for Toxic Language Detection
• An Investigation of Critical Issues in Bias Mitigation Techniques
• Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization