Swedish version of the SuperGLUE diagnostic dataset.

Manual translation of the SuperGLUE Diagnostic Dataset. The data includes all annotated original sentence pairs of SuperGLUE and their Swedish translations.

License: Creative Commons CC-BY 4.0 International (please refer to Språkbanken, University of Gothenburg, Sweden).

I. IDENTIFYING INFORMATION
Title* SweDiagnostics
Subtitle
Created by* Felix Morger, Gothenburg University (felix.morger@gu.se)
Publisher(s)* Språkbanken Text (sb-info@svenska.gu.se)
Link(s) / permanent identifier(s)* https://spraakbanken.gu.se/en/resources/superlim
License(s)* CC BY 4.0
Abstract* Manual Swedish translation of all 1106 sentence pairs of the SuperGLUE diagnostic dataset.
Funded by* Vinnova (grant no. 2020-02523)
Cite as
Related datasets SuperLim, SuperGLUE diagnostic dataset, FraCaS test suite
II. USAGE
Key applications Fine-grained analysis of system performance on a broad range of linguistic phenomena.
Intended task(s)/usage(s) Natural language inference.
Recommended evaluation measures Matthew's correlation coefficient.
Dataset function(s)
Recommended split(s) No split.
III. DATA
Primary data* Text
Language* Swedish
Dataset in numbers* 1106
Nature of the content* Pairs of sentences annotated according with their inference relation and the linguistic phenomena that account for their differencs
Format* Comma-separated
Data source(s)* SuperGLUE Diagnostic Dataset: Pruksachatkun, Yada & Nangia, Nikita & Singh, Amanpreet & Michael, Julian & Hill, Felix & Levy, Omer & Bowman, Samuel. (2019). SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems.
Data collection method(s)* See original source.
Data selection and filtering* See original source.
Data preprocessing* See original source.
Data labeling* Some data labels (annotations) were changed to fit with Swedish example, but in general the aim was to keep such changes to a minimum.
Annotator characteristics
IV. ETHICS AND CAVEATS
Ethical considerations See original data source.
Things to watch out for See original data source.
V. ABOUT DOCUMENTATION
Data last updated* 2021-06-04, v1.0
Which changes have been made, compared to the previous version* Full translation coverage.
Access to previous versions
This document created* 2021-06-04, Felix Morger.
This document last updated* 2021-06-04, Felix Morger.
Where to look for further details
Documentation template version* 1
VI. OTHER
Related projects
References