Publications

Show years

2024 21
2023 26
2022 21
2021 29

2024

The INLG 2024 Tutorial on Human Evaluation of NLP System Quality: Background, Overall Aims, and Summaries of Taught Units

A Belz, J Sedoc, C Thomson, S Mille, R Huidrom

Proceedings of the 17th International Natural Language Generation Conference …, 2024 (2024)

Paper

The 2024 ReproNLP Shared Task on Reproducibility of Evaluations in NLP: Overview and Results

A Belz, C Thomson

Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems …, 2024 (2024)

Paper

Reproducing the metric-based evaluation of a set of controllable text generation techniques

M Lorandi, A Belz

arXiv preprint arXiv:2405.07875, 2024 (2024)

Paper

Qcet: An interactive taxonomy of quality criteria for comparable and repeatable evaluation of NLP systems

A Belz, S Mille, C Thomson, R Huidrom

Proceedings of the 17th International Natural Language Generation Conference …, 2024 (2024)

Paper

Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems (HumEval)@ LREC-COLING 2024

S Balloccu, A Belz, R Huidrom, E Reiter, J Sedoc, C Thomson

Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems …, 2024 (2024)

Paper

Proceedings of the 2nd Workshop on Practical LLM-assisted Data-to-Text Generation

S Balloccu, Z Kasner, O Plátek, P Schmidtová, K Onderková, M Lango, ...

Proceedings of the 2nd Workshop on Practical LLM-assisted Data-to-Text …, 2024 (2024)

Paper

Proceedings of the 17th International Natural Language Generation Conference: Tutorial Abstract

A Belz, J Sedo, C Thomson, S Mille, R Huidrom

Proceedings of the 17th International Natural Language Generation Conference …, 2024 (2024)

Paper

Preliminary wmt24 ranking of general mt systems and llms

T Kocmi, E Avramidis, R Bawden, O Bojar, A Dvorkovich, C Federmann, ...

arXiv preprint arXiv:2407.19884, 2024 (2024)

Paper

On the Role of Summary Content Units in Text Summarization Evaluation

M Nawrath, A Nowak, T Ratz, DC Walenta, J Opitz, LFR Ribeiro, J Sedoc, ...

arXiv preprint arXiv:2404.01701, 2024 (2024)

Paper

High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Models

M Lorandi, A Belz

arXiv preprint arXiv:2402.12267, 2024 (2024)

Paper

Findings of the WMT24 general machine translation shared task: the LLM era is here but mt is not solved yet

T Kocmi, E Avramidis, R Bawden, O Bojar, A Dvorkovich, C Federmann, ...

Proceedings of the Ninth Conference on Machine Translation, 1-46, 2024 (2024)

Paper

Filling Gaps in Wikipedia: Leveraging Data-to-Text Generation to Improve Encyclopedic Coverage of Underrepresented Groups

S Mille, M Pronesti, C Thomson, M Lorandi, S Fitzpatrick, R Huidrom, ...

Proceedings of the 17th International Natural Language Generation Conference …, 2024 (2024)

Paper

Differences in Semantic Errors Made by Different Types of Data-to-text Systems

R Huidrom, A Belz, M Lorandi

Proceedings of the 17th International Natural Language Generation Conference …, 2024 (2024)

Paper

DCU-NLG-Small at the GEM'24 Data-to-Text Task: Rule-based generation and post-processing with T5-Base

S Mille, M Sabry, A Belz

Proceedings of the 17th International Natural Language Generation Conference, 2024 (2024)

Paper

DCU-NLG-PBN at the GEM'24 Data-to-Text Task: Open-Source LLM PEFT-Tuning for Effective Data-to-Text Generation

M Lorandi, A Belz

Proceedings of the 17th International Natural Language Generation Conference, 2024 (2024)

Paper

DCU ADAPT at WMT24: English to Low-resource Multi-Modal Translation Task

S Haq, R Huidrom, S Castilho

Proceedings of the Ninth Conference on Machine Translation, 810-814, 2024 (2024)

Paper

Common Flaws in Running Human Evaluation Experiments in NLP

C Thomson, E Reiter, A Belz

Computational Linguistics, 1-11, 2024 (2024)

Paper

Beyond Abstracts: A New Dataset, Prompt Design Strategy and Method for Biomedical Synthesis Generation

J O’Doherty, C Nolan, Y Hou, A Belz

Proceedings of the 62nd Annual Meeting of the Association for Computational …, 2024 (2024)

Paper

ATHAR: A High-Quality and Diverse Dataset for Classical Arabic to English Translation

M Khalil, M Sabry

arXiv preprint arXiv:2407.19835, 2024 (2024)

Paper

Assessing the Portability of Parameter Matrices Trained by Parameter-Efficient Finetuning Methods

M Sabry, A Belz

Findings of the Association for Computational Linguistics: EACL 2024, 1548-1556, 2024 (2024)

Paper

(Mostly) Automatic Experiment Execution for Human Evaluations of NLP Systems

C Thomson, A Belz

Proceedings of the 17th International Natural Language Generation Conference …, 2024 (2024)

Paper

2023

Using MT for multilingual covid-19 case load prediction from social media texts

M Popović, V Nedumpozhimana, M Gower, S Rautmare, N Jain, ...

Proceedings of the 24th Annual Conference of the European Association for …, 2023 (2023)

Paper

Towards a Consensus Taxonomy for Annotating Errors in Automatically Generated Text

R Huidrom, A Belz

Proceedings of the 14th International Conference on Recent Advances in …, 2023 (2023)

Paper

The 2023 webnlg shared task on low resource languages overview and evaluation results (webnlg 2023)

L Cripwell, A Belz, C Gardent, A Gatt, C Borg, M Borg, J Judge, M Lorandi, ...

Proceedings of the Workshop on Multimodal, Multilingual Natural Language …, 2023 (2023)

Paper

The 2023 ReproNLP Shared Task on Reproducibility of Evaluations in NLP: Overview and Results

A Belz, C Thomson

The Third Workhop on Human Evaluation of NLP Systems, 2023 (2023)

Paper

Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems

A Belz, M Popović, E Reiter, C Thomson, J Sedoc

Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems, 2023 (2023)

Paper

Proceedings of the 16th International Natural Language Generation Conference: Generation Challenges

S Mille

Proceedings of the 16th International Natural Language Generation Conference …, 2023 (2023)

Paper

Peft-ref: A modular reference architecture and typology for parameter-efficient finetuning techniques

M Sabry, A Belz

arXiv preprint arXiv:2304.12410, 2023 (2023)

Paper

Non-repeatable experiments and non-reproducible results: The reproducibility crisis in human evaluation in NLP

A Belz, C Thomson, E Reiter, S Mille

Findings of the Association for Computational Linguistics: ACL 2023, 3676-3687, 2023 (2023)

Paper

Mod-D2T: A Multi-layer Dataset for Modular Data-to-Text Generation

S Mille, F Lareau, S Dasiopoulou, A Belz

Proceedings of the 16th International Natural Language Generation Conference …, 2023 (2023)

Paper

Missing information, unresponsive authors, experimental flaws: The impossibility of assessing the reproducibility of previous human evaluations in NLP

A Belz, C Thomson, E Reiter, G Abercrombie, JM Alonso-Moral, M Arvan, ...

arXiv preprint arXiv:2305.01633, 2023 (2023)

Paper

Migrant communities living in the Netherlands and their use of MT in health contexts

S Valdez, AG Arenas, K Ligtenberg

24th Annual Conference of the European Association for Machine Translation …, 2023 (2023)

Paper

Medical Concept Mention Identification in Social Media Posts using a Small Number of Sample References

V Nedumpozhimana, S Rautmare, M Gower, M Popovic, N Jain, P Buffini, ...

Technological University Dublin, 2023 (2023)

Paper

How to Control Sentiment in Text Generation: A Survey of the State-of-the-Art in Sentiment-Control Techniques

M Lorandi, A Belz

Proceedings of the 13th Workshop on Computational Approaches to Subjectivity …, 2023 (2023)

Paper

Generating Irish Text with a Flexible Plug-and-Play Architecture

S Mille, EU Dhonnchadha, L Cassidy, B Davis, S Dasiopoulou, A Belz

Proceedings of the 2nd Workshop on Pattern-based Approaches to NLP in the …, 2023 (2023)

Paper

Exploring Variation of Results from Different Experimental Conditions

M Popović, M Arvan, N Parde, A Belz

Findings of the Association for Computational Linguistics: ACL 2023, 2746-2757, 2023 (2023)

Paper

Evaluating factual accuracy in complex data-to-text

C Thomson, E Reiter, B Sundararajan

Computer Speech & Language 80, 101482, 2023 (2023)

Paper

Enhancing factualness and controllability of Data-to-Text Generation via data Views and constraints

C Thomson, C Rebuffel, E Reiter, L Soulier, S Sripada, P Gallinari

Proceedings of the 16th international natural language generation conference …, 2023 (2023)

Paper

Do Humans Translate Like Machines? Students’ Conceptualisations of Human and Machine Translation

S Leena, AG Dorst, M Koponen, K Zeven

Proceedings of the 24th Annual Conference of the European Association for …, 2023 (2023)

Paper

DCU/TCD-FORGe at WebNLG’23: Irish rules!

AB Simon Mille, Elaine Uí Dhonnchadha, Stamatia Dasiopoulou, Lauren Cassidy ...

Workshop on Multimodal, Multilingual Natural Language Generation (MM-NLG'23), 2023 (2023)

Paper

DCU/TCD-FORGe at WebNLG’23: Irish rules!(WegNLG 2023)

S Mille, EU Dhonnchadha, S Dasiopoulou, L Cassidy, B Davis, A Belz

Proceedings of the Workshop on Multimodal, Multilingual Natural Language …, 2023 (2023)

Paper

Data-to-text generation for severely under-resourced languages with gpt-3.5: A bit of help needed from google translate

M Lorandi, A Belz

arXiv preprint arXiv:2308.09957, 2023 (2023)

Paper

Computational analysis of different translations: by professionals, students and machines

M Popović, E Lapshinova-Koltunski, M Koponen

European Association for Machine Translation (EAMT), 2023 (2023)

Paper

Barriers and enabling factors for error analysis in NLG research

E Van Miltenburg, M Clinciu, O Dušek, D Gkatzia, S Inglis, L Leppänen, ...

Northern European Journal of Language Technology 9 (1), 2023 (2023)

Paper

Adapting the CycleGAN architecture for text style transfer

M Lorandi, A Mohamed, K McGuinness

Zenodo, 2023 (2023)

Paper

A Pipeline for Extracting Abstract Dependency Templates for Data-to-Text Natural Language Generation

S Mille, J Ricci, A Shvets, A Belz

Proceedings of the Seventh International Conference on Dependency …, 2023 (2023)

Paper

A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization

L Zhang, S Mille, Y Hou, D Deutsch, E Clark, Y Liu, S Mahamood, ...

Proceedings of the 61st Annual Meeting of the Association for Computational …, 2023 (2023)

Paper

2022

User-driven Development of a Medical Note Generation System

T Knoll, F Moramarco, RY Alex Papadopoulos Korfiatis, C Ruffini, ...

Proceedings of the 20th Annual Conference of the North American Chapter of …, 2022 (2022)

Paper

Two Reproductions of a Human-Assessed Comparative Evaluation of a Semantic Error Detection System

R Huidrom, O Dusek, Z Kasner, T Castro Ferrera, A Belz

International Natural Language Generation Conference, 2022 (2022)

Paper

The Human Evaluation Datasheet: A Template for Recording Details of Human Evaluation Experiments in NLP

A Shimorina, A Belz

2nd Workshop on Human Evaluation of NLP Systems, 2022 (2022)

Paper

The accuracy evaluation shared task as a retrospective reproduction study

C Thomson, E Reiter

Proceedings of the 15th International Conference on Natural Language …, 2022 (2022)

Paper

The 2022 ReproGen shared task on reproducibility of evaluations in NLG: Overview and results

A Belz, A Shimorina, M Popović, E Reiter

Association for Computational Linguistics (ACL), 2022 (2022)

Paper

Reproducing a Manual Evaluation of Simplicity in Text Simplification System Outputs

M Popovic, R Huidrom, S Castilho, A Belz

International Natural Language Generation Conference, 2022 (2022)

Paper

Quantified Reproducibility Assessment of NLP Results

A Belz, M Popovic, S Mille

Proceedings of the 60th Annual Meeting of the Association for Computational …, 2022 (2022)

Paper

Proceedings of the 2nd Workshop on Human Evaluation of NLP Systems (HumEval)

A Belz, M Popović, E Reiter, A Shimorina

Proceedings of the 2nd Workshop on Human Evaluation of NLP Systems (HumEval), 2022 (2022)

Paper

On reporting scores and agreement for error annotation tasks

M Popović, A Belz

Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation …, 2022 (2022)

Paper

Leveraging pre-trained language models for gender debiasing

N Jain, M Popović, D Groves, L Specia

European Language Resources Association (ELRA), 2022 (2022)

Paper

Introducing EM-FT for Manipuri-English Neural Machine Translation

R Huidrom, Y Lepage

Proceedings of the WILDRE-6 Workshop @LREC2022, 1-6, 2022 (2022)

Paper

Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation

F Moramarco, AP Korfiatis, M Perera, D Juric, J Flann, E Reiter, A Belz, ...

Proceedings of the 60th Annual Meeting of the Association for Computational …, 2022 (2022)

Paper

GEMv2: Multilingual NLG Benchmarking in a Single Line of Code

S Gehrmann, A Bhattacharjee, A Mahendiran, A Wang, A Papangelis, ...

arXiv preprint arXiv:2206.11249, 2022 (2022)

Paper

Findings of the 2022 conference on machine translation (WMT22)

T Kocmi, R Bawden, O Bojar, A Dvorkovich, C Federmann, M Fishel, ...

Proceedings of the Seventh Conference on Machine Translation (WMT), 1-45, 2022 (2022)

Paper

Evaluation of Response Generation Models: Shouldn’t It Be Shareable and Replicable?

SM Mousavi, G Roccabruna, M Lorandi, S Caldarella, G Riccardi

Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation …, 2022 (2022)

Paper

DiHuTra: a Parallel Corpus to Analyse Differences between Human Translations

MK Ekaterina Lapshinova-Koltunski, Maja Popović

Proceedings of the 23rd Annual Conference of the European Association for …, 2022 (2022)

Paper

Consultation Checklists: Standardising the Human Evaluation of Medical Note Generation

A Savkov, F Moramarco, AP Korfiatis, M Perera, A Belz, E Reiter

arXiv preprint arXiv:2211.09455, 2022 (2022)

Paper

Building machine translation system for software product descriptions using domain-specific sub-corpora extraction

P Lohar, M Popović, T Habruseva

Association for Machine Translation in the Americas, 2022 (2022)

Paper

Automatic Multilingual Incident Report Generation for Crisis Management.

S Mille, G Casamayor, J Grivolla, AV Shvets, L Wanner

ISCRAM, 299-309, 2022 (2022)

Paper

A survey of recent error annotation schemes for automatically generated text

R Huidrom, A Belz

Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation …, 2022 (2022)

Paper

A Metrological Perspective on Reproducibility in NLP*

A Belz

Computational Linguistics 48 (4), 1125-1135, 2022 (2022)

Paper

2021

xR4DRAMA: Enhancing situation awareness using immersive (XR) technologies

S Symeonidis, S Diplaris, N Heise, T Pistola, A Tsanousa, G Tzanetis, ...

2021 IEEE International Conference on Intelligent Reality (ICIR), 1-8, 2021 (2021)

Paper

Underreporting of errors in NLG output, and what to do about it

E Van Miltenburg, MA Clinciu, O Dušek, D Gkatzia, S Inglis, L Leppänen, ...

arXiv preprint arXiv:2108.01182, 2021 (2021)

Paper

The ReproGen Shared Task on Reproducibility of Human Evaluations in NLG: Overview and Results

A Belz, A Shimorina, S Agarwal, E Reiter

Proceedings of the 14th International Natural Language Generation Conference …, 2021 (2021)

Paper

The human evaluation datasheet 1.0: A template for recording details of human evaluation experiments in nlp

A Shimorina, A Belz

arXiv preprint arXiv:2103.09710, 2021 (2021)

Paper

The gem benchmark: Natural language generation, its evaluation and metrics

S Gehrmann, T Adewumi, K Aggarwal, PS Ammanamanchi, ...

arXiv preprint arXiv:2102.01672, 2021 (2021)

Paper

Text-in-context: Token-level error detection for table-to-text generation

Z Kasner, S Mille, O Dušek

Proceedings of the 14th International Conference on Natural Language …, 2021 (2021)

Paper

Quantifying reproducibility in NLP and ML

A Belz

arXiv preprint arXiv:2109.01211, 2021 (2021)

Paper

Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval)

A Belz, S Agarwal, Y Graham, E Reiter, A Shimorina

Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval), 2021 (2021)

Paper

Proceedings of the Sixth International Conference on Dependency Linguistics (Depling, SyntaxFest 2021)

N Mazziotta, S Mille

Association for Computational Linguistics, Sofia, Bulgaria, 2021 (2021)

Paper

Proceedings of the 14th International Conference on Natural Language Generation

A Belz, A Fan, E Reiter, Y Sripada

Proceedings of the 14th International Conference on Natural Language Generation, 2021 (2021)

Paper

Paraphrase and parallel treebank for the comparison of French and Chinese syntax

R Poiret, S Mille, H Liu

Languages in Contrast 21 (2), 298-322, 2021 (2021)

Paper

On nature and causes of observed MT errors

M Popović

Proceedings of machine translation summit XVIII: Research track, 163-175, 2021 (2021)

Paper

On machine translation of user reviews

M Popović, A Poncelas, M Brkić, A Way

Proceedings of the International Conference on Recent Advances in Natural …, 2021 (2021)

Paper

Nl-augmenter: A framework for task-sensitive natural language augmentation

KD Dhole, V Gangal, S Gehrmann, A Gupta, Z Li, S Mahamood, ...

arXiv preprint arXiv:2112.02721, 2021 (2021)

Paper

Machine Translation Quality of User Reviews

M Popovic

(2021)

Paper

Genetic improvement of routing protocols for delay tolerant networks

M Lorandi, LL Custode, G Iacca

ACM Transactions on Evolutionary Learning and Optimization 1 (1), 1-37, 2021 (2021)

Paper

Genetic improvement of routing in delay tolerant networks

M Lorandi, LL Custode, G Iacca

Proceedings of the Genetic and Evolutionary Computation Conference Companion …, 2021 (2021)

Paper

Generation challenges: Results of the accuracy evaluation shared task

C Thomson, E Reiter

arXiv preprint arXiv:2108.05644, 2021 (2021)

Paper

Generating gender augmented data for NLP

N Jain, M Popovic, D Groves, E Vanmassenhove

arXiv preprint arXiv:2107.05987, 2021 (2021)

Paper

Evaluating a digital twin of an IoT resource slice: An emulation study using the ELIoT platform

F Granelli, R Capraro, M Lorandi, P Casari

IEEE Networking Letters 3 (3), 147-151, 2021 (2021)

Paper

EM Corpus: a comparable corpus for a less-resourced language pair Manipuri-English

R Huidrom, Y Lepage, K Khomdram

Proceedings of the 14th Workshop on Building and Using Comparable Corpora …, 2021 (2021)

Paper

EM ALBERT: a step towards equipping Manipuri for NLP

R Huidrom, Y Lepage

Proceedings of The fifth “Widening NLP” Workshop (WiNLP), EMNLP 2021, 2021 (2021)

Paper

Automatic construction of evaluation suites for natural language generation datasets

S Mille, KD Dhole, S Mahamood, L Perez-Beltrachini, V Gangal, M Kale, ...

arXiv preprint arXiv:2106.09069, 2021 (2021)

Paper

Assessing the syntactic capabilities of transformer-based multilingual language models

L Pérez-Mayos, A Táboas García, S Mille, L Wanner

Zong C, Xia F, Li Wenjie, Navigli R. Findings of the Association for …, 2021 (2021)

Paper

Another PASS: A Reproduction Study of the Human Evaluation of a Football Report Generation System

S Mille, T Castro Ferreira, B Davis, A Belz

Procceedings of the 14th International Conference on Natural Language …, 2021 (2021)

Paper

Agree to disagree: Analysis of inter-annotator disagreements in human evaluation of machine translation output

M Popović

Proceedings of the 25th Conference on Computational Natural Language …, 2021 (2021)

Paper

AfriVEC: Word Embedding Models for African Languages. Case Study of Fon and Nobiin

BFP Dossou, M Sabry

African Natural Language Processing Workshop, EACL 2021, 2021 (2021)

Paper

A Systematic Review of Reproducibility Research in Natural Language Processing

A Belz, S Agarwal, A Shimorina, E Reiter

EACL'21, 2021 (2021)

Paper

A Reproduction Study of an Annotation-based Human Evaluation of MT Outputs

M Popovic, A Belz

Proceedings of the 14th International Natural Language Generation Conference …, 2021 (2021)

Paper