publications

2026

1. EACL SRW

   Learning Nested Named Entity Recognition from Flat Annotations

   Igor Rozhkov and Natalia V Loukachevitch

   In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop), Mar 2026

   Abs DOI HTML Code

   Nested named entity recognition identifies entities contained within other entities, but requires expensive multi-level annotation. While flat NER corpora exist abundantly, nested resources remain scarce. We investigate whether models can learn nested structure from flat annotations alone, evaluating four approaches: string inclusions (substring matching), entity corruption (pseudo-nested data), flat neutralization (reducing false negative signal), and a hybrid fine-tuned + LLM pipeline. On NEREL, a Russian benchmark with 29 entity types where 21% of entities are nested, our best combined method achieves 26.37% inner F1, closing 40% of the gap to full nested supervision. Code is available at https://github.com/fulstock/Learning-from-Flat-Annotations.

2. CLEF

   Overview of the BioASQ BioNNE-R Task on Nested Biomedical Relation Extraction at CLEF 2026

   Igor Rozhkov, Natalia Loukachevitch, and Elena Tutubalina

   In Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2026), Oct 2026

   Abs HTML PDF Code

   

   Biomedical relation extraction is typically studied under flat named entities, keeping only relations that involve nested named entities, where one entity is contained within another, unexplored. This paper presents an overview of the BioNNE-R shared task on biomedical nested relation extraction, organized within the BioASQ 2026 workshop at CLEF. Building on the NEREL-BIO corpus and the earlier BioNNE and BioNNE-L shared tasks, BioNNE-R requires systems to extract semantic relations between nested biomedical entities in English and Russian PubMed abstracts. The task covers 8 entity types and 14 relation types and is organized into three subtasks under two evaluation tracks: a monolingual track, with separate English and Russian subtasks, and a bilingual track combining both languages. We received 20 Codabench registrations, and 7 teams submitted predictions during the evaluation phase. Most participating systems framed the task as pair classification with fine-tuned multilingual and biomedical transformer encoders; the best systems reached macro-F1 scores of approximately 0.50-0.53, substantially above the baseline given, with a document-level graph-based approach achieving the top results on the Russian and bilingual subtasks. These results indicate that relation extraction over nested biomedical entities remains a challenging problem with considerable room for improvement.

3. ECIR

   BioASQ at CLEF2026: The Fourteenth Edition of the Large-Scale Biomedical Semantic Indexing and Question Answering Challenge

   Anastasios Nentidis, Georgios Katsimpras, Anastasia Krithara, and 18 more authors

   In Advances in Information Retrieval, 2026

   Abs DOI

   Over the past thirteen years, the large-scale biomedical semantic indexing and question-answering challenge (BioASQ) has consistently driven the advancement of methods and tools that enhance access to the rapidly growing body of scientific resources in the biomedical domain. BioASQ offers a unique common testbed where research teams worldwide can evaluate and compare innovative approaches for accessing biomedical knowledge by organizing shared tasks on an annual basis and providing respective benchmark datasets that represent the real information needs of biomedical experts.

4. CLEF

   Overview of BioASQ 2026: The Fourteenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

   Anastasios Nentidis, Georgios Katsimpras, Anastasia Krithara, and 8 more authors

   In International Conference of the Cross-Language Evaluation Forum for European Languages (CLEF 2026), Sep 2026

2025

1. Извлечение именованных сущностей предметной области из неоднородного входного потока

   Н. В. Лукашевич, И. С. Рожков, and Б. В. Добров

   In Программирование и вычислительная математика: сборник материалов конференции, посвященной 100-летию со дня рождения Н. П. Трифонова, 2025
2. arXiv

   Methods for Recognizing Nested Terms

   Igor Rozhkov and Natalia Loukachevitch

   May 2025

   Abs DOI arXiv

   In this paper, we describe our participation in the RuTermEval competition devoted to extracting nested terms. We apply the Binder model, which was previously successfully applied to the recognition of nested named entities, to extract nested terms. We obtained the best results of term recognition in all three tracks of the RuTermEval competition. In addition, we study the new task of recognition of nested terms from flat training data annotated with terms without nestedness. We can conclude that several approaches we proposed in this work are viable enough to retrieve nested terms effectively without nested labeling of them.

2024

1. LRE

   NEREL: A Russian Information Extraction Dataset with Rich Annotation for Nested Entities, Relations, and Wikidata Entity Links

   Natalia Loukachevitch, Ekaterina Artemova, Tatiana Batura, and 8 more authors

   Language Resources and Evaluation, Jun 2024

   DOI
2. Предобученные языковые модели в задаче извлечения вложенных именованных сущностей

   Н. В. Лукашевич and И. С. Рожков

   In Ломоносовские чтения-2024: научная конференция, факультет ВМК МГУ имени М.В.Ломоносова. Тезисы докладов, 2024

2023

1. Bioinformatics

   NEREL-BIO: A Dataset of Biomedical Abstracts Annotated with Nested Named Entities

   Natalia Loukachevitch, Suresh Manandhar, Elina Baral, and 5 more authors

   Bioinformatics, Apr 2023

   DOI
2. LJM

   Machine Reading Comprehension Model in Domain-Transfer Task

   I. S. Rozhkov and N. V. Loukachevitch

   Lobachevskii Journal of Mathematics, Aug 2023

   DOI
3. PRIA

   Prompts in Few-Shot Named Entity Recognition

   I. S. Rozhkov and N. V. Loukachevitch

   Pattern Recognition and Image Analysis, Jun 2023

   DOI

2022

1. Dialogue

   RuNNE-2022 Shared Task: Recognizing Nested Named Entities

   Ekaterina Artemova, Maxim Zmeev, Natalia Loukachevitch, and 4 more authors

   In Computational Linguistics and Intellectual Technologies: Proceedings of Dialogue 2022, Jun 2022

   Abs DOI arXiv Code

   The RuNNE Shared Task approaches the problem of nested named entity recognition. The annotation schema is designed in such a way, that an entity may partially overlap or even be nested into another entity. This way, the named entity "The Yermolova Theatre" of type "organization" houses another entity "Yermolova" of type "person". We adopt the Russian NEREL dataset for the RuNNE Shared Task. NEREL comprises news texts written in the Russian language and collected from the Wikinews portal. The annotation schema includes 29 entity types. The nestedness of named entities in NEREL reaches up to six levels. The RuNNE Shared Task explores two setups. (i) In the general setup all entities occur more or less with the same frequency. (ii) In the few-shot setup the majority of entity types occur often in the training set. However, some of the entity types are have lower frequency, being thus challenging to recognize. In the test set the frequency of all entity types is even. This paper reports on the results of the RuNNE Shared Task. Overall the shared task has received 156 submissions from nine teams. Half of the submissions outperform a straightforward BERT-based baseline in both setups. This paper overviews the shared task setup and discusses the submitted systems, discovering meaning insights for the problem of nested NER. The links to the evaluation platform and the data from the shared task are available in our github repository: https://github.com/dialogue-evaluation/RuNNE.

2. Dialogue

   Machine Reading Comprehension Model in RuNNE Competition

   Igor Rozhkov and Natalia Loukachevitch

   In Computational Linguistics and Intellectual Technologies: Proceedings of Dialogue 2022, Jun 2022

   Abs DOI

   The paper studies machine reading comprehension model (MRC) (Li et al., 2020) in its application to extracting nested named entities (nested NER) in the RuNNE-2022 evaluation (Artemova et al., 2022). The model transforms named entity recognition tasks to a question-answering task. In this paper we compare several approaches to formulating ”questions” for the MRC model such as entity type names (keywords), entity type definitions, most frequent examples for the train set, combinations of definitions and examples. We found that using two most frequent examples from the training set is comparable in quality of nested NER with gathering qualitative definitions from different dictionaries, which is much more complicated. In the RuNNE evaluation, the MRC model obtained the best results among models without any manual work (rules or additional manual annotation of texts).

2021

1. ISTA

   NEREL: A Russian Dataset with Nested Named Entities and Relations

   I. V. Denisov, I. S. Rozhkov, and N. V. Lukashevich

   Intelligent Systems. Theory and Applications, Oct 2021

   HTML