#acl2024nlp — Public Fediverse posts

▪️ 7 papers authored or co-authored by UKP at this year's #EACL2024

▪️ 5 papers authored or co-authored by UKP at this year's #NAACL

▪️ At the #ACL2024NLP in Bangkok, Iryna Gurevych held a Keynote and the UKP Lab is part of two outstanding paper awards. Congratulations to the authors Indraneil Paul, Goran Glavaš, Jan-Christoph Klie, Rahul N., Juan Haladjian, Marc Kirchner, and Iryna Gurevych!

✨ As 2024 is coming to its end, we are happy to review some of the achievements of the UKP Lab ✨

▪️ We have established two independent research groups (AI and NLP for Mental Health, led by Shaoxiong Ji, and NLP for Expert Domains, led by Simone Balloccu.

▪️ At this Year’s #EMNLP2024 we presented 13 papers, including 11 in the Main track and 2 in the Findings track

▪️ 11 papers authored or co-authored by UKP members have been accepted for publication at this year's #ACL2024NLP in Bangkok 🇹🇭!

✨ As 2024 is coming to its end, we are happy to review some of the achievements of the UKP Lab ✨

▪️ We have established two independent research groups (AI and NLP for Mental Health, led by Shaoxiong Ji, and NLP for Expert Domains, led by Simone Balloccu.

▪️ At this Year’s #EMNLP2024 we presented 13 papers, including 11 in the Main track and 2 in the Findings track

▪️ 11 papers authored or co-authored by UKP members have been accepted for publication at this year's #ACL2024NLP in Bangkok 🇹🇭!

✨ As 2024 is coming to its end, we are happy to review some of the achievements of the UKP Lab ✨

▪️ We have established two independent research groups (AI and NLP for Mental Health, led by Shaoxiong Ji, and NLP for Expert Domains, led by Simone Balloccu.

▪️ At this Year’s #EMNLP2024 we presented 13 papers, including 11 in the Main track and 2 in the Findings track

▪️ 11 papers authored or co-authored by UKP members have been accepted for publication at this year's #ACL2024NLP in Bangkok 🇹🇭!

✨ As 2024 is coming to its end, we are happy to review some of the achievements of the UKP Lab ✨

▪️ We have established two independent research groups (AI and NLP for Mental Health, led by Shaoxiong Ji, and NLP for Expert Domains, led by Simone Balloccu.

▪️ At this Year’s #EMNLP2024 we presented 13 papers, including 11 in the Main track and 2 in the Findings track

▪️ 11 papers authored or co-authored by UKP members have been accepted for publication at this year's #ACL2024NLP in Bangkok 🇹🇭!

My #acl2024nlp Presidential Address is now publicly available. If you saw the slides & discussion of them in August, especially, please have a listen to the actual talk. It starts at about 43'50" in this video:

https://underline.io/events/466/sessions/18203/lecture/104931-test-of-time-awards-lifetime-achievement-awards-presidential-address

»[O]ur results do not mean that AI is not a threat at all« emphasized Iryna Gurevych. »[But future research should] focus on other risks posed by the models, such as their potential to be used to generate fake news.« (3/🧵)

Read the full press release here: https://nachrichten.idw-online.de/2024/08/12/independent-complex-thinking-not-yet-possible-after-all-study-led-by-tu-shows-limitations-of-chatgpt-co

#ACL2024NLP

The 2024 study, authored by Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi and Iryna Gurevych (BathNLP Lab | Ubiquitous Knowledge Processing (UKP) Lab), was just presented at #ACL2024NLP. It found no evidence of emergent abilities in LLMs that go beyond in-context learning.(2/🧵)

ArXiv: https://arxiv.org/abs/2309.01809

Our colleagues Iryna Gurevych, Yufang Hou and Preslav Nakov presenting the work of Max Glockner on #Missci at #ACL2024NLP 🇹🇭 , a collaboration with IBM Research Ireland and MBZUAI.

https://arxiv.org/abs/2406.03181

And consider following the authors Fengyu Cai (UKP Lab), Xinran Zhao (Carnegie Mellon University), Hongming Zhang (Tencent AI), Iryna Gurevych, and Heinz Koeppl (Computer Science, TU Darmstadt) for more information or an exchange of ideas.

See you at #ACL2024NLP 🇹🇭!

Check our paper and code!

📰 Paper:https://arxiv.org/abs/2407.12512
💻 Code: https://github.com/TRUMANCFY/geohard

(8/🧵) #ACL2024NLP #NLProc

We demonstrate that with the knowledge of class-wise hardness, class reorganization will lead to a more coherent class-wise hardness distribution, and further improve the model performance.

(7/🧵)

#ACL2024NLP #NLProc

☝️ Moreover, we theoretically prove that the intra-class hardness is associated with overfitting phenomena, leading to performance degradation in the training process.

(6/🧵) #ACL2024NLP #NLProc

🧠 GeoHard is stable across different semantic encoders and NLP tasks. This means that it generalizes well in measuring class-wise hardness.

(5/🧵)

#ACL2024NLP #NLProc

By computing the correlation between hardness measures and performance, we compare GeoHard with baseline metrics, specifically the aggregation of instance-level hardness metrics on eight NLU datasets.

GeoHard outperforms the instance-level aggregation by more than 59%! 🤯

(4/🧵)

#ACL2024NLP #NLProc

🤔 How do we measure the hardness of a class?

GeoHard to the rescue! It incorporates both inter-class and intra-class measures from class-wise semantics.

In the embedding space, greater diversity within a class and closer distances between classes indicate higher hardness.

(3/🧵)

#ACL2024NLP #NLProc

🎯 We've found a consistent pattern of class-wise difficulty across various language models, paradigms and human annotations on eight NLU datasets.

• Fine-tuned LMs: Roberta/OPT/Flan-T5
• In-context learning: LLama/OPT

Class-wise difficulty is an intrinsic feature! 🔍🤖

(2/🧵) #ACL2024NLP #NLProc

Are all classes in #NLProc tasks equally difficult to learn? 🤔
In our #ACL2024NLP paper, we analyze why this is not the case!
Please meet #GeoHard, a metric to measure class-wise difficulty 🔍📊 ! 🧵(1/9)

📆 Poster: Tue, Aug 13, 12:15 ICT

📰 https://arxiv.org/abs/2407.12512

Meet our fellow researchers representing UKP Lab at #ACL2024NLP: Iryna Gurevych, Qian Ruan, Justus-Jonas Erker, Indraneil Paul, Fengyu CAI, Sheng Lu, Haishuo Fang, Furkan Şahinuç, Kexin Wang, Haau-Sing Li, Andreas Waldis, and a very special guest from BathNLP Lab, @harish Tayyar Madabushi!

Interested in our research? At these times you can see the presentations of papers co-authored by our colleagues at #ACL2024NLP 🇹🇭!

ACL 2024 accepted demo papers are out! 🙂 Check https://2024.aclweb.org/program/demo_papers/ for the list of papers! #acl2024nlp

Congratulations to all authors! We look forward to seeing you in Bangkok 🇹🇭 this August!

➡️ https://www.informatik.tu-darmstadt.de/ukp/ukp_home/ukp_news_details_300736.en.jsp

(17/17) #ACL2024NLP #NLProc

»$\textit{GeoHard}$: Towards Measuring Class-wise Hardness through Modelling Class Semantics« by Fengyu Cai, Xinran Zhao, Hongming Zhang, Iryna Gurevych and Heinz Koeppl (16/🧵) #ACL2024NLP (arXiv coming soon)

»$\texttt{DARA}$: Decomposition-Alignment-Reasoning Autonomous Language Agent for Question Answering over Knowledge Graphs« by Haishuo Fang, Xiaodan Zhu and Iryna Gurevych (15/🧵) #ACL2024NLP (arXiv coming soon)

Also, 2 papers have been accepted to the Findings section of #ACL2024NLP. They are:

»On Efficient and Statistical Quality Estimation for Data Annotation« by Jan-Christoph Klie, Rahul Nair, Juan Haladjian and Marc Kirchner (13/🧵) #ACL2024NLP

https://arxiv.org/abs/2405.11919

»DAPR: A Benchmark on Document-Aware Passage Retrieval« by Kexin Wang, Nils Reimers and Iryna Gurevych (12/🧵) #ACL2024NLP

https://arxiv.org/abs/2305.13915

»Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revision« by Qian Ruan, Ilia Kuznetsov and Iryna Gurevych (11/🧵) #ACL2024NLP (arXiv coming soon)

»Dismantling the Misleading Narratives: Reconstructing the Fallacies in Misrepresented Science« by Max Glockner, Yufang Hou, Preslav Nakov and Iryna Gurevych (10/🧵) #ACL2024NLP (arXiv coming soon)

»Triple-Encoders: Representations That Fire Together, Wire Together« by Justus-Jonas Erker, Florian Mai, Nils Reimers, Gerasimos Spanakis and Iryna Gurevych (9/🧵) #ACL2024NLP

https://arxiv.org/abs/2402.12332

»Are Emergent Abilities in Large Language Models just In-Context Learning?« by Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi and Iryna Gurevych (8/🧵) #ACL2024NLP (arXiv coming soon)

»Systematic Task Exploration with LLMs: A Study in Citation Text Generation« by Furkan Şahinuç, Ilia Kuznetsov, Yufang Hou and Iryna Gurevych (7/🧵) #ACL2024NLP (arXiv coming soon)

… Alham Fikri Aji, Nizar Habash, Iryna Gurevych and Preslav Nakov (6/🧵) #ACL2024NLP

https://arxiv.org/abs/2402.11175