#eacl2024 — Public Fediverse posts

▪️ At this year's #eaclmeeting in Malta, two papers co-authored by the UKP Lab have won awards for Best Resource and Social Impact. Congratulations to Yuxia Wang (MBZUAI (Mohamed bin Zayed University of Artificial Intelligence) et al. for the Best Resource Paper and Tilman Beck, Hendrik Schuff, Anne Lauscher (Universität Hamburg) and Iryna Gurevych (Technische Universität Darmstadt) for the #EACL2024 Social Impact Award!

▪️ 7 papers authored or co-authored by UKP at this year's #EACL2024

▪️ 5 papers authored or co-authored by UKP at this year's #NAACL

▪️ At the #ACL2024NLP in Bangkok, Iryna Gurevych held a Keynote and the UKP Lab is part of two outstanding paper awards. Congratulations to the authors Indraneil Paul, Goran Glavaš, Jan-Christoph Klie, Rahul N., Juan Haladjian, Marc Kirchner, and Iryna Gurevych!

The two papers, authored by Luke Bates, Rachneet Singh Sachdeva, Martin Tutek and Iryna Gurevych, are among seven UKP submissions accepted at #EACL2024. They originate from the project »Fake News and Conspiracy Theories«, part of the @ATHENECenter research area #SeDiTraH:

https://www.informatik.tu-darmstadt.de/ukp/research_ukp/ukp_research_projects/ukp_project_fakenews_conspiracytheories/ukp_project_fakenews_conspiracytheories.en.jsp

Fighting fake news and misinformation together! 🔎 Our project as part of the German National Research Center for Applied Cybersecurity @ATHENECenter resulted in two accepted papers on the topic at this year's #EACL2024!

➡️ https://www.athene-center.de/en/news/news/accepted-papers-at-the-eacl-2024-1626

»Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting« by Tilman Beck, Hendrik Schuff, Anne @lauscher (Universität Hamburg) and Iryna Gurevych (@UKPLab) was awarded the #EACL2024 Social Impact Award!

More info on the paper in this 🧵: https://sigmoid.social/@UKPLab/112127298620459392

Learn more about »M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection« by Yuxia Wang et al., which was awarded the #EACL2024 Resource Paper Award, in this 🧵:

https://sigmoid.social/@UKPLab/112121773369035352

At this year's #EACL meeting, two papers involving UKP Lab have won awards for Best Resource and Social Impact. Congratulations to everyone involved! 💐

https://www.tu-darmstadt.de/universitaet/aktuelles_meldungen/kurzgemeldet/index.de.jsp#ukp_lab_konferenz

#EACL2024 #NLProc

At this year's #EACL meeting, two papers involving UKP Lab have won awards for Best Resource and Social Impact. Congratulations to everyone involved! 💐

https://www.tu-darmstadt.de/universitaet/aktuelles_meldungen/kurzgemeldet/index.de.jsp#ukp_lab_konferenz

#EACL2024 #NLProc

At this year's #EACL meeting, two papers involving UKP Lab have won awards for Best Resource and Social Impact. Congratulations to everyone involved! 💐

https://www.tu-darmstadt.de/universitaet/aktuelles_meldungen/kurzgemeldet/index.de.jsp#ukp_lab_konferenz

#EACL2024 #NLProc

At this year's #EACL meeting, two papers involving UKP Lab have won awards for Best Resource and Social Impact. Congratulations to everyone involved! 💐

https://www.tu-darmstadt.de/universitaet/aktuelles_meldungen/kurzgemeldet/index.de.jsp#ukp_lab_konferenz

#EACL2024 #NLProc

Haiming Liu calls for nominations for the Karen Spärck-Jones award. #ECIR2024 #EACL2024

This https://aclanthology.org/2024.eacl-long.5/ is a very important paper, published at #EACL2024. But the sad truth is that this would have been avoidable, if people would have followed well-known best practices in doing science: Avoid the hype, use local #llms in defined and controlled states. Reminds me of "Googleology is Bad Science" from 2010: https://aclanthology.org/J07-1010/

We are proud to announce that the contribution »Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting« by Tilman Beck, Hendrik Schuff, Anne @lauscher (Universität Hamburg) and Iryna Gurevych (UKP Lab) has just been awarded the #EACL2024 Social Impact Award!

The award was accepted by Anne Lauscher. More info about the paper can be found in this post: https://sigmoid.social/@UKPLab/112127298620459392

And consider getting in touch with the authors Tilman Beck, Hendrik Schuff, Anne Lauscher (Universität Hamburg) and Iryna Gurevych (@UKPLab), if you are interested in more information or an exchange of ideas.

See you this week in Malta!

(9/9) #EACL2024 #NLProc #Prompting #InstructGPT #LLMs

Takeaway? Sociodemographic prompting should be used with care ⚠️

If you are interested in the details of this work, you will find more information in the paper & code at:

📄 Paper: ​​https://arxiv.org/abs/2309.07034
💻 Code: https://github.com/UKPLab/arxiv2023-sociodemographic-prompting/

(8/🧵) #EACL2024 #NLProc

Sentiment and toxicity classification benefit from sociodemographic prompting in zero-shot classification.
Could we also use it to identify instances which will likely result in disagreement during annotation? Flan-T5 does a decent job with an avg 0.62 F1.

(7/🧵) #EACL2024 #NLProc

Let’s dig deeper into the influence of the model 🕵️: the predictions are dominated by the model family in use!
When prompted with different gender values, InstructGPT places at least 10% of its predictions on label 1 while OPT-IML none, independent of the gender value.

(6/🧵) #EACL2024 #NLProc

What are the effects of sociodemographic prompts?
- InstructGPT/OPT-IML are less affected than other models
- Shorter texts lead to more changes
- If the text led to disagreement among annotators, prompting results from different sociodemographic profiles tend to disagree.

(5/🧵) #EACL2024 #NLProc

We analyze 17 diverse instruction-tuned LLMs across seven datasets reflecting 4 different subjective NLP tasks (i.e., sentiment, hate speech, toxicity, stance).

(4/🧵) #EACL2024 #NLProc

❗ This is very promising, e.g. for dataset augmentation or human survey simulation.

How sensitive are different models to this prompting technique and why? Does it improve classification for subjective tasks, such as hate speech? And how robust is it after all?

(3/🧵) #EACL2024 #NLProc

Sociodemographic prompting refers to the idea of enriching a prompt with user profile information such as the gender, race, age, and education level. We expect the model’s output to be aligned with the sociodemographic profile described.

(2/🧵) #EACL2024 #NLProc

LLMs are increasingly prompted with different user profiles to solve subjective NLP tasks. What are the factors which determine what the model generates?

Discover it in our #EACL2024 paper – learn more in this 🧵 (1/8).

📰 https://arxiv.org/abs/2309.07034

#NLProc #Prompting

Interested in #ClinicalNLP? Our colleague Tobias Mayer presented his research on Monday at #EACL2024 in 🇲🇹 !

Find out more about "Predicting Client Emotions and Therapist Interventions in Psychotherapy Dialogues" in this thread: https://sigmoid.social/@UKPLab/112115994096462953

And consider following the authors Jan Buchmann, Max Eichler, Jan-Micha Bodensohn, Ilia Kuznetsov and Iryna Gurevych (@DFKI, @UKPLab, Systems@TUDA, @TU Darmstadt Darmstadt), if you are interested in more information or an exchange of ideas.

See you this week in Malta 🇲🇹!

#EACL2024 #NLProc

We provide open access to our code and results:

📄 Paper: https://arxiv.org/abs/2401.17658
💻 Code: https://github.com/UKPLab/eacl2024-doc-structure

(8/🧵) #EACL2024 #NLProc

➡️ Recap: long document transformers implicitly represent document structure. This representation can be enhanced via infusion of structural information, which leads to improvements on downstream tasks.

(7/🧵) #EACL2024 #NLProc

Evaluating whether two segment pairs belong to the same document section is a key capability. It is positively correlated with performance on downstream tasks ↗️.

(6/🧵) #EACL2024 #NLProc

Does this improved representation of document structure help in downstream tasks?

💯Yes, it does! But the most effective infusion strategy depends on the task.

(5/🧵) #EACL2024 #NLProc

What happens if we infuse structural information on node type and depth?

🚀 Accuracy improves on all probing tasks!!

Thanks to an improved representation of document structure.

(4/🧵) #EACL2024 #NLProc

We introduce a dataset and novel probing tasks and experiment with LED and LongT5.

Key insight 💡: Long Transformers have implicitly learned to represent document structure during pre-training

(3/🧵) #EACL2024 #NLProc

We view documents as graphs
🔵 nodes represent textual elements of different types (e.g. section headings or paragraphs)
➡️ edges represent the hierarchical organization.

(2/🧵) #EACL2024 #NLProc

Structure of a document helps humans understand its content. But for transformers, all documents are just a long line of text.

Want to fix this? 🛠️

Our #EACL2024 paper equips transformers with doc structure understanding – more in this 🧵(1/9)

📰 https://arxiv.org/abs/2401.17658

#NLProc

And consider getting in touch with the authors Luke Bates and Iryna Gurevych at @UKPLab if you are interested in more information or an exchange of ideas. See you in 🇲🇹!

(8/8) #EACL2024 #NLProc

Consider to look up the authors Tobias Mayer, Neha Warikoo, Amir Eliassaf and Dana Atzil-Slonim (Psychotherapy Research Lab/Department of Psychology, Bar-Ilan University) and @Iryna Gurevych (UKP Lab/Hessian.ai), if you are interested in more information or an exchange of ideas. (8/8) See you in 🇲🇹!

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

Consider to look up the authors Tobias Mayer, Neha Warikoo, Amir Eliassaf and Dana Atzil-Slonim (Psychotherapy Research Lab/Department of Psychology, Bar-Ilan University) and @Iryna Gurevych (UKP Lab/Hessian.ai), if you are interested in more information or an exchange of ideas. (8/8) See you in 🇲🇹!

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

Consider to look up the authors Tobias Mayer, Neha Warikoo, Amir Eliassaf and Dana Atzil-Slonim (Psychotherapy Research Lab/Department of Psychology, Bar-Ilan University) and @Iryna Gurevych (UKP Lab/Hessian.ai), if you are interested in more information or an exchange of ideas. (8/8) See you in 🇲🇹!

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

Consider to look up the authors Tobias Mayer, Neha Warikoo, Amir Eliassaf and Dana Atzil-Slonim (Psychotherapy Research Lab/Department of Psychology, Bar-Ilan University) and @Iryna Gurevych (UKP Lab/Hessian.ai), if you are interested in more information or an exchange of ideas. (8/8) See you in 🇲🇹!

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

We have encountered interesting challenges with regard to ambiguity in interpreting cultural slang in Hebrew. (7/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

We have encountered interesting challenges with regard to ambiguity in interpreting cultural slang in Hebrew. (7/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

We have encountered interesting challenges with regard to ambiguity in interpreting cultural slang in Hebrew. (7/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

We have encountered interesting challenges with regard to ambiguity in interpreting cultural slang in Hebrew. (7/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

🤔 But there's more – we uncovered ambiguities in IP and ER coding, which causes high confusion in some of the class predictions. (6/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

🤔 But there's more – we uncovered ambiguities in IP and ER coding, which causes high confusion in some of the class predictions. (6/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

🤔 But there's more – we uncovered ambiguities in IP and ER coding, which causes high confusion in some of the class predictions. (6/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

🤔 But there's more – we uncovered ambiguities in IP and ER coding, which causes high confusion in some of the class predictions. (6/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

📈 Results are in❗ Our best adapters achieve on-par performance with fully fine-tuned models, while having a fraction of the computational cost. (5/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

📈 Results are in❗ Our best adapters achieve on-par performance with fully fine-tuned models, while having a fraction of the computational cost. (5/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

📈 Results are in❗ Our best adapters achieve on-par performance with fully fine-tuned models, while having a fraction of the computational cost. (5/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

📈 Results are in❗ Our best adapters achieve on-par performance with fully fine-tuned models, while having a fraction of the computational cost. (5/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024

Experiments are conducted on a Hebrew dataset with fine-tuned language models and light-weight adapters. (4/🧵)

#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024