#surveyresearch — Public Fediverse posts

Is #Amazon going to sunset #mTurk?

As of July 30 2026, #AWS will no longer accept new customers (requestors).

"Existing customers can continue to use the service as normal."

https://docs.aws.amazon.com/sagemaker/latest/dg/sms-workforce-management-public.html

#cogSci #psychology #xPhi #AI #tech #news #crowdWork #gigWork #surveyResearch

Going back to the original question: does response to the "difficulty for men to conform to societal expectations" question vary by political leaning? Mostly no.

Now that sayDvoteR and sayRvoteD have been pulled out of all other groups GOP & Dem respondents don't seem very different from each other on this item.

Unaffiliated ('Oth') men have, by far, the highest response here. This is the part of the data story that is consistent with the first glance, a few days ago. Unaffiliated women have a good deal of variability but their average is everyone else's average. There doesn't seem to be much insight from the sayDvoteR group, and the sayRvoteD group has a potential gender split (men lower) though less dramatic than Oth and hovering around the overall mean.

Men in the 'Oth' group responded, on average, with significantly greater agreement with the item ("men have a harder time...") than Oth women (t=2.53, p<.05). There was no significant difference between men and women in the sayRvoteD group.

I think I'll stop here. I found that the patterns apparent in the first graph were due to at least one underlying factor, and partly disappeared when that was accounted for. I have more analyses to do with this quirky dataset, but I also have like a job or something. And a kid. You know, other things on the to-do list (some of which are far less fun than this).

/end (for now)

#uspol #survey #surveyresearch #gender #democrats #republicans

A few days ago I posted a graph with very minimal details. I'm going to try to do a more considered analysis here.

The original post showed (binary, sorry) male and female survey respondents' answers to a question about how hard they thought it was for men to conform to societal norms, by self-reported political affiliation. There was a potentially large surprise with politically unaffiliated women and (smaller effect) Democrat women vs men. That "other" category for politics can hide some interesting things, so I split things a bit differently.

I pulled out the participants who indicated they were Democrat or Republican (or leaning that way, for participants who said Independent or Other; there was a follow-up question). Then I compared self-reported D vs R affiliation/leaning with who they said they voted for in 2024. This created two new categories:

• sayDvoteR: Self-identified as Democrat or Democrat-leaning but voted for Trump
• sayRvoteD: Self-identified as Republican or Republican-leaning but voted for Harris

I did one more thing, still preliminary: I created a "MAGA score" by averaging the z-scores of nine questions that seemed explicitly relevant to MAGA ideology or affilitation. They were questions about whether the killing of an Iranian Islamic leader was justified, if the US violated international law when invading Iran, how much participants trusted Trump as a source of climate information, and whether they were proud to be an American. This a priori group of items clustered pretty tightly with Dem and GOP Ss, and less so with others.

1/

(Caveat! This is exploratory and I'm making some statistically non-optimal decisions I would at least explain more thoroughly if I were trying to publish this analysis)

#uspol #surveyresearch #survey #gender #maga

RE: https://things.uk/@eclectech/116640432702265723

These are some of the worst-written choices for a survey question I’ve ever seen. It seems like the person who wrote this doesn’t know how to write survey questions but also wanted to be deliberately misleading so they can get the kind of answers they want.

#SurveyDesign #FormDesign #SurveyResearch #research

#callforpapers #cfp #SurveyResearch
Who Produces #Knowledge? #AI Intervention in Survey Research between Paradigmatic Reaction and Epistemological Drift
When AI systems generate items, simulate respondents, or classify open-ended responses, measurement itself is shaped by interpretative operations embedded within the model. In relation to this interest, we are inviting contributors to this HSR special issue.

Important date: May 17, 2026: Submission of extended abstracts

https://www.gesis.org/hsr/ueber-hsr/detailanzeige-hsr-news/article/cfp-who-produces-knowledge-ai-intervention-in-survey-research-between-paradigmatic-reaction-and-epistemological-drift

Is checking students' online surveys multiple times in the same day technically stalking? I swear, I just want what's best for them. I just want them to stay safe. Other professors don't know how to treat their surveys.

#professor #surveyresearch

I feel like I'm constantly fighting an uphill battle when I raise awareness about any privacy issues. This email went out to my campus a few days ago from (ironically) the American Democracy Project. The link is below:

https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fforms.office.com%2FPages%2FResponsePage.aspx%3Fid%3DPAUkwXjwhEmPnTWuD3jFim27hL6NsGFBs-JLhn5poQ5URU4xM0tKRVM1S0w3UUNPQjdISDQwRzdIOS4u&data=05%7C02%7C[MyFullName]%40[MyUniversity].edu%7C426332fa16114117c45d08de8f73e19e%7Cc124053cf07849848f9d35ae0f78c58a%7C0%7C0%7C639105925031635084%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=V1ThxSS%2BVa236C7i03BCacXkisZdjcJLoopiOGDaZeI%3D&reserved=0

I removed my full fucking name and the name of my university; I'm sure there's shit that can be extracted by a knowledgeable person. Or maybe I'm paranoid and overreacting. I don't think all that text is necessary to simply route me to a survey's URL, even if it is on Microsoft Forms.

I replied to the all-campus email and said there was some data sent with that link, including my name. I said I'd like to participate in the survey if it was anonymous and asked if there was a simple, sanitized link that didn't send my name and other information with it. I got bounced to the Very Sincere Student Project Leader who assured me that there was no personal information collected, and I could verify this by looking at the questions in the survey.

On MS Forms.

Linked with all the stuff above.

Sigh. American Democracy, indeed.

#security #privacy #microsoft #survey #surveyresearch

gods save us from the college freshman edgelords who refuse to select any race/ethnicity categories in surveys and instead type in 'IRISH' (all caps).

You cost me 5 minutes of data cleaning, but I guess you'll add some good variability when you tell us how you feel about trans people.

#subtoot #research #survey #surveyresearch #asshat

I think I figured out why Qualtrics as a product hasn't lived up to anyone's expectations. If I had the time maybe I'd comb through their open issues and unaddressed feature requests to see if there's a little cliff in 2021.

https://www.investors.com/news/technology/qualtrics-ipo-initial-public-offering-trades-thursday-xm-sap/

#surveys #surveyresearch #qualtrics #research #enshittification

I've sadly come to conclude that #qualtrics (my beloved survey platform started by some fresh-faced Mormon boys from BYU 20 years ago) has become kind of enshittified.

For almost 20 years the company has had requests to add pretty reasonable features like (definitely not an exhaustive list):

• Vertical sliders
• Text orientation in labels
• non-numerical recodes (this has looked like it should for at least 15 years)
• Parity in options between matrix, sliders, and side-by-side questions

Several others, too. Nope. At least a dozen highly reasonable feature requests still unaddressed for 15+ years. I got off the phone with customer service and their best answer for vertical sliders or CSS-rotated slider/matrix labels was "check our community help forums, though we don't support anything they say." It's the Google/Microsoft model: convince people to crowdsource their own workarounds to your product while you rake in the dough from tens of thousands of organizational contracts.

Qualtrics' revenue is close to $2B per year, and they are valued north of $15B.

The sad thing is that there seem to be so few viable alternatives.

#enshittification #surveys #surveyresearch #fuckthis

@guyjantic If this question is the only thing you have, it is difficult to make an argument to support ordered categories.

In #SurveyResearch you could do it via previous qualitative work, e.g., cognitive interviews; or quantitative work (e.g., ask people to indicate the order of such terms). If you have specific rating scale anchors (I doubt it exists for the ones you gave as examples 😂 ), you could look for published work on them and at least argue that others have found these to be ordered.

Question for #SocialScience or #surveyresearch or #psychometrics or #statistics folks: How do I test the order of maybe-ordinal responses for some items in a survey?

I have done some scaling where the items are tested for ordered/monotonic properties (e.g., is item 1 really more of whatever than item 2?). However, this is a bit different, I think: I have been tasked with analyzing some items from a survey kind of like this:

I think pregnant women deserve the same rights as everyone else (choose one)
Absolutely
Hell yeah
Sure!
I don't know
Maybe
Nah
Nope

Is Absolutely actually more agreement than Hell yeah? Does a response of I don't know indicate more agreement than Maybe?

It's this kind of question. How do I test for the empirical order (if any) applied by about 800 participants?

#question #help #suggestion

https://winbuzzer.com/2025/11/24/study-shows-how-ai-agents-break-online-surveys-and-polling-posing-existential-threat-to-data-science-xcxwbn

Study Shows How AI Agents Break Online Surveys and Polling, Posing Existential Threat to Data Science

#AI #AIAgents #Cybersecurity #DataScience #AIResearch #AISafety #Polling #ElectionIntegrity #GenerativeAI #SurveyResearch

#jobs #stellenangebote #GESISjobs #jobfairy #SocialScienceJobs #PhDPosition #SurveyResearch #AcademicJobs

🎓 GESIS is hiring a Research Associate (PhD Candidate) in Questionnaire Development to join the Family Surveys team in Mannheim or Cologne, starting 01.09.2025.

Work on the FReDA panel and write your PhD in survey methodology or family sociology.

📅 Apply by 24.07.2025
🔗 https://www.gesis.org/en/institute/career/jobs/details/research-associate-in-data-preparation-phd-candidate

Have 15-20 minutes to think about gender? Participate in my lab’s latest #surveyResearch on gender expression across contexts to help develop new, better assessments of ✨ diverse gender experiences ✨ https://illinois.qualtrics.com/jfe/form/SV_5tHUpQzMeXbICmq please boost for reach, thank you!

#request - does anyone know of any panel services that are affordable? I need some survey participants (200 would be lovely) and quick, because of [swear words while thinking of why]. I'm looking into #mechanicalturk but I don't know if I can get this to happen quickly enough; I have one week.

I need young adults, English speaking (native fluency). North American would be fine, but European is great, too. Maybe near East (e.g., India, Pakistan) if the language fluency is there. Region/nation is less important than the age range and the language ability.

I'd be paying for this out of pocket, and I don't have much money, so hoping I can find inexpensive services somewhere (<<$500 USD? Maybe closer to $200? realizing this might not be possible because I've never looked into this until now). Qualtrics has panels but they're far too expensive for me. I'm googling, but asking here first.

#Recommendations?

#research #survey #surveyresearch #panel #participants #help #lastminute

Matthias Roth (@rothm) and I published a new paper on survey data harmonization.

"One harmonization fits all? – Impact of missing population invariance on harmonization error when harmonizing social science survey questions with equating"

The paper again corroborates that observed score equating outperforms linear stretching on average. However, it also shows that it is better to derive a harmonization solution for single-item measures from a sample that is drawn from a similar population to the population you are interested in researching.

https://doi.org/10.1080/13645579.2025.2481112

#surveyresearch #surveymethods #harmonization

#Microsoft's viral paper about *correlations* between #AI use and #criticalThinking also has "impact" in the title (despite admitting "Our analysis does not establish #causation"). 🤦‍♂️

#Confidence in #GenAI predicted LESS critical thinking.

SELF-confidence predicted MORE critical thinking.

PREDICTED ≠ CAUSED

https://www.microsoft.com/en-us/research/publication/the-impact-of-generative-ai-on-critical-thinking-self-reported-reductions-in-cognitive-effort-and-confidence-effects-from-a-survey-of-knowledge-workers/

#computerScience #LLMs #work #decisionScience #psychology #surveyResearch #management #logic

🗳️ Do Russians Really Support the War in Ukraine?

https://www.newyorker.com/news/a-reporter-at-large/do-russians-really-support-the-war-in-ukraine

archive link: https://archive.ph/indZg

#russia #war #ukraine #publicoption #socialscience #surveyresearch #focusgroup #interviewing

📊 New research finds minimal evidence that using a smartphone for surveys leads to lower data quality. Regardless of device, respondents provided consistent results.

#SurveyResearch #DataQuality #MobileSurveys https://www.surveypractice.org/article/127306-smartphone-or-laptop-device-type-choice-and-data-quality-in-surveys-of-commercial-and-residential-utility-customers

Conincidentally, this is actually how much of #SurveyResearch works #PoorlyDrawnLines

In a new paper, I simulate the consequences of combining survey data without proper harmonization techniques.

I demonstrate that there is a plausible risk of biased correlative analyses based on the integrated data, if we do not harmonize measurement units across different survey sources and instruments first.

#surveyresearch #surveymethods #harmonization

https://doi.org/10.5964/miss.11217

Recently, I've been exploring the nonprobsvy package (https://ncn-foreigners.github.io/nonprobsvy/) in R and am curious if anyone has experience working with it, particularly in combining probability and non-probability samples. I'd love to hear your insights or recommendations! #rstats #SurveyResearch #DataScience

People designing questionnaires for a German sample will find the just published new version of the "Demographische Standards" helpful. It offers guidance on how to collect varios socio-demographic variables (e.g., education, citizenship, employment) in a comprehensive manner.

https://doi.org/10.21241/ssoar.94099

#surveymethods #surveyresearch

Together with Cornelia Neuert and Tenko Raykov, I looked into "Assessing conceptual comparability of single-item survey instruments with a mixed-methods approach".
Innovations are using a confirmatory factor analysis on a collection of single item instruments and comparing it with qualitative findings from probing questions.

To illustrate the approach, we assessed the comparability of social (dis-)trust questions.

#surveyresearch #harmonization

https://doi.org/10.1007/s11135-023-01801-w

Well, @#$%. I'm going to have to learn some serious JSON skills (I'll try to use R packages to make it easier) to create #surveys in #Qualtrics from text files of hundreds of items.

I wish there was an easier way, I really do. Qualtrics is already a pretty amazing platform, so it feels greedy to want this, too, but I do want it.

Once again, $@#%.

#qualtrics #surveyresearch #Rstats

@pgmj #Rstats packages such as #catR #mstR (https://link.springer.com/book/10.1007/978-3-319-69218-0) #mirt and other such infrastructure have moved the field on the #Psychometrics side.

But there is no way to integrate them into the systems I have been using so far to collect for example online #SurveyData with.

Another hurdle:
The field of #SurveyResearch being disconnected from #LatentVariable Theory and related test developments. Something I re-visit each time I deliver training to deliver tests, items, interviews.

@gavintlbrown Thanks for making me aware. There are a number of good applications, especially in mental health and #HRQL research.

I think the problem lies more with what @pgmj points out in their response to this:
The lack of availability of tools/ systems to implement and collect data in this manner. And especially in Europe: to be able to tailor them to the specific security requirements.

#SurveyResearch