#prolific — Public Fediverse posts

Can researchers detect #AI bots taking paid surveys?

#Prolific tested humans and #LLM agents with various #dataQuality checks.
- The company says they caught 100% of the non-humans.
- My take-away: #reCAPTCHA and #mouseTracking caught 95%

https://www.prolific.com/resources/authenticity-checks-how-we-tested-the-most-accurate-method-for-identifying-agentic-ai

#surveyMethods

I forgot to share the #mTurk data quality result that got scooped:

“In late 2020…. Participants from the United States were recruited from Amazon Mechanical Turk, -#CloudResearch, #Prolific, and a #university. One participant source yielded up to 18 times as many low-quality respondents as the other three.”

https://doi.org/10.1093/analys/anaf015

#psychology #philosophy #surveyMethods #quantMethods #dataScience #qualityControl

Am I prolific?

A slightly battered boat in Ballycotton, Co Cork.

#Ballycotton #BlarneyPhotographyClub #Boat #boatDetails #Castletownbere #Cork #CountyCork #fishingBoat #fishingCooperative #fishingEquipment #fishingNets #Ireland #IrishFishingIndustry #IvanWildeLtd #maritimeHeritage #Photo #Photography #Prolific #Skerries #workingBoat

New #surveyMethods paper replicates and extends differences in #dataQuality, attention, naivety, decision style, etc. by
- online #research recruitment platform (#mTurk, #Prolific, #Qualtrics, #Pollfish)
- device (#mobile v. #desktop)
- person's incentive

https://doi.org/10.3758/s13428-025-02618-1

Excited to share YEARS of research about how to get people to think reflectively and how reflection impacts philosophical judgments at the 2025 #APA in #NewYorkCity (January 8 to 11): https://www.apaonline.org/mpage/2025eastern

Can't make it?
- More about my talk: https://researchgate.net/publication/370132037
- More about my poster: https://researchgate.net/publication/371248872

Thanks to the #APA, James Beebe, and the Experimental Philosophy Society for the opportunity!

#decisionScience #philosophy #xPhi #epistemology #bioethics #cognitiveScience #mTurk #Prolific #UniversityParticipants #surveyMethods #DualProcessTheory

A question for the #HCI folks (please repost for more reach):

Since the #LLM cat is definitely out of the bag now, do you feel that #Prolific, #MTurk, etc. are still a reliable source of survey participants and consequently, survey data?