#realevaltalk — Public Fediverse posts

This goes for any researchers and evaluators, even if IRB doesn't apply or if you have an exemption. I would think this would be obvious, but it's disappointing to learn everyday what isn't obvious to so many people.

Others are trusting us with their data. Can we act like we deserve and will honor that trust?

#evaluation #research #RealEvalTalk #NoAI #glam #museums #ScienceCenters #nonprofits

https://digipres.club/@dsalo/117201597199538074

In my experience, this is straightforward incompetence that shows their underlying values. They simply can't be arsed, because they (as an organization, supervisor, designer, researcher) don't truly care about the people taking the survey.

#RealEvalTalk #evaluation

https://fediscience.org/@josh/117157781034342818

You want blind and low vision visitors in your museum and in your programs? Invite them in the conversation, invite them into community. Make them feel welcome. Do what they say and believe them.

Don't offload all that responsibility and all that necessary work for *your* understanding onto a evil and extractive company like Meta. Meta will screw everyone, everytime.

#museums #RealEvalTalk #Meta #wtf

Y'all, sometimes (many times) I simply cannot deal with the ignorance of so many museum professionals. This time, it's via a forum post.

Simon Mused, of the Scott Family Amazeum, Arkansas, thinks it's ok to give Meta glasses to low vision guests at this children's museum. Simon tries to wrap this in utter bullshit by using real phrases like "phenomenological interpretation" along with AI.

I'd suggest not trusting that museum with jack squat, nevermind kids.

#museums #RealEvalTalk #Meta #wtf

Deeply disappointed and angry at arts evaluators who are seemingly okay, using surveillance tech, like Placer.AI, because they see it as a shortcut to more direct and relational data collection.

Yes, let's give money to these assholes who buy and sell your data by scraping location and financial data up indiscriminately from anyone with any sort of device.

But I guess I'm the stupid one, because I'm the one that's unemployed.

#evaluation #RealEvalTalk #privacy #ethics #surveillance

I totally get that producing the thing - whether it's a program, materials, or etc - is the priority.

But why is evaluating what actually happened, effects, and/or people's reactions so rarely a priority? I don't get why it's often a binary with so many people (e.g., it "worked" or it didn't). There's potentially so much more to learn from whatever happened!

Plus, you know, evaluation shows that you honor and care about people and effort they put in.

#evaluation #RealEvalTalk

I mean, the irony is so goddamn deep.

You want to have a personal, authentic discussion with people about their careers and how that interfaces with resistance and social justice - and you have a fuckin AI chatbot that's taking notes and sharing it with who knows what. Because convenience.

SIGH.

I literally do not know why I need to keep having this discussion with adults when literal KIDS get it.

Museums, we don't have to be this way.

#museums #libraries #glam #RealEvalTalk #FuckAI

Yeah, this husk of the IMLS that remains can get lost with this re-alignment with bigotry.

It'll be years if not a full decade of recovery. So many people have been screwed this past year, and now we should take this "opportunity" to do further harm?

Nah. How's about no.

https://www.propublica.org/article/institute-of-museum-and-library-services-grant-guidelines-donald-trump

#IMLS #RealEvalTalk #museums #libraries #glam #funding #uspol

@DrSaucy @404mediaco Agreed, yet I'm sure we both have seen all kinds of work done under the banner of science that doesn't use any sort of validation or ACQ. With all the recent cuts to social science and data analysis work, I'm guessing there's going to be more people looking for shortcuts, so they won't do the things we know to do. It'll get pushed to an intern or put on the docket of someone without the necessary knowledge base.

#research #evaluation #RealEvalTalk

This is so incredibly short sighted and intentionally ignorant, too. Laying off so many staff - and the entire evaluation department (!!!) - in response to fascist-led federal cuts will hurt the museum and surrounding communities for years. Evaluation staff at SMM did all kinds of good and thoughtful work all around the state and were committed to equitable evaluation principles.

https://www.mprnews.org/story/2025/07/09/science-museum-of-minnesota-lays-off-43-employees

#museums #glam #EquitableEvaluation #FundingCuts #UnforcedError #RealEvalTalk #imls #nsf

So, you know how there's been this whole museums are not neutral movement for damn near a decade?

I hope museum executives who resisted that movement - who resisted acknowledging the reality of that statement and what it asks of them and their organizations - now realize the cost of their inaction, the cost of clinging to their status quo.

I can't tell you how often (my) reports were ignored because of the risk or because change wasn't convenient.

#MuseumsAreNotNeutral #RealEvalTalk #FAFO

I'm entirely baffled by evaluators continuing to promote the embrace of generative AI. They almost universally dismiss deep ethical and practical concerns with a hand wave and/or a shrug.

When it comes to my profession, I'll remember 2024 as the year that I lost respect for many of my colleagues and professional organizations. I hope 2025 will be the year they wake up and change course.

#evaluation #research #GenerativeAI #RealEvalTalk

@peter_mcmahan If some researchers and evaluators can't be bothered to write the questions, analyze the data, and report findings

- and all of these are things I've heard researchers publicly crowing in pride about -

then why should people bother responding?

#evaluation #research #LLMfail #RealEvalTalk

Thanks for nothing, generative AI companies and their hype people. You've destroyed a tool that's *actually* important for NLP research and evaluation. Many packages in python and R depend on wordfreq, and this is probably the way things will be.

Volunteer maintainers will increasingly just say, nah it's not worth it or even possible to sort through the AI vomit.

https://tldr.nettime.org/@adoranten/113168878466009372

https://github.com/rspeer/wordfreq/blob/master/SUNSET.md

#research #evaluation #NLP #GenerativeAI #YaFoundOut #RealEvalTalk

I'm letting go of the work/project search for now. I'm tired of abuse and extraction, tried of terrible practice being rewarded. I haven't seen much in the nonprofit world in the last year that wants to fairly compensate me for my labor and expertise. It's been deeply discouraging.

If you're doing work that matters in a way that also matters and you need an evaluator or researcher (hint: you probably do), please contact me.

#evaluation #research #RealEvalTalk #museums #nonprofits

Hard to go through the self evaluation fan dance as a leader when what you feel the most is 'fuck it.'

#RealEvalTalk

It really is stunning how some evaluators are sprinting to use and promote salamis "AI." Then again, when I reflect on the field and the people involved, I guess I shouldn't be surprised.

When I called some of his specific claims into question as well as foundational issues, the author of a promo piece just suggested another.

Folks in evaluation are wasting a tremendous amount of time and effort on something that introduces more distance and confusion.

#evaluation #RealEvalTalk #FigureItOut

@sps @jason_m_kelly Museum evaluators in general - in my experience - are huge supporters of this... Though it's not implemented by the organizations nearly as often as it should be. It's usually the first thing that's cut. #evaluation #RealEvalTalk