#dtlsolar23 — Public Fediverse posts

"Sorry for the small graphic."

Sigh.

Paging EdTechHulk. Need some smashing of poor PowerPoint.

(Did the Hulkster make it off Twitter?)

[Hey, Derek, don't just complain, be useful.]

May I recommend Presentation Zen by Garr Reynolds? It will revolutionize your PowerPoint use. https://www.presentationzen.com/

#dtlsolar23

"These young people... their pets have Instagram pages!"

Um, my pet has an Instagram page. And I don't generally get "young" when people describe me.

#dtlsolar23

@nothingfuture Right there with you. So many generalizations happening here. #dtlsolar23

Frankly, there was a lot in this session about the logistics of gathering lots of student feedback, but not a lot of clarity on how student feedback can or should influence course (re)design.

For those interested in this topic, I would explore the Students-as-Partners model for involving students in course (re)design. https://www.centerforengagedlearning.org/books/power-of-partnership/

#dtlsolar23

I just heard a comment about dropping online program enrollment being the result of “screen fatigue” and I’m not sure I buy that assertion.
During the midst of quarantine, we watched late night tv get broadcast via shady webcams from hosts’ homes- but that was a thing we tolerated during that window of time. We can do better now (and so we do better…).
I wonder if students are increasingly intolerant of zoom-recorded-lectures and pandemic-panic-online designed courses and programs?
#dtlsolar23

I got an answer. Apparently part of the motivation for the async-only version of the course was the feedback the instructor received on the breakout room portions of the live sessions, that students were mixed on the value of those portions.

The instructor was also good with students who could skip live sessions and still complete the assignments. That was more reason for the async-only version (but not directly based on student feedback).

#dtlsolar23

Some of the presenters' takeaways: Having an active and engaged instructor interested in course redesign was key, especially if you're using weekly surveys. Also, those weekly surveys are likely to lower response rates for the institutional mid-term and end-term course surveys.

I've put in a question about whether the instructor has made any modality changes based on student input.

#dtlsolar23

In both the surveys and the focus groups, there was ambivalence over modality. Some students preferred pre-recorded videos, some preferred live video sessions, and some students were conflicted about the live sessions, seeing both pros and cons.

Btw, the qualitative data here (quotes from students from surveys and focus groups) are nicely compelling.

So, yeah, even the students are still muddling through modalities.

#dtlsolar23

@derekbruff This is sometimes difficult to talk to leadership about- hyflex appeals because they want every single student enrollment, but the disparities in experience and the load on the instructor are (at least in my opinion) unsolved problems. #dtlsolar23

What student feedback did the "agile" approach to course design gather and use?

* Weekly surveys from the instructor
* Week 3 and end-of-course surveys
* Voluntary student focus groups
* Interviews with the instructor and program manager

Why weekly feedback? Because the institutional survey didn't yield responses until after the course--when the next offering of the course was already live. And that feedback focused on the end of the course.

#dtlsolar23

Lots of questions here about modalities. Sounds like the instructor alternates offering a sync and async version of the course, and he doesn't like doing both at the same time. Also, HyFlex was a bust--he would commute an hour to the downtown campus and pay for parking and have 2-3 students show up in person. Sigh.

My takeaway: Many institutions are still muddling through the modality question, even now in 2023.

#dtlsolar23

This session is presented by Julia Scales and Casey Clevenger from the University of Washington's online / continuing ed division. They worked with an instructor to revise an online data analytics using student data and feedback. The course started on-site in Winter 2020 (uh oh), moved to sync online for a few years, and then piloted an async online version in 2023.

#dtlsolar23

Back at #dtlsolar23 for a session on building student feedback and input into online course design. Quick survey from the presenters asked for our roles on our campuses. I'm struck that the work I do is often similar to "instructional design" but I would never use that term to describe what I do.

#dtlsolar23

@derekbruff There was a story a week or two ago about people having used ML predictive modeling on smartwatches to predict Parkinson’s up to 7 years earlier (https://spectrum.ieee.org/amp/predicting-parkinsons-from-smartwatch-2662244110) so I totally *understand* why people want to jump to that… but I’m not sure relevant data sets of the size or confidence level exist in this space (yet). #dtlsolar23

One observation as this session wraps up: Many of the audience questions are about predictive analytics, in spite of Tom's warning at the start that they haven't built that out yet.

The #dtlsolar23 audience (instructional designers, educational technologists, online ed administrators) seem to be very interested in predictive analytics, maybe more interested in that than in dashboards showing what's actually happening.

Q from the audience: How do you communicate these tools and processes to students?

Tom C.: When we talk to students, they assume we're doing stuff like this already. They're surprised this is a new development.

#dtlsolar23

Let me emphasize this: This project is about making gradebook data (more or less) available to academic advisors. Faculty teaching these courses already have dashboard-like tools within Canvas.

Also, Tom says it's been important that a past provost mandated that faculty use their Canvas gradebooks. Otherwise they wouldn't have these data.

#dtlsolar23

On the roadmap at UCF: Developing some "at risk" indicators for students. Letting advisors sort by those indicators to aid proactive advising. Maybe push some messages to those students. And, of course, see if this is improving retention.

(Correction: It's currently being used by athletic advisors, not all advisors. That's on the roadmap, too.)

#dtlsolar23

They've rolled this out to advisors, including athletic advisors, as well as success coaches for online students.

Great feedback from advisors, but some limitations. Ex.: Every instructors has a different grading scheme, which takes interpretation. And not all post their syllabi to help with that.

Tom C.: If we can use this to automate the athletic reporting forms that faculty have to fill out every semester, we'll get 100% faculty buy-in.

#dtlsolar23

The resulting dashboard pulls out student engagement and performance data, the latter with comparison to class averages. Grades for individual assignments, noted as missed or late where necessary.

Advisors can access info for the students they advise, and the intent is to pull up that info before a meeting with a student. "High tech, but also high touch." No predictives yet.

#dtlsolar23

However, when they opened up the pilot to all athletes (not just two sports), Digital Learning didn't communicate that well enough to faculty. Example: Those advisors appear in the course rosters visible to faculty. "We had a freakout."

That led them to look at options for pulling learning data out of Canvas for advisors and others. Off-the-shelf products were fine, but not good enough. So they decided to build something.

#dtlsolar23

Up next at #dtlsolar23: Tom Cavanagh, vice provost for digital learning at U. Central Florida, on latest efforts toward a learning analytics dashboard to ID students at risk.

They had some existing data sets that were useful, but nothing that would work in real-time or near real-time.

First attempt used the Canvas Parent App, developed for K12 parents to "vulture" their kids, but maybe useful for advisors, esp. athletic advisors.

Good Q from the audience: If AI is going to take jobs, might it be taking entry-level white collar jobs? You know, the kind of jobs we send a lot of our college graduates to?

#dtlsolar23

Re: using AI to make the instructor “more present” in the discussion

I dunno, man. Like… is that really the *instructor* present, then? Is that really what students paid for access to?

#dtlsolar23

“And you can get citations”

Um. There are some lawyers that tried that and it’s not growing great for them.

https://news.bloomberglaw.com/business-and-practice/lawyers-ai-blunder-shows-perils-of-chatgpt-in-early-days

#dtlsolar23

My Q for the panel: Is there any practical way to pursue an academic integrity case against a student accused of using AI in an unauthorized manner?

Asim Ali: If the student admits to it, yes. Otherwise, no.

Tom C.: At UCF, I hear that the only cases that have been prosecuted are the ones where students confessed.

[I keep asking this question in different forums, and I keep getting the same answers. There's not a student misconduct solution here.]

#dtlsolar23

“Students are WAY out in front of us with using AI.”

I think that might have something to do with the profile of risk taking in people that age. Like, they also outpace my age group with car wrecks and unplanned pregnancies…

#dtlsolar23

Oh gosh, use AI to come up with strategies that your peers aren't using? These LLMs are precisely good at writing things that are commonly written. That's how they work. It will nail the genre but miss the specifics (sometimes dramatically).

#dtlsolar23

Since we’re talking about LLM and how they pass various exams, I’ll point out the two that they can’t pass (not even close, as it turns out): both of the high school AP English exams.
Recall? Yes. Think and have insights? Absolutely not. #dtlsolar23