#appliedml — Public Fediverse posts

Anyway, I keep meaning to write up a blog post on “falsehoods I have believed about measuring model performance” touching on #AppliedML issues related to #modelEvaluation, #metrics, #monitoring, #observability, and #experiments (#RCTs). The cool kids would call this #AIAlignment in their VC pitch decks, but even us #NormCore ML engineers have to wrestle with how to measure and optimize the real-world impact of our models.

Anyway, I keep meaning to write up a blog post on “falsehoods I have believed about measuring model performance” touching on #AppliedML issues related to #modelEvaluation, #metrics, #monitoring, #observability, and #experiments (#RCTs). The cool kids would call this #AIAlignment in their VC pitch decks, but even us #NormCore ML engineers have to wrestle with how to measure and optimize the real-world impact of our models.

Anyway, I keep meaning to write up a blog post on “falsehoods I have believed about measuring model performance” touching on #AppliedML issues related to #modelEvaluation, #metrics, #monitoring, #observability, and #experiments (#RCTs). The cool kids would call this #AIAlignment in their VC pitch decks, but even us #NormCore ML engineers have to wrestle with how to measure and optimize the real-world impact of our models.

Anyway, I keep meaning to write up a blog post on “falsehoods I have believed about measuring model performance” touching on #AppliedML issues related to #modelEvaluation, #metrics, #monitoring, #observability, and #experiments (#RCTs). The cool kids would call this #AIAlignment in their VC pitch decks, but even us #NormCore ML engineers have to wrestle with how to measure and optimize the real-world impact of our models.

Anyway, I keep meaning to write up a blog post on “falsehoods I have believed about measuring model performance” touching on #AppliedML issues related to #modelEvaluation, #metrics, #monitoring, #observability, and #experiments (#RCTs). The cool kids would call this #AIAlignment in their VC pitch decks, but even us #NormCore ML engineers have to wrestle with how to measure and optimize the real-world impact of our models.

You have a problem: you currently pick thresholds for model-based actions using some arbitrary heuristic.

Your solution: pick the threshold that maximizes expected utility (e.g. revenue, profit, ROI, …) instead. That’s the definition of the rational decision, right?

Hmm, for some reason you now seem to have several more problems.
#DecisionTheory #Optimization #rationality #AppliedML

You have a problem: you currently pick thresholds for model-based actions using some arbitrary heuristic.

Your solution: pick the threshold that maximizes expected utility (e.g. revenue, profit, ROI, …) instead. That’s the definition of the rational decision, right?

Hmm, for some reason you now seem to have several more problems.
#DecisionTheory #Optimization #rationality #AppliedML

You have a problem: you currently pick thresholds for model-based actions using some arbitrary heuristic.

Your solution: pick the threshold that maximizes expected utility (e.g. revenue, profit, ROI, …) instead. That’s the definition of the rational decision, right?

Hmm, for some reason you now seem to have several more problems.
#DecisionTheory #Optimization #rationality #AppliedML

You have a problem: you currently pick thresholds for model-based actions using some arbitrary heuristic.

Your solution: pick the threshold that maximizes expected utility (e.g. revenue, profit, ROI, …) instead. That’s the definition of the rational decision, right?

Hmm, for some reason you now seem to have several more problems.
#DecisionTheory #Optimization #rationality #AppliedML

You have a problem: you currently pick thresholds for model-based actions using some arbitrary heuristic.

Your solution: pick the threshold that maximizes expected utility (e.g. revenue, profit, ROI, …) instead. That’s the definition of the rational decision, right?

Hmm, for some reason you now seem to have several more problems.
#DecisionTheory #Optimization #rationality #AppliedML