improving how i (we) make decisions

The last time I wrote, it was about how AI will change education (and in fact, when I powered up my computer today, at the top of my inbox was this Education Week article: “AI could save school districts time and money—if they use it correctly").

AI uses algorithms to make decisions, and those algorithms operate by prioritizing certain factors over others. For example, internet advertising companies use an algorithm that prioritizes ads that are topically-related to your most recent browser searches (remember how spooky it felt when this first started happening?). It could just as easily prioritize ads for purple products, or ads related to food if you happen to log on at lunchtime.

Humans use algorithms when we make decisions, too, and that’s what I want to write about today.

Let’s say you’re the owner of a toy store and you want to open a second location. But where? There are lots of factors you’ll weigh, including your budget; how much space you think you’ll need; where your competitors are located; proximity to other businesses that would attract toy-buyers; your employees’ transportation needs; and more. Depending on which of these factors you prioritize in your own algorithm, you’ll likely arrive at different conclusions.

Then there’s the question of how you decide what factors to prioritize. If I’m the sole owner of the business, I could safely argue that I get to create the algorithm all by myself. Proximity to other businesses that would attract toy-buyers is top of mind for me, followed by the cost; if it’s a little smaller than I would like, or it’s a few minutes’ walk from the nearest bus stop, I can live with that.

​Now let’s say I am a public school district and I want to open some magnet schools in order to create more diverse school communities. (Set aside for the moment the question of whether there’s robust evidence that magnet schools are an effective way to desegregate.) Now I have to figure out what type(s) of magnet programs to offer. Some of the factors I could prioritize include:

• appealing to student interests (like  High School for Recording Arts in St. Paul)
• preparation for in-demand careers (like  Francisco Bravo Senior High Medical Magnet in L.A.)
• instruction that is particularly effective in addressing certain student needs (like ALLIES Elementary in Colorado Springs, which has a special focus on dyslexia)
• flexibility for students and families who need it (like Achieve Virtual in Indianapolis)
• leveraging unique community assets (like Normal Park Museum Magnet School in Chattanooga).

Any one of these factors would be a perfectly reasonable thing to prioritize in my algorithm—but they’ll yield different results.

In my magnet school example, who gets to decide what factors to prioritize in the algorithm? It’s not as simple as in the toy store example. Who is the “owner” of the business? Who is the “owner” of the decision about how to build the algorithm? Because it’s a public system, what special obligations does that create, if any, for community input?

District leadership, educators, school board members, parents, students, and community members all likely have ideas about what factors should be prioritized, and those ideas may not line up.

We talk a lot about “data-driven decision making” but I’m realizing that this may miss the mark. It’s becoming increasingly clear to me over time that our quarrel in decision making may not be with the data but rather with the algorithms the data are fed into.

​Sometimes if we have inaccurate or missing information we may have a data problem, but more often I think the friction is around:

1. Clearly stating what factors we’re going to prioritize in creating our algorithms. Often we lob pieces of data at each other and say, “How can you make that decision when the data show x, y, and z?” The data may show those things and those things may not be prioritized in the algorithm—both can be true.
2. Role clarity: Who owns that prioritization? Whose input influences it? Who needs to be informed (transparency) but doesn’t have a direct role?  Does everyone understand this at the outset?
3. Consistency: The prioritization should be stable in subsequent, similar decisions—unless there’s a logical and clearly communicated reason for the change. When this doesn’t happen, it breeds confusion and mistrust. Sometimes this takes the form of “Hey, you moved the goal posts!” or a sense that we are using data very selectively, and differently than before, to justify an end we’re seeking this time around.

This has been on my mind a lot this summer as APS staff, the community, the school board and its advisory councils weigh important, upcoming decisions about our facilities. It’s central to other aspects of our school system, too—for example, the school board’s annual budget direction is basically an instrument designed to prioritize factors.

​Maybe these insights have been obvious to everyone else and I’m late to the party (it wouldn’t be the first time). But it feels like this will be useful for me in my job moving forward—and perhaps it will be useful to you, too, particularly as we work to engage each other in constructive and trust-building ways.

Photo courtesy StartUp Stock Photos, pexels.com