Google Maps used predictions of what 10,000 other drivers would do in the next 20 minutes to route you around a traffic jam yesterday. The company ranked emails in your inbox in order of relevance to you alone. Netflix selected a movie based upon viewing patterns of 247 million subscribers.
These are the ones everybody knows about. Of greater interest — and what I have been documenting over the last couple of years — is the movement of decision-making frameworks created using competitive AI systems into tools that help ordinary individuals select better options for money, health, and daily life. Once stripped of technical jargon, these ideas are relatively straightforward.
Regret Minimization Idea in 47 Words
Research at the University of Alberta, 2007. Rather than attempting to identify the optimal selection, determine how much you would regret each selection after seeing the outcome. Adjust over time to minimize that regret. If you repeat this enough times you eventually arrive at selections that cannot be optimized against
This is the entire concept. The original research document is 12 pages of mathematical proofs demonstrating that this method works. The concept can be explained in a tweet.
This is already appearing in products that likely contain no indication that this technology exists.
Examples
Financial Planning Applications
Old style robo-advisors were focused on maximizing expected returns. Newer applications are asking another question: what portfolio would this person regret less than any other across the spectrum of all possible market outcomes? There is no difference in terms of their worst-case scenario that a 28-year-old and a retiree want to avoid. A 28-year-old wants to miss the bull run. A retiree does not want to run out of money. Both are using the same math — different data points — but both are creating very different portfolios.
Dynamic Pricing
When a hotel room costs $180 on Tuesdays and $340 on Fridays, there usually is a regret-minimization application behind it. The goal here isn’t to price you as high as possible, it’s to set prices that will work acceptably regardless of changes in demand. Demand-based pricing tends to create extreme boom/bust conditions. Regret-based pricing is typically more consistent and for this reason hotels and airlines like it.
Medical Decision Support
A medical doctor told me that she uses an AI system to analyze her treatment options. The old way: which treatment averages the best result? The new way: which treatment minimizes the worst-case regret for this specific patient? A treatment that performs wonderfully for 90 percent of patients but fails miserably for 10 percent has a significantly different regret profile than a treatment that provides reasonable results for all patients. The tool brings this distinction to light. She stated that it altered the way she considers trade-offs – not just in medicine.
Navigation
Sometimes Google Maps directs you down a road that may be slightly longer than normal to avoid a highway where traffic is unpredictable. That wasn’t a glitch. The app minimized your worst-case delay, rather than maximizing your potential delay. A 25 minute route that is always going to be 25 minutes beats a 20 minute route that could potentially take 45 minutes. That’s regret minimization in action. You simply aren’t aware of the name associated with it.
Three Principles For Use Without Any Math
Judge your Process, not your Outcomes
AI researchers demonstrated that optimal decision making processes produce poor outcomes frequently. Therefore if you lose money during a downturn, you didn’t make a mistake as a financial advisor; you merely experienced variance. Fire your financial advisor for losing money in one quarter and hire someone who performed well in one quarter; you’re now selecting for luck. This principle applies to many aspects of our lives including hiring, investing, parenting and career development.
Two years ago I made a career decision that did not ultimately work out. The position was not right for me and I left within eight months. My immediate reaction was to place fault on the decision I made. However after reviewing my decision process, I realized I had done due diligence relative to this position -- I’d interviewed numerous individuals in similar positions, I’d assessed the risks involved appropriately and I had considered all available information prior to deciding. The information required to change my decision (a major leadership shift two months post-hire) existed when I made my decision -- therefore I would make the exact same decision again.
Match Effort to Reversibility
AI systems devote almost no computational resources to reversible decisions and enormous amounts of resources toward irreversible ones. Choose a restaurant: choose in thirty seconds. Choose a career: spend weeks on this decision. Choose something to view on Netflix: choose instantly. Choose whether or not to have children: take whatever amount of time necessary to make this decision.
Most individuals get this upside-down. They agonize over choosing what restaurant to dine at and move quickly through choosing careers. You should be spending about as much time contemplating the decision as the likelihood of your error will affect you negatively.
Compound Tiny Edges
The most successful AI systems do not make dramatically better decisions individually; instead they make slightly better decisions (typically 3-5%) consistently across tens-of-thousands of decisions. While this margin is nearly impossible to recognize when evaluating any single decision over the course of one year it transforms your life.
The human equivalent: Plan your day for five minutes every morning. Evaluate one assumption regarding your work every week. Discuss honestly with one friend per month something that you have been putting off discussing with him/her. Each one is inconsequential when viewed separately from the others. Compounded over five years, however, these habits transform your life. I can identify specific occurrences in my career where I attribute specific outcomes directly to insignificant habits that I began performing several years ago and continue to perform as part of my routine.
Uncertainty
While I am fairly candid about what I find difficult about this framework; it necessitates acceptance that you will never have sufficient information — not because you haven’t searched sufficiently for additional information; but because such information does not currently exist. Information will depend on decisions made by others who have not yet made those decisions; events that have not occurred; random variables whose resolution is unknown.
Many individuals react to this reality by either becoming paralyzed (over-thinking), or they deny it altogether (using their gut). The regret minimization framework presents an alternative option: make the decision that would lead to the least amount of regret given the variety of possible futures — and then quit thinking about it.
I’m not going to pretend that I’ve developed mastery of this technique. I still over-think some issues and under-think others. Using this framework — even somewhat inaccurately — is superior to having nothing.
The researchers working on developing decision-making theories for competitive AI systems took nearly two decades of rigorous scientific study and documented what good decision makers do automatically. We humans can utilize their “cheat sheet.
