Tag Archives: systems

Black Mirror’s “Hang the DJ” Explained

Even if you haven’t seen it, there’s a pretty good chance you’ve heard of Charlie Brooker’s series, Black Mirror. In its first two seasons, Black Mirror used its standalone episodes to explore the dark side of upcoming and speculative technologies. New seasons, however, have included more uplifting episodes that depicts technology actually helping peoples’ lives such as Emmy-award winning San Junipero’s time traveling lesbian tryst or Season 4’s Hang the DJ (HtDJ).

If you haven’t seen it, you should. Otherwise, the Verge has a decent recap of it here and you can watch the trailer below:

As HtDJ came to an end with its twist ending, it dawned on me: the app in HtDJ is very much like the algorithm behind Nanaya! Nanaya’s algorithm was published during January 2015 and publicized by the New Scientist. Maybe it’s wishful thinking, but just maybe it was one of many different things that  inspired the script.

Nanaya and the system’s algorithm in HtDJ seemed like they had some similarities. But to actually understand what those similarities are I needed to reverse engineer the HtDJ’s algorithm.

This post offers an overview of how the algorithm behind HtDJ’s system and Nanaya work. Next week, I’ll post the issues with the HtDJ. In the following week I’ll post an even more detailed interpretation of how the HtDJ algorithm works.

How HtDJ’s algorithm works

To really evaluate the effectiveness of the HtDJ system, we need to understand a little bit about how the system’s algorithm works. In short, the algorithm creates matches real-life virtual profiles to see which two matches are most likely to lead to a successful relationship, while comparing it to other possible relationships. That’s not too different from how Nanaya works! The figure below shows the general steps of the HtDJ algorithm.



 

But to understand how Nanaya’s and HtDJ’s algorithms are different, we need to go into more detail.

For starters, HtDJ’s algorithm creates 1,000 simulations, each one a timeline starting with a 12-hour date between the real-life matches. These real-life users are represented by bots, like virtual Amy and Frank, in a scaled time simulation. After that first date with their real-life matches, both bots are assigned new relationships with different bots for a different duration. Here, the algorithm can be pretty complex – finding the right pairings that force bots to evolve and react the most.

In this way, the simulation can learn more from each relationship and improve accuracy. For any of this to be realistic, the bots need to accurately represent how the real-life user behaves and changes their behavior based on interactions.

After a series of relationships, a certain condition in the algorithm is reached and “pairing day” arrives. That morning, both real-life matches’ bots are given a final opportunity to meet again.  Interestingly, at least in HtDJ, the coach said both Amy and Frank would be matched with a “perfect match” they didn’t know. Essentially they have a choice: rebel against the system (which could conceivably be not accepting Frank and giving up on relationships!) or accept a random relationship. This choice as presented in HtDJ is something that is evaluated in Nanaya as well.

Ironically, their decision to be paired or rebel is a simulated emotional decision. Every dating service that uses numerical matchmaking or the Nanaya algorithm, which uses economic, modeling, defines a consistent and absolute standard of what is a good match. Typically, a good match is one that’s close in the highly dimensional matchmaking space (read more about that here). But in HtDJ’s algorithm, a good match is one that your simulated bot “feels” really good about. This is a paradigm shift in dating apps – emphasizing the emotional sensation of good than what is abstractly considered to be good.

But again, it’s simulated emotion. The emotional decision needs to be mathematically calculated. Some theory of decision-making needs to have been coded or numerically achieved such that a finite set of parameters can be fed into its machinery to come to a single, binary decision.

So how would you even go about attempting to simulation emotions? That’s:

The complicated part

In each relationship, the bots’ personalities can remember, evolve, and learn based on the experiences they have within the simulation, leading to the variability between timelines as different types and durations of relationships occur in each one.

These bot identities and personalities must be explicitly defined in a way that accounts for what we understand as identity and personality in real-life. For instance, there’s inevitably some parameter like extroversion. Imagine the designers of the app in HtDJ have found a way to relate your phone conversations, cell phone photos, text messages, and even biometric data like heart rate relate to extroversion. By snooping in, the app can find out how extroverted you are in a bar after a long day of work or at a public park on a sunny day.

Going deeper, the algorithm can do more if their ex’s are also users. With more information about their past relationship, it’s possible to start classifying what combinations of identities and circumstances just don’t result in healthy, long-term partnerships.

After some time, the app can build a very rich profile of not just how you’re extroverted but many other traits, values, and how it defines emotions. To account for all aspects of personality needed for the simulation, more than tens of thousands of variables are needed (The Nanaya algorithm on the site uses about 1,000 variables, depending on the user).

But this fabric of identity and emotion has to evolve in the simulation. Otherwise Amy and Frank would have rebelled at the start. To evolve all these parameters, many equations or numerical routines, like based on machine learning algorithm, would have to define how any interaction changes identity and emotion.

Between tens and thousands of variables and non-linear equations that relate them and how they change, there’s a lot of room for error. There’s also a lot of computation required, which runs up server bills!

In the next post, I’ll talk about the impacts of these errors and server bills as I describe the issues in the algorithm and what solutions exist.

 

“Should I Dump My Girlfriend? Will I Find Another?”

Well let’s not get too hasty.

Earlier this week, I posted the Nanaya white paper to ArXiv: “Should I Dump My Girlfriend? Will I Find Another?” Or: An Algorithm for the Forecasting of Romantic Options. The title is a bit melodramatic but it’s a reference to Peter Backus’s “Why I Don’t Have a Girlfriend.” Apologies to everyone, especially Dr. Backus.

So this paper summarizes how the Nanaya algorithm works. I genuinely wonder who will have actually read it. Theory papers are hard enough to get through on their own and then harder yet when it’s an interdisciplinary mush. So we’ll go over some of the key points here.

The first major issue is uniqueness. Did we do anything academically worthy? Well, a lot of people have worked on the Secretary Problem (to be covered in a future post), but no one has really tailored it for a situation like romance. There are simplifications that do little justice to the complexity and information available to people – but those aren’t useful. Let’s just call it overzealous journalism…

We also do something new: comparing a specific relationship to being single and the chance of any other relationship. This is options analysis and rather new to the problem.

The Nanaya process.
The Nanaya process.

The rest is explaining what’s going on in the algorithm. Let’s check out the first figure:

So let’s break it down by each step:

Assessment of Inputs

When the Nanaya Beta is up (pending funding, be sure to call your local, friendly Venture Capital fund and request them to fund Nanaya) we’ll be asking a bunch of questions to understand your life circumstances:

  • What is your personality and social and romantic history. This gives us an idea of how likely you’ll be successful in finding a match, attracting a match, and how happy you are when you don’t have a partner in your life. We also need to know how long you’ve been in the city and job you’re in as that has a huge impact on how many people you’ll be meeting. You can check out this blog post that goes over this detail.
  • What is the ideal This is a mix of personality and specific values that are shared.
  • If you’re with someone, how do you feel about them.
  • The groups you interact with, like at work, geographically, and in socializing.

Determining Match Probabilities

With all of that data we can use our database and others available to us to figure out what the chance is of finding a match in any given encounter, depending on the group.

These numbers are typically low. Something like 1-in-10,000 is not unreasonable. This is where statistics comes in to play and you have something like the birthday paradox to offset those low odds.

Sociological Modeling

This is where we mix the match probabilities with your social behavior to figure out what are the chances of actually meeting someone. This is a bit of sociology and a bit of statistics to solve a problem no one’s really touched before, at least in this context (if I’m wrong, let us know). The sociology is based on personality, group types, and the results of some of our prototype experiments. The probability is based on binomial distributions and a variation of the Urn Problem. This is how we describe our social interactions…with urns!

The Nanaya implementation of the Urn problem
The Nanaya implementation of the Urn problem

We admit we make some simplifications in the paper, namely use of binomial probability distributions and assumptions that populations are very large. For almost everyone, these are pretty good assumptions but not for people in small populations. This gets into a lot of theoretical combinatorics which is well understood but not yet implemented.

Utility Function Valuation

This is the ugly philosophically ugly part where we literally put values to things that ordinarily don’t have numbers associated with them: like compatibility and happiness.

Using personality test results for who you are, who you want, and who your partner is we can estimate how happy you’ll be in any given relationship. We can also use your personality and romantic history to gauge how happy you’ll be in time as you’re single.

To simulate a random match you meet through the course of your life, we run a Monte Carlo simulation based on the groups you interact with. Some groups will have their own types of traits that lead to specific archetypes or segments (in demographic-speak) – we want to respect what’s observed in reality so we use these archetypes in seeding the randomness of the Monte Carlo.

Reporting the Results

With all of the above done, all that’s left is showing the numbers in a way that’s easy to digest!

If you sign up and take our personality tests, we’ll be inviting you to join our Beta program when it’s ready! So be sure to take your personality test and start today!

 

Chaotic Love: What math can tell us about how relationships evolve

I’m very proud to say that Nanaya solves a problem that hasn’t been solved before: determining the value in time of relationships and the option of possible relationships. The Nanaya algorithm has to be pretty complicated to do all of that – a machine of gears, pulleys, and switches that work together to provide a result. The machine has many parts of which are some have been around for awhile.

For instance, the problem of determining relationship quality and stability has been worked on. My old textbook, Nonlinear Dynamics and Chaos by Steven Strogatz, works out an example. This isn’t exactly how Nanaya works but we can take this as a “simple” example.

To evolve a relationship in time, we need to know a few things:

  1. Who are the people before their relationship begins? That is to say, what are their personalities and life circumstance before the first kiss.
  2. How do their personalities interact?

Because this is math we need these to be quantified. We can do that with personality testing and asking a few other questions.

Now what’s next is to set up a system of differential equations. Specifically, there is an equation for each partner that shows how much their affection changes in time based on who they and their partner are. We can consider the simple example of “Romeo” and “Juliet” from Strogatz.

System of differential equations for our lovers.
System of differential equations for our lovers.

Oh dear. So what do these equations mean?

So R(t) and J(t) are functions of love/hate in time. Let’s use dR/dt as an example, though you’ll notice both equations are very similar. R(t) is Romeo’s affection toward Juliet. When R(t) is positive, he is happy with Juliet; when it’s negative, he’s not. dR/dt is simply how much his affection is changing at any specific point in time based on the shared emotions of the couple.

At time zero we have R(t=0), which is Romeo’s interest in Juliet before the relationship.  J(t) and J(t=0) are the same values for Juliet. The values a and b are factors that multiple the feelings of the couple for a specific person. The larger a is, the more Romeo is happy that he is in love. The larger b is, the more Romeo is happy that Juliet loves him.

Well that’s a lot of math. Of course any model like this is a huge oversimplification of reality. Here we look at two dimensions: how Romeo and Juliet feel about each other. These can be broken into many, many other dimensions based on personality and other factors. The oversimplification is welcome as adding more dimensions allows for very complex behaviors that aren’t very predictable. Essentially, the system becomes chaotic as described by chaos theory.

As complicated as it appears it’s very easy to see what happens for various personalities and initial conditions!

So we can consider a few personality types based on a & b:

  • Excitable lover: where a > 0 and b > 0. For the excitable lover, any positive emotions from either partner just make the person happier and happier.
  • Neurotic lover: where a < 0 and b > 0. For the cautious lover, self-doubt causes affections to diminish but love from a partner can over come it.
  • Narcissistic lover: where a > 0 and b < 0. For the narcissistic lover, personal feelings are more important than how a partner feels.
  • Hater: where a and b < 0. For the hater, any love is toxic. Even if they have early affection, that disappears. The more they’re loved the more they’ll run away.
  • There are special cases where either a and/or b = 0. We don’t like haters around here.

That gives us ten different combinations of relationship types, ignoring those special cases. Even for each pair of relationship types there are several different outcomes. For instance, how identical are their personalities and how attracted are they at the beginning of the relationship? Let’s see a few different test cases.

These are parametric plots. In all these cases the arrow indicates going forward in time. Positive values indicate affection and negative are annoyance or dislike (i.e. top right is mutual affection, bottom right is Romeo loves Juliet but Juliet does not love Romeo, bottom left is mutual dislike). Units are arbitrary!

 Two Identical Excitable Lovers With Mutual Attraction

Figure 1
a = 5; b = 5; c = 5; d = 5; R(0)=0.1; J(0)=0.1

This is the predictable outcome when two excitable lovers, attracted to each other, fall in love. Romeo and Juliet are attracted and that initial attraction just builds in time to create a positive feedback loop of mutual admiration. They go off into a bright, love-filled future.

Two Excitable Lovers With Juliet Not Initially Attracted

Figure 2
a = 3; b = 1; c = 1; d = 1; R(0)=0.1; J(0)= -0.25

 Without being initially attracted, Juliet simply isn’t interested even though Romeo is. Romeo has a persuasive personality, which causes her to change her feelings a little bit but she remains unconvinced. As time goes on, the relationship is simply unsustainable and the two part ways.

Now what happens if Romeo is just a little more persistent? We can tweak the a from 3.0 to 3.106 to find out:

Figure 3
a = 3.106; b = 1; c = 1; d = 1; R(0)=0.1; J(0)= -0.25

In this case, Romeo can cajole Juliet just enough for her to overcome her initial ambivalence or lack of attraction to fall in love. In the near future, it’ll be a one-sided relationship but it will ultimately become more equal in love over time.

A Neurotic and Excitable Lover

Figure 4
a = -2; b = 1; c = 1; d = 1; R(0)=0; J(0)= 0.1

Here we have a neurotic Romeo. A little bit of attraction at the start from Juliet enables a relationship to form but it ends up a bit lopsided. As time goes on, it’s clear to see Juliet does love Romeo “more.” There’s nothing wrong with Juliet per se, just that Romeo needs to learn to love himself apparently.

An Excitable and Narcissistic Lover

Figure 5
a = 1; b = -10; c = 10; d = 1; R(0)=0 ; J(0)= 0.1

 

 

A recipe for disaster, it’s an unstable relationship! As soon as the narcissist Romeo feels too loved he wants his freedom but as soon as Juliet pulls away he wants attention. Because Juliet will keep loving Romeo so long as he’s happy, the relationship just oscillates between love and hate. It’s unclear what happens in the end but it’s truly something to avoid unless you really like drama. Maybe someone could write a play about it?

So there you have it, we can try to predict the long-term behavior of relationships with mathematical modeling. Reality is a lot more complicated as human relationships span more dimensions than mutual affection. With those extra dimensions comes the chaos that makes predictions all but impossible. It’s even harder when there’s no truly objective means of assessing personality and integrating life circumstances with personality as a means of determining initial conditions.

But difficulty be damned. At Nanaya, we’ve worked on isolating those most important parameters and determining relative long-term behaviors. We’ve made unpredictable, computationally intensive problems tractable and realistic. With the help of volunteers, we’ve validated our prototype outcomes. As time goes on, we look forward to more data as a means of further validating and improving our theory.

All’s Nerdy in Love and War

Time to take a break from writing about relationships and psychology and write about something close to home: engineering. Specifically, this post connects fighting wars to forecasting love though an engineering history lesson.

If there’s something humans are born to do it’s stay alive. While almost all creatures have the biological urge to “fight or flight” in dangerous situations, humans are pretty unique in our ability to plan to survive. Planning is hard though. It’s instinctive to “run away from the angry wild animal we just poked with a stick,” but as soon as you add more wild animals or people the complexity increases along with the ability of a group to follow through with the plan. Planning also takes knowledge, but knowledge one day might be obsolete the next. People can spend years planning for business, war, and love only to have them fall through at the onset. To make a good plan takes knowledge of all the things involved in the plan, the system, and flexibility to account for all possible outcomes.

Arguably, people weren’t very good at planning till the 20th century. By about World War II, technology finally caught up such that we could get information from distant events in real time. The scale and interconnectedness of the war forced governments to accept how mind-bogglingly complex planning is. The destructive power of modern weapons and speed of battle in World War II also gave a bit of urgency in formulating good planning.

There’s nothing like self-preservation as inspiration. What resulted was a new discipline, systems engineering.

From a system engineer’s perspective war isn’t just a series of battles. The system engineer zooms out to see a web of interconnected parts: birth rates, the time to heal an injured soldier, grain harvest yields, tank production, and even innumerable things like morale. An example in war planning was the German Tank Problem where statistics helped Allied war planners determine the need for tanks on the ground by estimating the number of German tanks.

After World War II, things got even deeper. The destructive power of bombers and tanks gave way to nuclear weapons that can level nations in less than an hour. To prepare for nuclear war required rapid decision making based on limited information. Thus, the fields of game theory and decision analysis emerged.

As computers were integrated into war, it became vital to understand how information can be gathered and communicated consistently and meaningfully. An effort was undertaken by America’s Department of Defense to create such a framework. In the 1990s the Department of Defense Architecture Framework (DoDAF) was created to help understand, assess, and communicate the complex systems in war. DoDAF is a language to communicate how complicated things work but it’s also a framework of thinking about them.

An example of a DoDAF "Operations View."
An example of a DoDAF “Operations View.”

NASA isn’t a part of the DoD but space travel can be just complicated as military operations. Years ago, I used DoDAF to understand how to plan and cost the construction of Moon and Mars bases over the course of decades. Unfortunately, it’s not as easy as putting things on a rocket and launching it. Finding the cheapest way of doing it takes a bit of effort and worrying about the details, such as how much cheaper a certain module is to build after building the first one and the weight of food.

Snapshot from the Exploration Architectures Operations Cost Model. This brings back memories.
Snapshot from the Exploration Architectures Operations Cost Model. This brings back memories.

But women are not from Venus nor are men from Mars. Assessing the dynamics of dating and romance actually takes some rocket science. The Nanaya algorithm was designed much in the same way that decisions on the battlefield are made.

We can all do simple calculations to find out the chances of finding a partner much like I discussed previously, but that’s a single, meaningless probability that isn’t descriptive of the process involved in finding that person. If we want something meaningful, we need to address the entire process.

Specifically, with the help of DoDAF we can build something like Nanaya. Specifically, building a systems model, such as Nanaya, requires the following:

  • Defining the “building blocks” of a system
  • Determining how the blocks are connected
  • Understanding the values and qualities of each block and connection
  • Generating the rules of how all these parts change in time
  • Assessing the errors in knowledge and estimation

Once we follow these steps, we can get the equations to determine what actually happens in the system. With these equations, we can develop an algorithm for decision analysis.

Overview of Nanaya "systems."

Overview of Nanaya “systems.”

Strange but true: the very tool developed to help fight wars was used to make Nanaya, a service to help people make sense of their love lives.

But it makes sense if you ask me. If the first thing people worry about is self-preservation, the next is intimacy and love. If we’ve developed the tools for self-preservation, shouldn’t we develop tools for love? It’s time to make modern love just as nerdy as modern war.