To engineer serendipity, one must have frameworks for understanding it first.
The simplest framework I’ve found for it is the Birthday Problem.
The premise: there are 23 people in a room, what are the odds there are 1 or more shared birthdays (or birthday “collisions”)?
The answer? Slightly over 50% chance of 1 or more collisions.
How is this possible? The odds of a birthday is small, 1/365. So what’s going on?
What’s going on is there’s a lot of possible pairs in the group! Person A has 22 pairs in the group, Person B has 20 pairs, one less because their relationship with A is bidirectional. Person C has 29 pairs.. etc. In other words, the number of edges is a summation of this series. A simplification of this summation is (N (N - 1))/2. For 23 people, this is 253 possible pairwise combinations!
In the Birthday Problem the way to calculate 1 or more collisions is to calculate the inverse statement, what are the odds there are no birthday collisions for each person, the first person is 100%, the second is 364/365 (because there is one other birthday now), the third is 363/365, etc.
For 23 people, this is 49.3% chance of no collision, which means a 50.7% of at least one collision. At 57 people, it’s 99%.
(There is a nice formula which takes an arbitrary collision rate and the number of nodes, I think I’ve re-derived it but want to double check it).
If we were to generalize this to an arbitrary collision rate (ie replace 1/365), where the rate is C and the number of nodes is N it comes out to be:
P(match) = 1 - (Product of i = 0 until i = N-1, (1 - C*i))
One way of modeling the world is that it’s an unconstrained birthday problem — we’re not just talking about birthday collisions, we’re talking about all kinds of collisions, all kinds of potential connections, at the same time. This is happening all the time by default and yet there are often discoverability problems that make connection hard to find. Talking to a stranger at a party or on a bus requires this awkward phase of exploration to run into this inevitable serendipity. So what if it were possible to make this default opportunity always readily accessible?
Each moment could be interpreted as a massive multidimensional vector of possible dimensions of connection. In a world where such a vector is legible for each person, they could readily benefit from the nature of life’s unconstrained birthday problem and opt in to whatever flavor of serendipity they’re seeking at any time.
For me, it’s nearly impossible to imagine a world without electricity. Well, I suspect serendipity engineering will be our generation’s electricity.
There are several interesting observations from this birthday problem framing of serendipity.
Firstly there are two main “knobs” to tune: collision rate (e.g 1/365) and number of nodes (e.g. 23).
How does this map to reality? How does each knob map to different strategies? Lets look at social structures as an example.
Within every group, both strategies exist. The admittance process into the group involves a filtering process that’s focused on increasing the “collision rate”. But once a member, a high degree of transparency allows all the pairwise connections to potentially interact. The first step is a kind of conditional value on unequal footing, the second step is a kind of unconditional value on equal footing, where everyone can see everyone else.
I’m going to call the former “depth serendipity” and the latter “breadth serendipity”.
Different social structures emphasize these two approaches to different degrees.
Exclusive, small social structures (e.g. labs, startup, cults, subreddit etc) apply a high up-front cost so that the internal “filtering rate” and depth serendipity within the organization is incredibly high.
Broader commons (e.g. forums, markets, social media, etc) are highly inclusive and transparent so that the number of edges is extremely high and great for breadth serendipity, but the filtering rate is more variable and haphazard.
These strategies are applied in tandem.
Universities have an application process for a first-tier filtering process, within this is a commons, from there there are smaller communities and departments which apply their own filtering process, and there are deeper and deeper concentric circles, and as you get deeper serendipity is engineered to be higher and more specific. Each social context comes with different flavors of serendipity.
At the highest level, this framing assumes social groups are fully connected graphs — ie all nodes can connect with each other.
This isn’t true of all types of social structures. It may be true of commons, communities, families, neighborhoods, churches, etc, but it doesn’t apply to hierarchies. Hierarchies often look like trees, and as such, the number of edges within the group is intentionally minimized. While this is efficient from a relational and communicational standpoint, a tradeoff is naturally fewer edges, and with it, less emergent possibility of serendipity.
What are different flavors of serendipity? What constitutes a collision, connection? It’s broad.
Symmetry, reflection, complements (polarities?), are just three types.
Symmetry (in this specific context) arises out of similarities. Similar experiences, histories, dispositions naturally leads to compassion, fellowship, solidarity.
Reflection arises out of differences, contrast, asymmetries. When you travel, you learn more about home than where you traveled. Our defaults are invisible, and a new frame of reference offers self-awareness on the specific axis of contrast.
Complements are differences that fit together well. Gender polarities are an example. The marketer CEO - technologist CTO pair, etc.
These are just three types of connection, and there undoubtedly many other types.
Overlayed on top of this model, in community contexts there are specific critical numbers, which affects serendipity at each stage.
2 is great for depth, 3 is the smallest set necessary for triangulation and stable instability (it’s prime, which prevents equality deadlocks), 5 is great for emerging group dynamic, 7 is at a sweet spot of maximum relationships before the number of possible relationships explodes, around 15 is another critical threshold where process and culture trumps any one person’s influence on the group dynamic, Dunbar’s number is about 150 etc
Interestingly enough, this roughly maps to critical numbers in military organization:
- Buddy Team/Pair: 2 soldiers (the smallest element).
- Fire Team: 4–5 soldiers (typically 2 fire teams in a squad).
- Squad/Section: 8–12 soldiers (smallest tactical unit, typically led by a Staff Sergeant/Sergeant).
- Platoon: 16–50 soldiers (commonly 30–40, typically led by a Lieutenant).
- Company/Troop/Battery: 60–200+ soldiers (typically 3-5 platoons, led by a Captain).
These critical points can be applied to the model, giving a sense of differential serendipity at each stage. The smaller the group typically the more oriented around depth serendipity, and the more embedded it is within concentric communities, such that they benefit from compound filtering. The larger the group typically the more oriented around breath serendipity.
What are some opportunities that would arise out of serendipity engineering, beyond pairwise connection?
It begins to opens up a world of spontaneous coordination, instead of the synchronous coordination that arise out of standardized units of time and industrialization. Spontaneous coordination would see coordination as a kind of memetic chemistry and cybernetic feedback loops. In such a world, convergent alignment could have a viral feedback loop where it’s met with broader promotion — a group “flips” states when a critical mass of its members are available, when the group flips, there is a group vector that updates state, and these inter-states can be compared in exactly the same way that individual pairwise vectors are.
I have an intuition that it also opens up a world of multidimensional net-positive alignment over unidimensional compromise-based voting. Expression of preferences in a much more multidimensional ways effectively means everyone “show their hand” and in doing so, low-hanging solutions can be found.
What does a serendipity engineering scheme look like?
In terms of coordinating activities, I would propose it consist of:
Group + Mask + Status + Channel + Casting
Group: a set of individuals, either public or invite-only. Assumes the group is a “highly connected graph”.
Mask: set by an individual, it privately excludes members of a community that the individual does not wish to connect with.
Status: consists of several components. There is a user-defined signal (e.g. in a routine context, “gyming”, “running”, “dinner”, set to true, etc). A signal can be more than just true or false, it can be an arbitrary number of signal options. There is a time-window (e.g. “gyming” could be active for 1 hour, “lunch” 30 minutes, etc). There is a logic template — this is boolean logic which allows arbitrary connection types (e.g. symmetry, opposites, complements) to be defined for arbitrary signal options.
Channel: Different methods for updating a signal — in a world of hyper personalization any number of actions could trigger a signal changing — opening the door when you arrive home (assuming there’s an individual-specific code, or phone is used for opening door, etc), tapping a nfc chip with your phone, etc.
Casting: Do you want to broadcast or narrowcast this status? Broadcast shows anyone in the group this status change. Narrowcasting only notifies individuals in the group if they also have a complementary status. A complementary status is decided through the logic template which determines whether two statuses constitute a connection. Given the notification hell most of us are in, I’d assume everything would be narrowcasted by default.
On the backend these could be combined. Lets imagine there’s an nfc chip in my kitchen. When I tap the nfc chip with my phone (channel) this maps to backend logic where I’ve personally set this chip to update my status “Cooking Dinner” from false to true. Assuming “narrowcast” is true by default, when this happens, this status is checked with a specific group which is mapped to this status — essentially a lookup occurs on all its members and if there are two active statuses (status update within its time window) the logic template is run, provided that neither individual is masked*. If unmasked and the template evaluates to true, the two users are symmetrically notified.
One nice property of using logic gates (beyond allowing arbitrary user-defined signals and connections) is that there are cryptographic schemes that allows this whole process to be encrypted, such that the service running the computation cannot interpret the specific meaning of signal updates or connections (that being said, they would be able to see a connection took place based on symmetric notifications being sent out). That being said, in the prototyping phase I believe I’d stick to using in high-trust communities (e.g. coliving) with everything being in plaintext (except for maybe the social mask, that could be a good place to introduce encrypted and work up to it)
How would you roll out such a system?
The world is constantly changing and what worked a decade ago has radically changed.
I’m going to leave this for another email, but I have lots of thoughts on it. For this system to be powerful, it would be interwoven into one’s physical environment. Given the degree of embeddedness, I would strongly encourage this to be run on a self-hosted instance, run by the group. That could be an instance run by a coliving co-op, a third space, an apartment complex, even a group of friends.