Apologies for disappearing.

This chronicle has been the hardest to write so far. Procrastination galore. Not a problem to get started, but constant revisions, insecurities, what do I want to say.

To some degree it’s just shame and disappointment in myself, so perhaps it is important to get that out of the way: To you as a reader, I disappeared off the face of the earth, no notice, no goodbye. I was in the middle of weekly chronicles, a series on d3.js - and suddenly I was just gone. I realize that I’m not the first creator to do this, but I had never done that before - some of you reached out with genuinely caring emails, to which I neglected to respond. I’m sorry. Your caring deserved a better response than silence.

Regardless of whether you think I need to be or not, I am ashamed of dropping out like that. But I was just too broken, too burned out, too many entangled threads, too much stress, too much unresolved practicalities, too much financial troubles and too much accumulated disorder, too many neglected relationships, too much pent up need of rest, too much unexamined time, too much unattended, too many addictions still unbroken.

I am very proud of how far I’ve come in my maturity and self-development, I’ve spent more time in formal meditation over the last 4 years than many do in a lifetime. But what they don’t tell you about enlightenment (if that is what I can call my progress) is that all the actual accumulated entangled realities that have built up over the years are still there, and must still be dealt with. Untangling inner knots is necessary to untangle the outer knots effectively, but the outer knots will still demand time and effort to untangle.

Substack recovery.

If this is reaching you, it means that my migration of your subscription to funfun.email to Substack from customer.io has worked. I had quite elaborate plans to run a massively professional newsletter with personalized ads and stuff with customer.io, in a wonderful sovereignty-fuck-you to the big creator platforms. In reality, I was being arrogant about how much effort I could muster.

Possibly because there have been (several) times in my life where I had the energy and grit to deal with DKIM records and bounce rate optimization alongside figuring out how to make rent next month. To some degree, it’s an I’m-too-old-for-this-shit, but this is more that I have to come to terms with the fact that I am in recovery mode – and it has been made worse because I have been denying recovery to myself for so long.

So very, very long.

The Return of Fun Fun Function was doomed from the start.

Watching the announcement video, I am demonstrating suitable levels of YouTuber energy, perfectly masking my actual state. In reality, I was burnt out, desperate, at the end of my rope, both in terms of confidence, financial security, and quite frankly, identity. I’ve never been laid off from a job in my life, and during 2023, I was fired twice, which disrupted me so much more than I could have imagined - at one moment, I expressed to my girlfriend that I didn’t know who I was.

Restarting Fun Fun Function required a hero, but at that point, I was the one that needed saving. However, that kind of vulnerability was unfamiliar terrain to me. Up until that point in my life, I had been a Prince of The Internet - a happy dolphin swimming the digital seas. Computers and the world of computing had always treated me well.

Me and the Internet grew up side by side.

I was born in 1983. I was the last generation to do BBSes, my first phone was an Ericsson T18 and the site that I’ve wasted most time on is Slashdot. Eventually, in my later teens I found my people and place at the Sitepoint.com Forums, as a moderator and article writer - it’s where I published my first piece of authored code: A DHTML tree menu written for Intellected, which was a home page I built to document my interest in mnemonic techniques. It has long perished, but I actually found some remnants of it on Wayback Machine, demonstrating a wonderful Windows XP-like aesthetic and what was my favourite hex color back then: #99CCFF.

The timing of all this meant that I learned computers before computing was the shit. Last year, a friend of my father came up to me at a family gathering and apologized to me for telling me when I was a kid (and I have zero recollection of this advice) - “Mattias, this computer interest of yours is a good hobby, but eventually you need to also learn something that you can do to earn money”.

Programmers in 2010:
Rare, expensive, magical cattle.

I kind of coasted through the job market for about 25 years or so - my unintentional timing could hardly have been more impeccable. My peak was working for Spotify during the glory days of tech - long before Cambridge Analytica. All was fine and dandy. Programmers were a kind of rare, expensive, magical cattle that needed to be exercised at ping-pong tables and take regular baths in ball pits.

I was part of the Playlist team, so I maintained code that literally touched hundreds of millions of people’s everyday lives, which is something I had strived for since I learned programming, believing that meaningful work would make me content and fulfilled. But one day, wondering why I was depressed even though I had 500% objectively made it, I looked through my journal and realized that I had spent an entire year optimizing scroll performance of long playlists.

I don’t mean to belittle my work on scroll performance - that is genuinely important work, especially at that scale - I likely knew more about DOM redrawing than 99.99% of people on earth and I’m proud of that part.

In fact, I love and admire geeking out about insanely niche things - I found myself mesmerized and inspired from this story about Fil Riviere spending 4 years on digitizing the Bertin projection.

The problem with my work of playlist scroll performance, and what made my work different from Fil’s work on the Bertin Projection, is that his came from within him - it is part of what builds his unique oeuvre, not something he was commissioned to do.

Always start this weekend.

It is those inner embers that eventually compelled me to buy a Blue Yeti mic on a friday and shoot the first episode of Fun Fun Function that same weekend. Not even with a proper camera - I just used my normal laptop webcam. I had a TV production background from before, but I knew then that I would procrastinate the project into oblivion if I allowed myself to even google for one second what camera to get.

Just. Get. Started.

I said that I spent a year optimizing scroll performance - though if I’m honest, I spent a ludicrous amount of that time procrastinating from work answering questions on Quora.

Quora was the best place back then, incredibly pure, with chemists answering my questions about how to keep muffins fresh, and gut-and-heart-wrenching writing from inmates at San Quentin answering what the first day of a life sentence feels like. I found a niche answering JavaScript questions and managed to transfer some of the attention to Twitter, which became the seed for my first subscribers on YouTube.

Demons captured in commitment boxes.

Nowadays I have a clinical diagnosis of ADHD with medication, but back then I made do with various coping mechanisms. Being justifiably fearful of my tendency to procrastinate, I constructed a commitment box made of adamantium for myself. You may know it as the phrase “Good Monday Morning”, the line that opened every episode of Fun Fun Function — I know it as the prison I intentionally built for myself that allowed me to do what is still, in many ways, my life’s work.

I produced Fun Fun Function every week, largely without fail. For the first two years, during the weekends, and for three years after that as my full-time job. Some weeks it was mostly a delight, some weeks it was mostly a chore. But it was only the first few episodes, before I locked myself in with “Good Monday Morning” that felt creatively pure, if that makes sense. After that, there was always, to at least some degree, a struggle, a tug of war, between the intrinsic motivation and the duty of consistency. I never really got that intrinsic fire back, not completely.

Some of the more intense struggles showed up in the Losing Motivation video and Worst Hello World Ever, that were two examples of solutions to the problem “how can i make a video when i dont want to make a video”. I constantly hoped that I would find the tactic that would keep me engaged consistently, but my inner motivation refused to let me re-use a prior strategy. The demon of novelty would not be satiated for long.

A return to Fun.

Where am I going with this, you ask? I guess it is a restatement of what Fun Fun Function is, in a more authentic way. funfun.email will be at the center, I might produce videos from time to time, but that will be rare. AI and other tooling has augmented video production a lot, so producing great video is more efficient than ever, but it also means that it’s more competitive than ever.

Hypothetically, I could make something “good enough” but for some reason I am just not built for that. My creative neurology seems to only thrive in the point of crisis, in the crux of complexities, the tipping and turning points, the forefront. Whenever I try to aim “easy” production, and reap the rewards of my built skills, and “do a standard video with one hand tied behind my back” sort of thing, it just doesn’t WORK.

It’s like oxygen is drained from my blood, and I procrastinate until the last few hours and reflexively invent an over-ambitious production complication to raise the stakes and make something much more involved in less time - blasting past the deadline, the guilt of being late fueling a feverish fervor, generating the energy required to push me past the finish line with something convoluted enough that my inner demons deem adequately worthwhile.

I can find myself working for insane lengths of time on intrinsically motivated things, but if I introduce even a whiff of external motivation into the work, my neurology tends to drain the blood from my face. For years, I did talks happily across the world for free, but the first time I was actually paid for a talk, suddenly preparing it was like pulling teeth.

I really miss the days of creativity without duty or ambition mixing itself in, without the idea of something being a startup or that what you’re writing needing a certain number of views. I miss non-performative creative expression. Just doin’ something because I find it deeply interesting, following a subtle whimsy, respecting my own salience.

So I’m going to just be doing funfun.email for a while, and very intentionally NOT with a regular cadence, and writing when I want to about what I want to. And absolutely no ads or subscriptions for the foreseeable future, I want to see what it feels like to do this with absolutely no pressure.

When I say Return to Fun, I do not necessarily mean pleasurable or even light-hearted - but more about a stance toward how the world is changing at this time in history.

So this is not a return of Fun Fun Function, but it is a return to Fun.

A computing cavalcade.

It is very unclear where AI is going to be taking me (or any of us), career wise, in the coming years. In many ways, it’s liberating for me that AI breaks the economics of code, because in a way, the fact that coding was so valuable was what made me stop doing it for fun and instead doing it for others, paying me. There was a period in my youth where I would obsessively ideate business plans that I ought to do, and now it is incredibly hard for me to think of a single idea with even a reasonable moat.

Looking at my feeds, it seems clear that coding agents are doing incredible things in terms of art and software development, and human creativity in general - the amounts of ingenuity I see on a daily basis is absolutely staggering. Tooling is created at a dizzying pace, some useful, some delightful, by people that could not fathom coding before.

Which is quite disconcerting on so many levels, but also - all risks and disruption and collateral damage put aside - it’s quite something to see upfront - what a show.

Especially for someone that grew up with CPUs. I have silicon’s cell from the periodic table tattooed on my left arm — Si, element 14, the fourteenth thing the universe learned to be, and somehow the one that learned to think back. A reminder of how much these things have affected me, and how much of what I have and what I am, I have computing to be grateful for.

I find myself grateful that I am just old enough to have my first computer be an Amiga 1200, a very powerful gaming-oriented computer with mind-blowing graphics running on a Motorola 68EC020 @ 14 MHz and 2MB memory.

Leaning fully into this, what we live with today in terms of capacity is so awe-inspiring that my eyes well up with tears if I allow myself to fully feel it:

A single H100 does roughly 1,000 trillion BF16 floating-point operations per second. For the sake of argument, I’ll assume that you need 8 of those turkeys to run an inference shard of Opus 4.7, requiring around 8 petaFLOPS. Compared to my Amiga, that’s a factor of about a billion times more arithmetic per second.

If my Amiga had been computing one Opus token since the Big Bang, it would still be nowhere close to finishing the matrix multiplies that the actual hardware does between pressing Enter and the first word appearing.

I find myself deeply privileged to understand computing well enough to appreciate what is happening - but also understanding the upcoming challenges currently hidden because providers are subsidizing what compute really costs. I’m currently reading eagerly about Quantization and prompt caching because god knows we need to make these things run much, much more efficiently.

What if these are the good old days?

I would not for the life of me trade this for flying cars or jetpacks - there is no other age I would like to be in. Don’t get me wrong - I am not a person that is positive about the future. Nor am I negative. I subscribe to being optimist as David Deutsch defines it:

Last year, I wrote a chronicle about William Playfair, a statistician that invented the line plot, that lived in the era of Descartes, Napoleon, David Hume, Adam Smith, the French Revolution, the invention of the steam engine, protests (and hangings) against agricultural automation. Playfair lived during a titanic shift in history and knowledge, arguably in the greatest melting pot of engineering and science since Ancient Athens.

I wrote that chronicle from a hotel in Granada, being quite drained and feeling quite small in life, and I remember at the time feeling a certain pang of jealousy against Playfair for living in such a defining era of humanity. I realize now, with my feet a bit more on the ground, with more security around me, and some rest and silence, that someone might be writing about this point in time with that same reverence.

On some days, it is hard for me not to be overwhelmed with the meta-crisis that the world is going through, but on good days, when my mind is clear and sharp, and my heart beats strongly, I remind myself of the etymological origins of the word crisis:

Things are currently too much in flux for me to see clearly what my role ought to be in all this, and I suspect I share that experience with you, and most people that have spent their entire career in computing.

For now, we just have to stay open, experiment, take interest, play - and wait.

...which is, I suppose, another way of saying:

Stay curious.

The slow cartography movement and a bearded cajun grandma

This week, three stories caught my eye.

A cartographer spent four years implementing a projection he thought would take a weekend.

A systems scientist extracted networks from hundreds of corporate organization charts using no AI—just "two human brains and a laptop."

A New Orleans chef hosted Monday night dinners with exactly the same menu for decades, creating one of the city's most vibrant communities.

It's always hard to know why certain stories touch us, and quite honestly, it's almost always a post-rationalization - however, I do think a word that connects these three is "narrow".

The Cartographer's Patience: The Tale of the Bertin Projection

"I thought that automating the Bertin projection by computer code would be a weekend project. It took me around 4 years."

That is a little tidbit mentioned by Fil Rivière (posting on Bluesky as @fil.rezo.net) when interviewed in “We can always talk about maps”, an absolutely mind-bogglingly dense AMA with him on the Observable Blog.

In case you aren’t super into maps, you might think it alien that one would spend 4 years on that, so let me go into some background.

Aaron Sorkin leveraged The West Wing to edify his audience around various niche matters, and one of the best examples of this is a scene where he highlights how much the Mercator projection is the most common way of showing the world, but just isn’t very truthful.

This is a rather vast subject out of the scope of this chronicle, but if it piques your interest, I suggest What This 3-Minute Scene From ‘The West Wing’ About Maps Can Teach You About Leadership on Forbes.

Jacques Bertin was a French cartographer and theorist who revolutionized information visualization in the 1960s. His seminal work, "Semiologie Graphique" (The Semiology of Graphics), published in 1967, established fundamental principles for data visualization that we still use today.

Bertin developed a unique map projection that prioritized showing relationships between regions rather than precise geographic locations.

Unlike the Mercator projection (which preserves angles but distorts size dramatically near the poles) or the Peters projection (advocated by the characters in the West Wing episode, which preserves area but distorts shape), Bertin's approach focuses more on being conceptually correct - it preserves almost all of earth’s surfaces without distorting continents too terribly, and instead pushes distortion to the oceans.

Bertin would manually redraw maps to emphasize the data relationships he wanted to highlight—essentially treating cartography as an act of visual communication rather than geographic precision. His maps often looked "wrong" to the traditionally trained eye but revealed patterns that conventional projections obscured.

Bertin's maps embodied what he called graphic permutation - the ability to reorder visual elements to reveal patterns that would otherwise remain invisible. His matrices and reorderable matrices (what he called "dominos") allowed mapmakers to physically rearrange geographic data to discover correlations and structures.

Bertin would manually redraw maps to emphasize the data relationships he wanted to highlight—essentially treating cartography as an act of visual communication rather than geographic precision. His maps often looked "wrong" to the traditionally trained eye but revealed patterns that conventional projections obscured.

Bertin's maps embodied what he called "graphic permutation" - the ability to reorder visual elements to reveal patterns that would otherwise remain invisible. His matrices and reorderable matrices (what he called "dominos") allowed mapmakers to physically rearrange geographic data to discover correlations and structures.

As I was looking into Bertin, I found this gem of an interview with the late Jacques Bertin on WaybackMachine originally published on infovis.net, but now saved by WaybackMachine).

Interviewer: Your book “Semiologie Graphique” was published well before the boom of personal computing and computer graphics. Nevertheless the avalanche of data present in Internet and the impulse that computers have provided to visual communication mean that many people associate Information Visualisation with computers. In your opinion, what is the role that computers have to play in this context?

Bertin: The use of computers shouldn’t ignore the objectives of graphics, that are:

1. Treating data to get information.

2. Communicating, when necessary, the information obtained.

Computers are able to multiply useless images without taking into account that, by definition, every graphic corresponds to a table. This table allows you to think about three basic questions that go from the particular to the general level. When this last one receives an answer, you have answers for all of them. Understanding means accessing the general level and discovering significant grouping (patterns). Consequently, the function of a graphic is answering the three following questions:

A. Which are the X,Y, Z components of the data table? (What it’s all about?)

B. What are the groups in X, in Y that Z builds? (What the information at the general level is?

C. What are the exceptions?

These questions can be applied to every kind of problem. They measure the usefulness of whatever construction or graphical invention allowing you to avoid useless graphics.

The Bertin projection was created in 1953, but (in 2013) there was still no formula to describe it. It only existed as a drawing, with people passing around copies of a base map in Adobe FreeHand.

You might like to read a longer writeup by Fil about the Bertin implementation on Visionscarto, but one detail that I find wondrous is how short the implementation is in the end, when implemented for d3-geo (Github), only about 20 lines of code!

I think you'd be hard pressed to find a better example than this that  illustrates why D3.js seems borderline immortal as a library. This 4-years-of-research-distilled-to-4-lines-of-code showcases very well how the abstraction level and modularization that D3 provides is surgical to the level of uncanny:

On a grander scale, it is also an incredibly concrete example of how programming languages (and math) is human language for expressing very specific concepts, not the language computers (or nature, for that matter) speaks.

Programming languages and math is what we use to speak to one another, and to think clearly, both ourselves and together.

This is not really rocket surgery to realize but it really bears repeating, because it is not obvious to an outsider. You've probably heard the misguided but popular sentence "the next programming language is English" a couple of times in the last few months, and reading this implementation is a clear demonstration of why we have programming languages (and math). There is nearly no accidental complexity in this implementation, i.e. no technical aspects to hide. Expressing this in English would not only be awkward, it would be impossible.

When we automate, what loss is the most subtle?

I do use AI tools extensively (and proudly partner with some) but I think that an enormous amount of the excitement right now is actually the hope for "artificial work" or "artificial interest" or "artificial attention".

Fil: "When I started making maps, I thought that automation would be a good idea. I wanted to make a 'machine' that would save Reka a lot of time. I've since realized that there is a lot of value in making maps slowly."

Laziness is important. It is especially an important trait as a programmer, but cultivating a narrow specialization and respect and interest for what you do is almost a value in itself. Automation, especially the kind of automation that is currently receiving trillions of investment, almost inherently needs to gloss over details.

Fil: Sometimes I see maps using a dataset as the source of truth. If a point is missing in the dataset, then it's often excluded from the map, and people will just say, 'Oh, there's no data.' But that's not true. It's not that there's no data, there's just no data in this dataset.

_Take a statistical indicator of literacy by country. That value is likely not changing much from year to year for any given country, so we may be able to infer literacy data from another recent year. But, we tend to not do that research anymore. We don’t say “Oh, that’s an interesting question, I have to call someone to find the answer,” or try to interpolate or extrapolate values from existing data.
_

We probably need a slow cartography movement, similar to the slow food movement. And there are still people who are doing the work very slowly, even though it’s much easier to make quick maps that have gaps. Taking the time to go through everything carefully, and asking questions when something isn’t obvious, is really important.

God I love this guy, if I were a zombie I'd want to eat his brain so much.

"A couple of Human Brains and a Laptop"

This theme of narrow focus yielding unexpected depth appeared again in the Organizational Networks episode of the wonderful Data Skeptic: Graphs & Network podcast, interviewing the wonderfully interesting Hiroki Sayama (posting as @hirokisayama on Bluesky), a SUNY Distinguished Professor of System Science and Industrial Engineering.

Hiroki led a project to extract network structures from corporate organization charts in PDF format. If you've ever seen corporate org charts, you know they're a zoo of inconsistent formatting nightmares.

Some might assume that the solution here was to throw thousands of dollars of GPU at this problem. Hiroki's approach? Pure heuristic image processing—essentially developing logical rules step by step.

Hiroki: We didn't do any AI or machine learning kind of stuff. Everything was based on logical steps. And yet we were able to generate the graph object for 46% of the original images.

Hiroki isn't an AI luddite - they leverage AI extensively in other projects elaborated on in the podcast, but for the particular case that they had in the above, there just wasn't a 'right' way to train the model, the problem domain was just too ... unsolved, and then we need to roll up our sleeves.

The Soul of a Table: Pableaux Johnson's Monday Night Ritual

At the end of every Monday in New Orleans, people gathered for a meal, around a Formica-topped oval table.

"In New Orleans, supper is never just a meal. Nowhere do Louisiana's French roots show as deeply as in the centrality of food to its social life."

The table wasn't particularly elegant.

It wasn't from a formal dining room but from a kitchen—the one where Pableaux Johnson's grandmother had fed him and his 23 cousins across generations. As he would often remind guests, "Tables have souls after a fashion. They retain the traces of all those who have gathered. And this table needs to be fed at least twice a week."

So he fed it, creating one of New Orleans' most vibrant dinner communities through a ritual of radical constraint:

The menu never changed: red beans and rice, cornbread, and bourbon for dessert.

There were rules -

• no phones

• don't sit next to someone you know

• pay attention to those around you

Between 8-12 people would gather each week, a deliberately diverse group that Pableux curated with the same care as his slow-cooked beans, ensuring that the same constellation of people never occurred twice.

Pableaux died the other week, which made me aware of his existence because the foodie internet exploded in remembrance. I particularly enjoyed reading/listening to [“It’s not a dinner party, it’s just supper:” Monday nights with Pableaux Johnson over at The Splendid Table.

What's appeals to me about these dinners wasn't what they included, but what they deliberately left out. No menu choices. No technology. No sitting in your comfort zone.

The constraints weren't limitations, or I mean, they were, but they were the secret ingredient. By narrowing the focus, Pableux created an environment where the only thing to do was connect deeply with the strangers beside you and the familiar food in front of you.

Assumptions and defensive optimizations

Looking at the unassuming 20-lines of bertin.js projection, you'd be forgiven for missing that there were four years of research by a very competent person behind it. In the same way, when I read Pableaux talking about his praised dinners, the developer in me really appreciated the near-algorithmic thinking going on below:

Pableaux:

It didn’t start off quite as random. Before I lived in New Orleans, I lived in Austin [Texas] for about 10 years. My entertaining style in Austin was doing whole-day parties – big gumbo and red beans parties – that eventually 150 or 200 people would come to over the course of a Sunday.

When I went to New Orleans, I had my grandmother’s table. I decided to switch to weekly because one of the long-standing food traditions of New Orleans is red beans on Monday night. Instead of doing 150 people once every three months, it became 10 to 12 once a week. That changed the rhythm of the way that I cooked, and that changed the menu I served.

It adapted to something that was easier and lower maintenance to pull off on a Monday. People aren’t often doing things on Monday nights so it tends to be free and a good break during the early workweek. It’s something to look forward to, and that’s where it came from. Once that ritual and rhythm was established, you have 10 to 12 seats to fill every week.

It started off with a smaller group of friends, then it became bigger, and everybody rotated in and out until it was a large-format, kismet-driven, weekly event that everybody looked forward to.

[...]

Interviewer: I love that it’s the same thing, not just in terms of ritual, but you have this thing that you’re doing every single week on the same day. It’s not entertaining. It’s not a dinner party. It’s not, “How am I going to impress people? What am I going to do?” You’re just feeding people, and it’s a constant. It’s as much about the idea of ritual, community and family as it is “Hey, let me cook for you this awesome thing.”

[...]

Pableaux: There have been attempts at different times for people to do things like, “We should bring a salad. Can I bring a salad?” I try to gently discourage that. Part of it is I don’t have 12 salad plates. And once you start adding different complicated components to it, things get complicated very quickly. That’s also about twice as many things I have to clean up – and I’m cleanup adverse. I like having it be “serve yourself.” I put a roll of paper towels on the table for linens. I put the spoons in the middle. Everybody gets their own drink and it’s self-serve. If you want more, get up and get it.

The above exposition strikes me as both so down to earth yet also algorithmic. It is iterative, like good development - assuming nothing, generalizing and optimising defensively. I know from hiring a lot of career-switchers that journalists and lawyers make great programmers quickly, and I theorize that chefs do, as well.

Will you be narrow for me this week?

It is easy, in this particular day and age, for me to feel the pressure to know more, do more at scale, automate more, summarize more, absorb more.

But perhaps another tactic, another response, is to slow down, pay more attention, learn to be more manual, more precise, with deep engagement of a specific problem. Go narrow.

Be a light, not a judge. Be a model, not a critic.
And, as always, stay curious. 🧐🐒

Mattias Petter Johansson

Double video release today!

All right, I'm dropping two videos today! They premiere at 9AM PST / 6PM CEST, if you want to watch the premiere live with me and other FFFers, you can do so here - that first video premiere will automatically redirect to the second video.

You know what they say, a picture says a thousand words, a video says a lot more, but not as much as MPJ talking about two of his videos:

Video 1: Line Plots are Plot Lines - A data visualization spoken word cover of funfun.email #12

It was always a little dream of mine to take the best of the chronicle and make video "covers" of them, but what format that should be was less obvious.

One of my favorite chronicles last year was absolutely #13, the plot lines one, and when I started exploring how a video script for it would be, it just turned more and more poetry the more I worked it, so I decided to lean into the theatrics and let it be over the top, and it was so delightfully fun to make this one, I hope you appreciate it:

Line Plots are Plot Lines - A data visualization spoken word cover of funfun.email #12:

It is common for us to be touched by stories.

Rarely are we touched by digits.

But digits can be turned to line plots

and line plots can express a plot line.

Have you ever attended a spellbinding budget meeting?

I have.

It was at a meditation center board meeting.

I have been in many budget meetings in my life,

and they are usually dominated

by tables of numbers,

profit and loss statements,

revenue at the end of this quarter or that.

I also expect the audience to

covertly simulate

not zoning out.

This meeting was different.

The treasurer enthralled the audience - iconoclastically eschewing tables of digits and instead presented the metrics as time series,

plotted on a line,

line plots,

plot lines,

telling the story of how donations flowed to the center every year,

more in the summers,

and how liquidity increased after the summer due to a loan taken,

and how that liquidity was spent

buying materials for the construction

of a new meditation hall.

Line plots can captivate!

To survive and thrive,

we must collaborate,

and to collaborate,

we need truth and clarity.

But our world suffers a deficiency of quality truth.

Instead of speaking data,

we shout

pluralities of anecdotes.

An anecdote

is one point

told as a story,

and thus anecdotes enjoy

the privilege,

the prerogative,

the endowment

of story.

So anecdotes ascended to authority

dominating data

dubiously,

disproportionately.

How could a

nuanced

neutral

numerical

narrative possibly compete

with fiery fables of

feasting cats and dogs?

The digital revolution is new -

we only needed to start digit-

izing when we invented pottery,

merely 12,000 years ago.

In contrast, we've had stories for millions of years.

Stories are

primeval,

primordial,

visceral,

natural.

Pottery may be new technology,

but lines are not.

Developmental psychology studies show that

young children understand

time or progression

when they are presented

with a timeline or a path of dots.

Why is that? Perhaps lines even predate stories. One can imagine communication, prior to spoken language, our ancestors pointing and drawing lines in the sand, to plan hunting, making maps.

Line plot or plot line,

both come from the old English plot,

meaning a small piece of land

and, by the 16th century, the age of Shakespeare,

the word evolved to describe

a plan or scheme,

a mapping of events

in a narrative,

guiding audiences through a structured sequence.

When we plot a story or a time series,

we are mapping meaning onto time or space,

showing relationships and progressions.

Just as land plots are markers of territory,

story plots chart emotional landscapes.

Each plot point is a landmark in the journey

across the terrain,

whether that terrain is land,

the psyche,

server uptime

or a budget.

We turn numbers into lines,

digits into paths,

snapshots into flowing time,

plotting lines

so that we also, viscerally, understand -

not merely our System 2.

Video 2: From Physics to AI: Building a 200ms Coding Assistant | ft. Guy Gur-Ari, co-founder of Augment Code

The coding assistant / Developer AI that I've been using personally for the last few months now has been Augment Code, and I'm very happy to announce that we have an official partnership, something that I wanted to make happen immediately after I started using it.

Please do let them know I sent you, by signing up using this link, it helps support FFF out a lot:

fff.dev/augment

...aaand the video:

From Physics to AI: Building a 200ms Coding Assistant | ft. Guy Gur-Ari, co-founder of Augment Code

The video focuses on the narrative of how Augment Code is uniquely good at handling full codebase awareness for codebases of actual-workplace-size (and much larger too) at extremely low latencies for an LLM solution.

But if you are interested in some of my personal thoughts that didn't make the video, about choosing a coding assistant, read on:

I have been rather silent about my personal coding assistant use, because quite honestly I never found one to be quite good enough to warrant downsides of adding an assistant to my workflow. Don't get me wrong here, I think many assistants out there are good, they just aren't good ENOUGH - for me.

I'm perhaps a bit atypical so let me explain where I am coming from so that you can determine if this is relevant to you too.

I am a recovering tooling alcoholic (I realized that I had a problem around the era when ReSharper was a thing) and my attitude to these things is expressed in this video by much younger version of myself:

I don't like introducing too many things into my workflow, and when I do, I want it to be something that I can use for years and years and learn to use like a fine chefs knife, and be loyal to that brand. For me, dev tools are sort of like cameras, switching brand becomes a big thing because you just build it into your neurology a bit if you really lean into a tool.

Pretty much all developer AI solutions are cloud-based which means that you are introducing a network dependency into your toolchain, which means that you will have to change habits completely if you jump on a plane, or if you are out on the country with spotty wifi (which is for me quite often when I do an offsite retreat to focus on getting some project out the door).

I think this is worth it for Augment Code, but for me, Augment is the first assistant that was good enough to offset this downside. In this over-connected, over-distractable world, I really, really like to be able work offline if possible, it was a big decision to start depending on an LLM for me. Just the possibility of a network request failing means my flow is highly likely to be broken.

I know, I know, theoretically one can fall back to non-AI habits if a network connection is spotty, but I am not like that. Personally, I find that I am most productive when I lean into tools so heavily that the hotkeys are fused into my bloody spine. I want to focus on the problem I am working on, not the meta of the tool - that should just be an extension of myself, and be as simple and robust as possible.

Paying for capacity

I really like that Augment is priced... how do I put this... plausibly?

It is the best and most reliable assistants I've found, but it's also premium priced - at $30 for the individual license, and $60 for enterprise license (isolated env, soc2 etc). Also, when comparing it's important to take note that they charge per ACTIVE user i.e. if a dev is on vacation for a month, Augment won't bill your account.

$30 is an interesting price point - and there is only really one other software that charges that much, that I also happily pay for, and it's Superhuman. $30 is a magical price point that many companies avoid because it means you need to be able to withstand a tremendous amount of scrutiny compared to a $20 price point.

When I am considering tooling for my core professional workflows, I really really dislike to use products that are free or suspiciously cheap. I want what I pay to matter somewhat to the company that produces my dependency.

Considering that you can get an okay car for less than the more expensive Nvidia GPUs, the pricing strategies of many of the established actors has always seemed a bit fishy to me. Copilot was extremely suspiciously priced at $10 when it came out and Microsoft's GitHub Copilot Loses $20 a Month Per User.

I first heard the saying "if you're not paying for the product, you are the product" about Facebook around the time of the Cambridge Analytica debacle [Wikipedia], but that quote was actually originally about television advertising in the 1970's, and it's a truism that I think works well when it comes to AI too.

But for me, this is actually not so much about data security as it is about reliability:

For my non-coding, non-developer work, writing video scripts, copywriting, brainstorming etc, I tend to mostly rely on Claude, but I switch around a bit too. Especially when using GPT, the failing it is especially obvious that they are not having much spare capacity sitting around and it is very obvious that I am sharing the processing capacity with others with very little margin for capacity swings.

I don't know about you, but my experience trying to develop with a general purpose LLM gives a looooot of this:

While I am okay with interruptions like this for my personal stuff, and at least somewhat tolerant for it when I am doing professional creative writing, my developer time is more expensive.

If I am billing a client for my own dev time, or if I am paying a license to provide the developers on my team, I absolutely do NOT want interruptions by connection errors due to my provider using a loss-leader capacity strategy.

And don't even get me started on what incantations of murder I would invoke upon an AI provider that breaks down when I am awoken in the middle of the night by PagerDuty to fix a payment system failure.

MPJ is shorter than Michael Jordan

Finally, it's sooooo wonderful to have sponsors like Augment that are willing to be a little crazy with the format.

Me going "um hey, instead of doing the standard sponsored mention can you give me hours with your most technical co-founder to extract a 40-minute deep dive for my audience" and getting an enthusiastic "let's rock" is really cool to be able to do. Big shoutout to Emma, Head of Communications at Augment for making this happen.

I, of course, don't want to compare myself to Michael Jordan (I'm a much better programmer than him, and even if I wasn't, I'm also much shorter and vastly superior to him at traversing tightly cramped spaces such as ventilation ducts) but I was very inspired by the Air movie:

I saw this when I was thinking about what partnerships I wanted for this new generation of FFF -  it made me want to really evaluate partnerships more as a retailer looking for the best stuff to resell, rather than someone selling mention space in my videos, and this is the first experiment in this direction.

Onwards into 2025

Thanks for your nerdy interest in this weeks behind-the-scenes-meta chronicle. I am so grateful for having you all around while figuring out what new the FFF is about during last year. It feels like the vision is finally becoming clear, and looking forward to a very active 2025!

And, to be true to the form of a creator, I give you one final nudge with: fff.dev/augment

As always, stay curious 🧐🐒

Mattias Petter Johansson

hey lovelies

Have been working hard towards a new video release next Monday (two videos, actually!) and have not had the mental bandwidth to think out a cohesive narrative for this weeks chronicle, but I have been inspired by many things this week, so I figured I would share an allotment of them!

Defence Against Dishonest Charts

An article on FlowingData named Defense Against Dishonest Charts has been making the rounds on Bluesky (I’m @mpj.fff.dev in case you want to follow me) for good reasons.

It’s a very pedagogical walkthrough with some wonderfully canonical interactive illustrative examples built in d3.

Data Engineers hunting the Al Capones of our age

The Economist (I should really have some kind of partner deal with these guys because I push them a lot) published this fantastically inspiring piece: Cryptocurrencies are spawning a new generation of private eyes, a spin-off piece to their magnificent  investigative podcast series Scam Inc. If you're considering a career in data, this might be a (hardcore) angle that you perhaps did not think of before.

I binged Scam Inc. this week and it is a spectacular piece of reporting by Sue-Lin Wong (@suelinwong.bsky.social).

I’m not really sure if I can say that I recommend it, since it's not for the faint of heart, but maybe for someone that considers a career in data, it might be mandatory listening.

Space filling curves

I was made aware of the concept of Space-filling curve this week, not sure what to use it for yet, but incredibly beautiful and fascinating.

Annas Archive and data scoundrels and rogues

When I read in the news that OpenAI were all sad about DeepSeek stealing their hoarded knowledge, I started wondering where OpenAI stole all it's hoarded knowledge, especially the papers and stuff.

This week, I randomly ran across the concept of Shadow Libraries, such as Annas Archive, which seems to be a Pirate Bay of books and studies.

From their datasets page:

“Our mission is to archive all the books in the world (as well as papers, magazines, etc), and make them widely accessible. We believe that all books should be mirrored far and wide, to ensure redundancy and resiliency. This is why we’re pooling together files from a variety of sources.

Some sources are completely open and can be mirrored in bulk (such as Sci-Hub). Others are closed and protective, so we try to scrape them in order to “liberate” their books. Yet others fall somewhere in between.”

I found it quite interesting that they have this statement on the datasets page:

“If you are interested in mirroring this dataset for archival or LLM training purposes, please contact us.“

I had not really thought about it before, but it makes sense that the information-wants-to-be-free crowd (copyleft?)  would naturally really like broad OSS LLM ingestion, as it is massive evolution of the old and brittle “we just host hashes” defence against attacks from the copyright.

HUGE visualization of all the worlds books

The reason why I found myself in the above rabbit hold was that both of the concepts were referenced in this absolutely nutty data visualization:  Visualizing all books of the world in ISBN-Space - phiresky's blog. It uses a dataset of a kabillion ISBN numbers and creates a visualization with bookshelves and fills them all with actual little books, and allows you to zoom in and out, Google-Earth style. Fun!

M5Stack Cardputer

In my quest to rekindle my love for computers this year, I find myself being pulled constantly to (not so) borderline silly hardware, like the M5Stack Cardputer: ESP32-S3 Pocket Computer. I mean, it’s like a silly little dog or something, it’s adorable and I want it.

Repurposing Epaper price tags for everyday needs

It’s subtle, but many, many stores these days use e-ink displays for prices. Once you look, you see it everywhere. These are really, really, really cheap displays, that are surprisingly capable (yes, yes it runs doom sort of) and you can update them easily over Bluetooth LE. When they run out of batteries, stores sometimes just throw them out and you can get them for free.

There are some geeks that have made a point of repurposing these for all kinds of silly personal uses and I just love it: Using e-ink (epaper) Price Tags (Shelf Labels) for everyday needs:

Yep. That's it.

Do find a source of wonder this Monday.

As always, stay curious 🧐🐒

Mattias Petter Johansson

What has caught my attention this week  is, not uncommonly, an etymological quirk - the connection between optimizing and optimism. Both "optimism" and "optimization" derive from the Latin word "optimus" meaning "best" or "excellent" but I think there is something much more interesting in the overlap.

Workplace feedback: “Not positive enough"

On a personal note, a very common conflict in my workplaces has been that I tend not to get swept up in drum-beating positivity, instead feeling awkward as the sole skeptic in the room. At the same time, I have an even greater disdain against people who habitually express negative predictions about the future, justifying their reactionary position by pointing at lines trending downwards towards their prediction of gloom.

This group of people often pose proudly as intellectual realists, when in fact they are just cynics who lack the courage to put their heart on the line.

I've never quite been able to put my finger on what I am in these social situations. Am I some "centrist” floating between positive and negative people, a person that likes to see the shades of grey, nuances of situations? But it always felt distinctly incorrect to refer to myself  as “in between” - I don’t feel “balanced” "nuanced" - my volition is often strong and clear, not at all a “negotiator in the middle"

We likely won’t survive, so let’s go!

I often say that I believe in humanity in spite of itself. There are many reasons why we would not make it - Complex organisms tend not to. Empires tend to fall. The dinosaurs were killed by a big rock that they could do nothing to stop, and we could not, at our current state of advancement, do anything different.

Saying something like that often makes me come off as a bit of a downer, but I firmly believe that we can make it. This belief is based on  the fact that there are so many (at the time seemingly impossible) problems that we have since solved.  Humanity has proven countless Doomsday prophets wrong. I actually don’t believe it’s likely that we will make it - in fact, I believe that the probability is slim, but I also know  that we tend to bend such probabilities over the course of history.

This week I've kind of landed on an insight, I am not in between two points, I am a third thing. These are in fact three distinct things - a peculiar triangle of positivity, negativity, and optimism. It's kind of like how Nassim Taleb talks about systems - you have fragile systems on one hand, and then on the other side you have unmovable solid systems. But then he discovers that actually, the opposite of fragile isn't solid - it's antifragile (wikipedia.org/Antifragility).

I started out this chronicle trying to find a word for what I am, but I realize that there already is: Optimist. I have just been confused about it my entire life because there are so many people who say they're optimists but who are actually more… positivists? Eh, Not quite right since it has an established meaning in philosophy (logical positivism) which is quite different from what I’m trying to get at… maybe just sunshine pushers?

The Subtle Language of Optimism

The confusion is so subtle that it is fascinating - it’s commonplace to say"oh, I'm an optimist" or "he's a pessimist”, or  "oh that feedback is very positive" or "that is very negative commentary" - alluding that these opposites, but in most other situations, positive and negative is a mathematical or relative term - you have a baseline, and whatever is above that is positive, whatever is below that is negative.

What’s actually happening with people when they say they are "optimists" (or “pessimists” for that matter) - they're actually something more like "energyists" in the sense that they seek to perpetuate whatever energy is going on because that's the path of least resistance for them. This can be expressed as avoiding work by rejecting new ideas, or by perpetuating project momentum with cheer, a phenomenon that is very common if you hold stock in a company in a hype cycle.

What is funny and interesting is that I never hear people request "give me some optimistic feedback". I suspect the reason is that  this could be interpreted as an implication that this might not work, and instead we request “constructive” feedback (which I think is a term that, compared to real optimism, will ever so subtly bias away from challenging the current approach too much).

The Three Perspectives: Our World in Data

What prompted this inspiration was an interesting article on Our World in Data, written by Max Roser. It talks about maternal deaths as an example of perspective shifting, and how many diverse perspectives can be extremely true at the same time - the title is The world is awful. The world is much better. The world can be much better. (https://ourworldindata.org/much-better-awful-can-be-better)

It’s a fantastic article that is condensed neatly into this gorgeously dense illustration:

The article doesn’t use the terms, but I think the above illustration perfectly captures the pessimistic, the positive, and the optimistic. Roser talks about how crucial it is that all these three points are true at the same time, and I think it elegantly shows how optimism isn't just about being hopeful, but about holding multiple truths simultaneously - the reality of current problems, the evidence of past progress, and the possibility of future improvement.

I want to stress here that I am not arguing that these are different strategies or personalities, I am saying that all three are necessary components in a triad of reasoning required for progress.

You have to first be negative and be willing to truly see how severe the problem is, then how bad the problem actually was, which puts it in perspective, showing that we’re actually better off than we thought, which also implies that we can improve it even further, enabling us to verify that an optimistic outlook is genuinely plausible, that the problem is optimizable.

No, this is not a "different personalities" thing

To be clear here, I am not saying that “we need all three people”, I am postulating that in order to be a good thinker, you need to keep all three modalities in your head, you cannot think clearly if you slant into too much positivity or negativity, or even if you skip them as steps and rushing quickly into uninformed optimism (i.e. not considering whether it is within our particular power to optimize this particular problem)

Furthermore, you’re going to get into perpetual, unproductive fights with others if you’re hardlining one of these modalities. If you’re being principally positive, you’re going to be fighting the principally pessimistic without having a path to reach common ground.

• Principled Negativity: Cynicism masquerading as realism.

• Principled Positivity: Blind cheerleading that ignores problems.

• Uninformed Optimism: Premature optimism that lacks proper grounding through the negative and positive observations described in the maternal death example.

• True Optimism: The courage to see both the negative and the positive in order to create an informed belief that improvements can be made.

Why a simple plot can be so motivating

I want to pull back to a really concrete realization, about that little habit tracker from an earlier chronicle, the one that wasn’t really a habit tracker at all, yet I found worked better than anything I’ve ever tried before - it's just a time series of good behaviour markers:

There is no judgment, no streaks, no failure. It's pure observation leading to improvement, and I think that I accidentally stumbled on creating something that exemplifies optimism versus principled positivity.

The positivity approach is the classic habit streak tracker, that "don't break the chain" thing. Which, by the way, Jerry Seinfeld has been interviewed about many times and he has been clear that it’s something he has been very  misunderstood on.

Seinfeld expressed a statement around “do something sort of every day if you want to become very good at it" which was then somehow bastardized into an entire "don't miss a day in your streak" ideology which Seinfeld actually never expressed.

Bright deception: Enforcing Success

The problem with the streak-oriented mentality is that it assumes and focuses on success, and treats any break as a “failure”, as “incorrect”, as “bad”. Streak mechanics gives an energy to be sure, but leads to a massive amount of disappointments and lack of motivation if the stream is broken. That's why people who have this approach, need to be super momentum-oriented, to the point of being in denial.

You have to be very forgiving with your streaks because it's so devastating to break the streak that you can be like "yeah, I went down the corner store so now I can get a habit checkmark for my daily walk” or “hmmm the chicken/bacon salad was kind of a healthy meal I guess”. When you are targeting a specific result you’re likely to see what you want to see, instead of what truly is.

Dark deception: Harsh self-discipline

On the other hand, we can also motivate ourselves through flagellation. The dark energy  with ruthless self-talk about the critical self-monitoring that many of us fall into, where we instead track our failures. It’s less recognized as a valid strategy in cheerful self-improvement circles, but I think it is actually an extremely common strategy and negative self-talk is one of those things that might be underrated in its efficacy.

I just love this contentious video by Struthless, that makes a strongman argument for negative thoughts, talking about how the ultra marathon runner Turia Pitt envisions horrible things in order to scare herself into running faster:

However, just like the positivity case above, the longer you try to sustain negativity strategies like this, the more likely it is to eventually necessitate another brand of self-deception. This is triggered because being failure driven is exhausting -  if you are looking at yourself or the organization or the world saying “this is shit, we are shit, we must improve” every observation becomes painful.

Pain can be motivating in bursts, but on a longer scale, it becomes untenable and we start developing strategies to avoid the pain, essentially, make our lives not miserable. 🙂You can either solve this by distracting yourself with other matters (because observation is painful, remember) but you can also take the strategy to take the edge off your negativity-engine by formulating explanations to how certain parts of the problems are not within our power to solve, or philosophies of how we don’t deserve better, that the problem is impossible to solve, not worth it, etc. I.e. abdication of responsibility.

The Third Way: Optimistic Observation

Now, the third approach, just drawing a plotline of the quality that you're working with - that is optimizing. Simply observing and graphing - that's optimism in action - it acknowledges both the ups and downs.  The optimism that I am trying to describe here is defined by how it acknowledges both progress and problems at the same time, and crucially also maintains the possibility of improvement through better understanding.

I.e. you're not chasing streaks or flagellating yourself over misses - you're not even in a middle ground between those things. You're in a completely different place, a third place, where you are creating knowledge about your patterns. You are asking "how can this pattern become better?" but you're still very much looking (and choosing to actually see) what the pattern is.

A miniature model for civilization's progress

The habit plot is a little mini universe of how civilization actually makes sustained progress. It's not through relentless positivity or cynical negativity - sometimes they work - but that is mostly because of chance, or because the solution to the problem was blindingly obvious, which is exceedingly rare.

We didn't reduce child mortality from 50% to 4.4% by pretending everything was fine or giving up in despair - we did it through persistent observation, learning, and problem-solving.

Doomsday prophecy is the cheapest type of prophecy

There's this fantastic historical example that perfectly illustrates our positivity-negativity-optimism triangle - the story of Thomas Malthus and his predictions about population growth. In 1798, this guy looks at the data and makes what seems like an absolutely rational prediction: humans are fucked. Like, properly fucked.

His argument was mathematically sound - population grows exponentially, but food production (he thought) could only grow linearly. Therefore, mass starvation was inevitable unless we controlled population growth. It's the perfect example of the "intellectual realist" mindset - taking current trends, extending them forward, and concluding disaster is inevitable.

The core problem Malthus identified was real - feeding a rapidly growing population is a genuine challenge. A pure positivity approach would have just denied the problem existed. But what actually happened? Through a combination of scientific advancement, agricultural innovation, and systematic problem-solving, we dramatically increased our food production capacity.

The Green Revolution, modern farming techniques, better understanding of soil science, genetic improvements in crops - we didn't just solve the problem, we overshot it so much that in many parts of the world, obesity is now a bigger concern than starvation.

What is quite amusing is that we now have a whole new set of doomsday prophets warning about declining birth rates - the same mathematical "realism" that once warned us we'd have too many people is now warning us we'll have too few.

This pattern - identifying real problems, but then assuming they're unsolvable - is what separates cynicism from true optimism. The optimist doesn't deny the severity of the problem or need for strong action (that would be toxic positivity), but also doesn't assume current limitations are permanent. Instead, they look at the problem as something that can be understood, analyzed, and ultimately solved through better knowledge and systematic improvement.

It's the same pattern we see with child mortality today - we can simultaneously acknowledge it's still a massive problem, recognize how far we've come, and work towards the better outcomes we know are possible because some countries have already achieved them.

Positivity stole the credit from optimism

At its heart, this is about a fundamental misunderstanding we have about how to create progress. There's a widespread belief that positivity - that relentless, momentum-driven cheerleading - is what drives progress, but anyone that has come to that conclusion is likely not accounting for survivor bias, because positivity without observation is just luck.

Real progress comes from knowledge creation, and knowledge creation requires something far more demanding than positivity: the courage to see things as they truly are while maintaining an unwavering belief that they can be better. This is what separates the optimist from both the cynic and the positivist. The cynic sees the truth, but concludes nothing can be done. The positivist refuses to see the truth for fear of losing momentum. But the optimist? The optimist looks reality straight in the eye and says "Okay, now how do we make this better?"

True optimism requires perfect courage - the courage to accept what is while working toward what could be. It's like a negotiation - if you can't walk away from the table, you're not really negotiating. Similarly, if you can't accept the full truth of a situation, you're not really optimizing. You're just performing.

True progress comes from a bleeding heart

Perhaps most cynics were originally principled positivists that were hurt, betrayed or disappointed  - someone who once relied on blind faith and momentum, only to have reality break their stride.

But a true optimist is inoculated from becoming a cynic, because optimism isn't about faith or momentum - it's about the practiced ability to face reality while maintaining the conviction that improvement is possible through better understanding.

It's how we've solved countless "impossible" problems before, and it's how we'll solve the ones we face today -eschewing denial and despair,  and putting our hearts on the line through clear-eyed observation and methodical, intelligent action.

As always, stay curious 🧐🐒

Mattias Petter Johansson

Good Monday morning - how are you feeling today? Or rather, what are you sensing? What is going on in that body of yours?

I'm writing this from a rather grey February in Stockholm, caught in that transitionary period where the snow has abated but spring hasn't quite arrived. We get the occasional bright, sunny day, but mostly it's grey and nondescript.

While Sweden is a wonderful country, the weather isn't one of its greatest features. Nevertheless, let's dive into something that's been on my mind: measuring sleep data, and tracking subjective experience in the body.

The Curious Case of Sleep Tracking

Sleep data is fascinating because it sits at an interesting intersection. While precise measurements of deep sleep and REM cycles traditionally require electrode-based polysomnography, we can now get surprisingly accurate approximations through wearable devices. The Quantified Scientist [YouTube] has tested various wearables against professional sleep tracking equipment. The latest Apple Watch versions, for instance, come remarkably close to professional measurements.

These devices use a clever combination of proxy sensors - accelerometers, heart rate monitors, and temperature sensors - matched against professional-grade measurements to create meaningful approximations of our sleep patterns.

Rise: The Actionability Problem and The Social Element of Sleep Data

I've been tracking my own sleep for quite a while, including a stint with an Oura Ring [Company Site]. These devices provide all sorts of metrics about "readiness" and sleep quality. But I've struggled to find truly actionable insights from all this data collection.

The closest I've come to finding genuinely useful sleep tracking is an app called Rise. [Company Site] Unlike other apps that focus on complex sleep stage analysis, Rise takes a refreshingly simple approach: it just tracks total sleep time and sleep debt.

Here's where Rise gets interesting - it allows you to pair with your partner's data. You see your sleep debt graphs side by side, which creates a fascinating dynamic. You can tell when your partner hasn't been sleeping well, prompting you to perhaps be a bit gentler with them, avoid late-night discussions, or suggest an earlier dinner.

It's a beautiful example of using data in a simple yet meaningful way - creating accountability and providing a shared vocabulary for something that's typically hard to discuss precisely.

The Body's Built-in Sensors

There's an interesting counterargument to all this self-quantification: we already have incredibly sophisticated biological sensors - our own bodies. As one of the Hard Fork [Podcast Homepage] hosts pointed out (while playfully mocking his more quantified-self oriented co-host), you can simply check in with your body to see if you're tired and then go to bed early.

I have previously talked about how I stopped using my HidrateSpark [Company Site] smart water bottle after I reflected on how I was further exacerbating my dissociation from my body by buying a product that measures my body and blinks with an LED what I should be able to sense internally with no issues.

It's somewhat ironic that we have this incredible neurological system with billions of interconnected cells, yet we often trust a simple accelerometer strapped to our wrist more than our own bodily signals. We've become so disconnected that we rely on tech companies to do our observation for us.

Finding Middle Ground: Quantified Intuition

However, I think there's a middle ground between pure technological measurement and pure bodily awareness. The critics of quantified self often swing too far in the opposite direction, rejecting measurement and aggregation entirely.

What if we combined both approaches? We could start by actually observing ourselves in the morning - noting our energy levels, mood, and general well-being - and then recording and tracking that data. After all, sleep is complex with many contributing factors, and the ultimate metric we care about is how well we function and feel afterward.

In fact, no matter if you do tracking of the the proxy metrics of accelerometer, temperature and heart rate, and leverage derived metrics like deep sleep and REM sleep, or elusive “sleep quality” aggregate metrics that many sleep apps have, in the end we need to actually have a success metrics that we are gunning for, and reasonably it needs to be something self-reported that correlates with us thriving.

Yes, self-reported data has its problems, but when it comes down to basic quality of experience, if we are good or not, it must be self-reported.

The Path of Least Resistance

The reason we often default to automated tracking is simple: it's less effort. Anyone who's worked with data engineering and analytics in a corporate setting knows that the most commonly collected data is often what's gathered almost accidentally - website clicks, user interactions, etc. Intentional tracking tied to specific success metrics is rare because it requires significant effort and organizational consensus.

In personal tracking, though, we have complete control and can create simple systems to track this. We can decide to track our subjective sleep quality every morning, perhaps 90 minutes after waking (when the adenosine has cleared our system) with a simple sticker-marker system on paper and enter it later into a CSV, which we then pull into ObservableHQ [Company Site] or Hex [Company Site]. This direct measurement of our success metric - how we actually feel - could be more valuable than proxy measurements from our devices.

Maybe the simple act of tracking makes us more mindful of our sleep quality (“what gets measured gets managed, and all that, plus I suspect actually measuring as opposed to just reading graphs of measurements is a powerful distinction in this context). Plus we can still correlate this subjective data with the automated measurements we're already collecting, to see if deep sleep actually does make our days better.

A Glassdoor for Sleep Quality?

One of my wonkier data ideas was generated by a psychologist at Mindler [Company Site] - a European BetterHelp with only licensed psychologists where I worked as Head of Data for a couple of years. The psychologist mentioned something fascinating, but not exactly surprising: virtually every mental health condition and external stressor manifests in disrupted sleep patterns. Given that we spend a huge chunk of our lives at work, our workplace environment inevitably impacts our mental state – and by extension, our sleep.

What if we could leverage this connection by pooling it together? Imagine a community-driven platform where people anonymously contribute their sleep metrics and employer information into a shared pool. We could create something akin to a "Glassdoor for Sleep" – a way to see which employers have the most well-rested employees. Not just another corporate wellness metric, but a genuine insight into how different workplace cultures affect their employees' fundamental well-being.

The beauty of sleep data is that it's both deeply personal and remarkably universal. Unlike traditional employee satisfaction surveys, sleep patterns can't be gamed or manipulated – they're a raw, honest signal of how our work lives affect our biological rhythms. Of course, we'd need robust anonymization and aggregation to protect privacy, but the potential insights could be transformative for understanding workplace well-being (or at least, it would give you an idea of what workspaces lets you sleep well at night)

Maybe an FFF community project, who knows... It is raining today, after all...

Making Cholera Beautiful

On a final note...

I was recently made aware of 30 Day Map Challenge [Project Site]- a yearly community event that pushes the boundaries of geospatial visualization, where participants create and share one map per day based on daily themes throughout November.

There is a really cool #30DayMapChallenge hashtag going on on BlueSky, where among many other amazing works,  Alex Selby-Boothroyd posted this piece - quite an achievement to make cholera beautiful:

It seems, by the way, that Bluesky is the platform that is winning over the scientific community and I am also loving the starter packs feature [bluesky.com]

So it's highly likely that Bluesky is the microblog I will be using after X goes... Threads was working out for a while and then it went all algo on me and Bluesky is the only one that has that nice raw internet feeling that I like.

My Bluesky handle is @mpj.fff.dev, if you'd like to join me - not posting much yet, but a little.

As always, stay curious 🧐🐒

Mattias Petter Johansson

Good Monday-ish Morning to you!

Creating a resource

As I've been writing these chronicles, it's been important to have sources of inspiration, and I'm starting to build a mess of links and resources, and I figured that I would never really organize them well unless I shared them publicly.

I present to you my fledgling little Github repo for this:

https://github.com/funfunfunction/fff-dataviz-resources

For now it's just markdown files (PRs welcome) but will hopefully turn into a more sophisticated directory with a searchable index over time, etc. We'll see where it goes.

Because I am not in my most creative and energetic self this week and already late with the chronicle, I am going to reproduce the directory as this week's chronicle.

Again, please send in your tips as PRs or just to mpj@fff.dev, and I'll make sure it's to everyone's benefit. In addition, if you have projects of your own, please send them in and I'll start a "made by Fun Fun Function" category.

Without further adoooo

The Pudding

Aaaah...The Pudding! The Pudding transforms complex data into really captivating visual essays. Their projects span from AI-powered music criticism to 3D population landscapes. For a masterclass in interactive storytelling, see their Hamilton visualization by Shirley Wu, where Broadway meets data art.

FiveThirtyEight

FiveThirtyEight, this is a classic! Founded by Nate Silver, their visualizations elegantly handle complex datasets, from their election forecasts to NBA player projections. Even seemingly simple topics like ranking Halloween candy become engaging data stories!

The Upshot (New York Times)

The Upshot is The New York Times' data journalism wing which is a real masterclass in making complex data intelligible through the web. Their work demonstrates how modern web technologies can turn dense datasets into explorable spaces, making them particularly valuable for developers looking to level up their data presentation skills.

Reuters Graphics

Reuters Graphics is really impressive. Their work, like Drowning in Plastic and the Tokyo Olympics coverage, are case studies in transforming dense information into explorable interfaces that reveal rather than overwhelm.

FlowingData

Nathan Yau's FlowingData - this really stands out to me as a learning platform and a source of inspiration for data visualization! I love the projects of the evolution of love songs and workforce participation trends. The site helps me think of what we can do to be a bridge between statistical thinking and visual design.

Financial Times: Visual and Data Journalism

Financial Times: Visual and Data Journalism division sets the standard for data design in financial media. Their projects, from the Ukraine War Tracker to the Global Inflation Tracker, demonstrate how complex financial and geopolitical data can be made explorable without oversimplification.

Their COVID-19 Dashboard became particularly notable for showing how real-time data visualization can serve a critical public need while maintaining visual sophistication.

Information is Beautiful Awards

Information is Beautiful Awards - these awards have been around since 2012 and is a great living archive of data & design.

Their 2023 winners are just amazing examples of inspiration: basketball, insects, shadowy borders and toxic dust - it's all there!

Not to be confused with...

r/dataisbeautiful

With over 20 million members, the r/dataisbeautiful subreddit is really a window into the experimental edge of the field - this is where creators test new things and occasionally stumble upon entirely new ways of seeing data. For web developers, it's particularly valuable as a space to observe how different technical approaches resonate with a general audience. This is where you can see real-time feedback on what makes data visualization truly accessible and engaging.

Domestic Data Streamers

At Domestic Data Streamers they craft installations and interfaces that literally go beyond the screen. Their work, spanning from AI-driven memory reconstruction to the Child Marriage Data Portal, show how data visualization can extend beyond traditional web interfaces into physical and hybrid spaces. For developers exploring data's potential, their project Everything that is not eaten is a perfect example of an immersive data experience.

Graphic Detail (The Economist)

Graphic Detail, is the Economist's data journalism wing. Their work ranges from rapid-response analysis like wildfire trend mapping to deep system explorations, like their landmark Sun Machines piece on solar energy's exponential growth. Their reflection on their first year reveals the rigorous methodology behind their seemingly effortless clarity.

DataPhys.org

DataPhysicalization.org documents how information can take physical form. Their List of Physical Visualizations is both historical archive and inspiration gallery (at least for me!). They also have extensive resources on enabling technologies if you are ready for a new hobby or side project. This includes a comprehensive bibliography and teaching materials.

Observable's Data Visualization Ecosystem

Observable, founded by Mike Bostock (the creator of D3.js) and Melody Meckfessel, really represents a paradigm shift in how we approach data visualization development.

Unlike traditional IDEs, Observable introduces a reactive notebook environment that transforms the typically isolated process of visualization development into a living, breathing ecosystem of collaborative exploration. This fundamental reimagining of the development environment has made it the de facto platform for modern data visualization prototyping.

At the heart of this ecosystem lies three interconnected galleries that showcase the platform's capabilities. The Trending Notebooks section serves as a dynamic window into the community's collective exploration, highlighting notebooks that range from practical utilities to experimental visualizations. These trending examples often reveal emerging patterns in data visualization practice, offering insights into both technical implementation and design thinking.

Meanwhile, the Observable Plot Gallery demonstrates the power of Observable Plot, a high-level visualization library that strikes a careful balance between the simplicity of declarative charting libraries and the expressiveness of D3.js. Plot's gallery serves as both documentation and inspiration, showing how complex visualizations can be created with remarkably concise code.

The D3.js Gallery, curated by Bostock himself, stands as a masterclass in data visualization implementation. These examples, ranging from basic chart types to complex interactive visualizations, demonstrate not just what's possible with D3.js, but how to think about decomposing visualization challenges into manageable pieces. The gallery's examples are particularly valuable because they showcase D3's latest capabilities within Observable's reactive environment, offering insights into modern best practices for visualization development.

Together, these three galleries form a crazy comprehensive learning environment that caters to different levels of expertise and different approaches to visualization development. You havve Plot's high-level abstractions all the way to D3's low-level control. You can look at community experiments as well as polished examples. It's just a full spectrum of approaches to data visualization. For web developers venturing into data visualization, this ecosystem has both copy-paste examples and deeper learning opportunities. Can you tell I like it?

Take it easy this Monday... Or not.

I hope you are having a more energy-filled start to your week than mine, but hey, sometimes we are just tired, and I'll go get some of that zzzz now and roar back into the week rested.

As always, stay curious 🧐🐒

Mattias Petter Johansson

"Thank you, my Chinese spy. I will miss you forever."

Soooo TikTok got banned this weekend. Not particularly relevant to this blog but I really did enjoy the poetry by the Economist on the matter:

AS A SHUTDOWN looms, TikTok in America has the air of the last day of school. The Brits are saying goodbye to the Americans. Australians are waiting in the wings to replace banished American influencers.

And American users are bidding farewell to their fictional Chinese spies—a joke referencing the American government’s accusation that China is using the app (which is owned by ByteDance, a Chinese tech giant) to surveil American citizens.

It is also a nod to the personalised feel of each user’s “For You Page” (FYP), the app’s algorithmically curated stream of videos. “Thank you to my Chinese spy for perfecting my FYP and understanding me more than anyone else. I will miss you forever,” one video signs off.

(From: A protest against America’s TikTok ban is mired in contradiction - Economist.com)

Data physicalization

Anyhoo

This week I've been musing around data materialization, that's the best word I can find for it. Or perhaps data physicalization?

What I mean by physicalization is a process very similar to data visualization in terms of thinking and goals, but not limited to merely visuals, and specifically manifesting data as material in the physical world, often in interactive ways.

MPJ makes a papercraft heart bar chart habit tracker for 2025

I have created paper-based trackers with stickers before, and based on learnings from earlier versions this one had three specific design goals:

1. Should feel decidedly tactile, little dependency on writing, and specific use of stickers that bulged out of the page, and that you can run your hand across and get a distinct feel of (basically, you should be able to consume the artifact with your eyes closed)

2. The data should be tracked in a uniform format that could be easily compared over time. This is why the axis starts on day 6 and not day 1 of the month, and only shows 28 days at a time, as opposed to the full month.

   For the same reason, Mondays are clearly marked on the axis, so that one can easily compare 4 weeks to the 4 weeks before it. We really want to take the cyclicality of the week into account in this data since we are comparing habits here, where adherence tends to be heavily influenced by weekends, this way of comparing makes a lot more sense.

3. The artifact should be easily made by hand, pen and paper (well, and a ruler). This is for many reasons, but one important one is that it bounds the design process.

4. The artifact should be extraordinarily simple, but no simpler - have exactly the amount of design elements needed to achieve its goal, and no more.

That last point is implicitly about a fifth meta-goal; that it should disincentivize configuration - many habit tracking apps, such as Streaks on iOS, which is very minimalist as apps go and I want to tip the hat to, still has so much configurability and functionality that is seductive to me and makes me spend more time with the meta of the habit tracking than doing the habit itself. I know it has helped many, but it always trapped me in the phone rather than making habit change in actuality.

This is a sin tenancy, of course, isn’t limited to digital solutions. I am probably stepping on toes here, but I think the Bullet Journal is an example of a system that is way too configurable and allows you to procrastinate with the system instead of doing the actual thing the system tracks.

One could consider this a user problem, but I think that is a copout. The whole point of a system is to guide you to do the right thing, so we should go to great lengths to ensure that the system doesn’t become a culprit in the drive that it is striving to protect from.

And yes, I just called Bullet Journals a dirty cop in the  productivity police. 🤣

Preliminary results

After a few weeks of use, it is clear that this form of tracking is very, very satisfying, which is one of the main success metrics of creating a physical artifact instead of a digital habit tracker.

It does have a "coin collecting" effect and placing out markers for your habit occurrences is something the nervous system understands very well, and triggers the reward loop of the brain

It also does have, to a degree, a "streak" mechanic and it does incentivize you to not skip days, but doing so doesn't feel "broken" as many habit trackers tend to inherently model a skip day as. Even the ones that do model skipping as okay still tend to have a "failure" feel around it - the below is the Duolingo widget warning you about breaking a 706 day streak, imagine seeing this on a day when you have a deadline at work and while driving your cat to the vet:

The papercraft tracker is very much not that - it is simply creating a bar chart of hearts of how many times you perform the habit.

The papercraft bar chart tracker is observing yourself rather than "tracking" yourself (something that a hunter does to prey).

There is no “wrong” state - however, what is established is a visually clear baseline, that you can visually compare yourself against next month. i.e. What you are comparing towards is not someone else or a wishful thinking ideal.

The most natural comparison when we are drawing a plot is you, yourself, in the prior period.

How can I learn to love computers again?

This little project is, ironically, related to a life theme where I try to rekindle my love for computing.

I used to really, really LOVE computers. They were pure sources of joy for me.

When I was in my teens, building computers in ATX cases was something I did quite a bit, and it was a hobby, a tinkering project of mine. When I think back even further, to my old Amiga 1200 (Wikipedia), I am filled with warm fuzziness.

I also used to really, really love my smartphone. There was so much delight in my iPhone, but somewhere along the line I lost control over mine and it ran me for a while. These little devices are so versatile, contain so much of … us, and what we do, that it has become a priority for anyone that wants our money or attention to breach this thing.

Smartphones are pretty secure in terms of technical cybersecurity, but I find that there is another type of yet-to-be-named shielding that we need to figure out with these things, in terms of guarding our attention.

Apple is certainly doing a lot of work in guarding me, with privacy controls, screen time, notification filtering etc. but I feel like it is woefully inadequate, it feels like the whole system has design considerations that make it a Sisyphean task to shield my attention - notifications are just too powerful per definition kind of, that is something that my life partner and close family and friends should have, corporations should not even be able to gain that access, so much of the attention grabbing that apps do should not even be possible to request.

I've been seriously yearning for a Light Phone 3, and the fact that the product category is getting a third generation is really telling that I am not alone. This device is just gorgeous and it just fills me with calm just looking at it:

Introducing the Light Phone 3

Clawing back control, one use case at a time

I have been exploring ways to extract certain use cases from the phone, in this case -  habit tracking. It was something that I would have done via app earlier, but honestly, that very rarely worked for me - probably for the same reason why I find that Morning Pages loses much of its effectiveness for me when I try to do it on a digital device more advanced than a Remarkable.

I started doing a lot of things (such as habit tracking, timekeeping and alarms etc) on the phone because it was convenient, but all this functionality convergence had a cost that only emerged over time - that the entire world uses one of two operating system for their morning alarm click has made it very important for companies to get in there as much as possible in order to get as much as your prime attention as possible as early in the day as possible, because that attention is monetizable.

Now that many of us are treasuring time away from the phone, the use cases for which the phone is only kinda sorta good at (because it isn’t a dedicated device) become dubious. Below are two projects which I found inspirational this week:

They Never Shipped, So I Built My Own Focus Dial

Building My Dream Productivity Device

Munzner's wonderful definition of dataviz

In her epic, classic book, Tamara Munzner has a really cool definition of data visualization:

Computer-based visualization systems provide visual representations of datasets designed to help people carry out tasks more effectively. 
[...]
Visualization is suitable when there is a need to augment human capabilities rather than replace people with computational decision-making methods.

One interesting trait here is that visualization is only needed when an automatic solution isn't available - we don't need vis if a fully automatic solution exists and is trusted (we very much still need visualization to inspect the performance of a not yet trusted automated solution)

The same definition is true for materialization, but here we can extend the definition to also include when data gathering has to be done by collecting observations from persons, and also motivating and engaging the given persons to actually provide the data (or engage with and understand a materialization of it)

Pulling data out of the digital realm

Once I started down this rabbit hole I found other examples, I particularly liked Stefan Sagmeister's use of yellow gumball machines in The Happy Show (to be honest mostly because it is so yellow)

I also loved loci - a 3D printing software to visualize your TripIt flights, that is sadly no longer available (at least I couldn't find it):

After more searching, I ran across the fantastic Data Physicality Wiki Gallery, with hundreds of super inspiring uses of data (you really need to check this one out, it's amazing):

I also ran across this fascinating article titled Tangible Data Visualizations.

Looking at some of these I am intrigued by the idea of making someone able to actually feel a mathematical formula with their hands:

The same article made me aware of a yearly show called Of All The People In All The World, which is a wildly cool idea to make data evocative:

Of All the People in All the World, also known as the Rice Show, is an art installation and performance by a theatre company in Birmingham called Stan's Cafe, which utilises 112 tonnes of dry rice to represent the world's population, with one grain for each person (about 60 grains of rice—or people—per gram).

Two 20kg bags of rice became the number of people displaced in Darfur.

Individual piles of rice represent various statistics, such as "deaths in The Holocaust" or "the population of England". Topical events are also covered, such as a pile representing the people who lost jobs upon the bankruptcy of Lehman Brothers, which happened during the September 2008 exhibition.

Some of the piles are shaped to represent something associated with the statistic, such as the rice representing the crowd at a football match being in the shape of a stadium, with players in formation. Visitors, or people using the show's website, are invited to suggest new statistics for inclusion.

On arrival, visitors are each presented with a single grain of rice, to represent themselves.

Finally, this little tidbit from the article is quite on point if you ever doubt if it is really worth the investment to make a visualization interactive:

While there is no solid evidence that targeting specific learning styles (tactile, auditory, visual) in education produces better results, kinesthetic learning has been shown to increase overall understanding, as many people retain more information through 'doing,' rather than listening or writing.

As Edward Tufte notes "An especially effective device for enhancing the explanatory power of time-series displays is to add spatial dimensions to the design of the graphic.

On a final, totally unrelated note: I love myself an Amiga Burger

… I must say that the one algorithm that I am really good at making my friend is the YouTube algo.

My YouTube recommendations are the best, and sometimes they bring me incredible gold that I had no idea I wanted and I have no idea why anyone would ever make, that is just pure expression of humanity and defies all laws of capitalism and self-expression and just touches me in weird ways.

This week, it was definitely Four-byte Burger, a restoration of a 1985 Amiga artwork.

This was like nothing else and made me extraordinarily happy, I hope it does the same for you:

Have a spectacular week, and  - as always - stay curious. 🧐🐒

Mattias Petter Johansson

Premiere Today: Dirty D3 - Episode 2: Linear Scales

As usual, we're using YouTube premieres for this episode, so join me and fellow ferrets in the live chat to watch it together when it drops - you can see the exact time in your timezone here.

Cross-Field Terminology is Fun: "Formats"

I wanted to talk a little bit around video and give some thought today on the format, formalize some thoughts - production formats are kind of fun and I find it really elucidating to learn about a term that might not be known to me much in another field, but that exists in my own field but with a slightly different meaning.

As a JavaScript dev, the term closure has a particular meaning at the front of my mind, but did you know that there is such a thing called closure in the world of comics?  (Closure in Comics [comicsormanga.home.blog])

Sidenote: I learned this from the amazing book Understanding Comics which is just incredible, actually, and I really do thoroughly recommend it if you're looking for learning about something really different in the new year.

In the same vein, when I hear 'format' I tend to think "file/data format" or "A4, A3", or "italic/bold" but in this case I'd like to dive a bit into how I as a dual class programmer/theatre-TV monkey thinks about the Fun Fun Function formats, in particular Dirty D3.

When we say "format" in video production we refer to specific, replicable adaptations in a genre.

For example:

• In the sitcom genre, The Office had a format with elements such as "single camera without audience".

• In the reality tv genre, Project Runway and MILF Manor are formats.

For Fun Fun Function, it's a format I control so I'm not very strict with it and have changed it a lot over time as I've gotten bored, but in the genre of programming education I've had a rather distinct format.

During some periods I made use of big paper plates since animation was very arduous back then and tended to lead to a lot of procrastination for myself.

But before I go into the format, let's get a very important question out of the way:

Must I really learn D3? I don't want to.

When rebooting FFF with Dawn of the Data Developer one part of me really wanted to avoid D3.

I looked for "what is the next thing that is the D3 successor" or "what is a new easier way of doing things" and almost every single new thing I found was building on top of D3. It is really clear to me that if one is to be serious about data development and visualization, there is just no way getting around learning D3.js.

This is especially true for me and you, as advanced web app developers looking to get into data. Data is a rather saturated field with high sophistication in almost all areas, but one underserved area is dataviz - most visualization looks really bad, isn't interactive, isn't engaging.

The core reason for this is that, in practice, you need skills in JavaScript and web technologies, canvas more specifically. In all other areas of data, Python takes you very far, you can spend a very extensive professional life in data without anything but Python, but not if you want to do good dataviz.

And the reason for that is in turn, D3.js. D3.js is so good and so vast. Replicating it would be a herculean task, and the motivation to do so is dubious. D3 in JavaScript has a position similar to Pandas or NumPy in Python.

Theoretically, we can replicate them in JavaScript and use JavaScript for data processing, but the effort being made just doesn't take hold, because… well, why would it? Python is for processing, JavaScript is for visualizing, is that so bad that we need to lift heaven on earth to change it, or maybe we should just lean in?

And for us web devs looking for a way to kick in the door into data, it's actually really good for us that there is this massively powerful advanced tool with high maturity, that an entire industry is largely scared of.

The Shortest Fun Fun Function Format Yet  (The chicken McNuggets of Data Development)

Dirty D3 is the shortest format that I've ever done - each episode aiming to be in the single digits in length, which is quite a challenge for me - I often say that if the survival of the universe depended on me being terse, we'd be so fucked.

To be honest, short might be a bit misleading, perhaps bite-sized is a better term, because while the individual episodes will aim to be below 10 minutes, the whole series will be very, very long, hopefully hundreds of episodes over the coming years.

Embracing the Mess

It's called Dirty D3 because I wanted a format that invites mess. If I make weird cuts, change things around, make typos, or just downright say incorrect things and follow up with a correction in the next episode, it should not break the format.

This is a trick that I learned from the great Casey Neistat, who was awarded YouTuber of the year and vlogged daily for 800 days. He's actually an incredible filmmaker, has done stuff with HBO etc, but is also a master of creating high-expression formats of incredibly artistic quality, and the thing he sacrifices is elegance.

Some would say that his stuff isn't polished, which I disagree with, it's meticulous in many ways, but the reduction in elegance means that there are fewer points that need to be polished.

A great example is this vlog: What's Your Motivation? (skip to 5:38 for the best MacBook unboxing)

Another example of his messy style (albeit more elaborate production) is this one: Make It Count

But it's not only for production ease and artistic style and expression that I like messiness, it's also because life is messy - and software development and especially data is messy.

Anyone that has spent more than a few weeks working with data science, be it analytics reporting or advanced AI, knows that 95% of the work lies in generating clean datasets from a messy collection of reality.

Dealing with complicated messes is not glamorous or easy, but that doesn't change the fact that it is the most important job.

If you can teach yourself to enjoy dealing with complicated messes, that is job security for the rest of your life.

Embrace the mess! Embrace the dirt!

Self-efficacy (or: Breaking your personal field of gravity)

Fun Fun Function has always been about putting self-efficacy at prime priority, breaking through that invisible bubble of brain fog, doubt, insecurity, uncertainty and doing something that your intuition experiences as defying the laws of gravity in terms of your self-image.

Honestly, I've been meaning to learn D3 for at least 5 years now, but it keeps scaring me off. This series is a way for me to commit myself to thousands of people to finally become an expert in it.

Dirty D3 is all about becoming a D3 developer, right this week. No excuses, just jump in.

We'll be fine. Together.

The Pareto Principle is for babies

Dirty D3 is about going Dark Souls Hardcore difficulty mode, learning the hardest weapon right away, because if you do that, everything else becomes easy.

There are lots of results-oriented tools and courses out there, if you are interested in learning the absolute minimum to do the report that you need for work right now, which will NEVER be D3. Yet almost everything builds on top of D3.

Dirty D3 is the opposite of no-code, which in my experience is often driven by a desire to do no-work.

Dirty D3 is absolutely not the straightest path to creating charts, it's the path of highest resistance, but it is a path and I promise that if you follow it, it will lead to you becoming something you did not think you could.

Comfort is Prime

However, while this is not the easiest or most straightforward way to learn D3 (just the choice of D3 in the first place means it's per definition uneasy and unstraightforward) we'll make darn sure that we are comfortable.

We'll achieve this primarily by being nerds. As soon as we feel overwhelm we will halt and nerd the shit out of whatever overwhelms and confuses us, until we find it interesting instead.

In a way, I suppose this is core of what it means to be a geek, find a subject most avert their eyes from, and own it.

I shall provide order

Another key aspect to comfort is to reduce the option space. This is the main value that I provide to you with Dirty D3, to be honest - I'll be making a canonical decision path to learn this stuff.

There is no official order to learn D3 in general, but on the other hand there is very much a designated order to Dirty D3, there is an episode 1 followed by episode 2, and so forth.

There might be seasons in the future, but they will also be numbered. There is one place to begin, and from there, there will be only one next step. There is no tree.

hello D3.2025

I did not set out to make a new year's resolution, but I suppose that I have accidentally done a very hard one with Dirty D3 - now I'm ***ing doing this thing, and I hope you join me in it!

I have given heads up of it before, and I mention again that a lot of Dirty D3 will be paid content, as it is a highly valuable skill sought by people serious about their career efforts into data. However, the first ones are free and that includes this one - so enjoy!

Book Corner: The Body Keeps the Score

Finally, during the holidays, I've finally been reading a book I've procrastinated on reading for a long time, The Body Keeps the Score, an absolutely fantastic description of how trauma really works and is stored in the body.

I know about this book for a long time but didn't really understand how incredibly knowledgeable the author of this book is, and how long he has been in the field, he was treating veterans before PTSD was even a term.

It should be noted that this book is not without its controversial statements and as any book on the frontlines of treatment, should be taken with ample salt. I still think this book is a true gem, and will make you understand yourself, your partner and the world a lot better.

It is highly relevant even if you had a happy, uneventful childhood without emotional scarring (which is a lot rarer than one might think, I have learned from the massive amount of mind-blowing data that the author presents) and if you're the opposite, the data will make you feel a lot less alone. Or... well, I won't presume what you will feel - it made me feel a lot less alone.

As always, stay curious 🧐🐒

Mattias Petter Johansson

This is the last chronicle of the year, and the next chronicle will be on 13th of January. Writing these has been a lovely thing and I am looking forward to getting FFF rolling proper next year. 2024 has, to be honest, been a lot about getting back on the bike for this creator, and I am finally feeling I'm getting warmed up.

As a hint of what is coming, the final video of the year is dropping today, and it is the start of Fun Fun Functions "Dirty D3" series, where you watch my suffering in leaning into the the impossibly large API surface area that is D3.js, bit by bit, step by step, with my unfiltered frustration at every step of the way:

Depending on when you read email, you'll be able to watch it right now, or get notified of the premiere below:

Dirty D3.js Part 1

Dirty D3 won't be the most pedagogical series, or the most succinct, the cleanest, and certainly not the shortest, elegant or terse.

But Dirty D3 will be the weirdest, most vulnerable, most energetic, and most importantly, most sequential course. Course-ish.

Dirty D3 will provide (an) order. There is really no one right sequence to learn all the bajillion hojillion components of D3, but that is an awful optionality, so the course will just decide the order to do it. This is the order that we learn this shit, and thus, it is the right way, because it is the way that we choose.

Hopefully, under 2025, (almost) every week, you'll be learning a little bit of d3.js with me, and at the end of 2027, we'll be renowned world experts, or at least known for having collaboratively created the most stupid course on the matter.

Deeper Dirty D3 (D5?) is the main perk for the upcoming Fun Fun Founders membership

I've been very fortunate be able to run the Dawn of The Data Developer reboot experiment on savings and support from my partner during 2024, but during 2025 the experimental phase ends and I need to actually start making this a job, so prepare for me asking for your financial support in January. It won't be donations though, and I will handsomely rewarding for you - so read on.

In the past, FFF has been very selective with sponsors and only done it when it really, really made sense to promote to the audience. Historically, I've relied mostly on financial support via Patreon, and for the new FFF I've considered a lot of variants for how we clolud do it, and in the end, I landed on using YouTube memberships as the primary audience support mechanism instead of Patreon.

More info is coming up around this next year, it will be a lot more "perky" than the Patreon of former FFF - initially it will give you a lot of e_xtra live streams and member-exclusive material around d3.js.\_

The general content strategy will be that I'll be considering non-d3 material to be the mainstreamy free tier of FFF, while d3.js (especially deeper parts of d3) is, more or less per definition, for the people that are serious about a data visualization career and that also want to support the production of FFF)

Upcoming membership perks: Live streams (d3 support and live episode shoots) + member only Dirty D3 episodes

I aim to have significant amounts of Dirty D3 shot in front of a live stream audience, and in addition, offer monthly support live streams.

These two aforementioned live streams formats will be a perk to YouTube members (more on all that next year) and while I aim to publish 66.6666% of the edited d3.js episodes publicly for free, but the 33.3333% of Dirty D3 episodes that are more deep-divey will be the third main member perk.

The fourth main membership perk will be early-bird pricing and access to _r_eal-life Fun Fun Function events. In the first days of ideation for Dawn of the Data developer, the idea for a D3 and Destress event format  was born. An event in the north of Sweden, where we disconnect, ground ourselves, and focus on learning d3 and doing data projects in a beautiful Scandinavian setting with sauna, nature and silence, and meeting other FFF fans for the first time (it's really ridiculous how few of you I've actually met through the years).

D3 and Destress has been a dream of ours for a while now, and during 2025 we will make it happen. The first events will be highly prototypal and only available to a very small group, and first  members that sign up will be the first cohort we invite.

California is finally beheading the duck

The new Fun Fun Function is very much gravitating towards the niche of making data more graspable, more evocative, more… transferable.

Visualization is part of this, and interactivity, but also understanding how mnemonics and memetics work. Last week I ran across one of the most shining examples I’ve seen of a mnemonic curve - a plot expressing a mechanic of energy consumption in a memorable way. The Duck curve is a simple, evocative image which gives us a vocabulary to talk about and direct efforts towards a complex matter. The Duck joins the hall of fame of memetic plots, alongside the Hockey Stick, Bell, Bathtub and Elephant Curves.

The Duck Curve | Wikipedia, named for its resemblance to a duck's profile in grid consumption graphs, has been the bane of California's solar ambitions for the last decade (I2M Associates: Why is California Trying To Behead The Duck?). ‘The Duck’ told a simple but frustrating story: solar panels would flood the grid with power during sunny midday hours, only to fall silent just as evening demand peaked, creating a daily pattern that looked remarkably like a duck's belly and neck.

As with many things Solar this year, the trend was broken on June 1st. For 8.7 hours, wind, water, and solar didn't just meet California's demand – they exceeded it. (CleanTechnica: Solar Passes 100% of Power Demand in California!) And unlike in years past, when such abundance would have been a crisis requiring solar panels to be shut down, this excess energy found a home in California's growing network of grid-scale batteries (New York Times - Giant Batteries Are Transforming the Way the U.S. Uses Electricity)

Flashback: Birth of the Duck (2016-2018)

Back in 2016, Vox released a video that would become oddly prophetic. "The 'duck curve' is solar energy's greatest challenge," the title declared, and for good reason. The California Independent System Operator (CAISO) had noticed something peculiar in their grid data: a shape that looked uncannily like a waterfowl's silhouette.

The duck curve is solars greatest challenge

As the California sun climbed high, thousands of solar panels would spring to life, flooding the grid with electricity. This abundance of solar power would create a deep belly-like dip in net demand during midday hours. But as the sun began its descent, just as millions of Californians returned home and switched on their devices, solar generation would plummet. The resulting evening demand spike created a steep neck-like curve that, when combined with the midday dip, traced the unmistakable profile of a duck.

The duck curve caught on as a visualization of a problem statement, representing a serious engineering challenge. Like the (in)famous Hockey Stick, it provided a lighthearted mnemonic at the same time that it gave us a clear antagonistic challenge to a very serious problem. Grid operators needed to rapidly ramp up alternative power sources to match this evening surge, a daily ballet of supply and demand that grew more extreme as more solar panels were installed. The duck was waddling deeper into our power grid, and its neck was getting steeper.

The Hunt Begins: The Battery Solution

The batteries emerged as the perfect apex predator for our duck. The strategy was elegant in its simplicity: catch the solar surplus, store it, then use it to flatten the duck's neck.

Pretty slow-moving for a predator hunting birds, these white large boxes that really only sit around and don’t seem to do much, patiently absorbing the excess solar generation during those deep midday belly hours. But, as evening approached and the duck's neck began to rear up, the batteries, now fat with energy from excess solar power, would strike at the head of the duck, finally halting the previously seemingly unstoppable growth of the duck neck.

But what made this hunt truly remarkable was its scale. This wasn't about picking off a single duck – it was about transforming an entire ecosystem. As battery technology improved and costs plummeted, California's storage capacity grew exponentially. The duck, which had once seemed an immutable feature of the renewable landscape, found itself increasingly cornered by progress.

If you are financially inclined, you might wonder if the duck curve is an arbitrage opportunity, and you’d be right. In California power prices tended to crash around midday, when the state produces more solar power than it needs, especially in the spring when air-conditioning use is low.

Battery costs have plummeted by over 90% in the past decade. In addition to pure cost, modern grid-scale batteries can also cycle (charge and discharge) more times before degrading, handle deeper discharges, and respond to grid needs with almost instant precision.

This gives an innovative power company willing to invest in batteries a trading opportunity  - charging up when electricity is plentiful and cheap and then selling power to the grid when electricity supplies are tighter and more expensive. This opportunity becomes even greater if you specialize in becoming among the best among grid operators at orchestrating batteries for better efficiency and durability.

This opportunity was not lost on power entrepreneurs, and California now has 10,000 megawatts of battery power capacity on the grid, enough to power 10 million homes for a few hours, not only during the duck neck hours, but the additional capacity has also proven to help massively when the grid is stressed from heat waves and wildfires.

Watershed moment

On June 1st, 2024 for a record 8.7 hours, wind, water, and solar didn't just meet the state's power demands – they soared past them, peaking at a whoopin’ 136.4% of total demand.

In years past, such abundance would instead have been a crisis. Grid operators would have had to shut down panels, as solar production as solar panels don’t take well to not having anywhere to direct the energy. But this time, California's growing fleet of grid-scale batteries transformed what would once have been a liability into an asset. As CleanTechnica's data shows, solar alone managed to provide 102.1% of demand during peak hours.

The Power of a Pattern

The beheading of California's famous solar power curve teaches us something profound about pattern recognition. The "duck curve" – named when someone at California ISO noticed their grid consumption graphs traced a waterfowl silhouette – did more than just describe a technical challenge. It transformed an abstract grid management problem into something evocatively graspable: "Look, it's a duck!"

While probably just a joke at the time, the duck was suitable as a rallying point. When engineers, policymakers, and investors discussed grid-scale battery deployment, they weren’t talking just in the abstract about megawatt-hours and demand curves – they could refer to "flattening the duck”. The mission became memorable, actionable, almost playful in its framing.

Just like Schrödinger's Cat, the Butterfly Effect, or the Double Helix, the Duck Curve is a narrative device that gives us, as a group, a cognitive handle on a complex system.

The mind loves to find patterns, and sometimes those patterns become tools for understanding and action.

I am very honored that you've graced me with so much of your attention today and this year. Time is the only resource that we all have the same amount of , and attention is the only resource that has that same characteristic, but that is even more scarce, and I very much do not take yours for granted.

I wish you the very best next 2025, see you on the other side!

As always, stay curious 🧐🐒

Mattias Petter Johansson

The Rising Costs of AI (And Why That's Actually Good)

hey  👋

I've been thinking a lot about something lately, and while this isn't technically a sponsored newsletter, it's inspired by the forces in the world of an upcoming sponsor.

When I was brainstorming on the script for the sponsor video, I ended up with a side-musing that was so interesting that it warranted further exploration in todays chronicle.

The True Cost of AI is Finally Showing

AI has started costing more lately. OpenAI just rolled out some bizarre $200/month tier that gives you more Sora credits, and honestly - I feel like AI is finally starting to cost what my neurology feels it should cost.

There is a specific reason why I'm like this - I was born in 1983, which means I started playing Quake 2 at a particular time in my life, which was formative. Kids these days take 60 FPS for granted - we had to fight with blood, sweat and tears for a smooth 30 FPS.

In my teens I bought a GeForce 256 (Ref: Nvidia GeForce 256 celebrates its 25th birthday) with money I earned from selling collected golf balls before that turned into an elaborate industry, and I installed that GPU with my own pimply hands into an ATX case.

I even put custom coolers on my Geforce to squeeze out a few more frames per second in Quake 2 (I was garbage at the game, so I really needed that edge over my friends who were actually good at it).

I actually had one of these:

I don't want to pull the "I'm older and wiser than you" card (and by that I mean I totally want to pull that card), but I'm part of a generation that viscerally understands how much heat a GPU generates – it's literally fused into my childhood memories. We understood the value of these things because we had to slave away all summer to get one. And the GPUs we got? They were cheap compared to what's running in today's data centers.

The GeForce 256 was the first GPU ever, and I held that little treasure in my tiny, weak, insecure, trembling teenage hands. The value of a GPU was profound to me.

There is no such thing as free lunch GPU

So when someone with this background notices that OpenAI has a free tier? That's fishy. It's unnatural. It's a "don't take candy from strangers" situation. When somebody's giving out free candy in the city, you want to know why. That's weird AF.

The economics won't add up in a while. It's relatively well-known that Microsoft is losing $20 per $10 Copilot user. Some users even set them back $80 per month. GPUs are really, really expensive to run.

In my mind, I model (haha) large language models kinda like a sophisticated search – fancypants Googling. As developers, maybe it's a bit facetious to say that 90% of our job is Googling, but 90% of our time certainly is.

When we have LLMs built into our editors, this gets even more intense. We generate LLM requests as we type – every time code completion triggers, it now pings a large language model.

A programmer will easily make 10, maybe 100 times more requests to an LLM than the average user, like a lawyer or research scientist. We just keep pumpin' that LLM.

The case for specialised models everything

If you're using an LLM thousands of times per day, you really don't want to pay for more model than you need.

I don't want to pay compute costs for a billion parameters worth of cookbooks or the entire Library of Alexandria thousands of times per day when I'm trying to untangle some hellscape of Salesforce export modules written in 2010. I want that model to be speedy, cheap, and most importantly, not wasteful.

When was the last time you actually experienced that bigger was better? I'm personally not a huge believer in enormous monolithic LLMs like GPT and Claude. They're cool, but that $200/month pricing hints at their inefficiency problem. I'm far more interested in purpose-built models, and structures and systems that coordinate models.

One of my absolute favourites is SQL Coder, an Open Source LLM focused only on SQL – they recently released a new version with just under 70 billion parameters. For context, GPT-4 is around one trillion parameters (i.e. GPT is more than 10x the amount of parameters)

Here's the fascinating part: even though SQL Coder is so much smaller than GPT-4, it outperforms it because it's focused on one thing. It's not just better – it's also much cheaper. When you look at correctness per compute hour, which is what really matters, SQL Coder shines.

We live in the real world of time and energy. Even if I'm a wealthy developer who can throw money at the problem, there's still the issue of feedback and latency. If my autocomplete starts taking more than 100 milliseconds, that's a problem – I'm not getting feedback fast enough on my typing.

In general, to become more effective as humans, as economies, as complex adaptive systems, we tend to specialise. We don't typically succeed with big, general-purpose systems.

Sure, we might start that way – remember when we used Ruby on Rails for everything? (I loved those days!) Or ASP.NET, which, oh my god, did everything for you? There was even that era where everything web-related was done in Dreamweaver. Those were comfortable times, but we learned that specific tools for specific problems work better.

It's just how the universe works. Doctors specialise. Lawyers branch into human rights or corporate law or real estate. We don't use bazookas to kill flies.

The really interesting Context Window Assymetry Problem

Here's where we hit a really interesting challenge with AI coding platforms – the context window asymmetry.

We have these impressive LLM models like Claude with its 200k token context window. That's fantastic for research papers where you can dump in big PDFs. GPT-4 isn't far behind at 128k. Sure, it's slow and expensive when maxed out, but it works.

But for code bases? 200k tokens doesn't get you far. Take the Quake 3 engine – about 300k lines of code. At roughly 15 tokens per line, that's 22 times larger than Claude's maximum capacity. And Quake 3 is compact! Look at MySQL with its 12.5 million lines of code – that's almost 100 times larger than Claude's best context window.

Here's what gets me – On social media I keep seeing these excited posts about how fast people can prototype apps with some LLM. They're thrilled about doing something in hours that would've taken 10 times longer without the LLM.

I get it, I do use LLMs a lot, but I also kind of don't get it. As someone who's done countless prototypes, hack weeks at Spotify, startup hackathons, and side projects, I'm flabbergasted that people are THAT excited about LLMs being able to accelerate prototyping. Why would you want to automate away the best part?

Making the prototype, the first version, the MVP – that's the honeymoon phase of software development! It's pure creation, with no compromises or dependencies. It's ridiculously satisfying. Why would I pay money to skip the best part? Sure, its maybe 10 hours of work "saved", but that is ridiculously insignificant in the life of a piece of software.

The Joy of Creating vs. The Drudgery of Maintenance.

Here's a secret – in those early stages, you don't really need to pay a developer. As long as we can feed ourselves, it's pure fun. Getting paid to prototype is a delight, like playing a game.

I mean, I of course understand how it might seem overwhelming if you don't know coding, but so is painting if you've never painted.  Once you know how to code, it's a joy.

Automating the prototyping MVP away is, a little bit like speed-running the puppy part of having a dog.

What we actually need help with

The real work – the actual work - what is hard  - what I wouldn't do without pay – comes later.

When we have to maintain that big codebase, when customers start rolling in, when we need to handle all the edge cases, when we discover that weird concurrency bug that occurs every 14th hour... maybe... sometimes? That's the actual work.

Making an MVP isn't work – it's being in love. Being excited about automating that away is like wanting to skip the first 18 months of a relationship and jump straight into stability and moving in together.

Sure, the stable part is what is good and meaningful, but those first 18 months of being in love? That's a gift. Why would you want to remove that? Why rationalise it away? What is the actual point of that efficiency? Just because something eats up ours, does that mean that it is prudent to optimise it away, or is there another criteria for what we target for optimizing?

I don't want AI to remove the delightful work – I can handle that part myself. I want AI to help me with the drudgery. Prototyping isn't drudgery.

You know what is? Making repetitive changes across an enormous, brittle codebase that's 10 years old. That's what we need AI for.

Dreaming of Code Deletion

I won't give our upcoming sponsor away today, but I want to end with quoting their CEO:

"I dream of an AI that deletes code."

Now that is my kind of CEO - and I'll tell you why.

When I was at Spotify, we used GitHub so you could track your total code contributions.

In my last few months after giving notice, I was hell-bent on removing one of the most over-engineered pieces in the entire desktop client – a social sharing dialogue that was absurdly complex for what it did. It was my nemesis, and had been a blight in my code ownership for years. A true insult to nature, and I would not leave that monster for someone else, not even my worst enemy.

On my last day, merged a pull request that I replaced the entire thing with something that was ostensibly speaking equivalent to a small shell script (Slang background: "Go Away or I Will Replace You With a Simple Shell Script").

But point being, in that action, I removed so much code in one go that I ended up with a negative line count for my entire four-year career at Spotify. ☠️🤣

As always, stay curious 🧐🐒

Mattias Petter Johansson

🪵 Hello Hygge, My Old Friend

The darkness has settled over Stockholm, and I’m leaning into it.

If you don’t manage it proactively, it can get pretty rough emotionally. But with some well-placed Innolux lamps and a trip to Spain, I’ve managed to cultivate a cozy Christmas feeling—appreciating the darkness and the lights that emerge to surround us in Sweden. Think light shows, holiday markets, and Santa lattes.

Hygge isn’t really a Swedish word—it’s Danish & Norwegian—but it feels right for this moment. It’s sort of like “cozy,” but not quite. Hygge is a big, warm hug for your soul. It’s that feeling you get when curled up under a blanket with a hot drink or laughing with friends around a table lit by soft candlelight. It’s the little things—dry socks after trudging through the snow, old friends lighting a fire, or just the perfect playlist in the background. It’s about slowing down, being present.

Hygge is about finding comfort and joy in the everyday—especially when the world outside feels cold and dark. It’s about proactively creating those small, simple moments of happiness that make you think, “Yeah, life’s pretty darn decent right now.”

On that note, I’m sitting here writing in a café in southern Stockholm. It’s run by a couple of confused kids with the weirdest playlist, but it works for me—just the right amount of noise and people, a lagom level of entropy. I’m feeling pretty darn good about 2025 at the moment. I’ve finally landed on a decisive vision for what FFF will be on a practical level, and I’m excited about the things I’ll be sharing with you soon.

This week, I’m back with another collection of bullets I’ve stumbled upon. Kind of newsy, but not really.

📕 Book: Meditations for Mortals

I was first introduced to Oliver Burkeman through Time Management for Mortals, which was included on the Waking Up app—an app otherwise focused on “pure” mindfulness content. For some reason, I decided to pick up Meditations for Mortals.. (And this is “meditations” in the Marcus Aurelius “mic drop” sense, not the “sitting quietly with closed eyes” sense.) It’s a wonderfully wise book that fills me with calm and clarity.

I strongly recommend it for starting the new year, offering a fresh, forgiving perspective on yourself and maybe even shaking up a few patterns.

I particularly liked a concept introduced by Burkeman as of “Going to the Shed” as a mental model for tackling big procrastination challenges. It’s an alternative to the usual “break it into small chunks” or “just push through” strategies I’ve historically used with spotty success. Burkeman credits this idea to Paul Loomans, a Dutch Zen monk, presumably from his book Time Surfing. So, maybe that’s the book I’m actually recommending. 😄 Haven't gotten to reading that one yet though.

📗 Book: 'Rest'

One book cited by Burkeman that I did pick up is Rest by Alex Soojung-Kim Pang. It’s a refreshing take on rest—not as a break from the world or “self-care,” but as an essential part of the creative and production process. The book makes a serious, compelling argument for working less—not to improve quality of life, but to enhance the quality of your work and output.

Pang references numerous studies suggesting that the magic number for daily knowledge work is around three hours. The key, he argues, is preparing and cultivating space to make those three hours as productive as possible, rather than diluting your output across more hours.

🧳 Who Buys Luggage at the Airport?

My new favorite podcast is Search Engine by PJ Vogt, and their recent episode tackling this wonderfully silly question in Who Buys Luggage at the Airport? It's perfect holiday listening. It captures the oddities of airports—those strange, ritualistic spaces we all just accept—and showcases PJ’s brilliance.

If you’re unfamiliar with PJ Vogt, I highly recommend checking out his former podcast, Reply All. Maybe start with episode 69, “Disappeared,” which tells the story of how Azer Koçulu broke npm—and the internet. The super tech support series are generally incredible too.

🖋️ Superpoet Whyte

A few weeks ago, I mentioned that my trip to Granada was inspired by a poem in Consolations by David Whyte. Well, Whyte has released Consolations II and he has been doing rare interviews, including one The Tim Ferris Show: David Whyte, Poet — Spacious Ease, Irish Koans, Writing in Delirium, and Revelations from a Yak Mange

Whyte is an absolute delight to listen to, and his words are perfect for snowy, dark days.

To be honest, it’s a bit jarring to hear Ferriss’s “INTERVIEW AND DECONSTRUCT HIGH PERFORMERS” (cue techno music) intro segue into a Zen-trained poet. But that’s also the charm of our current cultural melting pot. 🤣

📊 Freakonomics: Abortion and Crime (Revisited)

In my early twenties, the abortion-and-crime study by Steven Levitt in Freakonomics had a huge impact on me. It demonstrated the power of data and narrative. If you’re unfamiliar, it’s a controversial study (original here) that correlated unwantedness with violent crime rates.

Freakonomics recently revisited the study, reflecting on its creation, the debates it sparked, and updated data that validates its predictions. It’s a fascinating listen, touching on abortion, lead poisoning, and crime rates. Levitt’s no-nonsense, data-first approach always stands out, and I can’t help but chuckle at his remark about receiving more death threats from the left than the right—classic Levitt to track those stats. Podcast and transcript here.

One key takeaway: much scientific debate revolves around interpreting data rather than the data itself. Levitt makes a compelling case for shifting the focus of public discussions:

LEVITT: I think it’s really hard for a layperson to be able to watch a scientific debate, or social-scientific debate, especially one that’s being mediated through newspapers, magazines, and blogs, so much being lost in translation, and figure out what’s really true.

It’s not even easy for me as an academic. And I think there is a much more intelligent way to discuss social-scientific research than is done now. So right now, maybe the most interesting way to portray an idea is to talk about the hypothesis. And then, almost absent a lot of discussion of data, ask people to make a judgment about whether the hypothesis is true.

I actually think we should flip that discussion on its head.

If we want intelligent laypeople to be able to make good choices about what they believe and don’t believe, then the basic premise has to start not necessarily from the hypothesis, but from the data.

So, if the way that social science was reported was to say, “Here are the five facts that are true about the world.” And then what those mean are up to people to agree upon, but that’s never the way that discussions happen. Maybe because it’s not interesting, maybe because it’s a little too complicated, maybe it takes too much time. But I think there’s actually a lot less disagreement about facts than about the interpretation of the facts.

I believe that for an educated layperson, given a set of facts, they can make a better judgment about how to interpret those facts than the current way the media treats things, which is to often not talk about the facts but just to talk about the interpretations and often to focus on really extreme emphasis on minor differences.

😉 Dataviz, anyone?

🏆 Is it impossible for you to be just a little  bit more Hygge this week?

Even if you’re swamped, take a moment to level up your workspace. Add some light. Replace that annoying cup. Clean your monitor. Blow the dust out of your keyboard. Throw on a blanket.

Speaking of blankets—lately, I’ve been trying to reconnect with my sense of temperature. Sounds odd, but as a lifelong “stoic productivity dude,” I’ve often ignored basic cues like hunger or cold. It’s messed up. I even used a Hidrate bottle to measure my water intake instead of just…listening to my body.

On a related note: next year, I might just learn how to build a sundial.

As always, stay curious 🧐🐒

Mattias Petter Johansson

.... but first, nuggets.

With another slightly late chronicle this week (my routines are not stellar lately, urgh), I decided to jump down the rabbit hole of finding cool visualizations and insight’s I’ve run across lately.

The main one that I want to highlight is the recent work by the Economist around where solar power is as technology, and it is MUCH further along than you’d think. It is an incredible example of hopeful narrative using data.

But first, a few cool nuggets!

Say Cheese With Your Chart

Background: The Financial Times is well known for its Visual Vocabulary that is free to download as a massive PDF here: Charts that work: FT visual vocabulary guide

The original is cool in itself, but Say "Cheese" With Your Chart by London School of Communication is a really cool print derivative.

It’s a hardcover book printed based on the FT Visual Vocabulary, with semicircular opening along the ridge to enable tactile referencing (and also add to the thematic visage that the book resembles a block of cheese when standing up).

It is completely printed in yellow and black, and it uses data from a fictional universe called 🧀 Cheeseland to enable concrete and pedagogical examples to contextualize the guide, to make it easier and more entertaining to understand the vocabulary.

I have unfortunately not managed to figure out where one can find a copy of the actual print edition, but you can read it for free online here (Say "Cheese" with your Chart - online)

The Child Marriage Data Portal

My partner works in the field of child abuse prevention, and by extension she has made me more aware than the layperson of the staggering size and complexity of child abuse as a global challenge.

That so much child abuse occurs in the world is a subject that most prefers not to spend too much time thinking about, but ironically that unawareness is what enables much abuse to occur.

I don't mean to imply here that visibility is everything needed to solve problems, but  most evil is committed in secrecy, shrouded by common ignorance, so data projects that shine a light on the matter, especially in engaging, accessible ways is very close to my heart.

Thus, The Child Marriage Data portal is an incredibly beautiful and interactive visualization. I really like how the main country selector is kind of a menu item, data visualization and hero banner rolled into one.

The Economist on Sun Machines

But what has been on my mind most lately is the Coverage of solar by the Economist. It was a special edition of the Economist with a series of articles, but the one that I caught my eye, a and particular visualization in it, was this essay: SUN MACHINES:  Solar, an energy source that gets cheaper and cheaper, is going to be huge

The article is unfortunately subscriber-only, but if you don’t want to get one, there is a good video cover of this article by Money and Macro below:

Money and Macro showcases the visualization I am talking about in the first 20 seconds, but when I watched the video, I had a hard time parsing it at the time of viewing, because the visualization is actually interactive and gradually revealed when you scroll down the article, and interestingly dependent on the interactivity to be parseable.

I am generally not super stoked on scrollbar overrides, but this was quite a cool use. I did a dirty screen capture myself to give you an idea, I did it slow so that you can use the video progress bar to "scroll" through it yourself:

I find this plot interesting because to be is quite hard (or rather, impossible) to parse when seeing it in it’s final state because of the overlapping prediction lines. When it is gradually revealed, however, the amount of data points becomes less overwhelming. In addition, since this plot really does tell a story, the gradual revelation really makes sense, making it a bit exciting to follow.

While this kind of sort of works in a video, there is something lost when the interactivity is removed. When I am allowed to scroll myself, revealing more data as my cognition absorbs each set of data points, the story is paced for me.

In addition, the suspense is built as the story goes on, and every reveal by me, triggered by my curiosity, is rewarded with more information, another step in the story. Plus we love to see charts go up, as well, it’s a very nice experience to go through, this little scroll-chart.

The Economist actually has 50% this Cyber Monday (not sponsored, I just love the Economist) so if you have been thinking about getting a subscription, today is the day.

Interactivity really is key to being able to “intuit” complex data. If you do it correctly, a layperson can figure out quite complicated things.

I picked up replaying Monument Valley last weekend, and that game is a fantastic example of how adding feedback mechanisms allows the player to intuit super awkward Escher geometry. It seems unbearably complex at first, but as you rotate the levels around, your brain quickly starts “wrapping” itself around the geometry.

It is quite fascinating that we do this - it’s like the human brain is evolved without any assumptions around how reality works, and can just kinda roll with whatever, walking in impossible Escher levels with ease after a while.

I found the Geometric Room level of Idas Dream particularly interesting in this regard, its just lines (0:59 into the video below):

In the same vein, we can also think quite flexibly around time, which games like Braid will show you after you go into the levels more deeply:

On the same note of “intuiting data” I started leafing through the 1991 edition of Virtual Realities: A Shadowrun Sourcebook just because I found it on a friend's bookshelf.

While I never played Shadowrun, I did play RPGs in my teens and back then I found it somehow soothing to just sit and read rules about how a world worked, in hindsight understanding that seeking a world with clear rules is an escape from the unpredictable, uncertain real world, where doing the right thing still might yield a bad outcome.

Now that I think of it, dice-oriented RPGs actually do add an element of randomness with critical misses etc.  Randomness is actually a prominent element of the game. For example, in Baldurs Gate 3, dice are featured VERY front and center. However, here, randomness is contained in one space, in a familiar form of dice, and you have various ways of affecting the randomness - compared the overwhelming randomness of the physical world we live in, it is very manageable.

However, as I leafed through the pages, I realized that I seemed to have lost my interest in reading rules like this as I grow older. Maturity is to a large degree becoming more at peace with the uncontrollable unpredictability and chaos that life is, and even though I sometimes feel like a child, encountering things that gave me comfort as a child but no longer shows that I have learned something at least.

I was about to close the book and put it back on the shelf, when I got to a novella part of the book, which is quite extensive - it is an incredible short story about a being (not clear if it is an AI or not) born in the Matrix. The Matrix in Shadowrun is a Virtual Reality construct, but it is not primarily the simulated world that is envisioned in other sci-fi, but more something like “the Grid” in Tron.

In Shadowrun, the virtual reality of the Matrix is a physical interface, a GUI, made to allow humans (cyberdeckers, hackers) to handle much more information by “translating” it to the nervous system, so that the decker would not be limited to the “conscious” minds processing power, and make make use of the greater bandwidth that the neurology of motor control, object vision and general intuition affords.

I.e. virtual reality in sci-fi, and our current explorations, is of skeuomorphic, trying to look like the real world, but this is not at all the world of Shadowrun (which I suppose has its roots in Neuromancer, but not sure) where things a intentionally symbolic, yet still made so that humans can intuit easily, such as often making use of polyhedra (particularly fascinating Wikipedia article btw) because humans can intuit those easily, etc.

I do believe that there is a lesson there, for anyone that seeks a career in data visualization - we do have human-computer-interaction as a field (which is sooo bizarre as a title when you consider that "computer" used to be a job title for humans), but the term human-data-interaction is not something you hear, even though that is kind of what we are talking about, in it's essence.

Nat Bullard & Halcyon

On a final note, I found a fascinating cookie that I started following - Nat Bullard, who is credited for some of the analytics in the economist article, has done a writeup about it here: Understanding the Sun Machine

Nat works as CSO for an incredibly interesting  stealth-ish startup called Halcyon that “is building an AI-assisted search and information platform that helps those navigating the energy transition find valuable new sources, understand shifting market dynamics and answer complex questions quickly”.

It seems to me to be a NotebookLM, but that is more of a professional tool that can pull together quantitative data.

You can sign up for the Halcyon waitlist here (also not sponsored, just soooo cool)

As always, stay curious 🧐🐒

Mattias Petter Johansson

Good Monday Morning!

Seriously, how the hell do we define “data”?

A canonical definition of data?

If your timezone is early and this affects you - apologies for getting this Mondays chronicle out a few hours late, this was a tricky one to get right and had to do a rewrite from my original idea. 🦦

I got into a debate this week around defining data. Even though it’s been about a year now since I started this channel, I realized that I did not really know of a good definition of the term.

Data is one of those words that I have used my entire adult life and more (and you too, I suppose) yet I don’t really know what it is. You might get more out of this chronicle if you now put your reader down, and write down what you think is a good definition, and then check a dictionary or two, and ask a couple of different LLMs. Anyway, I did, and the definitions are surprisingly inconsistent.

I was initially going to list the different definitions here and try to find commonalities, and compare them to terms like Experience, Observation, Data, Information, Fact and Truth, Explanation, but it all turned rather messy to be honest and after going down that route it felt like I was just planting more trees that were hiding the forest.

Can we use the dictionary definition?

I will quote one definition though that was the one that set me off that path:

“the quantities, characters, or symbols on which operations are performed by a computer, which may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical recording media.” (Oxford Languages)

It dawned on me that this is a great dictionary definition, but at the same time a horrible description of the essence of the concept.

The above definition is almost a caricature of what a colloquial and parochial (see wikipedia/parochial) definition is. It is very superficial as a description and I find it to be relevant that it completely disregards that computer used to be a common human job title, in the era before Alan Turing and Enigma. I find that this definition fails to capture anything about the essence of the concept.

I realized that I have, throughout my life, used dictionaries assuming that they (or at least strive to) describe concepts correctly. However, when you think about it, that is naturally not what they optimize for!

Dictionaries don't strive to describe concepts, dictionaries strive to describe meaning, and meanings are pointers to concepts, and these meanings are often broadened or altered in language over time.

For example, the word cool was (one speculates) initially only for describing the temperature weather, but over time evolved to also refer to the “je ne seis quoi” of Fonzie (wikipedia/Fonzie)

I.e. dictionaries (as they should) strive to be correct in the sense of

“How is the word most broadly used”

... as opposed to...

“What is the most useful and congruent definition of the essential concept?”.

One Datum, Many Data

As I bounced around LLMs, I realized that they also have the same problem. They end up on popular branches of definitions, and unless you add in a certain thinker (e.g., Hume, Popper) to narrow things for an LLM, it just goes for broad meaning.

Either way, Oxford's dictionary offered an interesting clue with the origins of the word: mid 17th century (as a term in philosophy): from Latin, plural of “datum.”

The word datum means

“something given” or

“a thing granted” in Latin.

The word datum means

“something given” or

“a thing granted” in Latin.

Its plural, data, literally means

“things given” or

“things granted”.

It’s quite fun that I didn’t know the singular of data until today. 🤷‍♂️

Data is what we take for granted

In philosophy, a “given” is something assumed to be true without requiring proof in the immediate context. For example, when constructing a logical argument, certain premises (data) are often “granted” as the basis for further reasoning.

This aligns with the idea that data is accepted without question or further justification, at least. Early philosophers viewed data as foundational “givens” to be worked with, analyzed, or interpreted.

Over time, data evolved to mean recorded observations or measurements—the “givens” collected through experience, experiments, or computation.

Today, data is often treated as raw material for analysis, but I think its latin roots in “things given” remind us that it originates from something granted or observed, not inherently interpreted.

What subject do you trust to measure reality objectively?

There are quite a few modern definitions (that also float around in LLMs) that suggest that data is “neutral”. I think this is a problematic and even downright incorrect way of thinking about data.

Because we have lots of automatic data gathering devices these days (smart watches, thermometers and google analytics etc) that makes it easy to build a collective delusion that data can be gathered completely objectively.

However, perfectly neutral data is an impossibility, as data must be generated by observing an experience, thus making it inherently subjective.

I should make clear here that it is perfectly possible for data to be more or less subjective, it just cannot be objective.

We can also compensate for subjectiveness to some degree, by gathering data from many subjects and creating aggregates, or by doing meta-analysis of studies, but that doesn’t make the data gathered neutral or objective - such efforts are working around the fact that data is subjective.

I think the evolution of the word data to also include an expectation of recording to be very important and useful.

It would be (at least colloquially) acceptable to argue that a non-recorded observation is data, but I don’t see the usefulness of a definition that broad, as it is very close to just being a “memory” or a “story”.

The magic moment of recording

I think what makes data magical and distinct from memories and stories is really what happens at the recording step - and here I am talking about record more in the sense of a “database record” rather than “a record of history” or “voice recording”. To be more specific, I think that what makes data truly distinct is when we observe reality and record it according to a schema.

As I’ve talked about previously, the word plot comes from the Old English plot, which originally meant small piece of land, but evolved to mean “plan or scheme”.

And scheme, in turn: “a supposed or apparent overall system, within which everything has a place and in relation to which individual details are ultimately to be assessed”

Data is, ironically, itself not a discrete/digital concept. It is quite a gray area. A story would be dubious to argue to be a piece of data, but a structured Captains Log … maybe a bit more so, and a mood log would “more data” than a plain text journal. I suspect here that scheme and schema plays a large role in data, and how strictly the recording of observations adheres to the schema affects how much “data” it is compared to being a story or a memory, and is an important quality vector of data.

The argument that I got into was with a person that argued that lived experience can be considered data. I said that I think that equating experiences to data would be like equating a cow to beef stock, or the cryptographic hash of a disk image to the disk image itself. Experiences are not data, because to transform experiences into data, we need to do observational work.

Data is, to me, an aggressive reduction/simplification of an experience according to a schema. It intentionally sacrifices away the overwhelming majority of the experience to record specific aspects of the experience, which now gives us new powers in that we can compare datum to datum.

Does that compute?

In addition, the definition of data has also grown to include that the recordings are qualitative or quantitative. Think about a mood tracker vs a mood journal vs general daily journal. Do you agree that one feels “more data”?

I would personally say that what we are actually gunning for isn’t inherently that records are qualitative or quantitative, but that we seek for the records to be computable.

This has gotten a bit muddled in the last few years because computing power has gotten to a point where we can treat any recording, as long as it is done in a language, as data. In a data set for an LLM, English or Spanish etc becomes a very broad schema that we can use for computation).

Forgiving Schemas: The curling parent of data

The problem with a forgiving schema is that it becomes ludicrously energy-inefficient.

LLMs can be extremely impressive, but anyone that has tried to make something cost-effective with an LLM becomes painfully aware that there are enormous efficiency sacrifices one accepts when operating a model on a dataset whose recording schema is very forgiving.

The energy consumption between asking an LLM to calculate 1234 * 5678 and running a Python script that does the same thing is easily on the order of thousands of times more efficient. How Much Energy Do LLMs Consume? Unveiling the Power Behind AI

I am the last person to take financial advice from, but if you want a sure way of making money on AI investments, put your money into renewable energy, because LLMs are an inherently inefficient way of computing.

OK ADD ALL THE LOGS SO THAT WE HAVE THE BIG DATA IN LAKEHOUSECLOUD

In addition, when we record data according to a schema, that schema also often has quality criteria connected to it.

We can remove this and “just ingest books and also anything that looks vaguely like a book” which adds effectiveness limitations on top of your efficiency limitations:  (Google’s AI Recommended Adding Glue To Pizza And Other Misinformation—What Caused The Viral Blunders?)

This kind of nilly-willy vacuuming of all available data reminds me of how logging was done at Spotify early on in our data journey. Back then, the idea was to LOG EVERYTHING on every single feature you rolled out, and then we would learn stuff from that data.

It’s quite an absurd thing to do a GDPR request for your Spotify data - you’ll get mousemove events since the dawn of time, it is an absolutely staggering amount of data.

I shamefully admit that I did not see the problems with this and thought it was a good idea at the time, but over the years I (and the organization) learned painfully clearly that…

A) logging “everything” is a pipe dream - reality is complex and even though we added events on every single thing, leading to gigabytes of logs per user over the years, in hindsight you almost always found yourself missing an important event when you did analytics on things.

B) the reason why we needed to log everything was because we were not honestly interested in falsifying the efficacy of our features. We looked to validate features that we rolled out, not to disprove that they didn’t (because nobody has ever gotten promoted for removing a feature they built because it was ineffective).

I understand now that what we were doing was justificationism, and as data “scientists” we were completely ignorant of the work of Karl Popper on critical rationalism (wikipedia/popper), so Spotify was at time actually hundreds of years behind the gold standard of doing science - we were performing it at the level that was the norm when chopping lots of heads with the guillotine was in vogue.

Data is not information

Another interesting thing I learned recently is that Data is not information.

We’ve established above that data, while it cannot be objective, data does purport to be collected within the bounds of its stated schema, and intentionally tries to be void of presupposed conclusion.

This is because the purpose of data is to serve as the material for data processing (typically an analysis) which in turn outputs information. Put in another way, the definition of information is data that is processed and contextualized, that acquires meaning through structured interpretation, revealing patterns, relationships, or potential significance within a specific conceptual framework.

Analysts turns data into information

It is often quite subtle what an analyst does in a company. As a decision maker, you constantly feel starved of information, and having to go through analytics often feels painfully slow and limited.

This has (in my maybe somewhat controversial opinion) unfortunately led to products like Snowflake gaining popularity, to a large extent because they offer easy querying of data decision makers without having to go through an analyst.

While that access isn't in itself a bad thing, I think it will lead to bad results if the decision maker conflates data with information.

In reality, there is lots of work left to do on data in order to turn it into meaningful actionable information that isn’t misleading. And then further work to make it widely digestible (good visualizations, summaries etc).

Your crazy hoarder uncle that would do well in Big Data

To make matters worse, data in organizations has often been collected with the Bigger Data = Gooder Data fallacy that I described above at Spotify, where the data is not all that more structured than the “enormous pile of internet” that we shove into gargantuan LLMs.

Often this is vulgar amounts of “tracking” data of users, vast amounts of clicks and likes and whatnot. It is technically data, but only very barely so.

The data in question has often been hoarded with the assumption that some magical data thingamabob down the line will be able to make sense of it. Hoarding data in a company is not really better than your crazy hermit uncles hoarding tendencies.

Big Data is expensive to store and index, and legally risky if it is personally identifiable information - which the majority of data is, unless you take (yet again expensive) measures to anonymize it.

The hoarder postpones a decision of what is important to observe. Observing, discerning is one of the hard things in life. The mammal is inherently lazy (energy conservation is good!) and wants to avoid hard things.

A broad schema of storing everything, be it hoarding clothes or microwaves or old DVDs or hoarding petabytes of arbitrary track events pushes the work of keen observation to “someone” else in the future.

Unfortunately, since we have recorded our “observation” with a very broad and speculative and hopeful schema, that future “someone” is unlikely to be able to use the data that we have collected, much like how all those clothes that you have “inherited” from the “estate” have not taken your size information in mind, let alone your taste.

Video that inspired me most last week: Hank Green on the history and mechanics of populism

I’ve been creating for the internet longer than most, and started following (and still follow today) Hank and John Green, who are just some of the best people period.

There are few men that just inspire pure hope for humanity in me. To make sense of the current state of things with a long loooong perspective, I really must recommend this one, it really made me happy just because they are so good:

Onwards to the week!

I hope all of that made you much more confused than you were at the beginning of this chronicle, it certainly did for me!

😉 Oh yeah and in case you scrolled down to the bottom for an easy answer, the Wikipedia definition of data is not half bad: “Data” - Wikipedia 🤭

🧬 On the note of how there is no neutral observation, it seems like there is no neutral presentation either: Nature: Adding a personal backstory could boost your scientific credibility with the public

As always, stay curious 🧐🐒

Mattias Petter Johansson

I need your help to find a certain report

Before I get on with today’s chronicle, I’d like to ask for your help. I’m working on a video concept that’s a kind of “cover song” for an existing data report out there. Ideally, it’s something published in the last year or so—not too fresh, not too old.

Something good and unintuitive that you feel didn’t get the reach it deserves. I want to take aspects of it and create a really impactful animated informatics video, with fantastic music and beat editing, to give some good data the presentation it deserves.

If you know of a report like that, send it my way (just hit reply to this email; it goes straight to me!).

Eratosthenes was so cool

Anyway, on to the chronicle!

One of my obsessions is how Eratosthenes calculated the circumference of the Earth almost perfectly back in 200 BC. There was no need to go into space and photograph our blue marble—Eratosthenes achieved this by measuring the angle of shadows and paying someone to pace the distance between Alexandria and Syene (the latter is what caused his calculation of 40,000 km to be slightly off from the actual value of 40,075 km).

How he did this is explained incredibly well by Carl Sagan in this video, which is one of the best examples of pedagogy I’ve ever seen:

Carl Sagan explains how Eratosthenes calculated circumference of the Earth

(A longer clip can be found here: Carl Sagan - Cosmos - Eratosthenes)

The religion of science

The reason I’m so fascinated by Eratosthenes is that I’ve come to realize, in recent years, that I’ve been no better than the religious people I’ve looked down on. If I’m being honest, I’ve been a believer in science. I’ve been a staunch advocate of science my entire life, but I can’t point to a single case where I’ve actually done any scientific verification myself. I just believe in scientists.

Naturally, we all have to do this to some degree—it would be ludicrous for everyone to demand their own time with the Large Hadron Collider to observe the Higgs Boson. But the fact that I haven’t done any basic science myself feels wrong.

If most people are like me and don’t know how to actually verify science, then are they really being taught science? I’ve come to realize that I wasn’t taught a scientific mindset in school so much as I was taught that scientific institutions are trustworthy. My education didn’t provide me with the tools to challenge what I was learning, had it been wrong—and good explanations do that. They come bundled with instructions on how to attack them.

Popping bad explanations

Karl Popper (as interpreted by David Deutsch) tells us that good explanations are courageous because they expose themselves to being disproven.

For example:

* Bad Explanation: The seasons happen in winter because of Demeter’s grief.

* Good Explanation: The Earth tilts on its axis by 23.5 degrees relative to its orbit around the Sun, producing predictable seasonal changes.

The bad explanation isn’t bad because it’s unverifiable (it actually is); it’s bad because it’s inoculated against falsification.

Sam Harris puts it beautifully in this quote from the video below:

“If your beliefs are not falsifiable—if there is no scenario that could convince you your most cherished opinions are in error—then that is proof you did not get them by being in contact with reality.”  @ 04:20

Teaching trust instead of scepticism

I believe one big problem today is that science has, to some degree, lost touch with its core and started functioning like a religion. Instead of teaching how science can be inferred, we defer to institutions and authority.

This is necessary to some extent, but if ALL the principles a person holds to be true come from authority, and none are inferred independently, why are we surprised when those principles crumble under the sway of a brighter, louder authority?

Take climate change, for example. Do you believe in it because you understand the evidence or because you trust the people around you? For me, it’s the latter—I wouldn’t confidently explain climate change to a skeptic. But Carl Sagan could:

Carl Sagan testifying before Congress in 1985 on climate change

His testimony in 1985 was incredible. Let’s pause to appreciate the foresight of this man. Cynics in the comments lament the lack of progress since, but I wonder how much worse things might’ve been without this testimony.

Anyway, his key argument is made at 04:13 very cleanly, but what I want to highlight is that at 09:05 in the video, we see an incredible example of Sagan’s pedagogy and down-to-earth (pun intended) scientific communication around the greenhouse effect of Venus’s atmosphere.

Sagan mentions, as a cautionary example, that Venus’s surface atmosphere is about 470 degrees Celsius 470°C (900°F) due to its dense CO₂ atmosphere. While doing this, he explains WHY we can be sure of this:

“Soviet and US spacecraft have landed on the surface and dropped a thermometer down.”

Many communicators would not bother citing sources for the surface temperature of Venus, but I really love that he does here, and it’s very Sagan (just like how he explained that Eratosthenes hired someone to pace the distance between Alexandria and Syene).

It is a really cool move to drop in a couple of credible data points like that, assuming the audience to be skeptical and taking responsibility to cite sources, with a very understandable and plain explanation (given that it is in space).

Now, the two measurements aren’t exactly replicable by the average person, but the two measurements he cites are from two sources that are unlikely to collude (US and Soviet).

Zen and Science

Sagan finishes the testimony with this quote:

“I think that what is essential for this problem is a global consciousness. A view that transcends our exclusive identifications with the generational and political groupings into which, by accident, we have been born. The solution to these problems requires a perspective that embraces the planet and the future because we are all in this greenhouse together.”

Maybe some people feel that I am yapping on about a lot of hippie-dippie-adjacent stuff lately, but I can’t help but think that this statement is eerily similar to something Reverend Angel Kyodo Williams (author of Radical Dharma) said in an interview on Search Engine earlier this month: How do you sit quietly in the middle of a storm? @ 34:36.

Background: Reverend Angel is an ordained Zen priest (which is as close to certified enlightened as one can be) and has, in some sense of the word, gone rogue and returned to society to create broader change than the normal Zen priest route might offer.

Looking at Sagan’s finishing words and hers side by side, there is something very strong and universal about seeing the world for what it is, not what we want it to be that seems to be a core shared value among meditation practitioners and scientists—something we need more of in the world.

Rev. Angel:

“We are not going to be able to deal with the crises of the world and the planet if we don’t have more people mature, right? Become more mature human beings, more kind human beings at a more accelerated rate. And we need something that will operate—forgive me for saying this—at scale.”

PJ Vought:

“Enlightenment at scale!?”

Rev. Angel:

“We need to be able to. I don’t know if we—I don’t think we need a thousand more Buddhas. I think we need millions more people that are, I like to say… aware, more aligned, and more alive. And when we do that, it is not that all of our problems will disappear—but we will learn to conflict well with each other.

We will learn to conflict honestly and with integrity and with a willingness to meet life as it is, and not to abdicate to fantasies and the fictions that we’re creating and getting ourselves looped into.”

I wish you a week of shadows

I’ll leave you with that.  When you find yourself walking in the sun this week, take a look and find some time to marvel at the fact that the angle of shadows is data.

PS. In my research this week I learned (and don't even ask why I got on that tangent) about Operation London Bridge which was the codename for elaborate plan in event of the Queens death (and the sub-operation Operation Unicorn which was for if the queen died in Scotland) DS.

Stay Curious.

Good Monday Morning!

William Playfair: Inventor of statistical graphics and scoundrel extraordinaire

It has not eluded any reader that I have gotten obsessed with time series plots lately. I can reveal that the next video that I am working on is a spoken word piece about time series plots, and in this weeks chronicle, I aim to further dive into this rabbit hole, and start writing on the story of William Playfair (Wikipedia), my new favorite criminal and data visualizer.

Absurdly enough, it was only this week that I ever considered who actually invented line plots. It’s one of those things that seem so obvious that it them having an inventor seems unintuitive.

A wonderful reader of last weeks chronicle sent me some clippings from the book The Visual Display of Quantified Information by Edward Tufte, that made me do a double-take:

“Hang on a freaking minute, are those hand-drawn?”

Being a total data street urchin, I did not know about Tufte, which is a character in himself - besides being a data visualization luminary, he is an artist with an enormous sculpture garden:

Hogpen Hill Farms: Amazing sculpture park hidden in the hills of Woodbury

But I digress - ANYHOO - the data drawings above were not made by Tufte, but are reproductions of the works of William Playfair, who was the first to devise and publish all of the common statistical graphs - the pie chart, the bar chart, and the statistical line graph.

Yes, you read that right, William invented or improved 3 of the 4 fundamental types of statistical graphs (the scatter plot appearing first at the end of the 19th century.

Sidenote: Is it only surprising to me that the scatter plot came AFTER the line plot? I always have assumed that line plots naturally involved on top of scatter plots.

But I (yet again) digress from the arguably enormous digression that I intend to make, which is how incredibly esoteric William Playfair was.

Where do I even begin?

From the book "Statisticians of the Centuries":

"Despite the importance of Playfair's innovations, his name is largely un__known, even to professional statisticians, and those who have heard of him know little of his life.

One might expect a life of the inventor of statistical graphs to make dull reading, but Playfair pursued a variety of careers with such passion, ambition, industry, and optimism that even without his great inventions, he would be judged a colorful figure.

He was, in turn, millwright, engineer, draftsman, accountant, inventor, silversmith, merchant, investment broker, economist, statistician, pamphleteer, translator, publicist, land speculator, convict, banker, ardent royalist, journalist, editor, and blackmailer.

His business activities were sometimes questionable, if not downright illegal, and it may fairly be said that he was something of a rogue and scoundrel (Spence & Wainer, 1998)."

What. The. Actual. Fuck.

Sitting in the early morning in a cafe next to my hostel, reading into the history of the origins of statistical graphics, this was not what I expected.

okidokey then - not only am I going to spend this weeks chronicle entirely on Mr Playfair - this is going to be a recurring series that will hopefully be a series of videos eventually*.

*  = Fair warning: This content creator has promises like this once or twice before.

I was initially going to talk about William in just the intro of this chronicle and then make a list of data resources, but Williams story is too vast, set in the era of Descartes, Napoleon, David Hume, Adam Smith, the french revolution, the invention of the steam engine, protests (and hangings) against agricultural automation - Playfair lived during a titanic shift in history and knowledge, arguably in the greatest melting pot of engineering and science since Ancient Athens.

If I personally had to pick an era in history to live in, I would have picked the one William lived - I can't envision someone more actual steampunk than this man.

The book quote above omits one title that of Playfair that is more contested: Spy.

Playfair was said to have acted as a secret agent for British government and devised a plan to inflate the french war chest away by counterfeiting and distributing one hundred million Assignats (the french currency at the time).

Not only is that insane in itself, but it actually has seemed to also work - by 1795 the French assignat had become worthless and the ensuing chaos undermined the French government.

YES THE INVENTOR OF THE LINE PLOT IS BELIEVED TO BE A BRITISH SPY THAT TOOK DOWN THE FRENCH CURRENCY

To be clear, while Playfair was a lot of things, the spy thing is most likely and disappointingly untrue (the book The Flawed Genius of William Playfair contests it decidedly). However, I love the story, and it does paint Playfair in a light that makes my point easier to make so I'll keep it in here - even though I admit it to be false - a trick I will henceforth try to popularize as "Feasting on cats and dogs". 😉

When you consider what the line plot actually is, I think it required the mind of a really scruffy polymath to envision. While a line plot might seem like a fairly obvious thing to us now, repurposing Cartesian systems to visualize datasets was unheard of in 1786 (and pretty much visualizing data in general) and was a really weird thing to think.

Playfair’s impure use of Cartesian coordinate systems challenged the prevailing standards of his time, which largely dismissed such graphical representations as unscientific or unsophisticated for academic discourse.

Basically, you'd need to be little a bit nuts in your thinking to invent line plots. I imagine that if Playfair had been a modern-day employee at CERN, he'd be chilling out looking at the Large Hadron Collider and toying with thoughts like "Hmmm, I wonder if one could use the LHC to open a can of Pepsi".

Rene Descartes introduced the use of a coordinate system (X and Y axes) that allowed equations to be plotted as curves or lines, representing abstract, mathematical relationships. This laid the foundation for analytic geometry, where graphs typically represented theoretical or algebraic relationships rather than empirical data.

But for Descartes and his successors, graphs were used to illustrate precise, known mathematical functions. Cartesians did not at all intend for this to be used for exploring patterns in raw, real-world data. This tradition set a rather formal tone around graphs, aligning them with the rigorous, logical depiction of mathematical truths—not with the messier, sometimes ambiguous representations of empirical observations (e.g., data collected from nature or society).

Playfair, however, used graphs to depict statistical data—actual empirical observations, such as trade balances, national debts, and economic trends. This was a radical departure from the Cartesian approach, where graphs were seen as purely mathematical tools. Instead of plotting functions, Playfair plotted real-world numbers, encouraging readers to observe trends, patterns, and relationships in data, and published them outside of scientific journals.

Playfair broke away from this tradition, suggesting that graphs could indeed help us understand real-world phenomena. It is hard to overstate what a wonky person and mind is required for this to happen - someone with the technical know-how that also comes from another direction than the tradition, AND the aesthetical skill to produce these as drawings.

Playfair rejected the part of Cartesian tradition that postulated that graphs should only illustrate abstract functions, pushing instead for a new paradigm where graphs could explore real-world patterns, helping people to see data-driven insights in entirely new ways.

William had a rather gritty childhood in Dundee, Scotland. It was a modest household, shadowed by the early death of his father, Reverend James Playfair. This means that he was largely raised by his elder brother John, one of Britain's foremost mathematicians and scientists as Professor of Natural Philosophy, Mathematics, and Geology at Edinburgh University.

Having lost his father at a defining time in his life, I imagine William to have had a quite active and practical mind, as minds growing up in less secure environments often become.

Threshing Machines

William would have been 17 years old when he landed an apprenticeship with Andrew Meikle (Wikipedia), inventor of the threshing machine, an extremely influential step in the history of automation.

This hands-on experience would have been formative, grounding him in practical engineering, and also likely influenced Williams later interest in both innovation, agriculture and economics - without which he would not have invented the visualization tools we enjoy today.

Meikle’s threshing machine stood out among inventions at the time - it mechanized the very labor-intensive process of separating grain from stalk, a job once done by hand with flails. The machine increased productivity so dramatically that it foreshadowed the broader mechanization wave that would soon reshape society.

Automation anxiety in Scotland

Threshing machines stirred anxieties similar to those around automation today: they threatened traditional farming jobs and stoked fears of technological unemployment.

Scottish farmers (and eventually English ones, too) rioted against their widespread adoption, viewing them as symbols of dispossession and displacement. These reactions echo today’s concerns about AI and automation, where technological efficiency collides with social stability and employment security.

Before threshing machines, threshing was an intensely laborious, backbreaking task. It was typically done by agricultural laborers, often hired for short-term work during the winter months, and involved beating harvested grain with a flail to separate the kernels from the stalks. It was dusty, repetitive, and exhausting, requiring long hours and constant physical exertion.

For these workers, threshing was crucial because it provided seasonal employment and kept them afloat during the colder months when other farm tasks slowed. So, while it wasn’t particularly pleasant work, it had a critical role in rural economies: it gave laborers a reliable, albeit low, income in an otherwise difficult time of year.

With the advent of the threshing machine, however, this winter work began to disappear. Andrew Meikle’s invention mechanized what had been a manual process, massively speeding it up and allowing fewer people to process more grain in far less time. This productivity boost benefited landowners, who could make larger profits with reduced labor costs.

But for the workers, it was a disaster. Their dependable winter job was being erased, and with few alternative opportunities, they faced severe financial hardship.

The Swing Riots

The loss of work led to widespread anger and unrest. In some regions, desperate laborers took matters into their own hands, sabotaging threshing machines and destroying property in an effort to stop their spread.

Much later, a few years after William Playfairs death, this movement became part of the larger “Swing Riots” - a name derived from Captain Swing, the mythical figurehead of the movement, that was the signatory of threatening letters sent to farmers, magistrates and parsons. 'Swing' being a reference to the swinging stick of the flail used in hand threshing.

(I personally got introduced to the concept of a Flail (Wikipedia) as a weapon in D&D that does 1D8 bludgeoning damage, and did not know about it's agricultural roots)

It's kinda wild (and sobering) to learn about how automation really has done quite significant disruptions throughout history.

In response to the sabotage of machinery and the movement in general, landowners and authorities reacted with ruthless suppression. The landowning class in England felt severely threatened by the riots and responded with harsh punitive measures.

Nearly 2,000 protesters were brought to trial in 1830–1831; 252 were sentenced to death (though only 19 were actually hanged), 644 were imprisoned and 481 were transported to penal colonies in Australia.

Not all rioters were farm workers since the list of those punished included rural artisans, shoemakers, carpenters, wheelwrights, blacksmiths and cobblers. One of those hanged was reported to have been charged only because he had knocked the hat off the head of a member of the Baring family. Many of the protesters who were transported had their sentences remitted in 1835.

If you are familiar with UK banks and wondering if the Baring family is the namesake for the infamous Barings Bank (Wikipedia) that collapsed in 1995 due a massive derivatives trading fraud, then yup, it’s the bank they started with them sweet threshing machine money .... and also had someone hanged for knocking off a family members hat, there is that.

This brutal response left a lasting impact on rural communities, spreading fear and entrenching social divides.

In many ways, it highlighted the tensions between technological progress and social stability—a dilemma that still resonates today as automation affects various industries.

Graphing National Debt

Even though this crescendo happened after William Playfair died, he likely saw these conflicts and trends early on, informing his future work in economics and statistics, where he was deeply interested in understanding economic patterns, labor markets, and the broader societal impacts of industrialization.

One of Playfair's most striking visualizations that resonates with the plight of agricultural laborers is his Chart of National Debt (depicted above).

This chart is part of his seminal work, The Commercial and Political Atlas, published in 1786. It used time-series plots to depict the growth of Britain’s national debt alongside trade balances.

By visualizing this economic data, Playfair was able to illustrate the broader impacts of policies on British society, particularly the pressures of war financing, taxes, and trade deficits, all of which affected common laborers deeply.

During the period Playfair charted, Britain was amassing debt from its conflicts, notably the American War of Independence, which led to an increase in taxes. These taxes fell heavily on lower-income groups, including rural laborers, intensifying economic hardship at a time when mechanization was also reducing job security.

I can't help to think that there might be a lot to learn from this period of history, and that the issues being discussed today might not be as new as they seem.

Nugget of the week

While doing research for this chronicle, I ran across Data by Design, a project and print book coming out in 2025 that seems absolutely amazing and their site is kind of a online beta version of the print book, really cool project.

It has an incredible segment on Playfair called "Visualization as an argument", that features a really inspirational scrollbar-design for data.

And may you have a great week!

Stay Curious.

We all want to know how the world works, but only a few have the courage, tenacity and integrity to do the very hard work of finding out.

Explanations are to a penny, but good explanations are very hard to come by

My girlfriend constantly says that she feels in direct competition with David Deutsch because I talk so much about how awesome he is, which always reminds me that I don’t write enough about Deutsch in the chronicle because he is so goddamn awesome.

In the video below (which you should absolutely not start watching because you won’t get back to this chronicle this month, possibly this year) I love Deutsch’s definition of explanations: “An explanation is a statement of what is there in reality, of how it works, and why.”

He continues: “explanations are to a penny, but good explanations are extremely hard to come by, and this is what the growth of knowledge is actually all about”

David Deutsch - What is Truth?

In this chronicle, I’d like to delve a little bit into one specific component of good explanations, which is the production of causal proof. Oh man.

I’m going to talk about two examples of causal proof war stories, trying to convey how causal proof is even more powerful than it seems, but also why it is so bloody hard to produce it and why so few people rise to the challenge of actually doing it.

Stories of Courage and Causality

My thesis for this chronicle is that you need integrity, and the courage of a lion, as well as have ice in your veins in order to produce causal proof. If you are a person for whom it is critical to be liked and experienced pleasant, don’t get into the business of trying to produce causal proof, and instead leave that to truly hardcore people and groups.

I am going to tell a story about ludicrously large investments into music streaming latencies, but first I will tell a story about combating child sexual abuse.

What initiatives actually protect children?

Combating child sexual abuse is one of those deeply challenging fields where everyone wants to help, yet finding truly effective interventions has been daunting. I know people who work at the World Childhood Foundation, and their mission is to protect children from exploitation and abuse. They’re driven by scientific rigor, which can often be a rare thing in fields like this, where well-meaning initiatives are plentiful, but evidence-based solutions are sparse.

Now, there are countless programs aimed at helping children. But the effectiveness of many of these, especially in reducing child sexual abuse, has historically been hard to measure or even determine. Take orphanages, for instance—a well-intentioned model funded by people worldwide. Orphanages receive massive donations, and yet, from an evidence-based perspective, they don’t serve children’s best interests. Supporting children within their families or local communities, or at the very least in family-based care, has been shown to produce better outcomes.

Now, there are countless programs aimed at helping children. But the effectiveness of many of these, especially in reducing child sexual abuse, has historically been hard to measure or even determine. Take orphanages, for instance—a well-intentioned model funded by people worldwide. Orphanages receive massive donations, and yet, from an evidence-based perspective, they don’t serve children’s best interests.

Supporting children within their families or local communities, or at the very least in family-based care, has been shown to produce better outcomes: Institutionalisation and deinstitutionalisation of children (The Lancet)

But the problem persists: without clear proof of what works specifically to reduce child sexual abuse, it’s tough to justify or attract large-scale investment in any given solution.

Recently, though, one of World Childhood Foundation’s partners - Parenting for Lifelong Health - made a breakthrough. They conducted a large (n=4800), placebo-controlled study on a parenting app aimed at under-resourced communities, designed to help parents with support, education, and practical tools. Here’s where it gets remarkable: the study revealed, well within the bounds of statistical significance, that families who used this app reported a marked reduction in sexual violence vulnerability and victimisation of children. Even better, it showed a cascade of other positive impacts, from improved family dynamics, poverty reduction and mental health improvement.

This is a remarkable feat in a space where there are extremely few initiatives that are cost-effective, can be scaled, and show a predictable effect against child sexual abuse, and it cannot be overstated how hard and unrewarding something like this is to get in place.

Halo of Indisputability

Most people that have worked in a larger organization have frustratingly noticed the effect that if you’re working on a high-priority effort, especially an urgent one, the initiative is less likely to be scrutinized for efficacy. The reasoning usually goes like this: “This cause is so important that any work towards it is automatically assumed to be good work”

In child protection, this effect is taken to its extreme. Protecting children from harm, and especially sexual abuse, is an issue that resonates so deeply and broadly, that it requires nerves of steel to raise concerns about efficacy. Even though orphanages are highly dubious compared to many other more effective efforts, pointing that out is not exactly going to make you a hoot at parties.

On a greater scale, merely creating a controlled study opens you up for being criticized for being heartless. If you believe an effort to be effective in helping families, in order to prove that, you need a control group, which gets a “placebo” effort that you know is not effective. This is the way we must do things to know things scientifically, but it is a significantly less direct, gratifying or flattering way than donating money to an orphanage. Proving that your way works is a LOT harder than just doing the status quo of what is currently accepted as being good.

A great crusade for 30 milliseconds

When I was told this story, it took me back to my time at Spotify and the studies done that led to an enormously complicated initiative called head files.

When I worked at Spotify, we rolled out so many features in A/B tests that it was ludicrous. Everyone was targeting the grand North Star Metric, which at the time was Second Week Retention (the % of new users still streaming after 14 days of signing up). We had figured out that this metric was a strong predictor that someone used Spotify basically forever, meaning that if one even nudged this metric even ever so slightly with a feature, that made Spotify a hojillion kabillion dollars (approx.)

The problem was, this metric was near impossible to budge with any singular rollout, at least to a statistical significance, so A/B tests were kind of pointless towards it. In order to tackle this, the data team had set out to create proxy metrics that were more feasible to affect with an individual change to the software, but that we knew IN TURN to affect the North Star metric.

You can learn more about The North Star Framework at Amplitude but because I am the best at this stuff I have summarized everything you really need to know about the North Star framework in this chart:

In experimental ethics, can free tier users really be considered to be human?

In order to do this, the data team did something that was extensively criticized internally as unethical, which was to intentionally induce crashes and slowdowns in the client, just to test if that made users leave. Not exactly unethical enough to send one to the Hague, but arguably it was kind of torturing users for science, and Spotify's internal discussion groups were quite fiery during this era of the company.

Either way, the results of this experiment were incredible - the data team created unequivocal causal proof that three metrics were proxy effectors of the north star second week retention metric. I am drawing from memory so this might not be completely accurate but I believe they were:

• Number of client crashes

• Number of plays that took more than 1 second to start

• Stutter during the first week

If these went up, second week retention went down. Ergo, if one does an initiative that moves these proxy metrics in a positive way, that initiative can be incredibly expensive to do, and still be easily worth it.

OMG guys someone scienced it LFG

While this might seem common sense, there is a huge difference between knowing that something is “most likely” so and “we’ve proven this causally with massive amounts of data” in how much you are willing to bet on something.

In this case, it allowed Spotify to do an absolutely ludicrously boring and expensive and ungrateful project that I do not see how it would be done otherwise. In order to get streaming latency down (and remember here that Spotify has absolutely INSANELY GOOD streaming latency as it is) we wanted to put the music on CDNs closer to the users. The problem with this is that the licenses that Spotify has with labels require song files to be individually encrypted to the user before it is sent over the wire in order to prevent privacy.

So what Spotify had to do was to go to ALL the labels (and there are a lot of them, and they are INCREDIBLY fierce negotiators) and convince them to give Spotify the first couple of thousand bytes of each song unencrypted, so that these “headfiles” could be created for every song in the entire catalog (100 million songs give or take) and replicated to CDNs all over the world. All of this had to be implemented at the very core layer of the player stack, meaning that every single client in the fleet (desktop, mobile, Sonos systems, CarPlay, playstation, etc) was affected.

The result of this effort? I think it was a 30ms latency improvement or something to that effect. It seems silly, but I strongly believe that this kind of stuff is why Spotify, as a small tech company from Sweden, could hold its own against its titan competitors. I have a yearly little ritual to try out Apple Music every year, and being used to the subtle yet extreme snappiness of Spotify brings me back every time.

And without being backed by the proxy metric causal proof that the data team had produced, there is no way such a huge effort for such a (seemingly) insignificant improvement could be justified, but knowing it, this was heroics.

And without being backed by the proxy metric causal proof that the data team had produced, there is no way such a huge effort for such a (seemingly) insignificant improvement could be justified, but knowing it, this was heroics.

This is the power of causal proof.

Mostly, the people that actually create the causal proof are unsung heroes - the praised are often the first ones that leverage the proof to confidently blaze through what was previously unknown territory.

Tonight, from my terrace in Granada, Spain, I raise a glass to you all, the unsung, ice cold, hardcore, causality creators. Thank you!

Stay Curious.

This day has been a long time coming, a new video is dropping today. Watch the premiere together with me and fellow ferrets here at 20:00 CEST (16:00 Rio, 7PM London, 12PM San Francisco, 00:00 IST).

After the premiere I’ll be hosting a chill live stream (with likely horrifying technical issues), so stick around after the premiere - YouTube will forward you automatically:

Untutorial: Recreating Hans Rosling's famous data visualization using Observable Framework

The Health And Wealth of Nations

As I’ve talked about before, this one is a trial-by-fire, a gauntlet for yourself, an Untutorial to throw yourself into the deep end of getting into data development, by implementing one of the most famous visualizations of all time, and arguably the dataviz that made TED what it is today - The Health Of Wealth of Nations.

The original on youtube (you can also find a better quality version on Ted.com)

For some of you, you are already past this point, and then the video might do nothing for you, but otherwise, I hope that this can be what you need to jump off the cliff into the cold water, showing your skeptical neurology that this is indeed well within your abilities, moving data development from a maybe to a been-there-done-that.

The Untutorial experiment

Having focused on the release of this video this week, I have absolutely zero, zilch energy to write the normal chronicle, but I can talk a little bit about the background and reasoning about this particular video, a behind the scenes,  behind the mind, if you will.

The Untutorial is a new intentionally highly experimental format and deviates a lot from the old FFF formats, and there are many facets of this particular production that I will keep and many that I will change, so don’t hold back on the feedback.

The fear of the baseline

YouTube has changed quite a bit since I started 8 years ago, or whenever it was. It is ridiculously hard to re-start something - first time for yours truly, I realize in hindsight, I’ve only ever started new things, perpetually, which is a creative kevlar of sorts, because comparing yourself to yourself and what you should be now is a very uncomfortable proposition. If you are just doing new things all the time, the point of reference disappears.

In a way, it is like a time series plot without a baseline to compare with. If you just launch a completely new product every year, then you cannot compare the sales to last year. Doing apples and oranges and pears protects you from self-inspection, to see if you are truly growing, or just creating the illusion of it, by moving horizontally.

What is the distinction between

generating and genuine creation?

I have been thinking a lot about how to position myself as a content creator / author / intellectual professional, in an age where production capabilities become so much easily accessible. There is just so much material out there, and more is being generated every day, by humans, and to a greater extent assisted by  generative AI.

In such an age of abundance of derived information and content, what creative skills should be honing?

1. Modeling

As developers, I’ve already mentioned in the Dawn of The Data Developer that Modeling is what I consider to be the core skill of a developer, i.e. figuring out exactly what to tell the computer (and then adapting and maintaining the integrity and quality of that model over time, in collaboration with other programmers and stakeholders).

Coding has always been, and still is, a very small part of a senior developers job. This doesn’t mean that LLMs won’t cause issues, as junior devs can to a large degree be replaced with LLMs, so we might be seeing problems with the inflow of engineers - how would someone actually grow to senior if there are no junior jobs?  We might end up in situations similar to NASA, where, after the Apollo era, the lack of new engineers learning from experienced ones led to a loss of expertise in lunar missions. When it was time to return to the Moon, they had to rebuild this capability from scratch because the knowledge hadn’t been passed down to a new generation.

But even in that hypothetical scenario, my hypothesis that  modeling is going to be a valuable skill in the future is even truer, since it will be even harder skill to acquire without the junior dev positions that teaches modeling incidentally to people.

2. Discernment

Modeling, in turn, requires discernment. By discernment, I mean the essence in you that determines quality - your taste. What is good code? What is good writing? What is a good bottle of wine? What is good keyboard?

Yes, one can make a kind of checklist, or scoring system to answer these questions, it can only really tell if something adheres to that scoring system we just created. Like how a map of a city doesn’t really describe how the city actually looks and feels, a scoring system can only be a very simplified facsimile of quality. Yes, we can have static analysis of code, and classification systems of wine, but it doesn’t capture the essence of these things any more than google maps captures the essence of Brooklyn, New York.

For this, we rely of experts, connoisseurs, tastemakers. However, somewhere along the line this got quite messed up - a torrent of content and personalized algorithms provides us with a sea of advice, information, products, possibilities.

One area of articles that I find reaches me on YouTube a lot is "advice". I have certainly been guilty of generating many of these myself with my musings, but I can't help but think about a quality problem with these - how do we tell if advice is actually good or not? After all, it's not like recommendation algorithms go out in the world and try out life advice and only then pushes it to people. It only checks if the video is liked, shared and viewed.

I was randomly watching two videos by Mark Manson (the guy who wrote the Subtle Art of Not Giving a F*ck) the other day, and was quite amused that he posted one video praising journaling and dumping on therapy 6 months ago, and then 2 months after that one he posted another video that does a 180 and pushes therapy.

The videos are incredibly well produced from a videography standpoint, but as you might guess from the rather wild delta on something such as therapy from month to month, you might not be surprised to see that this is the level of advice in the videos:

You might also not be completely unsurprised that one of these videos is sponsored by Day One, and one of the videos is sponsored by Betterhelp. 😉

Now, I don't want to shit too much on this - I know friends that have had the Subtle Art of not Giving a F*ck help them in their life, it is an evocative book and narrative that can certainly be useful - sometimes we just need someone like Yogi Bryan to pull us out of a state, so I am not saying that it's bad that this is spread, but it is concerning that we don't really need much substance for things to spread.

This spreads because it is algorithm gold, and most likely the people involved in the production are avid fans of Made to Stick.  It feels good, it sounds good, it spreads good. Mark is VERY punchy. It’s excellent production.

How do we tell if someone truly has good discernment, or has he just found things that sound acceptable to a broad base and lumped them in a ball?

I am not too familiar with him, so I can't tell broadly, but he says in one of the videos that "verbalizing your anxieties makes them lose power over you" in relation to therapy - as a person who has spent lots of time in therapy, hundreds of hours of yoga and breathwork and by now close to a thousand+ of hours in meditation, I can at best call that statement lacking in nuance and quite honestly borderline damaging and irresponsible. It is known that merely verbalizing anxieties vents them, but they generally come back if you don't deal with them.

It is a lot harder to find people that have actual, strong discernment of what ACTUALLY works for things (and not just listing 22 things that sounds like they will work). And, ironically, to find them in the first place, you need discernment yourself. You cannot rely of personalization to find quality material, because algorithms have no way to filtering for quality - they filter to engage you, and that it does by showing your material that is agreeable to you.

3. Authenticity

Strong discernment, in turn, comes from a strong authenticity. By that, I refer to the skill of being true to oneself. Finding out what it is that YOU like, and for good reasons.

Eating something because you truly like it, buying a record because you like it in your soul, buying keycaps because you have discerned that particular clicky-clacky sound that you love (but your colleagues hate).

The thing about quality is that it is, in the end, subjective - we do not operate in a conceptual reality, we live in the actual world, where there is physics and individual context and preferentially - quality arises in the moment of experience, and it a wonderfully ephemeral quality that is one the the essentials of being human.

As David Deutsch, the inventor of the Quantum Computer, points out about AGI -  discernment is something that AI will only be able to do if it has autonomous experience (i.e. not enforced robot laws etc, how would an intelligence be able to know how to make the laws in the first place, or know their goodness, if it cannot be allowed to experience the learnings behind them) - discernment can only be built by experiencing the world with agency (and this is why we are so far from AGI, it is more of a liberty problem than a technical one).

The book Zen and the art of Motorcycle maintenance deals with the subject of quality in great detail - a book I thoroughly recommend giving a read (and should probably revisit by now, was 20 years since my last read through)

4. Curiosity

Authenticity, in turn, requires curiosity. Being genuinely interested in the world, not ever thinking that you are right (because then you are not being scientific, as Karl Popper teaches us), never being to sure of yourself, and disciplining yourself to see the interesting in new situations. This sometimes requires a large amount of abandonment, courage to see uncomfortable mistakes one has made, and a willingness to be wrong all the time, and, if you are a creator, making those mistakes in public.

If you do not have curiosity - your authenticity cannot be robust, only through curious conjecture can we have a strong authenticity. Without authenticity, our discernment is just strong, but not deep and will be moving unpredictably around depending on the whims of the situation, hype or sponsors,

Anyway, pulling modeling, discernment, authenticity and curiosity together, allows us to express ourselves, producing material that is truly discerning models of reality, based on actual broad experience, communicated from a point of authenticity, because we as authors believe what we are saying, with good authority, and not something we communicate because it is what agreeable and rewarded to communicate.

The Augur, The Ought

I guess that this is my very long-winded way of saying that I hope that I can provide you with a strong spark - a jolt that inspires you to move forward in a way that you truly ought to do.

A jolt that is convincing not because of my flair (though I hope I have panache and humor too), but because you see that I it comes from a tremendous amount of thought and curious wandering, that I say this because this is something I believe you and more people ought to learn.

I truly believe that the world is suffering from deficiency of quality truth, and it will get worse before it gets better. I hope that today can be a day where I say what I think ought to be done, and that you join me in making that ought, your ought, our ought.

To survive and thrive we must collaborate. We cannot collaborate without truth and clarity. To have truth and clarity, data must win over the anecdotes.

To surmount the anecdotes, data ought be made elegant and delightful, charming and accessible, beautiful and democratized, authentic and sharp.

That is what I think we ought to do.

Stay Curious.

Today, I would like to muse about game theory combined with anattā (no-self), and explore what practical implications that such a mental model could have in the ways we treat ourselves.

Anattā is actually not quite what I am describing, it is just the closest word I could find, so you have to forgive me for deviating a bit from the concept. I'll try to elaborate more specifically what I mean:

I recently found myself rewatching an old video (embedded below) by Dr. Alok Kanojia (or Dr K, as he is known as on the internet, due to his YouTube channel HealthyGamer) on why it is so hard to be consistent.

This particular video is one of the more shining examples of why I admire Dr K so much, in that he brings together his training as a psychiatrist with his background as a monk, with his avid interest in Gaming, into a beautifully integrated narrative:

In this case, he makes an exposition of Avidya that I had not quite heard before. Avidya, in Sanskrit, means “ignorance” or “lack of knowledge and refers to misunderstanding of what the self is.

There are multiple different interpretations of Avidya in Hinduism and Buddhism, and Dr K’s explanation of it is so peculiar that I cannot really tell if it comes more from the buddhist side or the hindu side or if it is his personal take, but either way, the mental model that he postulates in the video is that we constantly spawn into someone else's body and mind.

I.e. every morning you are actually a new “operator” spawning into somebody else's body. You inherit memories etc, but the essence of you is new, and stays around for a rather short time.

If one entertains this mental model, it provides an explanation why it is so hard to do things that give rewards in the future - if we are new selves that all the time only get to exist today, if the self is essentially a mayfly, then why not use our credit card to buy a pizza today, especially since a prior version of ourself did that too.

If we save money and eat something healthy today, the next us is better off for our sacrifice, but why would we do that when we can have pizza now and have the future us deal with the consequence.

I personally find this mental exercise delightful and interesting. I used to be quite provoked by much religious terminology and ideas, but lately I have started evaluating ideas (especially ideas that we cannot validate) according to the criteria of usefulness rather than correctness.

What if you thought of yourself as a completely different person compared to yesterday?

Basically, the mental exercise here is, what if your consciousness actually isn’t bound to your specific being, but actually rotates around in everyone all the time?

I find it to be reminiscent of the scenario expressed in ‘The Egg’, a wonderful short story by Andy Weir’s that has been beautifully captured in animation by Kurzgesagt: The Egg - A Short Story

The Nucleus Accumbens and Future Discounting

Shifting away from the spiritual perspective, Dr K also talks about the neurological reasons for this, and it has to do with the workings of the nucleus accumbens, part of the basal ganglia.

This part of the brain is, roughly simplified, the brains reward center - instrumental in mediating pleasure, reinforcement learning, and motivation and decision-making. It’s a dopamine hub, sort of.

A well documented behavior of humans related to this is called Future Discounting and refers to the tendency of individuals to value immediate rewards more highly than future ones—essentially prioritizing short-term gratification over long-term benefits.

In Future Discounting experiments, participants are presented with choices between receiving a smaller amount of money immediately or a larger amount at some point in the future. The idea is to assess how much people “discount” the value of future rewards.

Participants are asked something like:

“Would you prefer $50 now or $100 in a year?”

“Would you prefer $50 now or $80 in 6 months?”

Before you continue reading, I really, really encourage you to spend a few seconds to self-inspect your own intuition to the two  questions above. Then do some quick math on what the interest rate you are given in the above scenarios, and see if you think your first intutition matches your second-guessing rationale.

Writing this chronicle can be very elucidating - prior to writing this one, I thought of Future Discounting as an example of humans being irrational:

In studies, people consistently tend to heavily discount rewards as the delay increases. For example, someone might prefer $50 now over $100 in a year, even though the delayed option offers a much better return.

Hyperbolic discounting is actually rational

One detail that I’ve previously overlooked when reading about this is that discounting isn’t linear, but hyperbolic (which is kinda like parabolic and exponential but not).

In the early 1980s, psychologist George Ainslie discovered something peculiar. He found that while a lot of people would prefer $50 immediately rather than $100 in 6 months, they would not prefer $50 in 3 months rather than $100 in 9 months.

The handwavy explanation for this behavior is that  both options feel “far away”, but it turns out that this is selling our automated System 1 neurology short. Our “conscious mind”, System 2, is often thought of as “the rational” mind, but when it comes to heuristics for  complex decision scenarios with unpredictable factors (such as, you know, reality) then System 1 often has some surprisingly well-oiled approaches.

In this case, the intuitive decision making of humans is shown to follow a non-linear hyperbolic discounting curve. I.e. it’s absolutely not arbitrary and seems evolved to this specific point, especially when you consider that rats, pigeons and monkeys also do hyperbolic discounting.

Hyperbolic discounting can seem irrational at first glance - the question examples I mentioned earlier (“Would you prefer $50 now or $100 in a year?” and “Would you prefer $50 now or $80 in 6 months?” ) are actually 100% and 120% APY respectively, so on paper the reasonable thing to do is to pick the deferred amount, which is often why these experiments makes some economists go "hoomans are so stupid loollll".

However, us fleshy beings do not live on paper, we live in reality, where we have to factor in the interest variance and risk of the deal falling through.

There are mathematical models for doing that the ultra rational way (for example including a random walk for interest rates), and when you factor that in, it’s pretty much the same as hyperbolic discounting, which means our intuition factors in so many factors that it seems irrational, which is FUCKING SUPER COOL.

I strongly recommend this fantastic article by Chris Propel, which makes the argument in a very robust way: Hyperbolic discounting — The irrational behavior that might be rational after all

Trusting your gut: A rational decision

There is millions of years of evolution behind the brain and in this case, what seems like the irrational decision is, in fact, the wise decision, and our brains do it completely automatically.

I often find it hard to trust my own intuition, second-guessing myself all the time, and part of me rolls my eyes when I hear sentences like “trust your gut” or “follow your heart” or “trust in the wisdom of the body”, but when I read articles like this, I realize that I should pay more respect to the heuristics of my human intuition.

Our intuitions are studied, hijacked and exploited by companies, especially in recent years, and that has perhaps painted intuition in a bad light. I realize that I might have blamed my intuition for being stupid when in fact we should be blaming exploiters of it for being sinister.

What if you realized that you just started your life, and only have 5 minutes to live?

Speaking of sinister opponents, let me drag us from this dangerously far-off digression back to track (as it is also quite frankly mudding up a point I am later about to make), and move us to how this relates to game theory.

In his video on consistency and avidya, Dr K makes an analogy in gaming, and paints a scenario where players take turns playing 5 minutes each in a game. I.e. you randomly get dropped into a game, you play 5 minutes, and then control of the game character moved to another player.

I find it really interesting to think about how this would change how I play. For example, I would focus way less on the end result, and more about enjoying myself and the moment. At the same time, I would be acutely aware of that the next player is the same as me and will get my game state, so I would try to leave it in an enjoyable state. If we win “together” I would also be more inclined to be disciplined to do my best, I think.

A particularly interesting thought is that the everyday gaming behavior of typing “gg” and quitting as things started to go sour, as I would do when I start losing a game of Hearthstone, just becomes inconceivable.

If these these are the only 5 minutes I have to enjoy the game, then playing as well as I can in a given situation, taking as much delight and learning as possible from the way down, becomes rewarding in itself.

Plus, I can do so without the guilt/shame of having to put myself in the bad situation, and can completely accept that it isn’t my fault, but my responsibility. As such, I will do my best and maybe even turn the ship around.

However, as Dr K points out, a game like this would go sour when one player “defects”. An example of this would be a player losing control of their presence, patience and equanimity, and impulsively going for a high-risk, quick win. This way, they get the chance of sweet victory within their 5-minute window. It will likely fail, and cost everyone the game, but since you only have 5 minutes to play, isn’t that the “rational” choice?

I think this very eloquently illustrates the concept of Avidya, where seeing oneself as separate from others, causes suffering to oneself, and also makes everyone involved (including oneself, assuming everyone defects), to be objectively worse off.

The Prisoners Dilemma: MUCH more to this one than I thought

But another thought that comes to mind is that the scenario Dr K paints is also highly reminiscent to the Prisoners Dilemma in Game theory.

Many will be familiar with the The Prisoner’s Dilemma, but in case you aren’t, I’ll briefly describe it here so that you don’t have to open a browser tab:

Imagine two prisoners, each isolated from the other, with a choice to make: they can either collaborate (stay silent) or defect (betray the other).

• If they both collaborate, they get a light sentence.

• If one defects while the other collaborates, the defector goes free while the collaborator gets a heavy sentence.

• If they both defect, they both get a harsh punishment.

The logical, self-serving choice is to defect, because it minimizes risk in an uncertain situation.

However, if both prisoners follow that line of reasoning, they end up worse off than if they had trusted each other. It’s a maddeningly simple scenario, but it highlights how real life—like this dilemma—isn’t a single isolated event. We’re always part of repeated interactions, where being nice (cooperating) actually leads to better outcomes over time. You can almost feel the tension between short-term gain and long-term success humming underneath it all.

To be honest, I’ve historically had a bit of a disdain for the Prisoners Dilemma, as it is often used as an example of how self-serving behavior is better for the individual. It always felt rather sad to me, but also wrong in a way that I could not verbalize until I ran across Robert Axelrod’s work on the Prisoners Dilemma.

What Game Theory Reveals About Life, The Universe, and Everything (Veritasium)

This video by Veritasium interviews Axelrod where he describe the work that is the basis for his book The Evolution of Cooperation. It is a really good video, but I’ll describe the relevant context here to keep you from tabbing away:

Axelrod’s work flipped my perspective on the Prisoner’s Dilemma - rather than focusing on isolated decisions, he explored what happens when the dilemma is repeated over time—a much more realistic scenario, given that life isn’t a one-off event.

In his famous tournaments, where different strategies (simple bots) were pitted against each other in repeated rounds of the dilemma, and what emerged was surprisingly hopeful.

Nice, Forgiving and Retaliatory guys win in the end

Self-interested strategies actually consistently did the worst in these tournaments. The most successful strategies weren’t those rooted in ruthless self-interest, but rather those that embodied three key qualities: they were

• A) nice (they began by cooperating),

• B) forgiving (they were willing to move past betrayals if the other player returned to cooperation), and

• C) retaliatory (they weren’t pushovers—they punished defection, but only after it occurred).

What’s uplifting about this is that, despite the conventional wisdom that “defecting” is logical, it turns out that in repeated interactions (which is a facetiously succinct definition of “society” 😉), being kind, fair, and a little tough consistently wins out in the end.

Adapting Game Theory to self-care/discipline in a game of conciousnesses

Can we learn something from this that we can apply in our collaboration with ourselves?

If we humor the notion that our selves are mayflies of consciousness, teleported into our bodies and brains, just for today - and maybe even only until lunchtime, or the end of this pomodoro - then choosing between pizza and salad does, in a way, become a game theory experiment.

Can we somehow take inspiration from the prisoners dilemma, and Axelrods winning bot principles of Nice, Forgiving and Retaliatory, to ourselves?

Seems like a stretch_,_ but stretching analogies is my fourth favorite type of stretching, so we’ll have fun trying!

1) A Nice Strategy

Let’s roll with Nice first. This one is rather simple to envision - this would entail doing something that is decidedly nice for your future self.

This might entail clearing out your email inbox at the very end of the day in order to give yourself the grace to do guilt-free creative writing in the morning. Or, going to sleep early instead of watching the newly released episode of Severance (I am looking forward to the second season more than life itself)

Nice is not the same as Altruistic or Ascetic: I want to highlight that viewing ourselves in this wonky mayfly-conciousness-game-theory model makes it nonsensical to be self-abnegating (Renouncing one’s own interests for the sake of others.)

I.e. spending your day preparing for a joyless presentation the next day does not make sense for a mayfly. Your next-day-self will enjoy having it done, but if they also self-abnegate their entire day, the being's entire life will be joyless (i.e. this would be the inverse of a scenario where all mayflies opts for the always “do cocaine, eat pizza, watch the office, future self will deal with problems” strategy)

2) A Retaliatory Strategy

A more interesting challenge is how we can be Retaliatory to ourselves?

This introduces the need of the concept of holding a grudge into the model (if there is no concept of a grudge we cannot model forgiving) and for there to be a grudge we kind of need to have the concept of another. So for the purposes of this thought experiment, I suppose we will think of our future and past selves as “the others”.

Let’s assume that we wake up in the morning of our important presentation. We feel horrible, realizing that we have slept very little, having stayed up watching the new season of Severance all night.

We open up the presentation slides, that is half-finished at best, and we realize that we have to scramble to get it done.

In that situation, our former “the others” didn’t behave in a way that was supportive for us. In a game theoretical sense, they certainly defected.

While there are objectively bad strategies for the prisoners dilemma, there isn’t any “best” strategy, but a strong one is Tit For Tat, which is simple, when the other defects, Tit for tat immediately defects too, and after that goes back to being nice (we’ll get to Forgiving in a bit)

When we are dealing with ourselves, punishing is a bit awkward to model (people certainly do it, with penance etc) but it kind of only makes sense if you think of the other as your evil twin - if the others are a group of consciousness, then retaliation doesn’t quite work, but if we think of it as crackdowns, it might work?

For example, it might entail canceling the streaming subscription, or if this defection has occurred several times before, throwing out the TV completely (something that yours truly is seriously pondering).

Maybe you also schedule some spinning class in the morning the next day, or maybe an even harder commitment box?

As an example of a crackdown in my own life: To force our own hands in getting started with exercise, me and my girlfriend pre-purchased 3 months of sessions 3 times per week with a personal trainer.

That was truly a commitment box of steel - if I did not drag myself there in the morning, I'd have wasted a lot of money, my girlfriend would be stood up,  my personal trainer disappointed, and they would both be at the same time and space with my tardyness as a most handy discussion topic.

A prison for myself of my own design that I am incredibly proud of.

3) A Forgiving Strategy

Forgiving is the third key aspect of a successful Prisoners Dilemma strategy.

If we are playing life as the Tit for Tat strategy, after we have thrown out our TV, we’ll let bygones be bygones and be kind to our future self the next day (assuming they were kind to us even though we threw out the TV)

A tit for tat playing against tit for tat will just be nice to each other all the time, but let's be honest, life is hard and we cannot really be Tit for Tat all the time. Sometimes, we are just not very good, and we play as “Tester — start by attacking; if retaliation is received, then back off and start cooperating for a while; then, throw in another attack; tester is designed to see how much it can get away with.”

Tit for Tat lets bygones be bygones, but the problem with it is that it might get stuck in loops where punishing just goes back and forth. Therefore, a modified version of Tit For That, called Generous Tit For Tat, emerged in the field. It is basically a Tit for Tat that nearly always retaliates, but sometimes it doesn’t.

I am acutely aware that my analogy is getting quite stretched out at this point, but I think that there is something to be said for randomly forgiving your former self sometimes, sometimes the day is just randomly bad and crackdowns will just make things worse, and the removal of the TV will just make the others sadder, not nicer, as they were already trying to be nice,  but randomly couldn’t.

Idea: Receive a letter from you yesterday every morning

On a final note, this whole experiment gave me the idea of evening journaling in the form of a letter to the others. In the prisoners dilemma, a crucial component of the dilemma is that the prisoners are not allowed to  communicate.

We cannot really talk to our future or past selves, but we can send a letter to our future selfes to read. I imagine, perhaps, a journal entry that describing what your experience of the day was like - how you experienced that you were supported (which can be thought of as a specific form of gratitude journal entry) and what you did to support future you (or how you cracked down hard on something because you did not like picking up the slack of other yous).

It’s a kind of cool format in general for a journal, writing what expectations you have of the next you for the coming day (and perhaps what you thought of the expectations placed on you by your predecessor).

Kindness or Crackdown today?

I hope that your Sunday self was kind to you, giving you a well-supported Monday. If not, will you let it be bygones and hazard to be kind to Tuesday, or will they be facing some tough love with a much needed crackdown?

Follow tip: Milos Makes 3D maps (with R)

If you find delight in maps and dataviz - I ran across a really cool niche account this week, Milos Makes Maps - "I paint the world with R and teach you how to unleash your inner map artist." Beautiful work and extensive R tutorials.

Be a light, not a judge, be a model, not a critic

It is highly possible that news has reached you the .io top level domain (TLD) is going away.

This story is actually about another story but long story short is that The United Kingdom is relinquishing control over The British Indian Ocean Territories (Article: UK will give sovereignty of Chagos Islands to Mauritius).

All that means that the country ceases to exist, and thus the .io TLD goes away as well.

So, congratulations, Mauritius, on your new island(s)! 🎉

"Well this is a bit of a pickle"

A country and it's TLD being disbanded happens to be a bit  of a pickle for many tech companies (and many more crypto scams), who have adopted the domain because of it's input/output connotations, rather than having anything to do with the Indian Ocean territories.

I’m sure many that use the .io domain had no idea about the connection. Not just for domains one owns yourself, but also ones you use by extension - there is a lot of github.io pages (the domain github uses to provide free subdomains for hosting github pages)

I can't blame them - honestly, I myself use mpj.me and I had not thought about the fact that it is TLD for Montenegro (which is related to the later Balkan part of this story). As a Swede, I've also owned many .nu domains in my day, a TLD that is  is one big shenanigan featured in this story.

Digital "Anitya"

I do love learning about TLD shenanigans. I think it is because they remind me that nothing in life is stable. TLDs seem so robust, like something that transcends the physical in permanence, so when they go away, it is like digital anitya - a common buddhist term which means "Impermanence" in Sanskrit.

It gives me a weirdly nice and liberating feeling that the concept of a country and its domain just goes away. I think it’s for the same reason that I find pleasure in the fact that Limnic Eruption exists.

If you’re unfamiliar, Limnic Eruption when big carbon dioxide pockets at the bottom of a big lake gets dislodged by small earthquakes, and when the bubble comes up to the surface it pushes all the oxygen out of its way for a few kilometers and everything that was strolling around the vicinity of the lake suffocates. It is one of the most rare natural disasters that exists - in recent history, this phenomenon has been observed twice.

The first recorded limnic eruption occurred in Cameroon at Lake Monoun in 1984, causing asphyxiation and death of 37 people living nearby. A second, deadlier eruption happened at neighboring Lake Nyos in 1986, releasing over 80 million m3 of CO2, killing around 1,700 people and 3,000 livestock, again by asphyxiation.

While technically horrible, Limnic Eruption reminds us that the universe is absolutely neutral and owes us nothing, and the fact that ANYTHING works at all is wonderful.

But I also love TLD stories because of the same reasons I like heist movies - TLDs is an area where a lot of  (really clever) shadiness has been going on, and people often get away with it. Clever cookies in the early days of the Internet made deals with governments to run their TLDs before it was widely known how much money there was in running a registrar.

In the .io case, this cookie was Paul Kane (now CEO of CommunityDNS), and he got the TLD in 1997, and operated it until April 2017, he sold the Internet Computer Bureau holding company to Afilias for US$70 million in cash.

In July 2021, the Chagos Refugees Group UK (yes, the same island mentioned at the beginning of this chronicle - that is now under control of Mauritius) submitted a complaint to the Irish government against Paul Kane and Afilias, seeking repatriation of the .io domain, and payment of back royalties from the $7 m per year in revenue generated by the domain.

A detail I noticed while looking into this was that the The Chagos Refugees group co-filed this complaint with the Crypto Currency Resolution Trust, which is a crypto scam victim fund that sues for damages because .IO was a popular domain for crypto scammers to use, because the British Indian Ocean Territory lacks a financial crises unit, and can’t  really prosecute any crime committed on .io.

I also learned that said Paul Kane was selected to safeguard one of seven keys to restart the internet in case of a catastrophic event.

From the article: “In the event of a security breach, Paul may be required to travel to a secure location in the US where he will meet five other key holders, to recover the master signing key.”

In light of all this, I propose that we henceforth refer to Kane as “Domain Name Saruman”.

How to scam an small pacific island by leveraging a Swedish foundation

For Swedes, the story of the .nu domain is rather well know. The .nu domain, which corresponds to Niue, a small island in the Pacific with a population of 1681, has been the subject of a long-standing legal dispute over its control and the financial benefits associated with it.

In the 1990s, Niue’s government entered into an agreement with American businessman Bill Semich (Domain Name Sméagol?), who helped register and manage the “.nu” domain.

The domain, which coincidentally means “now” in Scandinavian languages, became particularly popular in countries like Sweden, Denmark, and the Netherlands.

Semich’s foundation, IUSN, promised to provide Niue with free internet access in exchange for control over the domain. However, over time, Niue realized that they were missing out on substantial revenue from domain registrations, which could have supported the island’s economy.

What is especially absurd with this story s that Niue still doesn’t have control over it’s domain - in 2013, the management of .nu was transferred to the Swedish Internet Foundation (Internetstiftelsen), sparking further dissatisfaction in Niue.

The Niuean government has since been fighting to regain control of the domain, arguing that they were misled and that the agreement was unfair. They even pursued legal action in Swedish courts, seeking damages for lost revenue estimated at around $30 million.

However, in 2020, the Stockholm District Court dismissed their case because a foundation cannot be liable to damages, it seems), and Niue was ordered to cover legal costs. Despite this setback, Niue remains determined to reclaim the domain and continues to appeal for redelegation through ICANN.

Can I just say: Yarrrrr. 🏴‍☠️

From .yu to .me

BUT my favorite story is that of .yu, because it has a bit of a different turn. In 1989, Borka Jerman-Blažič  (Domain Name Gandalf?) and her team in Ljubljana began working to connect Yugoslavia to the internet (it is a bit abstracted away nowadays, but during the early days of internet, a country effectively needed a TLD to connect to the internet.

At this time, Jerman-Blažič registered the .yu country code top-level domain (ccTLD) for Yugoslavia, a story she tells in the wonderfully internet-retro documentary from yu to me:

I am really glad that the .io debacle made me find the above documentary, as it is a wonderful piece of edge history about the Internet, and a really cool portrait of Borka. Her story is a very pure example of "ask for forgiveness, not permission" where she basically said to herself "well someone needs to pull internet into this country" and did it.

Connecting a country to the internet is a project that Borka basically managed on her own initiaive, and two years later, in November 1991, Yugoslavia was connected to the internet.

Due to the early nature of things at this point, we have many stories about people like Borka - the first person to realize that this was needed, and they pulled Internet in. Some were more entrepreneurial than others, and I think Borkas story was unusually ego-less and she is obviously a personality more driven by "this needs to occur" than personal financial gain or glory.

Anyway - Yugoslavia was unfortunately only peacefully connected for less than a year. After that, things got messy.

It is important to understand that during the two years it took Borka to actually use the TLD to actually connect Yugoslavia to the Internet, Yugoslavia crumbled quite a bit as a country, and in June 1991, Slovenia declared independence from Yugoslavia.

Already in February the following year, Slovenia manages to get international recognition and becomes a member of the UN. As you can see, it is a lot easier to pull off anything in the nineties.

The Slovenian government created the Academic and Research Network of Slovenia (ARNES) to manage its new .si domain. However,   while they were waiting for the .si domain to become active, they needed access to the internet, so ARNES members broke into Jerman-Blažič's lab, copied the .yu domain software and data, and cut off the line connecting her computers to the internet.

From How does one connect a country to the internet?:

“During one weekend in July 1992, the people who were later employed by ARNES broke into my laboratory and, using scissors, cut off the line connecting my computers to the switch located in the cellar of the institute, then copied the zone file for .yu.

They did know how to run the domain server,” she said. "Though they only used the network for email, ARNES secretly kept .yu running for the next two years, ignoring requests from a rival group of scientists in Serbia who needed the domain for their work.”

So yeah, in the nineties, (even if you didn't know how to properly configure DNS), you could break into a lab and physically heist a TLD by cutting a network cable with a pair of scissors and nabbing the zone file.

All of .yu went dark on March 30, 2010, but after that it was acquired by the museum of Yugoslav History,  similar to the Museum of Modern Art in New York's acquisition of the "@" symbol. The Museum of Yugoslav History views the .yu domain as its first virtual artifact.

Be a scoundrel

Early internet stories have so many stories of people who did things that they weren't supposed to (not not supposed to, there were NO rules at the time) and weird and unpredictable things occurred.

I don't mean to glorify any of these as examples of how things should go, but hey, they did, and it's cool, and the internet did end up existing!

Sometimes, its good to play it safe and be amicable and polite, and sometimes, there is treasure to be found, and adventure to be had, for the scallywags, rascals and scoundrels.

By all means, try to play nice, but if things are frustrating for you this week, even though you're doing all the things right and by the book, then maybe, just maybe - consider if instead it might be appropriate with some shenanigans.

Beautiful News

Oh, and one final thing - I just now discovered Beautifulnews by Information is Beautiful. This is great, especially since it redefines the usage of the word "news" as "news to you maybe" instead of the colloquial "recent sensational anecdote".

Lovely, simple:

Have a great week, and stay curious