Not a newsletter: Getting to the bottom of things

A refuge from urgency: data tools worth knowing, the joy of fundamentals, and why this is not a newsletter

Hey babes (I have been watching the new Dr Who and I am now saying 'babes' at the end of every sentence in a futile effort to achieve 5% of the charm of Ncuti Gatwa)

😊🙇💛 Writing in email format is a new experience for me, and the first one that I send out after such a long time is a lot of pressure, so I am going to resist the temptation to write some masterpiece and instead just get going by writing about a few assorted discoveries from the last three years, in a long-read-ish format.

🛋️📚☕️ I've written this in a flow, from the heart - so give yourself a break from skimming and scanning and stressing, and sit down and enjoy on a tablet with a warm cup in your hand. Go scout out a good chair - your attention is worth the attention.

"I have been a happy man ever since January 1, 1990, when I no longer had an email address. Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to get to the bottom of things."

- Donald Knuth, Author of "The Art of Computer Programming", from Knuth vs Email, July 1997

😎🧘🏼⛅️🍂

While there are some fresh tech nuggets below, my general ambition is to try to bias away news and the latest and greatest and hot new things when writing funfun.email.

This is decidedly NOT a _news_letter. I'll be referring to funfun.email as Chronicle (or perhaps Almanac?) and my goal is to produce something that gives you a safe haven from urgency and novelty - a refuge of fundamentals (puuuun intended).

On the response to the relaunch of Fun Fun Function 💖🤗

I have to mention this. Oh my.

For a number of reasons, 2023 was an incredibly hard year for me, especially for my confidence and energy. You cannot possibly know how precious it has been for me to experience the incredible storm of loving words and attention that you have generously given me in the last few weeks.

As of writing, the relaunch video has amassed 44,762 views, 4,929 likes and 864 comments. The return announcement tweet has 121,000 impressions, 1,800 likes, 250 retweets, and 177 comments.

And the most flattering thing of all - just under 4000 people have signed up for funfun.email.

I am humbled to my core.

Thank you 💛

Highlights from MPJ's data toolbox: ClickHouse, DuckDB and Observable 2.0

During the last 10 years, data tooling has become gradually more accessible, powerful and robust, and this shoulders-of-giants trend is a big part of what inspired what eventually led to the Dawn of the Data Developer narrative taking shape in my head.

There are four tools that I want to give especially honorable mentions to, that I experience as genuine game-changers, and that you are likely to see on the channel a lot:

ClickHouse. The saying "an overnight success takes 10 years" has hardly ever been more fitting. This open-source marvel is the first contender that can finally challenge the reign of BigQuery. It is mind-bogglingly fast, eats petabytes for breakfast, is enormously storage efficient, and can operate in real-time. It can run on your laptop and scale to massive deployments, and has impeccable documentation and wonderful community. I am impressed and so happy that ClickHouse exists.

This video showcases it very well: What Is ClickHouse?

DuckDB. A high-performance, in-process SQL OLAP database management system, designed for efficient analytical workloads. It has an incredibly tight footprint and is designed to be shoveled into almost anything, and remarkably, it can run directly in the browser using WebAssembly (WASM), which allows us to bridge high-performance data analysis capabilities to the frontend side in a way that was pretty much impossible before.

🤣 And such a funny community: DuckDB Tutorial For Beginners

Hex. When I was introduced to the field of analytics many years ago, Jupyter Notebooks was a remarkable tool, that allowed chaining cells of data together in order to create ad-hoc dashboards. Jupyter was the golden standard for a long time, but was always a bit of a data engineering nobility thing, because they needed a dedicated machine being deployed and Pandas has an esoteric vocabulary and the plotting libraries were equally arcane.

It is only recently that Analytics Notebooks as a category has gotten productified well, and Hex is one of the leaders here - with collaborative features and incredibly data+code aware AI assistants that removes many previous roadblocks and pushes the power of the Notebook to a much larger group.

Seats are not cheap unfortunately, but the free tier can be used on a small personal scale and gives you a taste of how much easier and pleasant analytics have become these days.

Observable 2.0. On the Open Source side of Notebooks, Observable Framework is extremely exciting and I have been using it a lot. I have been a big fan of Observable 1.0. For the newly initiated, Observable 1.0 was a kind of client-only JavaScript playbook, built by a company founded by Mike Bostock, the creator of D3.js, aimed at advanced data visualization, a kind of fancy REPL for D3.js if that sentence makes sense to you.

For me, Observable was always frustratingly close to incredibly useful, but some of its architectural limitations gave it a bit of a toy vibe. It was not easily deployable outside of their walled garden, and no Python support meant that I never really got professional analysts interested in it (turns out that there is a limit to how much JavaScript you can shovel down the throat of a Pythonista, just as the inverse is true 😀).

Observable Framework solves these issues, and is a fantastic environment, that I find myself using instead of Hex for many tasks, and the Observable Plot library has over the years matured to a very sophisticated plotting library. Observable provides a great hosted service, but Observable Framework is open source and generates static assets and can be deployed on any web host, even as static files on shared web hosting platforms - even if you can mix TypeScript and Python in the data loaders, they build as static assets at deploy time - wonderfully clever.

Observable Framework's announcement video:

Behind the scenes: Motion Canvas and programmatic animation

The Dawn of the Data Developer episode was a grueling experience from a production perspective, and took an incredibly long time to make due to a number of factors (technical, mental, motivational, health, skill, creative hardships, I had em' all!). But one surprising aspect (that is usually a massive time sink in video production) stood out as an absolute breeze: Animation.

I completely credit that to Motion Canvas, which is the secret behind all the animation you see in Dawn of the Data Developer. None of that is After Effects or similar. Initially an internal tool created by the creator of the indie game Astortion to create the illustrations for his Patreon devlogs easier, it has been released as an open source framework that is now at an impressive maturity and flourishing community.

Early demo video of Motion Canvas illustrating the concept:

Motion Canvas is an absolutely mind-blowing animation framework that leverages JSX (but not React), TypeScript and JavaScript Generators in a genius manner that frees you up from keyframing and long render times, working completely in a web application.

I have used the React-based Remotion before but I always found it slightly awkward even though I am extremely familiar with React. Motion Canvas is fundamentally different and generators and the programming style that it enables maps so well to how I express narrative and story.

There is also something about Motion Canvas that allow you to express yourself more beautifully, the stuff that the Motion Canvas community produce is overall very beautiful and I find it to be a delightful digital brush.

Some more demos of what it looks like:

If you want to get into Motion Canvas (which you should if you are doing any kind of animation) you should check out my series on JavaScript Generators - they are simple in essence but simple concepts are often hard to wrap your head around:

Replacing Notion, Tana and Obsidian with TypeScript and SolidStart for creative work

The following is probably my most wild and esoteric discovery and workflow change, that I feel somewhat lonesome in, so let me know if you're doing something similar. 🤗

As a person that to a large extent has rested my career on JavaScript, it took me a long time to warm up to TypeScript, especially for personal projects. I could see the appeal for teams, but why one would use it for small projects was lost on me for a long time, especially for tiny throwaway projects. I just felt that the type system was a rabbit hole of distraction and procrastination that kept me from expression and productivity.

The way Motion Canvas approached things changed my viewpoint on this fundamentally. I found that the code I was writing WAS suddenly the UI.

The types were so well-designed that it was superior to the guidances that a Graphical User Interface would have been inferior. With Github CoPilot rolling in at the same time made animation a wonderfully flowy process and often I did not have to look at the documentation because the guardrails were now being used as "guiderails".

This experience got me thinking… a lot.

Could this sorcery be summoned elsewhere?

At the same time as I was getting into Motion Canvas, I had been getting into Personal Knowledge Management (PKMs) tools a lot, jumping between Notion, Obsidian, Tana, Reflect, LogSeq etc.

PKMs was in general a destructive habit for me, to be honest - it was always the same pattern, I started a system in them, took great delight in it, but over time it crumbled and turned out to be another wasteful disappointment that made me feel like a procrastinating failure.

When working in these tools, I often find myself in a weird place where I'm simultaneously doing software development, knowledge work, and creative work.

A commonality in all these tools is that they all try to be a development environment and a creative environment at the same time.

That is like being able to edit the source code of an app at all times, with the ability to change the underlying structure using buttons - buttons that sit right next to the buttons that change the content. It messes with my brain and does this without any of the benefits of a proper development environment, such as version control or a clear distinction between dev mode and creative mode.

So, I figured - if I'm doing development anyway, why not just use TypeScript? At least then I have the benefits of a development environment, source control, styling and version control, etc?

This worked way better than anticipated, and I ended up replacing the bulk of my Notion and Tana use with small apps running in hot reloading dev mode using the absolutely amazing SolidStart and TypeScript. I don't even use a database, I just write everything as typed objects and the hot reloading as the interface.

Anything I would create a structure for in Tana or Notion, I now do with object structures using structured JSON and edit directly in Visual Studio Code. It creates a very fast and flowy process, much faster than these tools, and I can customize it however I want. Bonus is that I am saved from Notion's incessant AI upselling and pushes to switch to their Calendar with proprietary database sync that works only there.

Even though all of my work is done in my code editor, this method of working has also (ironically) made it MORE clear when I'm in "system editing" mode or "content editing" mode than when I am working in Notion or Tana. Even when writing in TypeScript, I separate my writing into separate files and keep the framework and other components in separate libraries, version controlling them separately. This way, I know if I'm changing my system, working on my system, creating content, or producing something.

I wish I had a cool demo video for this one to paste here - I might create some episodes on this once I have a more tangible structure, as it's currently super experimental and inherently rather dirty, but in the mean time, I really urge you to think about what else you could use TypeScript for because it is delightful.

Rocket Yoga

On a more real life note, I have heavily gotten into Rocket Yoga this year. I got quite a bit of remarks on the release video that I have been looking fit - I haven't been to the gym in a year, it's all Rocket (I switched once I discovered that I did more in deadlifts after my gym trainer came back from a vacation and during that period the only exercise I had been doing was Yoga).

I don't really do it to be fit, though, but rather because I find that it sharpens my cognition and presence. Plus it's a million times more fun than lifting weights (and I do actually find that somewhat fun).

I have done quite a bit of Yoga half-heartedly here and there during my life, and always thought it was decidedly "okay" until I discovered Rocket Yoga. If you are a near-Yogi-but-eh, I thoroughly recommend it.

It's a very, very fun and forgiving type of yoga, but it's also very, very tough, and often done to techno, and will keep you very bloody present no matter how strong your monkey mind is -

Rocket Yoga is a "you think, you fall over" kind of activity that I've only really experienced in skiing prior to Rocket.

Many Mondays to come ...

There is so much more to write, I realize, but I am running out of time and we should keep the length of these newsletters somewhat sane. 😛

Give me your thoughts on this format, it will be shaped by the feedback from you all, and please give ideas on subjects! Just hit reply to this email (I cannot promise a reply because I need to sleep and stuff, but I do almost everything that is nice).

And keep on keeping on, my intrepid friends. Keep learning, keep laughing, and keep being the amazing humans that you are.

Until next Monday morning - stay curious.