What if you did ALL your writing with the same tools you write code in?
Writing a video series in an IDE — why treating prose like code unlocks surprising production superpowers
Gooood oneday morning
This chronicle is a bit late and missed most of your Mondays, but it is quite a meaty one so I hope it makes up for the wait! 😊
The current status in my life is that I'm still working on revving up main pipeline for the dawn of the data developer format. While it is in many ways going well and I'm making strong progress, I've also realised that this process is going to take a little bit more time before the main format is ready.
But the software-assisted production thinking (more on that later in this chronicle) generates a lot of interesting insights, so I've decided to take inspiration by the Devlogs that many indie game devs do while they are making their game and feed you learnings I get about the process as I move forward.
LFG
What I am working on: The Untutorial Format
I’m currently working on a new format for fff, codenamed “untutorial”, a completely new production format that I have never seen anyone do before. I won't go too much into the untutorial format in this chronicle, but briefly:
I dislike tutorials, I don't think they teach things very well, they are like opening a walkthrough the minute you hit a boss in Dark Souls.
The joy in, and the key skill as a developer, is to figure things out. So the Untutorial is the tutorial turned upside down + it doesn't tell you anything about HOW to do something, but it gives a strong opinion of what you what you should learn and why, pumps you up to do it, and gives you a sense of achievement through objective win conditions.
The Untutorial isn't concerned with teaching you how - it cares about convincing you that you should and that you can - and giving you and objective measure to determined that you pulled it off. A loving gauntlet, of sorts.
Integrated Writing Environment
But this chronicle is not about the untutorial, but about automating writing - what I mean by that is that we have all kinds of automation and tooling around writing code, but when we write other things than code, we throw all of that out the window and shove the entire document into a what is essentially a glorified string variable (be it a markdown document of a google doc).
But what we didn't? What if we treated all writing that I do with the same level of respect and process that we give code? What if we could re-use use ALL the powerful tooling that we have around writing code, to all our writing?
I've historically written the Fun Fun Function manuscript in a plain markdown editor, but what if I instead created a domain-specific language on top of TypeScript, and wrote it in an IDE, like Cursor?
Here is a screenshot of how it looks like when I am writing this very Chronicle:
This way of working has grown on me a lot, and I'd like to spend this chronicle raving about it, because it wasn't obvious to me from the start how incredibly many benefits this has - it just keeps on giving.
Software-Assisted Fun Fun Function production
Video production is so incredibly time consuming. When I did Fun Fun Function back in the day, most of my time was spent on the editing and post-production, and a very small share of all the work I did on Fun Fun Function was really core content.
The dawn of the data developer video was particularly absurd in this regard, I shot it in one day, and post production was about 6 months. And even though I am a developer, there has historically been almost 0% of automation of my own process.
A precept of automation. Because the default intuition in video production is not to, one of the strict guidelines I have placed on myself on this new generation of Fun Fun Function is that the production process should be heavily software assisted and automated.
Basically, I want to create a set of software tools that helps me make Fun Fun Function. In a way, I want to treat the production of the show like a game developer treats the development of their game. Game developers are notorious toolbeasts, they build tons of tools for themselves, very often building their own game engines.
I will naturally avoid building my own video editor - that seems like a good way to never produce any actual video - but I am intending to have a lot custom software doing things for me. As part of this philosophy, for the video and audio parts I'm switching from Final Cut to Davinci Resolve because it is much more programmable and extendable than Final Cut.
Pre-production automation. However, it's not just post-production that I want to automate. I am also aiming to programatically generate a sizeable amount of the actual videos from my manuscript.
And no, we are NOT talking about "AI generated video" here. While this involves LLMs as a component, what I am about to describe is very different.
An API so good that it obviates GUI
The experience that made me seriously entertain the idea that I might be able to CODE a lot of the production was using Motion Canvas.
Motion Canvas is controversial in the sense that it very firmly takes the stance that animation is best expressed using code. Motion Canvas has the philosophy that the code and the API is the primary UI.
Motion Canvas still has GUI, it renders your animation instantly and has a timeline to show you the output so you never have to 'simulate code in your head', but there is hardly any input boxes or buttons in the interface regarding your animation, all of that done in TypeScript.
When I first heard this idea, I was a bit sceptical. Interesting idea for sure, but it is very novel - tooling like After Effects exists for a reason and has a strong legacy.
However, after using Motion Canvas for over a year now, I can tell you that if you are a person that works with visualising complex ideas, and you know of TypeScript and like Flexbox, Motion Canvas is the greatest thing since bread.
The ergonomics of the Motion Canvas API is just amazing, using Flexbox and Generators for positioning and animation flow a joy to work with compared to the traditional pain of keyframing values.
TypeScript, especially coupled with IDE Copilots, actually makes the user experience arguably more discoverable and intuitive than say, the rather daunting GUIs of After Effects or Davinci Resolve.
AI Assist for complex animation: Having the user interface be a typescript API also gives an LLM CoPilot working with you be incredibly powerful. My copilot will often infer what I am about to do next in motion canvas: but an AI assistant doing the same in the After Effects GUI seems unthinkable.
How would AI suggest below even work in a graphical keyframing GUI like After Effects?
I can't think of any way that would not trigger unwanted renders, as one advantage with code is that the suggested autocomplete shows intent in text without actually triggering an expensive re-render.
Instant iteration. One of the main reasons that animating in code is so powerful is that you can iterate - you have logic loops.
For example, in After Effects, if you want to make 90 circles with headers in them, representing the different parts of my manuscript, you often have to painstakingly animate all of them, at least to some degree.
But, in Motion Canvas, you just do a loop:
If you are a web dev, the yes, all of this is as modular as you might suspect from looking at that code. It bloody fantastic.
Creating reusable components in traditional animation software is not really a thing, at least not in the sense that developers are used to composability. In motion canvas, I can just render 30 circles in a for loop and specify a Flexbox flow and it just renders, hardware accelerated in real time to the canvas.
And it happens instantly, because with hot loading tools like Vite. Vite changes everything here. Using TypeScript used to be a slow feedback loop with manual save and rebuild steps - now it is instant gratification.
Here's some teasers of what my work on the Untutorial looks like:
But even through I could create 30 circles for my manuscript headings in a loop, I realised that I was still copy-pasting in headings from my original script (that was a markdown file), into my motion canvas project. This inertia led me to consider if I could apply the same principles that Motion Canvas applied to the animation to the scriptwriting itself.
I.e. What if I could programatically feed my manuscript directly into Motion Canvas, and use it as data in the for loop doing my rendering?
Okay. Hmm. Well, in order to iterate over anything, we need to first translate that thing into some data structure that actually can be iterated.
This is part where I try to use an LLM to squeeze blood from a rock
I initially toyed with the concept of parsing the markdown, but I quickly realised that markdown headings isn't a very expressive representation of the structure of a document, not all headings are headings that you actually want to render out, so headers effectively have implicit metadata.
Plus markdown is made to be rendered, not to be parsed as data, so it's rather awkward to deal with programatically when you try to do that. Plus, quite frankly, markdown is a very light format that is SUPPOSED to not get in your way with structure, so when I write in markdown I just tend to be sloppy and not be structured, in one way, thats one of the nice things about markdown.
I actually tried generating a structure from the unstructured script with an LLM, but when I did it was quite obvious that the problem was that my manuscript didn't have good structure, and good structure requires knowledge and reasoning, and LLMs don't really have that.
LLMs are very useful for certain tasks, and seems like they are omniscient sometimes, but this is one of situation that really clearly shows that they are indeed glorified autocorrect with more data and compute. LLMs only imitate structuring, they don't ACTUALLY know anything about how to structure writing.
LLMs know how to mimic organised writing, which means that tehy are very good at producing results that LOOK organised. It's a really weird experience, because at first (and even second) glance it looks really well organised, but when you inspect it closely, it the text equivalent of when generative AI illustrates a cat perfectly but with 6 fingers on one paw, and the cats are hovering one centimeter above the ground, plus there is also trees with both walnuts and acorns in the background.
In a way, the results of LLMs trying to organise material reminds me about some experiences at larger workplaces, where you sometimes run across a human that you are awestruck with for the first 6 months because they have developed a way of projecting that is completely optimised for sounding good, and when dare to feel stupid and ask them "what do you mean by that" and repeat a few levels in, and then it become obvious that they don't really know how many legs a cat actually has.
But I digress - if all that fails, then how can we ensure that our manuscript is structured data?
Narrative and Pedagogy as structured types
Documents with actual, structural types. My original background is in theatre, so I tend to work with scripts. Scripts in theatre and screenplays are conceptually speaking structured as hell - if you watch Studio 60 on the Sunset strip you'll understand how much structure there is to comedy.
In fact, all good writing has structure. The above is true not just for YouTube-programming-show manuscripts - there is a lot of structure to pretty much all good writing. I recently watched an interview with Dennis E Taylor, the author of We Are Legion, We Are Bob, who is a former software engineer, and in the interview he remarks that software engineering and novel writing is very similar in that you are working with a structured script:
Journalism has a tremendous amount of structure (and I've noticed that journalists, among with lawyers, are among the quickest career-switchers into programming)
Corporate Strategy is admittedly known for being a bullshit-writer-laden field, but the ones that know their stuff are very clear about their structure, Richard Rumelt and Peter Drucker for example.
OKRs are another example. They get a bad rap because unfortunately OKRs are often in game-of-telephoned into meaninglessness in organisations, but in my opinion the OKR framework is actually a very clear, precise and useful vocabulary if you go to the source.
Poetry is often made out to be very loosey-goosey and emotional, but any poet knows that poetry is one of the most structured types of writing (arguably in some cases more restrictive than programming).
But even with my background that makes me acutely aware of this phenomena, AND my background in software engineering, when I put have my writing into digital form, I've habitually just disregarded this structure and crammed it all into one datatype: a string. Dennis Taylor also seems to not have experimented much further either in his process.
I am 40 years old and have worked with software almost my entire adult life, and it has never struck me that there might be a more sophisticated way of writing manuSCRIPTS.
Claude's XML tags:
An unexpected muse
I spent a lot of time with Anthropics AI Claude in the last year, and it has an interesting peculiarity. When you get into advanced prompting of Claude, you learn that Claude likes it when you wrap concepts in XML tags to give the AI an idea of the structure of your prompt. And when you as a prompter are prompted to wrap a part of your prompt in an XML tag, you per definition have to NAME that XML tag. I can't help to suspect that it's not actually the XML tags themselves that improves the result, but the fact that it makes the prompter think more carefully about what the hell they are trying to do.
When I worked with the Claude API I found myself with something more structured, more explicit discrete declaratons. It was more structured than a string of markdown, but it was still rather cowboy typing. And when I was looking at it, I realized that this kind of kind-of-typed document is a much broader problem, hardly just manuscripts for youtube episides - we have a lot of this structured-ish data in organizations and our life in general, random lists, notion/tana/obsidian hellscapes with whatever productivity system or team workflow we thought was going to be the end-all be all this quarter, excel spreadsheets mapping vacations in teams, and my personal favourite: shared word templates with a 'version history' at the top. Yum!
Are these "Taglets" Data?
I struggle to find a good grouping terminology for the above phenomena, so I'll invent and refer to them with the horrible name "Taglets" in this chronicle. So sorry about inflicting that word upon you.
Are the above mentioned taglets data? I would intuitively say that, colloquially speaking, yes, they are.
Or at least in the sense that Transformers: Age of Extinction can be called a 'movie'.
But if these Taglets are data, what is a definition of data broad enough that includes Taglets, but that is more specific than just 'information'?
Really - what is a good colloquial definition of "data"?
Data: "A collection of information organized in a certain way"
I figured that the best way to find out was to ask someone that isn't a data professional.
I had a new friend visiting this week, who is a lawyer whose next career level is Judge - extraordinarily smart and intellectual, but not at all digital in their thinking.
Fun Fun Function came up, and she said that it's not really her domain and that she doesn't really know what "data" is. Not an uncommon or unreasonable statement but not really a correct statement one, because she, like any knowledge worker, deals with it all the time.
I believe that data is not really a digital concept per se, but an emergent concept from collective intelligence, and I suspected that she would have a good intuition about it, and asked her - how would you describe the concept of data to a child, as you relate to it in your personal and professional life?
She instinctively gave me the definition 'A collection of information organised in a certain way' - which I think is pretty darn bang on for a colloquial definition.
What separates a "taglet" from "real"data?
But then - if both Taglets* and quality datasets are data, then way is it that tagles are missing?
* = again, so sorry about that word, I am really already deeply regretting this nomenclature
What is the quality distinction between a neatly organised scientific dataset that we can actually do something with, and the Mordor of Taglets that Notion turns into after a few years of use?
There are, of course, many contributing factors - one being the dark pulsating sentient meat living on floor minus 8 that among other evils conjured the Notion Calendar strategy.
However, on a conceptual level I think what makes this data low quality is the lack of schema.
Another way to think about it is that these Taglets lack strict typing.
TypeScript benefits
So I started running with the wild thought - what would be the potential payoff is I stopped using WYSIWYG editors and experimented with write my manuSCRIPTS in TypeScript?
Before you think "MPJ has gone full mad scientist here and is probably into reiki healing too" let me tell you that I am not into reiki healing and and also:
TypeScript is a tool specialised at adding schemas on top of weakly typed data structures.
TypeScript has a massive ecosystem and is very well known, potentially allowing us to leverage the knowledge and toolset towards our writing process.
As a type system, TypeScript is extraordinarily expressive and can be molded in very dynamic ways - what I mean by that is that TypeScript is distinct from say, Rust, which is very oriented around safety and speed. Comparatively, TypeScript has a peculiarity that it prioritizes communication first and speed and safety second. This is a contentious point for many, but for our purposes here, it is perfect.
TypeScript is open source, and is just code as text in normal files, and as such I can use a code editor with all the bells and whistles and organization features built for codebases with millions of lines of code and hundreds of files.
When I think about it, the price to and quality feature discrepancy of writing software to editors like cursor, vscode, zed etc. is downright absurd. I've paid comparatively enourmous amounts of money for software like IA Writer, Ulysses, Scrintal, Scrivener etc.
A lot of these pieces of software are incredibly well made, but comparing the power and value that Cursor or competing editors adds to to coding, with the value that these dedicated writing apps add to writing, it is quite clear that coders get A LOT more bang for their buck.
Breaking free from invisible platforms
Even though I haven't thought of a lot of the things I've used as "platforms" even standalone apps have lock-in effect when compared to software development, which is a field incredibly allergic to anything that smells like dependency.
Many editors have special formats, custom cloud sync, and various types of lock-in - even Obsidian that is best-in-class in this regard, that has tried to be as portable and plaintext as possible, has to make certain concessions because Markdown as a format doesn't really have mechanics for interlinking and any importer realistically needs to understand the Obsidian format in order to do the import.
Git becomes the one cloud. When I write in TypeScript I can just use Git as my cloud storage. App-specific cloud storage that locks in your data with the app is no longer an issue, because I can use git for my work, which is dirt cheap and enterprise-grade and basically a commodity. Yes, Obsidian CAN use Git, but it just uses it as a Dropbox and wildly commits on save and it isn't really version control in the controlled sense that I have integrated in an IDE.
Real versioning. And that brings me to the fact that TypeScripted scripts gives me ACTUAL version control. Sensible diffs, branches. Even PRs. When I've tried to use Git for Markdown documents, it's clear that doesn't really "get" completely flowing text. This is not really Gits fault, its simply that flowing text is not really structured - it's just a big-ass string.
Git hosting is a commodity and commodity = good and cheap. The ridiculous value that comes from using Git instead of lock-in proprietary formats cannot be understated.
Before I go on I just to make this absolutely clear: I used to love Notion. So, so much. I was a very early user, and I lived in Notion. I know every nook and cranny. Me and my girlfriend organised our relationship in Notion. We literally had Notion dates.
Me and said girlfriend are still together, but me and Notion are going through a very bad separation.
As the last few years have progressed I have never felt more betrayed and extorted by any company. Notion seems to be proactively be seeking to become a caricature example of what Enshittification looks like. I am a pretty chill and understanding guy, and it's a tall order to piss me off, but Notion has been very diligent and focused in pulling bullshit consistently, and they have claimed their rightful seat on the brown throne.
Exhibit A:
The above is a history of a document on an account that I have specifically upgraded back in the day to receive version history (when that stopped being free), and nowadays they are asking me to pay more then further back I want to go.
What does the enterprise tier cost?
I AM NOT GLAD YOU ASKED
"HI PLEASE PAY SEVEN DISNEY PLUS SUBSCRIPTIONS in case you want to access old diffs of your own data that you have given us".
Disney gives you the David Tennant as Scrooge McDuck, and Notion gives you old diffs. For seven times the price.
Notice that Notion aren't shy to show me that they have no problem storing the versions in front of my face. Notion can do this because it is extraordinarily cheap to store diffs, and which is why plain Git commodity storage gives you this for free.
If Github started charging this way, whatever bank Gitlab is using would go offline from all the money they would receive at once. But Notion can pull this thinly-veiled-ransomware move because they know you cannot realistically migrate away to anything.
Okay, sorry about ranting, total digression - you did not come here to hear me rant about pricing schemes.
I will stop crapping on Notion now.
NOTION SUCKS
Notion does not support you to use your OpenAI or Anthropic key API key, but fear not, instead they offer this $25/month plan for their AI (model not defined but I'm sure it is totally comparable to GPT 4o or Anthropic).
So, paying for this captive-audience AI model and the tier that gives you full version history OF YOUR OWN GODDAMN DATA THAT YOU BY THE WAY ARE LEGALLY ENTITLED TO UNDER GDPR will set you back $362 per year. Per user.
In contrast, Cursor offers an insanely fast model with amazing GUI for $20 per month which actually includes a decent amount of GPT and Anthropic use (and, like Zed, they allow you to use your own keys too if you don't want to pay for their model).
And the AI integration is ludicrously good, and can be used almost completely with keyboard nav, I love the alt-enter thing Cursor has for "quick questions":
HOTKEYS and predictable ergonomics. One thing about an IDE is how familiar you get with it. I know the hotkeys of my IDE by heart and when I moved from vscode to cursor I could transfer my custom bindings.
On mobile, I write a lot of my stuff in Bear, which is a notes app that I do think is fantastic, but whenever I edit in it, I struggle to remember what hotkeys do what in that app, and all purpose-built writing apps have their own idea of what is good.
Code editors are much less arrogant and incredibly subservient to their user base, and if something is off, there is a slew of customisations and plugins to make it work like your neurology expects it to.
If you are a vim user, you know how much all of this means to staying in the zone and being productive - being in an editor that your neurology is wired to compared to one that isn't is night and day.
And if my scripts are structured data, I can automate the rest of my creative workflow on top of it - a chapter, a character, a concept - these things would actually now be discrete, programmable concepts that I can write automation flows on:
Publishing writing that is stored as TypeScript in git gives a treasure trove of cheap, robust options for publishing and sharing - Github pages, Netlify, Vercel etc. so much cheaper and less convolutes than Notion pages or whatever limited publishing flow is built into any given purpose-built editor.
Notion Pages is a very new feature so they have not started extorting people super aggressively yet, but I am screenshotting this here so that we can compare it in 24 months:
Custom rendering: I can display my scripts with react or solidjs, and use css frameworks, or ANY other output with npm modules and make any output look like I need it to. This is an enourmously sharp constrast to trying hack notion into looking not horrible with shoddy chrome extensions.
Custom export pipelines: I can run filtering functions on the script, getting just headers, making indexes, etc.
A concrete example below: In preparation for the shoot video version of the chronicle I am writing, wrote a small script that filtered and outputted the script in a format suitable to paste into Elgato Prompter software. This which would otherwise have required a painstaking amount of error-prone copy paste, because the Elagato app is very particular about line breaks an formatting:
(Avert your eyes from that last code, LLM POC, but hey it works! :D)
Powerful navigation and focus control. A code editor is designed to handle overwhelming amounts of complexity, which also works on writing if it is structured in TypeScript.
Below is an example of how sticky scroll pins sections of my script at the top when I scroll, and how I can collapse and expand sections when I want to focus on certain parts:
You can also imagine how tons of code navigation plugin you use can be used similarly. If you move things into variables, you can click them to navigate, and so on.
Staying in one, low-distraction app. Staying in the editor has been suprisingly good for my focus - going to the web is a rather dangerous affair for my attention these days , even with a powerful adblocker I find myself pulled away from my work, a text editor is SO serene and focused on supporting me - web platforms, that want to ENGAGE me, not so much. Not having to go through google (or even tab out from the app) to do quick research, has done wonders for me when it comes to uninterrupted flow:
I could probably go on and on with many of the benefits of working with way with writing, but this chronicle has already went far into the night, and if you made it this far, you are one brave and nerdy soul, and I am very humbled and I love you very much.
This week is sponsored by undefined
A final note - there was no sponsor sold for this weeks chronicle, so I thought I'd take the opportunity to ask you:
What company do you think should be sponsoring funfun.email?
Is there any particular product or service that you are a fan of, or perhaps you work for one that would be a a good fit? I'd much prefer to have sponsors that you all like or already have relationship with. If so, it would be very valuable to me if you sent your suggestion to partnerships@fff.dev 💛
I hope you have a great week, and as usual, I love hearing from you, so don't be shy about replying with your reflections.