So anyway, bjornstack is my opinionated take on how to write code, aimed at constraining and guiding genies to get their work done. And while working on it, like I have been whenever I’m working with genies, I got thoughts that I wanted to explore later, or ideas for other things I’m working on. I realized that since I started using beads capturing in this project has gotten easier, while chatting I just mention we need “a new bead about x because y provide all the context we have here” and then there’d be a new bead linked to the work we were doing, and I can just pick it up later.

But as soon as it was something for another project, or even just “I need to remember to pack the charger when I leave the office today,” then I had to break out to capture the thought, and blow my stack and lose my flow. And if I don’t capture it, I lose the thought.

Somewhere on Saturday I realized this is how Gas Town seems to be using beads: You sling beads across projects, they get filed, and they can be queried across everything you’re working on. There are ephemeral beads and long-lived ones. It’s beads all the way down.

So maybe the breadcrumb app I’ve been thinking about, crumb for short, doesn’t need to be a custom database with all the trimmings. Maybe, as a first version, it’s just a thin wrapper around beads. A skill I can give to an agent that says: this is how you record something into my breadcrumbs.

What I’ve been missing #

I’ve been thinking about building breadcrumb for a while now, and I was stuck on how to get started. The idea is simple, capture stuff when you realize it, but it’s hard to do without breaking flow. I was thinking of doing it when committing because it’s a point of “I’m done” but I’m not really committing as much anymore on my own now.

Back before I started doing a lot of my work in chat windows with genies I would do the majority of my thinking in Roam Research. It was structured, it had bidirectional links, and I could find most things super easily. I had workflows to help me think. Glorious. ✨

But when I started building with agents, it would break me out of what I was doing to go leave breadcrumbs in Roam so I could track what I had been up to. So I stopped. And honestly, it was something I noticed when I worked on meaty problems when programming too, I would get caught up in what I did and not take notes. To my detriment later.

So, I wanted to get back to having my interstitial journal workflow I loved in Roam, to be able to look back on a day and get a feel for how it went based on the notes I left myself. And also have a better trail of things I learned as I worked.

And I also wanted to restart the strong weekly review routine I had, which was where I realized a lot of the stuff I did that turned out to be good ideas at work. I have also been reading Cal Newport and I wanted to start doing his daily shutdown routine, and the daily planning with a startup routine. So let’s see if I can bake those into this too.

The thing I want crumb to be is kind of the flywheel that helps me grow. Daily capture feeds into daily reflection. Daily reflection feeds into weekly review. Weekly review is where I compound what I learned. The whole point is being able to retrace your steps. I did something, I learned something, and months later when I need it I can actually find it.

And over time the system starts producing things I actually care about: a brag document that writes itself from the work I did (even as my own boss, it feels good to see how much I got done), TILs worth sharing, a personal solution database I can query when I hit a similar problem months later. I don’t have to buy into “journaling is good for you” to get value, I just need to care about any one of those outputs.

What I built #

I knocked out a first version today. It’s a global Claude Code skill that I install on my machine. In any session, in the middle of whatever I’m working on, I just write (crumb: <whatever thought I had>) and it gets sent over to my cottage project. It can be anywhere in the prompt because I told it to look for either /crumb … or (crumb:…) and it just worked.

One thing I learned getting this to work: the skill needs to cd into the cottage project directory and run beads from there when creating. Otherwise beads doesn’t know which project context to file things into.

So I made a small wrapper around bd (the beads CLI) that I could allowlist globally in my Claude Code settings:

#!/bin/sh
# Wrapper that runs bd against the cottage project database.
# Works from any working directory.
cd "$HOME/workspace/cottage" && exec bd "$@"

The key ideas:

• It finds a daily journal bead in the cottage project (or falls back to creating a plain bead)
• It captures provenance: which project you’re in, which bead you’re working on, so you can backtrack to it later if you want more context
• It files the thought as a comment on the daily bead, then moves on, and the nice part of the comment is that it gives me when, a timestamp, and it doesn’t matter if I’m working on several repos at the same time because it’ll just be combined anyway in the daily bead.

---
name: crumb
description: "Quick thought capture — adds a comment to the user's daily journal bead in their cottage project. Trigger when the user says /crumb, 'crumb:', or uses the inline form '(crumb:…)' as a parenthetical aside. File the crumb FIRST before handling anything else in the prompt. The user is in flow; receive, file, move on."
---

# Crumb — Quick Thought Capture

## How to run commands

This skill includes a wrapper script `crumb-bd` that runs `bd` against the cottage database regardless of your current working directory. Use it for all cottage operations. Resolve the path relative to this skill's base directory:

```bash
scripts/crumb-bd <bd args...>
```

Every invocation needs `dangerouslyDisableSandbox: true` (Dolt needs raw TCP to localhost:3308).

**If any command fails, you MUST show the error to the user and echo back the crumb text so nothing is lost.** Don't silently swallow failures.

## Priority

File the crumb BEFORE responding to anything else in the user's message. The crumb is a side-channel capture — handle it first, confirm in one line, then continue with the main conversation.

## What this does

Adds the user's thought as a comment on their daily journal bead (a bead in the cottage project with label `daily`). The thought gets processed later during review — your job is just to file it.

## Parse the input

Format is `category: text` where the category is optional.

```
/crumb idea: what if protocols were composable
/crumb friction: dolt keeps dying on me
/crumb just a raw thought with no category
(crumb:reminder: pack the charger before leaving)
(crumb:idea: what if we split beads by domain)
```

Categories are freeform — `idea`, `friction`, `reminder`, `meeting`, `learning`, `question`, `todo`, whatever the user wrote. Don't rename or recategorize. No category is fine. `(crumb:idea: blah)` is valid — the category is `idea`.

## Find the daily bead

```bash
scripts/crumb-bd query "label=daily AND status=in_progress" --json
```

**Found:** Use it.

**Not found — fallback chain:**

1. Most recent closed daily that hasn't been reviewed yet:
   ```bash
   scripts/crumb-bd sql "SELECT i.id, i.title FROM issues i JOIN labels l ON i.id = l.issue_id WHERE l.label = 'daily' AND i.status = 'closed' AND i.id NOT IN (SELECT l2.issue_id FROM labels l2 WHERE l2.label = 'reviewed:daily') ORDER BY i.created_at DESC LIMIT 1"
   ```
   Comments on closed beads work fine.

2. No daily bead at all — create a standalone bead:
   ```bash
   scripts/crumb-bd create "<the thought>" --type task --priority 3
   ```
   Tell the user: "No daily bead open — filed as standalone bead."

## Build the comment

### Provenance

Capture where the thought came from — project name, and what the user was working on.

**Project name:**
```bash
basename "$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
```

**Active bead:** Use this escalation — stop at the first that works:

1. **Session context:** If the conversation has claimed or discussed a bead, use its ID and title directly.
2. **Query local db:** `bd list --status=in_progress --json 2>/dev/null` — use ID and title if exactly one result. Multiple results → pick the most recently discussed, or ask briefly. Zero → no active bead.
3. **Folder fallback:** If no bead is found and no beads db exists, record the working directory path. This lets the user query session logs for that folder and time later.

**Format the comment** with a provenance header on its own line, then the content:

```
(from bjornstack, bs-xyz — Refactor auth middleware)
idea: what if protocols were composable
```

When working in cottage, still include provenance if you're working on a specific bead — the point is capturing something different from current focus:

```
(working on in-mlk — Create the /crumb skill)
idea: what if we made categories queryable
```

If no bead is found and no beads db exists, use the working directory:

```
(from ~/workspace/some-project)
idea: what if protocols were composable
```

### Multiple thoughts

If the user drops several thoughts in one message, file them as a single comment separated by newlines. One provenance header, multiple entries:

```
(from bjornstack, bs-xyz — Refactor auth middleware)
idea: what if protocols were composable
friction: dolt server died again
reminder: pack the charger
```

## Add the comment

```bash
scripts/crumb-bd comments add <daily-id> "<the comment text>"
```

**If this fails:** Show the error AND echo the full crumb text back to the user so they can capture it another way. Example:

```
Failed to file crumb (connection refused). Your crumb text:
> idea: what if protocols were composable
```

## Confirm and move on

One line. The user is in flow.

```
Crumb filed → idea: what if protocols were composable
```

Multiple:
```
Filed 3 crumbs on <daily-id>
```

Don't engage with the content. If the thought is so terse that future-them won't understand it, ask for one line of context in the same breath — "Filed. Quick — what's the context? Won't make sense later without it." — then move on regardless.

## When to create a bead instead

Only if the user explicitly asks: "make this a bead", "track this", "this needs its own issue", or "bead: ...".

```bash
scripts/crumb-bd create "<thought>" \
  --description "Captured from <project>, working on <bead-id> — <bead-title>" \
  --type task --priority 3
```

The provenance bit is what makes this useful later. When I’m doing my shutdown review, each crumb tells me where I was and what I was doing when the thought hit. I don’t have to reconstruct context, it’s already there.

And it felt really nifty when I ended the day and the shutdown routine showed me: these are the beads you closed today, and here’s the note you left yourself when you finished that session. A mini-review of my own work that I barely had to think about capturing. Because the majority of my work is already flowing through beads, it all just becomes queryable. It doesn’t matter if I was working across several repos at the same time, the daily bead combines it all.

Why beads and not something custom #

When I was thinking about building this tooling, I assumed I’d have to build something tool-agnostic (Roam/Obsidian/Apple Notes/etc), and because this stuff is so sensitive, local-first with no centralized server. By adopting beads and dolt, I get all of that for free. Dolt pushes all the data into git, and there’s an existing nice CLI interface, and more people are building on top of beads already.

I’m admittedly kind of abusing beads here. It’s not really designed for this kind of journaling. But most of what I’m capturing are tasks I did, and having a daily bead where I add notes as the day progresses, that’s kind of how research tasks work anyway. You add comments, you talk to people, you figure it out, and then it becomes a written conclusion. So yeah, maybe bending it a little, but it’s solving a whole lot more than it’s bending.

A lot of the things I was originally planning to build already kind of exist, just in a packaging that isn’t necessarily optimal for what I want right now. But it’s a hell of a good start that I can adjust and adapt as I learn more about how I want this to work. And then get started on the part of “how do I extract more knowledge from all the information I’m creating?”

What’s next #

I’m going to try this out for a bit and see how the daily and weekly routines feel. If you’re reading this and interested in trying it out before I share it publicly, send me an email and I’ll give you a preview.

The work itself was about making my “scraps,” that is this site’s tweets, be sent out raw to my syndication targets. I feel that sending a link to something you see in the preview, or it could be inline in the post, is a bit much, I should just let it show up in the platform itself.

I have a couple of things planned for this site, and this was chosen as the first step because it would force me to think about how will I maintain my n8n workflows.

So now, instead of writing some Javascript code, thinking really hard, and then test it by tooting something out, I instead have some unit tests and a way to programmatically update my workflow. So as I go down the rest of my list of tasks I want to do working with n8n will be much smoother.

I had a couple of decisions to make, so I listed them out using the Choice algorithm of thought, and as always it forced me to dig deeper into the problem, even if I ended up doing what I guessed going in. But, the benefit of this process is being intentional and explicit, it gets me into the mode of “I’m trying to list out options,” and to explain what made me make the decision. Since this is now recorded in my notes I can easily go back later to find why.

The kind of highlights of what I got done:

• Updating n8n programmatically: Extract the code from a Javascript source file (glorified grep), and then wrap it so it’ll work in n8n.
• Creating reusable tests in Javascript: which turned out to be a bad idea, because the code was the same but diverged too much, so keeping the tests the same would’ve needed too much conditional logic.
  So I ended up copy/pasting instead of creating a maintenance nightmare. This pattern is useful to know, though, I use it in other codebases where the code is duplicated but some shared logic needs to stay the same. For example, adapters for databases implementing a common interface.
• Telegram Instant Views: Learning about and configuring. Create an Instant View template for this site and learn a bit more about XPath.
• n8n’s update API: Dig into what logic there is calling the update workflow REST endpoint, so I understand what I might break with how I decided to perform updates. Created a guard-rail to try and catch if I’m about to overwrite click-ops changes I hadn’t accounted for when changing locally.

Background and scope of work #

I will (for now) not add images to scraps until I have native support for images in my posting (so they either only exist in a special content type, which gets linked, or they’re attached when posting as a scrap).

I do this because I feel linking to these short-text things that really could just live on Masto/Bsky/etc. is a bit much, be a bit more native to them. Do use them to publish my other content but don’t only send links when they’re not necessary. It feels like being a bad citizen to them when it’s only a sink that way.

The way I’ve setup the sending in n8n right now, is that I have two steps in the status: 1) truncating the body and deciding if we need more, 2) and then adding the link, so I need to figure out a better maintenance pattern.

• Choice: Manage n8n workflow with source management
  • Constraints:
    • Be able to recreate my n8n setup if something goes belly-up
    • Keep as much as possible in source (but don’t overdo it, I’m a single person doing this and can manage myself)
  • Options:
    • Keep n8n fully clickops and keep code snippets in blog
      • Advantages:
        • Super simple and what I’m doing, the status quo choice
      • Disadvantages:
        • If I have to recreate the n8n setup from scratch, painful
    • Create the workflow file in my blog and upload it for changes
      • Advantages:
        • All my n8n stuff is in my blog, so it’s easy for me to deal with, and it’s something I can easily share later so that’s a win for sharing
      • Disadvantages:
        • More work to figure it out right now
          • But probably worth it, I’ll download the workflow file and look at it to see what it says
          • looking at the JSON file it looks pretty straightforward, and using JQ I should be able to pretty easily inject what I need into it, so I can create a small file/script to help me out here that’ll be worth it I think
  • Decision: Create the workflow file in my blog and upload it for changes
    • It feels like a good step towards a maintainable setup and minimal effort to get it going

Doing the choice here wasn’t necessarily super useful, I kinda knew that I wanted to start storing the workflow in my repo, but the point was to step back and consider what I was doing. I was in my head starting to go down the path of just doing what I have been doing, it has been fine. But when I decided to look at the options “the engineer in me” reeled back enough that I felt that this time investment will be worth it.

Okay, the following jq will let me update a specific field in my workflow file, so I will have to make it do it “in context”, so that’s a good start for what I’m doing.

jq '.nodes[] | select(.id == "e1bd320b-1222-4324-9168-22372d2e667c") | .parameters.status |= "m000"' src/n8n/bjorn.now-feed-distribution.json

So I think I want my maintenance to be: full status to be handled in one javascript block, so I can extract that and use jq to update the json file.

I haven’t decided how I will ensure the javascript and json are up to date, maybe I’ll just do a makefile because it’s easy for now? Or maybe I do a template that doesn’t store the “production data” so that I never have the actual prod version checked in and only create it on the fly as I’m updating to encourage avoiding the disconnect. I’ll decide later.

Coding the Mastodon logic #

With a short lunch break I’m now back at it, so let’s start by creating the logic I want so that I can let JQ pick it up.

Designing the solution #

Worry: I’m not sure exactly how I will inject this into JQ right now, will I do a full JS file? Extract it smarter and use JQ as a library since I anyway has setup Javascript in the blog…? I don’t know, I’ll look into that AFTER I have finished updating the code I want and verified it manually, then I’ll do this.

Okay, so what do I have available in the current setup?

I get injected $('RSS Feed Trigger').item into my inline function, which contains all the information from the RSS feed, so that’s perfect, that means I don’t need to send in anything more, I just need to decide what to output.

To make my testing life a bit easier I will now go and take my test data and make that available in a test, so I can TDD this thing.

Looking at my code that I’ve used in mastodon (I just remove the let mastodon = when using) I think it’ll work well as-is, so let’s start by backfilling some missing tests, so I have a clean place to work from, the only problem is that I’m calling it as I should in mastodon because I was only using it manually, so that’ll have to go. For now, I’ll remove the self-calling and figure out how to deal with this when I inject with jq, since it feels like as long as I can get the function definition I can very easily wrap it in a self-call with the param I need.

Backfilling tests for the existing code #

let mastodon = ((item) => {
    let msg = item.json.contentSnippet,
        msgLength = 500-23-1, // the default mastodon post size is 500, -23 chars for the link, and -1 char for newline
        isTruncated = item.json['content:encodedSnippet'] !== item.json.contentSnippet,
        truncateIndicator = '\n… more';
    let wordTruncate = (str, n, indicator) => {
        if (indicator === undefined) indicator = '…';
        if(str.length <= n) return str.trimEnd();
        n -= indicator.length;
        if(str[n] === ' ') return str.substring(0, n).trimEnd();
        let ret = str.substring(0, n).trimEnd(),
            i = ret.lastIndexOf(' ');
        // when there's no spaces at all, just cut in the middle of the text.
        if (i === -1) return ret.substring(0, n);
        return ret.substring(0, i).trimEnd();
    }
    msg = wordTruncate(msg, msgLength, truncateIndicator);
    return isTruncated ? msg + truncateIndicator : msg;
})($('RSS Feed Trigger').item)

Cool, I have a first test showing the standard behavior of not including any links, so let’s continue backfilling everything and then I’ll add in the rest.

That was only two tests, nothing much to write home about but this covers the behavior I have and it’s enough to characterize it for me, so these are the changes I want to do now:

Output the link as well, on a newline of it’s own.

Implementing the new logic #

So I’ll do this first, and make sure it’s done correctly for both tests, I will start by making both tests fail, because that feels like the best way.

If the URL contains /scrap/ then only output link if truncated.

Done, the code is there, I have some tests, it’s nothing too spectacular, but it does what I want. I realize that I should install standard and run the formatter regularly because I’m being inconsistent and I don’t like it. gofmt and black has ruined me 😁.

Updating n8n programmatically #

Now to figure out how to merge this into my workflow file.

• Choice: How to merge my javascript into my workflow file
  • Constraints:
    • No manual steps
    • Need to return the function to run, and then transform it into a “self-run” (((item) => { item.json.link })($('RSS Feed Trigger').item)) so it’ll work
  • Options:
    • Is there some easy AST stuff I can use with node to extract it?
      • Let’s ask claude

        • is there a function in node to parse the AST of a javascript file and to get the content of a variable as a string?

      • Advantages:
        • Exists, will be robust
          • I got back a code example but I’m hesitant to pull in these dependencies

            • const parser = require('@babel/parser');
              const traverse = require('@babel/traverse').default;
              const generate = require('@babel/generator').default;
              const fs = require('fs');
              const code = fs.readFileSync('functions/processData.js', 'utf8');
              const ast = parser.parse(code, { sourceType: 'module' });
              let functionCode = null;
              traverse(ast, {
                FunctionDeclaration(path) {
                  if (path.node.id.name === 'processData') {
                    functionCode = generate(path.node).code;
                  }
                },
                VariableDeclarator(path) {
                  if (path.node.id.name === 'processData') {
                    functionCode = generate(path.node.init).code;
                  }
                }
              });
              console.log(functionCode);

          • Disadvantages:
            • Huge set of dependencies for a small piece of work
        • Mark it up with text and do a text replacement so let mastodon = disappears
          • Advantages:
            • Simple to implement
          • Disadvantages:
            • Possibly brittle. Will rely on me not changing the wrapping of the function
              • But, I will notice pretty quickly if I get that wrong
        • Something else? Search and see how others solve this
          • Because I tend to get too carried away at just solving the problem and not look around enough, because honestly, I think just adding some text markers is enough for what I need and I’m inclined to just do that, because it’s good enough for my needs.
          • But, let’s start by doing a quick search and also ask Claude to help me search, since it’s usually good at coming up with unique search patterns
          • Prompt to Claude:

            I am working with n8n and want to maintain my workflow as code.
            My workflow uses custom javascript in some of the nodes so I have them stored on my filesystem and I want to extract the function in question and inject and run the function in my node.
            Are there existing solutions for doing this?
            My spontaneous reaction for how to solve it is:
            1. Add markers in text in my file
            2. Extract the content between those markers
            3. Strip the prefix/suffix around the function required for testing the function
            4. Wrap the remaining stuff in what I need to then use it in the workflow and inject it into a workflow template using jq

          • Claude is questioning my approach as brittle, which is fair, it suggests I package it as a node module instead or some templating that injects the source code of a function. If I can just inject the source code of a function then that’d be awesome, then I’ll just make a little node script that outputs the correct thing for me
            • Right, so Claude dreamt up a suggested way of implementing my marker idea. And a custom npm module would mean maintaining the code on the server and injecting it into n8n. It totally makes sense if I’m doing more advanced stuff, but for my current situation, I think having a single file trumps that.
    • Decision: Mark it up with text and do a text replacement so let mastodon = disappears
      • My use-case is so small, this feels like the pragmatic choice until I have more advanced needs
      • I wonder if I should just do all this work in javascript or if I should do what my gut says: a small bash script.
      • Quick overview in my head: upside of using javascript is that I can just load the json into a file really easy, and then output to stdout in a pretty-printed form. The logic to process between the markers isn’t that hard and I already have the testing setup
      • The other option is to do it in Python since the rest of my blog is in python. But since n8n is more js maybe it makes sense to keep this stuff in the one language
      • And if I keep it with JS then I don’t have to deal with jq, I can just do the whole thing in one pass by in JS. I think that’ll be easier.

Designing the extraction #

Okay, so what I’ll do is create a little helper script that.

1. I can point at a js file and it’ll extract out the marker I have specified,.
2. Using the content of the marker, change it slightly so it’ll work in place (make it self-calling)
3. Load the n8n workflow file, replace the key in the right place.
4. Pretty-print the json to STDOUT.

I’ll make the marker format //<EXTRACT:name>, the go convention of programmatic stuff doesn’t start with a space and all :p.

Building the extractor #

I wrote up my first tests, ensured the first “do nothing” test passes, and now with my second test in place I’m gonna ask Claude to implement it, because I don’t know these node APIs. And claude did that quickly and the implementation seems sane. Claude decided to wrap everything so no errors would bubble up, so I removed that catch because any errors should bubble up because I don’t want to miss out.

describe('extract.fromFile', () => {
    test(`if the markers aren't found, then nothing is returned`, () => {
        var actual = extract.fromFile('testdata/no-markers.txt', 'test')
        assert.strictEqual(actual, '', 'expected an empty string when no match found')
    })
    test(`if the markers are found, the content is returned`, () => {
        var actual = extract.fromFile('testdata/plain-text.txt', 'test')
        assert.strictEqual(actual, 'Hello, World!\n', 'expected the content between the markers')
    })
})

I also discovered when done that my tests weren’t passing in intellij because it was running the tests from my repo root, so I changed my Node.js test runner config to default the working directory to src/js/ for this repo, because I am deciding now that I’ll only ever have javascript stuff there, and I’ll only run the tests from that directory. It feels like a decent solution. Maybe I have to revisit that if I ever introduce a package.json and decide whether it belongs here or in the root, but it probably belongs in src/js/, but that’s the future 🤷.

Now, Claude did one thing I didn’t intend yet (because there isn’t any test), which is that it will ignore the whole line the markers are on (which I indicated in the docstring) but I don’t have a test for it yet. I want this behavior so that I don’t have to do any special handling to ignore whatever comment marker there is. So, let’s add a test for this even if the code is already implemented. I guess I could remove that comment and reimplement, but that feels like waste, so I won’t. Time to be pragmatic instead of dogmatic about the testing.

Hah! Adding the test was good, because I skimmed the code too quickly and it hadn’t done what I expected, it was checking the full line. So I actually do have to change the code.

Cool, so I have my extract function.

Building the workflow updater script #

Now I can create a script file and use it. I think I’ll keep the script in src/js/ folder and it’ll take the path to the reference workflow file and read it in and parse, and then I’ll work it from there.

I won’t TDD this file because it should be at the level that I’ll notice problems when using it. We’ll see how much I’ll regret that decision later.

In hope that Claude can do this for me, I started by documenting at the top of my script how I want it to work:

/*
 This script's job is to:

Extract the mastodon and bsky functions from n8n.copies.js.

Read the JSON file in the first argument and then replace the nodes for Mastodon and bsky respectively with their functions.

Output the updated JSON object on stdout, pretty-printed
Example: ./update_n8n_workflow.js ../bjorn-now-feed-distribution.json > workflow.json
Will update the node id `e1bd320b-1222-4324-9168-22372d2e667c` so the parameter.status is `={{ ((item) => {... })($('RSS Feed Trigger').item) }}
*/

Then I asked Claude to implement it:

implement @src/js/update_n8n_workflow.js as described in it's first comment, use @src/js/extract.js to extract the information. Only implement the mastodon portion right now 

My example contained too much information, so claude suggested that it needed to wrap the extracted function as: ={{ ((item) => {extracted code})($('RSS Feed Trigger').item) }} but it should be (...)($('RSS Feed Trigger').item), so I told Claude to plan again I was unclear: the wrapping is "(extracted code)($('RSS Feed Trigger').item)".

Implementation Plan for update_n8n_workflow.js
 1. Import dependencies: Import fromFile from extract.js and Node.js fs module
 2. Read command line arguments: Get the workflow JSON file path from process.argv[2]
 3. Extract mastodon function: Use fromFile('./n8n.copies.js', 'mastodon') to extract the function code
 4. Read workflow JSON: Parse the workflow file
 5. Find and update node: Locate node with id e1bd320b-1222-4324-9168-22372d2e667c and update its parameters.status field to wrap the extracted function as: ={{ ((item) => {extracted code})($('RSS Feed Trigger').item) }}
 6. Output result: Print the updated JSON to stdout with pretty formatting
 The script will be executable with: node update_n8n_workflow.js ../bjorn-now-feed-distribution.json > workflow.json

The plan now looks correct, it says “arrow function” instead of “extracted code”, which I guess is because it added a step to strip let mastodon = from the line, so I’ll assume it does the correct thing, either way, this is close enough that manually fixing it will likely be trivial. allons-y!

Importing the updated workflow #

A quick skim of the code and it looks correct, the output also seems correct to me, so let’s save it to a file, run jq to verify it’s valid json (I assume it is since node didn’t bork out), and then upload and see what goes. My saved file locally is what’s live now, so if it goes wrong, I can back out easily.

Right, interesting problem: n8n supports importing from a file and then it adds the file content alongside what you already have. So now I have a copy of all my existing nodes and with a 1 at the end. The content of my new Mastodon status node DOES look correct, though. So that’s a win Not quite what I had in mind, so I feel like I’m doing something wrong. Let’s start by asking Claude about this workflow and see if they can push me towards some online docs. or, n8n has these boxes “I wish this node would…” I can just ask there and see what goes.

Seems like this is working as intended (d’uh) and the way to handle this is doing it by API. And… fair enough, can. so I asked Claude to make me bash script that I could use. Then it started complaining that I didn’t have a name in my file, which I do, so there seems to be a mismatch between the downloaded workflow and what the API expects. D’oh, I had missed a line in cURL -d @- to read the file, so that’s on me.

Then I had issues because apparently my downloaded workflow doesn’t just work with the API, so Claude and I did some back and forth and finally ended up with this that limits the fields I want to update:

cat $1 | jq '{
           name,
           nodes,
           connections,
           settings
         }' | \
  curl -X PUT "${N8N_HOST}/api/v1/workflows/${workflow_id}" \
    -H "X-N8N-API-KEY: ${SANITARIUM_N8N_API_KEY}" \
    -H "Content-Type: application/json" \
    -d @-

Weirdly, I had to update the settings so it only contained the keys executionOrder and errorWorkflow as the other two values there weren’t okay. Whatever, they didn’t seem important so 🤷.

Validating the updated workflow #

Now I have uploaded my changed thing, and it’s time to verify it works as I want. I’m not entirely sure what I should post to verify it :p.

I was looking at the result from running the update, and unless I always save the result of the update I might rerun from a prevous point when updating. Or, no, because it’s a top-level field that I’m ignoring when updating. So it should be fine. But, I think for now, I’ll anyway pipe the update stuff into the saved file so it’s there and see what changes between runs.

Committed, and tested by posting a scrap about “wow… I really spent 5h doing this? That’s longer than I expected.” But, I guess the thing I have to keep in mind that this is the difference between “engineering” vs. “just throwing stuff together.” If I hadn’t invested in the infrastructure stuff for managing this, I would’ve been done after 30-40m when I updated the code. The point was the investment in robustness I think I’ve gained.

Repeat for Bluesky #

Handle the no-link with Bsky too.

Designing the change #

Now, with all that done, time to look into Bsky, there I don’t append the links but instead attach a “website card.” There, I guess I’d have to change it so:

1. The body gets the same logic/setup.
2. The logic is repeated in the “Website Card” field, assuming that having an empty value there will do nothing (which I assume it will, Javascript being very used to weird cases of keys not existing, but I should test (and executed with an empty body there, it just worked), so just repeat the same logic in that place)

this is a bit more annoying, will have to do some copy paste, but I can at least handle the consistency by creating a reusable test.

But I’ll work on this either tomorrow or later today, it’s after 16, and I feel like taking a break. And heading home feels like a good way to clear the head and then continue.

Implementing reusable tests #

20:03 And it’s after dinner, let’s see if we can get this in so we can push it through.

What I think I’ll do is just do a quick duplication of what I have, don’t be clever, just do the dumbest thing possible. With one caveat: I know the tests are mostly the same, so let’s make the tests reusable and configurable and see if that pattern works in node (I bet it does). Meaning, I want my existing tests for Mastodon to be shared with Bsky but change the limit to 300 char instead of 500.

The way I’ll test that is that I’ll make a function that returns a closure over my config, then I’ll run it by passing in my code to execute and then see if the assertions match. I’m not sure how it’ll go with the test runner, but let’s see. I.e. something like:

function reusableTest(name, maxLength, urlLength) {
  return function(toTest) {
    return () => {
      test(`it does stuff`, () => {
        let actual = toTest(arg)
        assert.strictEqual(actual, "whatever I want")
      })
    }
  }
}
describe('mastodon', reusableTest('mastodon', 500, 23))

And another part is, I don’t know how Bsky does links. There’s something about facets and how you can use them to reduce links but a big 🤷 in understanding that. I don’t know if I’ll care to look this up or not.

I could potentially do some smarts and just attach the URL as a card and remove if it’s a single link in the post, which I could make as standard for myself.

Anyway, let’s get the tests working now and see what goes first.

Okay, that kinda just worked as expected, a little surprised at how smooth it was. The commit.

Is this really reusable? #

I am getting a bit worried that the setup might be… too fiddly, but I’ll feel it out a bit more as I add in bsky. This feels like it’s hitting the point where code looks the same but it isn’t really reusable, because it looks similar but differs in intent so as we diverge it’ll just add more if-statements. But, we’ll continue going and feel it properly.

the first case: there isn’t any clear “a link is always 3 chars” so it’ll be counted as whatever the raw text is, so I’ll use -1 to indicate don’t do anything (an if, urgh). Spontaneous thought: I could pass in a function to handle the link calculation instead and just assume it sends me the link, so that would sort that problem out nicer and it keeps the test consistent. My favorite the strategy pattern showing up and making my day cleaner, again.

I’ll just do that, it’s much better than a magic number, and I can create that magic number as a variable from the function call. I know I have to implement this now, so it’s the right time to “Tidy First” á Kent Beck. 🙂.

And there we hit the problem it: it’s all very similar for good reasons, BUT the logic has to be different because the default way where I add the link at the end for Mastodon doesn’t apply for bsky, there I can just attach it as a card. So all the URL counting stuff don’t matter. So, I’m better off copying the tests and changing them slightly, and calling out that they’re similar but importantly different, any optimization here is folly.

I only spent ~15m on this, so I don’t feel too bad about it. In some other cases it would’ve worked out, and it does feel good to keep coding instead of just thinking a whole lot before doing anything.

So, git reset --hard on those files and a revert on the commit. Now let’s start by copying and changing.

And, to get what I need later to decide whether to do the card I’ll just look for msg.endsWith('… more') and if so, attach the link if it’s a /scrap/ link. So that’s pretty straightforward, it does mean I have to include the info, but that’s okay.

Refactoring after working tests #

And done, but looking at how I’m deciding whether something is truncated, I don’t get what I’m doing. Why don’t I just look at the string and check if it changed? That seems way more straightforward than comparing two fields from the RSS….

So I’m just gonna do that and see what happens, but first, commit my changes, then change the format of how I work.

That was quick, maybe 5m to make the change. I looked at the workflow file and the difference between contentSnippet and encoded:contentSnippet seems to have to do with newlines and other formatting, so that’s not gonna be very useful. Seems like a major brainfart that accidentally ended up working 🤷 (likely that those that didn’t need truncating were on a single line)

Okay, next up, now to make sure that update_n8n_workflow.js handles bsky as well and updates it.

Went through and updated it, I mostly kept the very procedural code and just added on, maybe I’ll clean it up a bit more if I loop back and add Telegram (for Telegram I’m thinking I’ll just do the preview in app button intead? but maybe it makes sense to put the scraps in there too, it’s no different from the others and there’s less links :))

Testing bsky live #

And I made a post that would fit 300 chars about helping my brother install some RAM, where the point behind is that having done it before gives confidence that instructions just don’t give. Whether installing RAM or relying on your tests 😜. And it gave me a chance to see that the logic works.

What does the n8n update endpoint do and how does it work? Becuase it feels like not understanding how it’s designed might come and bite me later, so let’s not just rely on the vibe coded stuff that seems to be working.

Wednesday EOD #

21:39 And done, I’ve finished bsky. Tomorrow I’ll be hanging with the niece all day so I’ll format and post this devlog when I’m in the office again 🙂 Okay, scrap that (heh), I’ll finish Telegram before posting. Then for formatting to post, I’ll do it by creating a script alongside Claude to figure out the formatting. I’ll have to brainstorm how then.

Update Telegram logic #

Add the scrap logic to Telegram too (if it’s a scrap, only post the text, otherwise the link)

Configuring Instant Views #

Time to get Telegram to be a bit nicer. I overall don’t want to force people to click through to my site if they’re happy reading where they are. Telegram has these Instant View buttons and I can translate my links to use those instead, so I figure I’ll do that in my channel so you can just click and see the full thing without loading my site. I guess this will mess with my statistics a bit (because I use Plausible which is doing it all in javascript), but 🤷, for now I’m just happy if people wish to to engage with anything I post.

I did create a preview a couple of months back so I started by taking a link to my site and then copying in what I did last time:

~version: "2.1"
site_name: //*[@class="masthead"]/a
article: //article
title: $article//*[@class="title"]
subtitle: $article//*[@class="subtitle"]
author: $article//*[@class="author"]
image_url: //meta[@property="og:image"]
published_date: $article//*[@class="published-at"]
body: $article//*[@class="content"]

That looks pretty good, but I’m not seeing the published time in the live preview, the xpath expression looks right to me. And I similarly don’t see the image_url, and looking in the template website I don’t see that the <head> section is part of the example document (From the future: Nope, that’s wrong, I was looking at the containing doc and not the iframe, it’s all there. Telegram will always do a GET for your page to get the current version and then format the page using these rules). So maybe that’s why? I’ll skim through the documentation and see if I can find why.

I see that the published_date says “unix time” in the docs, so maybe that’s the problem? I gave it the element and it’s in readable, but I do have the RFC3339 format string in the <time> tags datetime attribute. Let’s keep digging.

Right, so the docs shows that there’s a function @datetime that can take an xpath expression and return a unix timestamp, so I likely will have to give it the attribute and then specify the format so it can be converted.

So with that, the pattern should be something like this, which doesn’t work, and I think I’m trying to access the property incorrectly in xpath.

@datetime(-2, "en-SV", "yyyy-MM-ddTHH:mm:ssZ"): $article//*[@class="published-at"]['datetime']
published_date: $@

I asked Claude how to access it in xpath:

I want to access the property `datetime` on the element matched by this xpath expression (for Telegram Instant View), how?
`*[@class="published-at"]

Which gave me the answer that I need to access it by saying /@datetime, which to me reads that I “dig into” the element and @ seems to have to do with attributes/properties of a tag. When I replaced ['datetime'] with /@datetime and saved my match then Telegram showed my publish date!

TIL: Use /@<name> to access the value of an attribute when you’ve selected an element with XPath. With the expression //*[@class="published-at"] you will get the HTML tag that has class="published-at", and if you then want the datetime property you add /@datetime to get it.

And now that I have the published_at working I’ll see if I can figure out how to access my preview image which is available in <meta property="og:image" content="<url>">, and I bet the problem is that I’m not going into /@content the way I need to. Changed and that worked!

There’s this Instant View property called kicker which I don’t know what it means, so I asked Claude and it said:

kicker is the small text that appears above the main headline in article layouts - typically used for:

• Category labels (“Politics”, “Technology”)
• Section names (“Opinion”, “Analysis”)
• Eyebrow text (“Breaking News”, “Exclusive”)

I guess this is some publishing term and it sounds like my “content types” would be fitting here. I.e. post, crumb, link, scrap, etc. which I actually don’t show at the moment on my single pages. Maybe I should add that in there? Or should I just skip it? It’s part of the menu. Although, I have added it into the article’s class attribute what category it is, because I have been planning to do custom styling of scraps (pre-emptive code, I know bad, but turns out this bit of unnecessary future planning might be helpful now.) So, question is, can I strip away the post and single values from that string only leaving the unique piece?

In the list of functions there’s a @match function that takes a regex, so that could work. Let’s see if there’s anything else before I go into making a regex (which admittedly can be pretty simple given how it’s setup right now)

It seems that’s the only function I have. So right now, I’ll do it super simple and since my class attribute looks like this: class="post devlog single" I’ll just do post ([^ ]+) and then extract the second, the single also isn’t needed, but since I’m doing this on the article element I don’t feel the need to do more, I’ll just have the rule that the first class is always the post and this will work. The @match function’s second argument is which capturing group you want to use. While debugging it was a bit non-obvious to me that the debugger’s output included class="devlog" but it seems that it shows the full context, but when you add it to the kicker as $@ (which is the return value of the last function call) then it’ll just do the string and not where it was from. 🤷.

@match("post ([^ ]+)", 1): $article/@class 
kicker: $@

Next up, author_url which is a property on the <a> tag which contains my name, so let’s dig into it: author_url: $article//*[@class="author"]/@href.

Using Instant View links or not? #

And with that, I have the Instant View working as I want in the preview. Now to figure out how I use it when posting so others can see it. The documentation for it says:

Note that if you send the resulting t.me/iv?url=...&rhash=... link to other Telegram users, they will see an Instant View page built using your template (the rhash parameter in the link determines which template is used to create the page).

So all I have to do is wrap my URL to this and it’ll work. The downside of that is that then if you click that URL it’ll lead to Telegram instead of my site, and it’s not obvious to me how to click through to my site. I added the document_url but it seems it doesn’t show the real URL. I don’t like this. MAYBE what I could do is add a second link that allows the instant preview, as a way of making it simpler for Telegram users. But iunno, it also feels clunky. I could try to submit my template and see if it gets approved and then it just works. I asked Claude and it says the same thing, it’s not possible, and if you want that then you need to submit it. So that’s the only real option, I don’t want to completely nerf the readability for Instant View while I also want to make it simpler for others to read it where they are. The same reason I have the full content in the RSS feed. :) So I added 10 links that should work, looked that they seemed correct, and added them as checked. Then I added the homepage and blog post listing pages and saw that they don’t have any Instant View, which is what is supposed to happen.

Final Telegram logic and decision #

Which leaves my Telegram posting logic as:

1. If the URL of the item contains /scrap/ then post the content in full (I’ll deal with images later, but for now scraps are text-only with no formatting)

2. Otherwise, post only the link so it can be clicked through and if the Instant View is approved then that’s available.

And that took about 1.5h to do. Next up, to research how the update endpoint works for n8n, and then to do the formatting of this post and post.

Understand how n8n updates work #

I left a note for myself earlier that I wanted to check how n8n handles the updates, so I can feel more comfortable in not overriding stuff.

Digging into the source code #

I figure it’ll be a bit annoying to find this, so I’ll just ask Claude to find it for me:

where is the n8n source code for the `PUT` logic for workflow updates?

And it did some searches and told me path in the repo is packages/cli/src/public-api/v1/handlers/workflows/workflows.handler.ts so I went to GitHub and opened.

It does special handling for settings and will only change settings I have set and otherwise let existing ones be.

Okay, working my way to the WorkflowRepository it doesn’t implement update itself, so it’s probably in the base class, so let’s see what’s there. The base class comes from @n8n/typeorm and I don’t know where that is. I looked in the @n8n folder on GitHub but didn’t see it. I tried pressing t and typing it but it doesn’t seem to be part of the path. The search showed that there’s a CLAUDE.md so maybe it contains something useful? (I’ll dig a bit first before asking Claude to find it for me). And there I find this: **Backend:** Node.js + TypeScript + Express + TypeORM which seems to indicate that TypeORM might be a general package? I searched for TypeORM and saw a linter step to ensure we only use it in @n8n/db folder, so is there a definition of @n8n/typeorm in the db/ folder? The package.json lists the dependency as "@n8n/typeorm": "catalog:",, so maybe there’s something in this catalog folder (or maybe package?) I had a gut feel since I saw pnpm mentioned and something around monorepos, maybe this is a special pnpm thing so I did a search for “pnpm catalog” and found explains it’s a “workspace feature for defining dependency version ranges as reusable constants.” So if I get that right, it’s how you ensure the same version of TypeORM is used across this entire monorepo. Sounds neat. So, that means I’ll just assume we’re doing the bog standard TypeORM stuff so I’ll look at that. But I will assume it’ll simply do: replace the named keys if they’re part of what comes in, because a generic package usually doesn’t do more than that.

TypeORM’s documentation for update says: update - Updates entities by entity id, ids or given conditions. Sets fields from supplied partial entity. And I read that to mean: we’ll replace all keys we see, and the others will not be touched. So I won’t accidentally unset any fields I don’t change. SO, maybe then I should try and limit what I send even further since I only really want to update the nodes portion, so could I get away with only sending that? And the answer is no, because then it gives an error that I need to provide connections.

Adding “only update if version is what I expect on the remote” #

But, it’s fine, I’m anyway saving the version in the repo after each update, so I would only loose things if the version live is different from what I have. Which for now, I don’t worry about. I COULD do a check to see that the current version is the same as what I have downloaded or warn. I guess that would be pretty tiny effort, and therefore worth it since all you’d have to do to sort it out is run get-workflow and then recreate the update and check it. So… letting me know there’s a mismatch WOULD be useful.

The way to do that then is: jq -r '.versionId' bjorn.now-feed-distribution.json gives me the version id, so then I would have to pull down the current version of the workflow and check the version number is the same: ./get-workflow | jq -r '.versionId' and then compare it, so… screw it, let’s put that in the update script.

# Only update the workflow if the `.versionId` ISN'T the same as the server,
# because we're downloading the workflow and then applying our updates to it.
# This intends to avoid overwriting anything we didn't expect.
localVersion=$(jq -r '.versionId' "$1")
serverVersion=$(workflow_id=$workflow_id ./get-workflow 2>/dev/null | jq -r '.versionId')
if [ "$localVersion" != "$serverVersion" ]; then
  echo "The local workflow file ($1) is stale to the server version" >&2
  echo "  local version : ${localVersion}" >&2
  echo "  server version: ${serverVersion}" >&2
  echo >&2
  echo "Hint: Pull down the latest version as and recreate:" >&2
  echo "  ./get-workflow > bjorn.now-feed-distribution.json" >&2
  echo "  ./create-update" >&2
  exit 2
fi

Added a hint to myself in case I forget.

Final thoughts #

This entire endeavour got a bit more robust than what I’ve had before with my n8n setup, but that’s because I’m planning to add more functionality to my POSSE setup. So I wanted to make sure I don’t create more of a mess for myself as I change things, and create more things to check.

This isn’t 100% the production level I would do but it’s good enough for where I am. The “create the update” script is pretty rough, but I also expect that I will very rarely change it, and it’s built around a couple of assumptions that I don’t see changing. So keeping it hacky, or more as disposable, feels okay to me. If I had more people working on this I might not be, but for now, there’s enough guard rails.

I’m happy I decided to add the optimistic check that we’re updating against the version on the server, it’s the kind of thing that was really straightforward and simple to add and it’ll rarely fail, but it’ll warn in case I end up doing click-ops and forget to sync back to the repo.

But, I discovered that despite being within the character limit, one of my URLs were truncated when posting to Mastodon. This is because Mastodon does something non-obvious: it treats all URLs as 23 characters, no matter if they’re shorter or longer.

This is because they copied this behavior from Twitter, which ran all links through their t.co link shortener. The reason is that Mastodon doesn’t want you to use a link shortener, since it hides the real URL (i.e. makes rick rolling easier 😜).

So the module truncated the text in the status. Since it wasn’t URL-aware, all I got was a broken link.

I decided to fix it myself, because dammit, I’m a programmer. It’s an unfamiliar codebase I know nothing about, so I rolled up my sleeves and asked Claude, “Pretty please, can you fix the problem and add tests?” (ah the modern world we live in)

The key constraint: I needed to understand every line of code before shipping it, and it had to pass my quality standards because my name would be on the PR. Happily, that worked out becuse my PR was accepted. 🙂

Here’s how I worked through it over three days. All commits are here if you want to step through them. I really intend to share these devlogs earlier in the future, but I got lazy 😅

The rough notes #

Lightly edited for the web, and I’ve kept the day breaks in.

2025-10-08 #

I discovered that my mastodon poster truncated the URL (which seems to indicate that the n8n plugin does a naïve counting of characters, whereas it should only count 23 chars/URL, because Mastodon don’t penalize longer URLs so we don’t need link shorteners.

So I looked at the source for the package in question and found the 500 limit (I had looked at this before) and knew it was hardcoded, and then I decided to download the repo and ask Claude to find the bug and suggest a fix:

this is a plugin for n8n, and @nodes/Mastodon/status/StatusMethods.ts seems to have a bug where if I post a status with a link then it will truncate the status assuming the link takes up the full
length it has, instead of only 23 characters as is defined by Mastodon. Identify where this bug comes from, explain it, and come up with a plan for how to fix it. Also suggest tests to add that
fits with existing tests in this repostiory.

It then came up with a plan that I rejected because I wondered if there any existing packages that implement this logic, since the Javascript community is quite keen on tiny packages

Fix Plan

1. Create URL-aware character counting utilit

- Add new method ValidationUtils.calculateMastodonLength() that:
  - Detects URLs in the text using regex
  - Counts each URL as 23 characters (Mastodon's characters_reserved_per_url)
  - Counts remaining text normally
  - Returns the effective character count

2. Update validateStatusParam function

- Replace simple sanitizeStringParam with URL-aware validation
- Use the new calculateMastodonLength() to check if status exceeds 500 chars
- Only trim/reject if the effective length (with URL shortening) exceeds limit
- Keep the full URL text intact (don't truncate it)

3. Add comprehensive tests

Following the existing test pattern in __tests__/status.test.ts, add:
- Test for status with single URL that would exceed 500 chars without URL counting
- Test for status with multiple URLs
- Test for status with URL at the beginning, middle, and end
- Test that status correctly rejects when effective length > 500
- Test that long URLs are preserved in the request body
- Test edge cases: URL-like text that isn't a URL, localhost URLs, etc.

4. Optional enhancement

- Consider fetching instance configuration (characters_reserved_per_url) dynamically
- For now, hardcode 23 as it's the Mastodon standard

So I asked Claude to look for a package:

search and see if there's an existing npm package that does this calculation of post length per mastodon so we don't have to write it ourselves

It searched and didn’t find anything (it was a bit slow, so I decided to look as well, and didn’t find anything obvious), when I looked back after giving up it asked to download the original Ruby source for the status length validation and then came up with new search patterns

It then found twitter-text and suggested to import it and use

Fix Plan for Mastodon URL Character Counting Bug

1. Add twitter-text dependency

- Run npm install twitter-text (version 3.1.0, stable and well-maintained)
- This library provides the same URL detection/counting logic that Mastodon uses

2. Update ValidationUtils in Mastodon_Methods.ts

- Add new method ValidationUtils.countMastodonLength(text: string): number
  - Use twitter-text's parseTweet() or extract URLs manually
  - Count each URL as 23 characters (Mastodon's standard)
  - Return effective character count
- Keep existing sanitizeStringParam for other use cases

3. Update validateStatusParam in StatusMethods.ts

- Replace line 16: Instead of truncating, validate the effective length
- If effective length > 500, throw error with helpful message
- Don't truncate the actual status text (preserve full URLs)

4. Add comprehensive tests to tests/status.test.ts

Following existing test patterns, add:
- Test: Status with long URL (>100 chars) that fits within 500 char limit
- Test: Status with multiple URLs counted correctly
- Test: Status that exceeds 500 chars even with URL shortening
- Test: URL preservation (ensure full URL is sent to API, not truncated)
- Test: Edge cases (URL at start/middle/end, multiple URLs)

This approach uses a proven library rather than reinventing URL detection regex.

I wasn’t sure about all this because looking at the repo it looks like a big dependency, and maybe it has too much extra stuff that doesn’t really come into play for Mastodon?

will `parseTweet()` introduce any other logic that might not work with mastodon? Might we be better off copying the code around links (and tests) and adding it to this repo (do the licenses match?)
We *cannot* break the existing behavior and throw an error instead of truncating, but we can change the truncating behavior to the effective length instead of actual length. 

It then went off and did a bunch of searches around east-asian specific character encodings (Japanese and Chinese alphabets as far as I can tell)

I then got a new plan that looks better, but I still wonder about the when possible for preserving URLs (I expect it is that if there’s not 23 chars left, then it has to cut, so I will have it clarified in a more detailed plan)

Updated Fix Plan (Mastodon-Compatible)

1. Copy URL regex from twitter-text (with attribution)

- Add file header comment crediting twitter-text (Apache 2.0 license)
- Copy only the URL detection regex pattern (battle-tested, handles edge cases)
- No dependencies needed - just the regex

2. Create ValidationUtils.calculateMastodonLength() in Mastodon_Methods.ts

- Extract all URLs using the copied regex
- Replace each URL with 23-char placeholder ("x".repeat(23))
- Count remaining characters using standard .length
- Return the effective character count
- No CJK weighting - Mastodon treats all chars equally

3. Update validateStatusParam() in StatusMethods.ts

- Calculate effective length using new method
- If effective length > 500, truncate at boundary that fits within 500 effective chars
- Preserve full URLs when possible (don't cut URLs)
- Keep existing truncation behavior (no breaking changes)

4. Add tests to tests/status.test.ts

- Status with long URL (>100 chars) that fits in 500 effective chars
- Status with multiple URLs
- Status requiring truncation with URL preservation
- CJK characters counted same as Latin (no weighting)
- Edge cases: URL at boundaries, multiple URLs

I asked what when possible means and guessed some cases (I forgot to save the prompt)

⏺ Cases Where We Cannot Preserve URLs:

1. Theoretical impossible case (highly unlikely in practice):

Text: "Check this out: https://example.com/article"
Actual length: 44 chars
Effective length (URL=23): 39 chars ✅ Fits!

But if someone tried:
Text: 480 chars of text + " https://example.com/very/long/url"
Actual: 520 chars
Effective: 480 + 23 = 503 chars ❌ Over limit!

To fit within 500 effective chars, we'd need to:
- Remove 3 chars from somewhere
- We can't split the URL (would break it)
- We'd have to remove text BEFORE the URL

2. Multiple URLs scenario:

Text: "See https://example1.com and https://example2.com and more text..."
If effective length > 500, we might need to:
- Keep URL1 (23 chars)
- Keep URL2 (23 chars)
- Truncate the "and more text..." part
- OR drop URL2 entirely if there's not enough room

3. The actual problematic case:

Text: 450 chars + "Visit https://example.com/some/long/url for more info and keep reading..."
Effective: 450 + 23 + 35 = 508 chars ❌

To fix:
- Keep first 450 chars ✅
- Keep URL (counts as 23) ✅
- Truncate "for more info and keep reading" to fit in remaining 27 effective chars

The Real Answer:

We can ALWAYS preserve URLs! Here's why:

- URLs only count as 23 chars in the effective length
- If someone has room for 23 effective chars, they have room for the full URL (even if it's 200 chars long)
- We just need to truncate the non-URL text to make room

The algorithm:
1. Find all URLs and their positions
2. Calculate effective length
3. If over 500:
  - Truncate text BETWEEN or AFTER URLs
  - Never truncate the URL itself
  - Work backwards to find a safe truncation point

Edge case that might force URL removal:
Text with 22 URLs: Each counts as 23 = 506 effective chars
Even with ZERO other text, we're over the limit!

In this case, we'd have to drop some URLs entirely (remove the last URL(s) until we fit).

Should I proceed with a plan that preserves URLs when possible, but drops trailing URLs if necessary when there are too many?

So, now to handle this I will ask it to write this high-level plan into high-level.plan.md and then to create a detailed plan that could be implemented by a junior engineer, so we then can ask sub agents to implement them step-by-step. I saw the plan and it suggested to drop URLs that went over the limit, so I asked it to instead keep them.

I haven’t added any general/background context to this repo about my preferences, so I wonder if this will be enough to give me something useful in this repo:

create a deatiled implementation plan that a junior engineer could implement.

Ensure each step is written in a TDD fashion and that the tests are passing, changed to show the expected new behavior (red),
and then the change is implemented to make it pass (green).

Only implement as much as is needed to get each test passing and no more. 
Cover the copied code from Twitter with tests too.

It came up with a detailed plan, it looked good enough (skimmed it), so I asked claude to:

execute the test suite to verify it runs, if it runs successfully then document how to run the tests in the plan and then save the plan as plan.md

Then it couldn’t run any commands, likely because I never ran npm install in the folder so I did that and verified that npm test worked and then asked Claude to try again.

Next up, execute the plan: Now plan to execute @plan.md with subagents and marking off which steps has been done in the plan as they finish..

Strategy

Use general-purpose agents in parallel where possible to implement the plan steps efficiently. Track progress by updating plan.md with checkmarks as each step completes.

Phase 1: URL Extraction (Steps 1.1-1.6)

- Agent 1: Implement URL extraction utility with all tests
 - Create __tests__/url-extraction.test.ts
 - Add extractUrls() to ValidationUtils in Mastodon_Methods.ts
 - Iterate through all test cases in steps 1.1-1.6
 - Mark steps 1.1-1.6 complete in plan.md

Phase 2: Length Calculation (Steps 2.1-2.5)

- Agent 2: Implement Mastodon length calculation with all tests
 - Create __tests__/mastodon-length.test.ts
 - Add calculateMastodonLength() to ValidationUtils
 - Iterate through all test cases in steps 2.1-2.5
 - Mark steps 2.1-2.5 complete in plan.md

Phase 3: URL-Aware Truncation (Steps 3.1-3.9)

- Agent 3: Implement truncation with URL preservation
 - Create __tests__/url-truncation.test.ts
 - Add truncateWithUrlPreservation() to ValidationUtils
 - Iterate through all test cases in steps 3.1-3.9
 - Mark steps 3.1-3.9 complete in plan.md

Phase 4: Status Validation Update (Steps 4.1-4.4)

- Agent 4: Update status validation to use new utilities
 - Add tests to __tests__/status.test.ts
 - Update validateStatusParam in StatusMethods.ts
 - Verify all new tests pass
 - Mark steps 4.1-4.4 complete in plan.md

Phase 5: Final Verification (Step 5)

- Run npm test to verify all tests pass
- Update plan.md with final completion status
- Report summary of what was implemented

I then set Claude to execute and left some questions for myself to answer after we were done:

• Are there more places we should use this new validation/truncation logic?
• Is our new code implemented similarly to the rest of the code in this repository?
• What questions should I ask you to verify that we don’t break something?

00:34 It’s late, so time to let Claude spin away while I prepare for sleep.

00:50 Okay, while prepping for sleep it finished and I had a look on the new tests the tests have some problems that we’ll need to iterate on:

• Doesn’t assert the exact values we’re looking for
• Doesn’t make the 500 or 23 constants and are instead magic numbers, let’s name them
• The truncation logic is too smart and it should just cut the URL and leave an ugly “scar” if that’s what’s required, that’s fine. We can iterate on that

I think the way to work with this is to create a fork and do my changes in a repo with small focused commits, and then show the work as I commit, so I indicate how I grew the solution

Now, sleep for reals

2025-10-10 #

Okay, back at it again after a day’s break, got some tunes on and now to review.

Okay, to help me document how I’m working on this I will commit the change I got from Claude, and then I’ll do my changes from there and commit as I do them. Doing this so that it’ll be easier to blog about it, and I don’t know if that workflow will be what I want/need, so we’ll see.

Reviewing #

Starting the review by looking at the status.test.ts since I’m supposed to have good tests that tell me what’s going on, so that’ll be the lense through which I review this.

First off, I got header comments explaining the change that I don’t need, so nixing ’em, and I think all these tests belong in a describe that shows that the tests belong together. Commit

Then, let’s get going with the first test “should preserve long URL when using URL-aware character counting” doesn’t do one that will hit the problem, and the test doesn’t check the count is what we expect it to be (which is also a problem). okay, changed the test to only contain a URL and then verify that the expected status is exactly 23 characters long, which passes. Commit

Next is a test with multiple long URLs, which is useful, make sure we don’t do this for only one URL in the status. The URLs are not pushing it as far as it should, so I made both URLs more than the total length of a status (80 -> 500)

const url1 = 'https://example.com/' + 'a'.repeat(80);
const url2 = 'https://www.example.org/' + 'b'.repeat(80);

The text is a bit annoying so simplifying it for my own sake to make the length calculation simpler, it’s a list of links one per line (2*2+1 = 5 + 23 + 23 = 51):

const text = `- ${url1}\n- ${url2}`;

Commit.

And about now I’m annoyed at the way the mock implementation has so much setup, it doesn’t follow my coding style (too much repetition), but this isn’t my project, and it seems to be how all other tests are written in this file so we keep it. Consistency and avoiding surprises is more important than my personal style.

Now comes the really nasty bit: Claude decided to truncate text instead of URLs which isn’t what I want, if the URL is placed so that it’s unavoidable that it will be cut, then cut it. Don’t remove it, and don’t remove text before it. It’s better to indicate that something went wrong (by showing a cut URL) than silently dropping it (I’d rather go boom than be silent, been saying this for a very looong time, mostly living that philosophy :p)

So I think this particular test will have to change. And about now I’m regretting that I didn’t commit already because I feel I should’ve done it per test instead. Because now I’m about to make the test red, so I want to make sure I can change it cleanly. So, I’ll undo my changes (thanks IntelliJ for having infinite undo in the same session), commit, and then redo per function. That took ~5min to fix, and while not really necessary, I feel a lot better from going back and redoing and creating clean commits to show the work (so the commits linked was made before this).

Okay, now, this test I’ll also change up a bit:

• Loong text, ax480, a space, and a url which means the effective length will be 505 so the URL will be cut
• I changed the assertion on the sentStatus to be the naively cut status instead (since the URL can’t ever fit now, so it will just have to mangle the URL)
• And as expected, the test is now failing, so that’s good. Now, let’s ask Claude to fix the failing test I’ve created
• I highlighted the test in IntelliJ and opened up Claude and said (mostly because I haven’t looked at the code, this might actually be faster to do manually, so I’ll look at it as the fresh Claude session is figuring out what’s going on):

  • The highlighted test is failing correctly, I want to adjust the business logic to match. Confirm the test is failing by running `npm test` and then plan how to fix it 

Found the place where this is, it’s in nodes/Mastodon/Mastodon_Methods.ts and the new truncateWithUrlPreservation.

And I see that Claude is spinning and reading what it’s doing I realize I made a logical error: if Mastodon always counts URLs as 23 chars, I can’t truncate a URL and make it work. The logic Claude made was reasonable since I told it to never raise an exception. So that means I need to make this case an explicit negative case, that it will fail the validation, and I could exemplify that better by making it a really short but valid URL (ex https://example.com which is 19 chars)

Okay, asked Claude to get to it, but it’s struggling with the logic and I’m realizing that I should just add another test alongside this where we have a long URL at the start and a really long text at the end, that we can just truncate as needed, so it can see the logic better.

BUT, for right now, I’m gonna start a stew so I’ll go do that and then let it… stew for a coupe of hours so I can continue then

20:15 back and doing stuff again

I added another test that swaps the order so it’s <url> <text> and now it’s failing too, so let’s clear the session for Claude and start over

these two tests are failing because the business logic is incorrect, suggest how we can fix the business logic

Claude came back with a plan to let the server fail instead of us truncating long tests, which won’t make the tests pass, so I prompted:

before our changes this library truncated any text >500 chars, so we need to retain that logic. BUT if we end with a URL let it through and let the server

Claude then suggested a flow that seems to work:

New logic:
1. First check actual length (not effective length)
2. If text.length <= 500 → return text as-is (no truncation)
3. If text.length > 500:
  - Check if characters after position 500 are part of a URL
  - If yes (would break a URL) → return full text unchanged
  - If no (normal text overflow) → truncate at position 500

This preserves backward compatibility:
- Original behavior: truncate at 500 actual characters
- New behavior: same, BUT skip truncation if it would break a URL

And Claude implemented the fix and did it TDD style and highlighted that we have more code than necessary to pass, so I need to write another test case that exercises all the extra code we have in truncateWithUrlPreservation

2025-10-11 #

14:43 The next day, I ended up watching TV yesterday, and today I’m feeling sluggish

But I was reflecting about this yesterday and I’m a bit unsure about how to write a test that refuses the current “fix” of " if the actual is less than 500 let through" because it’s true since the edge-cases it won’t do it for is basically where the URL is wrong, BUT, the URL could be wrong at the start and the text after is okay to be truncated. I.e. cases where say https://example.com (19) x 21 = = 399 actual but the effective length is 483 and if I add 18 characters of text it’ll fail and could be truncated.

Maybe that’s the case to do, because it’ll force it to handle it correctly.

I ended up changing the “this can be truncated” test so the URL is shorter and the text is exactly so it should pass, and then pointed out that we need to hint that we need the special logic if there are URLs in the text.

run the tests and suggest how to fix the them while not changing any tests, the underlying business logic is incorrect and too complex. Hint: consider only doing the special handling logic if there is a URL in the status text

Claude came back with a plan:

New simplified logic:

1. Extract URLs first to determine which path to take
2. If NO URLs in text:
  - Simple check: text.length <= 500 → return as-is
  - Otherwise → truncate at 500 actual characters
3. If HAS URLs in text:
  - Calculate effective length
  - If effective ≤ 500 → return as-is
  - If effective > 500:
      - Calculate how many actual characters to remove: excess = effective - 500
    - Target length: targetLength = text.length - excess
    - Check if truncating at targetLength would break a URL (cut it in half)
    - If YES (would break URL) → return full text unchanged (let Mastodon fail)
    - If NO (safe to truncate) → return text.slice(0, targetLength)

And that update gave me the logic I wanted, so now let’s see what the change looks like and it’s good.

As I now look at it and am preparing the commit message I realize, this doesn’t fit the original design of this library/n8n node: it was blindly truncating, so I could also just truncate just before this URL so we can post. It doesn’t fit with how I discovered this problem, which was a truncated URL (which was still longer than 23 characters but wrong).

So, with that, I will instead say: nix the URL because it goes with the grain of the library and people like me have to ensure they truncate correctly. It’s the least surprising thing we can do. But, I have done all this work and despite realizing that I went down the wrong path, I should commit what I got so I can see the changes. I learned something and that’s still a win.

Okay, done , and I asked Claude to look at the comments I’ve tried to add to clarify the logic and it suggested some improvements to them which I’ve folded into the commit.

review the changes we've done, are the explanations in text clear enough that those reading the code would be able to understand?

Committed.

I’m not sure if using Claude here has saved me time, but I have been doing it in low energy and while watching TV, so upside is that it got done compared to that I likely wouldn’t have put in the full energy otherwise.

Although, partly that’s because when fixing the bad stuff I didn’t want to go in and learn how all the code worked, becasue I’m relying on the tests to tell me what’s working and not, so that’s pure laziness on my part. And the fact it dug through the full codebase and made a first pass when I was turning in for the night is sooooo worth it. I’m sure I would’ve spent more time figuring out the codebase on my own otherwise.

But, I am also saying this before I have manually tested any of this. I have only run the tests so far and not plugged it into n8n and seen if it works. So my optimism might still be premature.

Okay, updated one test file and the corresponding business logic, let’s start with the next test file (which I changed a little for the previous task, so I know there’s work to do).

2025-10-12 #

I did some final cleanup yesterday and didn’t document it all, it’s only available in the commits, but I basically went over the changes as I would any PR at work and then raised it. The maintainer looked at it during the day and then left a nice note thanking me for being thorough and documenting my code well 🥰

Hey Björn!

Thank you so much for your outstanding contribution with PR #4! 🚀

Your pull request to fix the URL character counting behavior was absolutely fantastic - it addressed a real problem that many users probably didn’t even realize they were experiencing. The fact that Mastodon counts all URLs as exactly 23 characters regardless of their actual length is such an important detail, and your implementation handles it perfectly.

What I especially appreciated about your work: #

✨ Thorough Research - You referenced the actual Mastodon source code to validate the behavior
🧪 Comprehensive Testing - 19 new tests covering all the edge cases!
📚 Excellent Documentation - Clear comments explaining the “why” behind the logic
🔧 Smart Implementation - The decision to remove entire URLs rather than break them is brilliant
🔒 Clean Integration - Minimal changes to existing code while adding powerful functionality

The Impact: #

Your fix means users won’t have their URLs unexpectedly broken when posting longer content. That’s a real quality-of-life improvement that will prevent confusion and frustration.

Code Quality: #

The implementation is clean, well-tested, and follows all the best practices. It’s the kind of contribution that makes maintaining open source projects a joy!

Would you be interested in contributing more to this project? Your expertise with Mastodon’s APIs and attention to detail would be incredibly valuable.

Thanks again for taking the time to identify this issue, implement a robust solution, and provide such thorough testing. This is exactly the kind of contribution that makes the open source community amazing! 🙌

Best regards

As they said, these interactions make open source fun because it’s people working and building stuff together 🙂