<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xml:base="https://bjorn.now/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Golang on bjorn.now</title><link>https://bjorn.now/tags/golang/</link><description>Recent content in Golang on bjorn.now</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Fri, 20 Mar 2026 09:23:00 +0100</lastBuildDate><atom:link href="https://bjorn.now/tags/golang/feed.xml" rel="self" type="application/rss+xml"/><item><title>gum: clean user interactions for shell scripts</title><link>https://bjorn.now/til/2025-08-20-gum-is-a-cli-tool-that-helps-your-make-your-scripts-easy-to-interact-with/?utm_medium=feed</link><pubDate>Wed, 20 Aug 2025 23:02:03 +0800</pubDate><guid>https://bjorn.now/til/2025-08-20-gum-is-a-cli-tool-that-helps-your-make-your-scripts-easy-to-interact-with/</guid><description>gum simplifies making shell scripts interactive: no more wrestling with read commands and ANSI escape codes for user input: just proper text editing, defaults, and clean UI.
Their demo:</description><content:encoded><![CDATA[<p><a href="https://github.com/charmbracelet/gum">gum</a> simplifies making shell scripts interactive: no more wrestling with read commands and ANSI escape codes for user input: just proper text editing, defaults, and clean UI.</p>
<p><a href="https://github.com/charmbracelet/gum/blob/9ab722ca4fe1e248283091626bd9fe5ecea65e1b/examples/demo.sh">Their demo</a>: <figure><img src="/img/2025/gum-demo.gif"
    alt="gum gemo">
</figure>
</p>
<p>For example, if you run:</p>
<div class=" copy-enabled">
  <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">gum input --width <span class="m">50</span> --header <span class="s2">&#34;Favorite author?&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> --value <span class="s2">&#34;Terry Pratchett&#34;</span></span></span></code></pre></div>
</div>
<p>It will give you a question with a sensible default that you can then change as you want, and what you wrote as you hit <code>enter</code> will be returned so your script can then use it.</p>
<p>If you instead has some options where you want to select one:</p>
<div class=" copy-enabled">
  <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">gum choose --header <span class="s2">&#34;Favorite book?&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> <span class="s2">&#34;Hogfather&#34;</span> <span class="s2">&#34;Thief of Time&#34;</span> <span class="s2">&#34;The Night Watch&#34;</span> <span class="s2">&#34;Small Gods&#34;</span></span></span></code></pre></div>
</div>
<p>Which gives you a selection box, you can configure to allow multiple selections, and the one you pick is then returned.</p>
<p>And one I really love, have it show a spinner to indicate that yes… something is still going on:</p>
<div class=" copy-enabled">
  <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">gum spin --title<span class="o">=</span><span class="s2">&#34;Counting down from 10&#34;</span> sleep <span class="m">10</span></span></span></code></pre></div>
</div>
<p>Built on <a href="https://github.com/charmbracelet/bubbletea">bubbletea</a>, a Go TUI framework, use that directly if you need these components in Go code rather than shell scripts.</p>]]></content:encoded></item><item><title>Working with Go's test cache on CI — be fast by avoiding work, while doing the important work</title><link>https://bjorn.now/blog/2025/07/07/working-with-gos-test-cache-on-ci/?utm_medium=feed</link><pubDate>Mon, 07 Jul 2025 21:50:00 +0800</pubDate><guid>https://bjorn.now/blog/2025/07/07/working-with-gos-test-cache-on-ci/</guid><description>I was trying to speed up our slow CI by caching Go builds. The easy win was caching Go’s module downloads (via GOPATH), but when I added GOCACHE for the build cache, I got a pleasant surprise: the tests were caching too. 🥳
I shared the change for review, and a colleague raised a great point: “What about our black box integration tests?” These tests hit APIs and external services that Go can’t track as dependencies. If they cache when they shouldn’t, we might miss real failures: the tests would pass because they didn’t re-run, not because the code actually works.</description><content:encoded><![CDATA[<p>I was trying to speed up our slow CI by caching Go builds. The easy win was caching Go&rsquo;s module downloads (via <code>GOPATH</code>), but when I added <code>GOCACHE</code> for the build cache, I got a pleasant surprise: the tests were caching too. 🥳</p>
<p>I shared the change for review, and a colleague raised a great point: &ldquo;What about our black box integration tests?&rdquo; These tests hit APIs and external services that Go can&rsquo;t track as dependencies. If they cache when they shouldn&rsquo;t, we might miss real failures: the tests would pass because they didn&rsquo;t re-run, not because the code actually works.</p>
<details>
<summary><strong>TL;DR:</strong> I found three ways to handle Go test caching on CI, they are:</summary>
<table>
  <thead>
      <tr>
          <th>Approach</th>
          <th>Changes Needed</th>
          <th>Test Caching</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>go clean -testcache</code></td>
          <td>CI config only</td>
          <td>❌ None</td>
      </tr>
      <tr>
          <td><code>-count=1</code></td>
          <td>Every test invocation</td>
          <td>❌ None</td>
      </tr>
      <tr>
          <td>Environment variables</td>
          <td>Shared code/library once</td>
          <td>✅ Full</td>
      </tr>
  </tbody>
</table>
<p><strong><code>go clean -testcache</code></strong> — wipe all test cache before running. No code changes, no test caching.</p>
<p><strong><code>-count=1</code></strong> — skip cache for a specific <code>go test</code> invocation. Need to add it everywhere.</p>
<p><strong>Environment variables</strong> — read your CI&rsquo;s commit SHA (e.g. <code>GITHUB_SHA</code>, <code>DRONE_COMMIT_SHA</code>) in your test setup. Go auto-invalidates the package&rsquo;s test cache when the value changes. Unit tests still cache, integration tests re-run on new commits as long as you read the variable once in the package. A helper to ensure it doesn&rsquo;t cache:</p>
<div class=" copy-enabled">
  <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">IsIntegrationTest</span><span class="p">(</span><span class="nx">t</span><span class="w"> </span><span class="o">*</span><span class="nx">testing</span><span class="p">.</span><span class="nx">T</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">t</span><span class="p">.</span><span class="nf">Helper</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">_</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="nx">os</span><span class="p">.</span><span class="nf">Getenv</span><span class="p">(</span><span class="s">&#34;GITHUB_SHA&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span></span></span></code></pre></div>
</div>
<p>Key behavior:</p>
<ul>
<li>Invalidation is <strong>per-package</strong>, not per-test</li>
<li>Reading env vars through library calls still triggers invalidation</li>
<li><code>go test</code> with no package target never caches</li>
<li>Debug caching decisions with <code>GODEBUG=gocachetest=1</code></li>
</ul>
</details>
<h2 id="the-fundamental-problem">
  The Fundamental Problem
  <a href="#the-fundamental-problem" class="heading-anchor" aria-label="Direct link to The Fundamental Problem"><span aria-hidden="true">#</span></a>
</h2>
<p>When my colleague warned me about integration test caching, I realized we had a bigger problem than just my PR. How could I fix this across our entire org? We&rsquo;ve got ~250 repos, and asking every team to remember special flags or change their test execution wasn&rsquo;t going to fly. I needed something that would just work everywhere with minimal changes.</p>
<p>The fundamental issue: our black box integration tests don&rsquo;t &ldquo;declare their dependencies&rdquo; to the Go compiler, so it doesn&rsquo;t know that something the test is critically dependent upon has changed and requires a rerun (and practically, neither do we really), because they test network services and external binaries that Go has no visibility into. I always have to run these tests.</p>
<p>I wanted the caching benefits without the risk, ideally with minimal changes since we have lots of codebases that could benefit from this caching improvement. Since our black box integration tests use a shared internal library, I was hoping to find a solution I could implement once there rather than asking teams to modify individual tests or change how they execute them.</p>
<h2 id="so-how-do-i-make-this-work">
  So&hellip; How Do I Make This Work?
  <a href="#so-how-do-i-make-this-work" class="heading-anchor" aria-label="Direct link to So&hellip; How Do I Make This Work?"><span aria-hidden="true">#</span></a>
</h2>
<p>I started by looking for the obvious solutions, there&rsquo;s probably a flag to ignore the test cache, right? And there were two straightforward options: <code>go clean -testcache</code> and <code>-count=1</code>. But here&rsquo;s the thing, both of them throw away ALL test caching, which felt like using a sledgehammer when I needed a scalpel.</p>
<p>Then I remembered reading something about environment variables, and files, affecting test caching. I went into <code>go help test</code> and I remembered right, tests that read environment variables get invalidated when those variables change, so since this is about CI, and I know that CI systems give us unique commit SHAs as env vars… and we have a shared library to help write black box integration tests… so I can make the fix once in that library and everyone gets this benefit!</p>
<p>Alright, that gives me three options, how do they compare?</p>
<p><strong>Simple Option: <code>go clean -testcache</code></strong></p>
<ul>
<li>Pro: Dead simple - add one line to your CI config, no test changes needed</li>
<li>Con: Zero caching benefit for tests</li>
<li>When to use: Safest solution that still gives you build benefits without changing scripts or tests</li>
</ul>
<p>This works by creating <a href="https://github.com/golang/go/blob/665af869920432879629c1d64cf59f129942dcd6/src/cmd/go/internal/test/test.go#L844-L848">a file with the current Unix time in nanoseconds</a> and then, whenever Go runs tests, it checks if the cached test is newer than that timestamp. I appreciate the simplicity: the cache cleaner doesn&rsquo;t need to understand which cached items are tests (and there are only hashes in that folder), it just sets a &ldquo;tests are invalid after this moment&rdquo; marker and the code skips it or not.</p>
<p><strong>Explicit Option: <code>-count=1</code></strong></p>
<ul>
<li>Pro: Granular control per <code>go test</code> invocation</li>
<li>Con: Need to change test execution and remember it everywhere, easy to miss (especially in monorepos with many go.mods)</li>
<li>When to use: Only some tests need cache invalidation</li>
</ul>
<p>This is the one you&rsquo;ll see when people ask how to make Go not cache tests, for our situation not super useful, but mentioning it for completeness.</p>
<p><strong>Elegant Option: Environment Variables</strong></p>
<ul>
<li>Pro: Automatic cache invalidation on CI when needed, maximum caching benefit when possible</li>
<li>Con: Requires thoughtful test design</li>
<li>When to use: You have enough tests that the speedup matters and you&rsquo;re willing to be intentional about cache invalidation</li>
</ul>
<p>Or laid out visually:</p>
<table>
  <thead>
      <tr>
          <th>Approach</th>
          <th>Changes Needed</th>
          <th>Test Caching</th>
          <th>Safety</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>go clean -testcache</code></td>
          <td>CI config only</td>
          <td>❌ None</td>
          <td>✅ Safest</td>
      </tr>
      <tr>
          <td><code>-count=1</code></td>
          <td>Every test invocation</td>
          <td>❌ None</td>
          <td>✅ Safe</td>
      </tr>
      <tr>
          <td>Environment variables</td>
          <td>Shared code/library once</td>
          <td>✅ Full</td>
          <td>✅ Safe*</td>
      </tr>
  </tbody>
</table>
<p>*<em>When implemented correctly</em></p>
<h3 id="how-the-environment-variable-approach-works">
  How the Environment Variable Approach Works
  <a href="#how-the-environment-variable-approach-works" class="heading-anchor" aria-label="Direct link to How the Environment Variable Approach Works"><span aria-hidden="true">#</span></a>
</h3>
<p>This works because Go invalidates the entire package&rsquo;s test cache whenever an environment variable it depends on changes, and most CI systems expose a unique commit SHA as an environment variable. For example,</p>
<ul>
<li>GitHub Actions: <code>GITHUB_SHA</code></li>
<li>Drone CI: <code>DRONE_COMMIT_SHA</code></li>
</ul>
<p>When your integration test reads this variable, Go automatically invalidates the cache for that package whenever you push new code. The key insight: you don&rsquo;t need to <em>use</em> the variable meaningfully, just <em>read</em> it. One thing to keep in mind: consider keeping integration tests in separate packages from unit tests to avoid invalidating unit test caches unnecessarily when only the integration tests need cache invalidation.</p>
<div class=" copy-enabled">
  <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="kn">package</span><span class="w"> </span><span class="nx">integrationtesting</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">func</span><span class="w"> </span><span class="nf">IsIntegrationTest</span><span class="p">(</span><span class="nx">t</span><span class="w"> </span><span class="o">*</span><span class="nx">testing</span><span class="p">.</span><span class="nx">T</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">t</span><span class="p">.</span><span class="nf">Helper</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// This ensures that we invalidate the cache</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// on CI whenever we have a new git SHA</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">_</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="nx">os</span><span class="p">.</span><span class="nf">Getenv</span><span class="p">(</span><span class="s">&#34;DRONE_COMMIT_SHA&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// ... another file relying on integrationtesting</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">func</span><span class="w"> </span><span class="nf">TestAPIIntegration</span><span class="p">(</span><span class="nx">t</span><span class="w"> </span><span class="o">*</span><span class="nx">testing</span><span class="p">.</span><span class="nx">T</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">integrationtesting</span><span class="p">.</span><span class="nf">IsIntegrationTest</span><span class="p">(</span><span class="nx">t</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Your actual integration test</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">resp</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nf">callAPIEndpoint</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">assert</span><span class="p">.</span><span class="nf">Equal</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span><span class="w"> </span><span class="nx">expected</span><span class="p">,</span><span class="w"> </span><span class="nx">resp</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span></span></span></code></pre></div>
</div>
<h2 id="understanding-how-it-actually-works">
  Understanding How It Actually Works
  <a href="#understanding-how-it-actually-works" class="heading-anchor" aria-label="Direct link to Understanding How It Actually Works"><span aria-hidden="true">#</span></a>
</h2>
<p>I was getting worried about edge cases. The Go docs said tests won&rsquo;t cache if environment variables change, but what exactly did that mean? Would reading an env var in one test invalidate <em>all</em> tests everywhere? Just that package? What if the variable is read through a library call?</p>
<p>I did some searching and found the <a href="https://github.com/golang/go/blob/6c3b5a2798c83d583cb37dba9f39c47300d19f1f/src/internal/testlog/log.go#L49-L54">testlog package</a>, which is how Go implements the tracking of when env vars are read and files opened. That gave me a good sense of how it should behave, so I made an experiment to validate what I expected from the documentation and the code I found.</p>
<h3 id="the-experiment-results">
  The Experiment Results
  <a href="#the-experiment-results" class="heading-anchor" aria-label="Direct link to The Experiment Results"><span aria-hidden="true">#</span></a>
</h3>
<p>I created <a href="https://github.com/gaqzi/experiment-go-test-caching">three test packages</a> to understand the boundaries:</p>
<ul>
<li>One that reads <code>DRONE_COMMIT_SHA</code> directly</li>
<li>One that reads it through a library call</li>
<li>One that doesn&rsquo;t read any environment variables</li>
</ul>
<p>Then I ran <a href="https://github.com/gaqzi/experiment-go-test-caching/blob/main/test.sh">nine scenarios to test my assumptions</a>: What happens when I set the variable? Does package listing matter (<code>./...</code> vs manual package lists)? What about <code>go test</code> with no arguments?</p>
<p>Key findings:</p>
<ul>
<li><strong>Package-level invalidation</strong>: All tests in a package get invalidated if <em>any</em> test reads the variable and the variable changes,</li>
<li><strong>Library calls count</strong>: Reading env vars through dependencies still invalidates the cache</li>
<li><strong>Package listing doesn&rsquo;t matter</strong>: <code>./...</code>, manual lists, even <code>./</code> all cache the same way</li>
<li><strong>No arguments = no cache</strong>: <code>go test</code> without targeting never caches</li>
<li><strong>File-level targeting = no cache</strong>: <code>go test file_test.go</code> also never caches</li>
</ul>
<p>The big takeaway: Go runs tests per package, and packages are run in parallel. If any test in the package touches an environment variable, the entire package&rsquo;s cache depends on that variable&rsquo;s value, so the cache is invalidated when the value is different on the next run.</p>
<p>Or as a semi-flowchart:</p>
<div class=" copy-enabled">
  <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Package reads env var? 
</span></span><span class="line"><span class="cl">    Yes → Env var value changed? 
</span></span><span class="line"><span class="cl">              Yes → Cache invalid
</span></span><span class="line"><span class="cl">              No  → Cache valid
</span></span><span class="line"><span class="cl">    No → Cache valid</span></span></code></pre></div>
</div>
<p>This made me comfortable that the approach was predictable and wouldn&rsquo;t have surprising edge cases, especially because the way <a href="https://github.com/golang/go/blob/6c3b5a2798c83d583cb37dba9f39c47300d19f1f/src/internal/testlog/log.go#L49-L54">testlog</a> works is by simply recording that <em>something</em> called <code>os.Getenv</code> with this value, and it doesn&rsquo;t know which test or from where, just that in the course of running these tests this happened. Nice and simple.</p>
<p>If you want to explore how Go makes decisions around caching yourself, you can run tests with <code>GODEBUG=gocachetest=1</code> to see Go&rsquo;s caching decisions in real time. There are more debugging variables available in <code>go help cache</code>.</p>
<h2 id="the-results">
  The Results
  <a href="#the-results" class="heading-anchor" aria-label="Direct link to The Results"><span aria-hidden="true">#</span></a>
</h2>
<p>We cut our CI times in half with this caching approach, and I&rsquo;m confident it&rsquo;s safe to roll out widely because I understand exactly how it works and I can make the required changes to our shared integration testing library so it will just work for the standard case.</p>
<p>What I appreciate about Go&rsquo;s design here is how it avoids clever optimizations in favor of predictable behavior. I knew caching wasn&rsquo;t just on/off, but I wasn&rsquo;t sure if there would be complex edge cases to worry about. Instead, Go picks the straightforward solution: package-level invalidation that&rsquo;s easy to reason about.</p>
<p>The &ldquo;bigger packages than you think, but still not huge&rdquo; philosophy means we get plenty of caching benefit, while keeping the invalidation scope manageable, and if we need finer control, we can always split into more packages.</p>
<p>And that&rsquo;s the thing, Go just picks the simple and boring solution that works, it&rsquo;s why I love working with Go. 😁</p>
<hr>
<p><em>The <a href="https://github.com/gaqzi/experiment-go-test-caching">full experiment is available</a> if you want to dig into it yourself.</em><br>
<em><a href="https://speakerdeck.com/gaqzi/working-with-gos-test-cache-on-ci">Presentation slides</a> from the Singapore Go meetup</em></p>
]]></content:encoded></item></channel></rss>