chris-kipp.io Blog

Claude Code, Metals, and NVIM

Chris Kipp — Wed, 13 Aug 2025 00:00:00 +0200

Claude Code, Metals, and NVIM

An overview of using Claude Code, Metals, and NVIM

I hated AI assisted development for a long time.

I remember when Copilot was first released and people lost their minds. This was apparently the beginning of the end for human developers. The next couple years new tools came out constantly promising a revolution in developer productivity and replacement... I hated it. All my initial attempts at using these tools were awful. Either I felt like I wasn't in control, the editor integrations for nvim were terrible, or I was just unsatisfied with the results. When new promising editors were released that were more AI-centric I couldn't find it in myself to pull away from a setup that I felt so invested and productive in.

I started a new job earlier this year that is quite AI-centric in their approach to development, not in an overpowering hype-driven way, but in a this should make you more productive... use it sort of way. While skeptical at first, I've found in the past few months that I've landed on a flow developing Scala that I'm quite happy with that allows me to stay in my comfortable nvim/tmux/terminal environment while benefiting from assistance that still allows me to feel like I'm in control and boosts my productivity.

My Setup

I still live in the terminal for development. On a daily basis developing Scala I'm using Metals through nvim-metals. I heavily use claude code with Metals as an MCP server, and that's what I'd like to talk about and illustrate here. If you're not familiar with MCP servers, the basic idea is that they expose a set of tools that can be used from your agent when the deem it appropriate.

As of Metals v1.5.3 Metals can act as an MCP Server. This matters because it can greatly speed up your workflow for some of the most common things you'll find claude doing:

make sure it quickly and accurately finds what it needs in your code base
making sure you code compiles after it changes it
make sure you tests pass after your changes

I get giddy about protocols and things working together, and I love the way that this ends up working on the hood. Let's assume a couple things in the following flow:

You're using your build tool as your server (let's say sbt). This matters because whether you're doing work in your editor, running things from the tool from the command line, or having Metals compile your code for you via the MCP tools, your code is always in the same compilation state. I'll illustrate this down below further. If you didn't know there is a setting to tell Metals to default to this if your build tool supports it instead of relying on bloop. In your metals settings just set defaultBspToBuildTool to true. I recommend you do this regardless of whether your using Metals as an MCP server or not.
You're starting Metals as an MCP server. You need to set startMcpServer to true in your settings for this.
This example uses claude code, so in order for Metals to write the correct config file so claude picks it up when you start it, you'll want to set mcpClient to claude in your settings.

While I use other MCP tools at work, we'll focus on this flow to illustrate the benefits of Metals here. If you have everything setup like I do above when you start Metals by opening your project you should be greeted with a nice little message like this:

LSP[metals][Info] Metals MCP server started on port: 64402. Refresh connection if needed.

And then if you start claude in the root of that same directory you should see a little prompt with the message:

New MCP server found in .mcp.json: metals

And then you need to respond if you want to allow it or not. This choice is saved then for your workspace.

It knows to start this because you'll notice a newly create .mcp.json file at the root of your project:

{
  "mcpServers": {
    "metals": {
      "url": "http://localhost:54640/sse",
      "type": "sse"
    }
  }
}

Letting Metals do its thing

for these examples i'll use the metals codebase, which isn't huge, but it illustrates this well. let's pretend that you are using another editor that doesn't use bsp to edit your code meaning that while you're editing your code it's fine, but if you switch to sbt, you need to re-compile your code. the same can be said for using bsp with bloop instead of sbt as well. even though your code is fully compiled with metals and with bloop, when you drop down into sbt you need to recompile your code. in my opinion this is one of the biggest benefits of defaulting to your build tool to be your build server: you don't need to pay that cost of compiling twice. so down below, what you're seeing is claude issuing a command to compiling your code with sbt.

⏺ The compilation completed but with some errors at the end. Here are the results:

  Total compilation time: 48.052 seconds (48s)

  The compilation mostly succeeded - it compiled all the main modules including:
  - mtags-interfaces
  - sbt-metals
  - mtags-shared
  - mtags-java
  - mtags
  - metals (main module)
  - tests modules
  - metals-docs
  - metals-bench

  The timing breakdown shows:
  - Real time: 48.052 seconds
  - User CPU time: 131.41 seconds
  - System CPU time: 7.94 seconds
  - CPU utilization: 289% (indicating parallel compilation)

However realistically part of your code will have already been compiled, but so incrementally compilation will have kicked in and it will be a bit faster than this. I'll have claude just add a single method to a file and then recompile and report back.

⏺ Incremental compilation time: 7.307 seconds (7s)

  The incremental compilation was much faster! Here's the comparison:

  - Full compilation (first run): 48.052 seconds
  - Incremental compilation (after small change): 7.307 seconds

  The incremental compilation only needed to recompile:
  - 1 Scala source in the metals module (the Time.scala file I changed)
  - 3 Scala sources in the metals module (likely dependent files)
  - 1 Scala source in the tests/unit module (probably a test that uses Time)

  This demonstrates SBT's incremental compilation working effectively - it only recompiled the changed file and its dependents, rather than the entire codebase, resulting in
  an 85% reduction in compile time.

Now realistically it's fair to say that it'll be changing more code than this. So we'll use 7 seconds as the bottom datapoint to prove the point that every time claude is dropping down to issue a bash command to re-compile your code it's going to at least take this much time.

Now if you're using Metals as your LSP server with MCP enabled and it's hooked up to claude every change that claude is making to your codebase ends up being registered and accounted for (although this does see to have slightly differently behavior depending on your editor for some reason if you read the issue here). In general Metals doesn't just rely on notifications from the LSP client about file changes but it also has it's own file watcher. When Metals changes a file even without any LSP notification being sent this basic flow will happen:

OS detects file change → PathWatcher receives event
File filter check → watchFilter(path) validates if file should trigger events
Event queued → FileWatcherEvent.createOrModify(path) added to BlockingQueue
Callback triggered → didChangeWatchedFiles(event) called
onCreate invoked → onCreate(path) in MetalsLspService.scala
Compilation triggered → compilers.didChange(path, false) in MetalsLspService.scala

So as claude is making changes your Metals state is consistently compiling and being updated to match what's happening on disk. Due to this if you get to a point where claude tries to compile your code it's already fully compiled and should basically return almost instantaneous with either no errors letting you know your code is fine or with diagnostics about what is wrong. The MCP logs for this looks like this:

[Trace - 18:16:29]  Sending request
{"method":"tools/call","params":{"name":"compile-full","arguments":{},"_meta":{"claudecode/toolUseId":"toolu_01ENHHfWxBMUZXTWFxEyASyg"}},"jsonrpc":"2.0","id":2}
[Trace - 18:16:29]  Received response
event: message

[Trace - 18:16:29]  Received response
data: {"jsonrpc":"2.0","id":2,"result":{"content":[{"type":"text","text":"Compilation successful."}],"isError":false}}

And if you have issues:

[Trace - 18:17:08]  Sending request
{"method":"tools/call","params":{"name":"compile-full","arguments":{},"_meta":{"claudecode/toolUseId":"toolu_01DJYsYhGJbh6KQm2nbZ9nWX"}},"jsonrpc":"2.0","id":3}
[Trace - 18:17:08]  Received response
event: message

[Trace - 18:17:08]  Received response
data: {"jsonrpc":"2.0","id":3,"result":{"content":[{"type":"text","text":"Compilation failed with errors:\nmetals/src/main/scala/scala/meta/internal/metals/watcher/FileWatcherEvent.scala L13-L13:\n'=' expected."}],"isError":false}}

Again, in both of these scenarios the compile command was basically a no-op and Metals was able to return immediately with information so that claude can either move forward with validation knowing nothing is wrong, or to start and fix whatever issue it introduced. The time this alone saves in a long editing session is immense.

This same concept can be applied for testing. If your codebase is already fully compiled then starting a test should also basically be instantaneous.

> Can you run the ScalaVersionsSuite test with Metals

⏺ metals - test (MCP)(testClass: "tests.ScalaVersionsSuite")
  ⎿  
     tests.ScalaVersionsSuite:
       + idempotent-minor-release passed
     … +55 lines (ctrl+r to expand)

⏺ All 54 tests in ScalaVersionsSuite passed successfully. The test suite completed in 42ms with
  no failures.

Behind the scenes the request in MCP looks like this:

[Trace - 18:20:43]  Sending request
{"method":"tools/call","params":{"name":"test","arguments":{"testClass":"tests.ScalaVersionsSuite"},"_meta":{"claudecode/toolUseId":"toolu_01BXjFLMVkLjAEQuwwjAobSh"}},"jsonrpc":"2.0","id":4}
[Trace - 18:20:43]  Received response
event: message

[Trace - 18:20:43]  Received response
data: {"jsonrpc":"2.0","id":4,"result":{"content":[{"type":"text","text":"\ntests.ScalaVersionsSuite:\n  + idempotent-minor-release passed\n  + retain-pre-release-version passed\n  + drop-typelevel-vendor-suffix passed\n  + recommended-future passed\n  + recommended-212 passed\n  + recommended-211 passed\n  + recommended-213 passed\n  + future-213 passed\n  + not-future-213 passed\n  + future-212 passed\n  + not-future-212 passed\n  + not-future-211 passed\n  + future-214 passed\n  + future-315 passed\n  + 2.12.11-comapatible-with-2.12.5 passed\n  + 2.12.5-not-compatible-with-2.12.11 passed\n  + 2.12.7-compatible-with-2.12.5 passed\n  + 2.12.5-not-compatible-with-2.12.7 passed\n  + 2.12.11-compatible-with-2.11.12 passed\n  + 2.11.12-not-compatible-with-2.12.11 passed\n  + 0.9.10-not-compatible-with-0.10.0-M4 passed\n  + 0.9.10-not-compatible-with-0.10.0-RC2 passed\n  + recommended-3 passed\n  + compare-3.0.0-M1<=3.0.0-M2 passed\n  + compare-3.0.0-M2>=3.0.0-M1 passed\n  + compare-3.0.0-RC1<=3.0.0-RC2 passed\n  + compare-3.0.0-RC2>=3.0.0-RC1 passed\n  + compare-3.0.0-RC1<=3.0.0 passed\n  + compare-3.0.0>=3.0.0-RC1 passed\n  + compare-3.0.0-M1<=3.0.0 passed\n  + compare-3.0.0>=3.0.0-M1 passed\n  + compare-3.0.0-RC1<3.0.0 passed\n  + compare-3.0.0>3.0.0-RC1 passed\n  + compare-3.0.0-M1<3.0.0 passed\n  + compare-3.0.0>3.0.0-M1 passed\n  + compare-3.0.0-M1<3.0.0-RC1 passed\n  + compare-3.0.0-RC1>3.0.0-M1 passed\n  + compare-RC1<=RC1-SNAPSHOT passed\n  + compare-RC2>=RC1-SNAPSHOT passed\n  + compare-RC1<RC2-SNAPSHOT passed\n  + compare-RC2>RC1-SNAPSHOT passed\n  + compare-NIGTLY passed\n  + not-future-3-M1 passed\n  + not-future-3-M2 passed\n  + not-future-RC passed\n  + smth-library_2.13-21.2.0-sources.jar passed\n  + scala3-compiler_3-3.0.1-RC2-bin-20210310-4af1386-NIGHTLY-sources.jar passed\n  + scala3-library_3-3.1.0-RC1.jar passed\n  + scala-library-2.13.1.jar passed\n  + cool4.4_2.13-3.0.jar passed\n  + scala3-library_3-3.0.0-sources.jar passed\n  + munit_3-0.7.29-sources.jar passed\n  + tested-3.0-sources.jar passed\n  + newer-RC passed\nExecution took 42ms\n54 tests, 54 passed, 0 failed, 0 skipped\n"}],"isError":false}}

The cool (or not so cool if you hate how many moving pieces there are) thing is that the Debug Adapter Protocol (DAP) is also being used here behind the scenes. When this request comes into Metals the flow is:

The flow:

Claude Request → MCP (mcp__metals__test tool)
MCP → Metals (McpTestRunner.runTests)
Metals → DAP (TestSuiteDebugAdapter creation)
DAP → Build Server (BSP test execution request)
Build Server executes tests via sbt/mill/whatever build server you're using
Build Server → DAP (test results)
DAP → Metals (formatted results)
Metals → MCP (test summary)
MCP → Claude (final response)

Or visually:

┌─────────┐    ┌─────────┐    ┌─────────┐    ┌─────────┐    ┌──────────────┐
│ Claude  │───▶│   MCP   │───▶│ Metals  │───▶│   DAP   │───▶│ Build Server │
│ Request │    │Tool Call│    │  LSP    │    │ Adapter │    │     (BSP)    │
└─────────┘    └─────────┘    └─────────┘    └─────────┘    └──────────────┘
     ▲              ▲              ▲              ▲                  │
     │              │              │              │                  │
     │              │              │              │                  ▼
     │              │              │              │         ┌──────────────┐
     │              │              │              │         │ Test Run     │
     │              │              │              │         │   (sbt/etc)  │
     │              │              │              │         └──────────────┘
     │              │              │              │                  │
     │              │              │              │                  │
┌─────────┐    ┌─────────┐    ┌─────────┐    ┌─────────┐             │
│ Claude  │◀───│   MCP   │◀───│ Metals  │◀───│   DAP   │◀────────────┘
│Response │    │Response │    │ Results │    │ Results │
└─────────┘    └─────────┘    └─────────┘    └─────────┘

These two feature alone have saved me a ton of time. There are also more tools that can be used in a variety of ways. If you're using claude you can view these via /mcp and selecting metals. The existing tools as of today are:

compile-file
compile-module
compile-full
test
glob-search
typed-glob-search
inspect
get-docs
get-usages
import-build
find-dep
list-modules
format-file

Many of these can be used either as alternatives to things like web-search for finding deps, hooks for formatting, or finding actual usages of symbols across your codebase in a way that can not only speed up your workflow, but provide you a more accurate one.

To summarize, if you use Metals and you use a tool like claude, use Metals as an MCP serer. It will save you some time.

An intro to Flow Fields in Scala

Chris Kipp — Sun, 25 Jun 2023 00:00:00 +0200

An intro to Flow Fields in Scala

This article is an interpretation in Scala of the wonderful Flow Fields article by Tyler Hobbs. Getting started in generative art can be difficult and intimidating, and I'm extremely grateful for artists like Tyler that open up their process for others to either learn it for themselves or to just help admirers better understand the process. Lately I've been attempting to go from the "admirer" to the "learner", so I was drawn to this article since it not only gives a clear introduction to Flow Fields, but also uses pseudocode to illustrate the concepts. Since I primarily code in Scala, my goal with this post was to really understand the concepts by translating his pseudocode into Scala code utilizing the Java Processing library. I'll also add in some extra resources to help fill in some gaps if you're new to Flow Fields, like I am. When you see quotes, unless noted otherwise, they're quotes from his article.

If you're curious to try running some of this code, I have each stage capture in this repo with instructions on how to run them. If having a fresh palette to start from is more your thing, check out my Giter8 template here that can be used to quickly get you started with Scala and processing. Processing 4 isn't published to Maven Central, so this will basically do some necessary setup for you to use Processing via Scala.

A Grid of Angles

Flow fields are based around a grid. Roughly speaking, this grid covers the whole image/canvas. At each point in the grid, you store an angle. Your grid should be stored as a 2D array of floating point numbers. Each item in the array holds an angle and represents a point in the grid.

Just like the article, we'll take this step by step taking the pseudocode and translating it to Scala. While the pictures will for sure help illustrate the steps, if you're more of a visual person and would rather watch a video to fully introduce the concepts before we dive in, I can highly recommend this video by Chris Courses. It does a fantastic job of going through many of the concepts covered in this article.

Now, to start this out, we'll do just like the article says and we'll initialize a grid. The grid will be 1000 x 1000, but in reality we'll make some extra margin around the image so you don't see everything abruptly end at the edge of our screen. The idea here is to just visualize the grid that we've created as well as a default angle that we'll give to every section in the grid (Pi * 0.25). We'll draw an arrow to visualize this angle.

import processing.core
import processing.core.*
import processing.core.PApplet
import scala.math.Pi

class One extends PApplet:
  // The actual size of our canvas
  val Width = 1000
  val Height = 1000

  // The actual size we'll work with for our flows with extra margin
  val leftX = (Width * -0.5).toInt
  val rightX = (Width * 1.5).toInt
  val topY = (Height * -0.5).toInt
  val bottomY = (Height * 1.5).toInt

  // The resolution, which will impact how many cols and rows are on the canvas
  val resolution = (Width * 0.01).toInt

  val numCols = (rightX - leftX) / resolution
  val numRows = (bottomY - topY) / resolution
  val grid = Array.ofDim[Double](numCols, numRows)

  override def settings(): Unit =
    size(Width, Height)

  override def draw(): Unit =
    background(255)

    // Drawing out all the arrows in the grid
    for col <- 0 until numCols do
      for row <- 0 until numRows do
        val x = (col * resolution)
        val y = (row * resolution)
        val angle = Pi * 0.25
        drawArrow(x, y, angle, resolution - 2)

  /** Given x and y coordinates we "GOTO" that part of the grid and then draw an
    * arrow showing the given angle.
    */
  def drawArrow(x: Double, y: Double, angle: Double, len: Double): Unit =
    pushMatrix()

    translate(x.toFloat, y.toFloat)
    rotate(angle.toFloat)

    val arrowSize = 2
    val lineLength = len.toFloat - arrowSize

    line(0, 0, lineLength, 0)

    triangle(
      lineLength,
      0,
      lineLength - arrowSize,
      (arrowSize / 2).toFloat,
      lineLength - arrowSize,
      (-arrowSize / 2).toFloat
    )

    popMatrix()
end One

Just to fill in a possible gap, a popular thing you'll see is that often when drawing a shape on the grid you won't actually store every point at each location on the grid. Instead, you'll store the points to draw the shape in a single location on the grid, and then move the grid around continually drawing the same shape. This is what you're seeing with the pushMatrix() and popMatrix() in the drawArrow method. You can read more about this here in this processing tutorial about 2D transformations.

When ran, the image produced will look like this:

Generated from One.scala

Now, all these arrow are just pointing in the same direction, which isn't what we want. So we'll introduce a slight turn as we iterate throw all the rows. The for loop we have that calls drawArrow can be changed slightly to do this:

for col <- 0 until numCols do
  for row <- 0 until numRows do
    val x = (col * resolution)
    val y = (row * resolution)
-   val angle = Pi * 0.25
+   val angle = ((row.toDouble / numRows.toDouble) * Pi)
    drawArrow(x, y, angle, resolution - 2)

Ran with this change, you'll now see the arrows curving:

Generated from Two.scala

Drawing Curves through the Field

The next step of this is to use this curve to guide us in drawing a line following the flow of the arrows. To make this a bit easier for the moment we'll simply plot a dot in red to easily show where the line would be drawn. In order to make this work we change the code around a bit to no longer just draw out the angle when we iterate over it, but instead we store the angle in the grid because we need to iterate over the grid again to draw the dots. The full code for this looks like this:

import processing.core
import processing.core.*
import processing.core.PApplet
import scala.math.Pi
import scala.math.cos
import scala.math.sin

class Three extends PApplet:
  // The actual size of our canvas
  val Width = 1000
  val Height = 1000

  // The actual size we'll work with for our flows with extra margin
  val leftX = (Width * -0.5).toInt
  val rightX = (Width * 1.5).toInt
  val topY = (Height * -0.5).toInt
  val bottomY = (Height * 1.5).toInt

  // The resolution, which will impact how many cols and rows are on the canvas
  val resolution = (Width * 0.01).toInt

  val numCols = (rightX - leftX) / resolution
  val numRows = (bottomY - topY) / resolution
  val grid = Array.ofDim[Double](numCols, numRows)

  override def settings(): Unit =
    size(Width, Height)

  override def draw(): Unit =
    background(255)

    for col <- 0 until numCols do
      for row <- 0 until numRows do
        grid(col)(row) = ((row.toDouble / numRows.toDouble) * Pi)

    for col <- 0 until numCols do
      for row <- 0 until numRows do
        drawArrow(
          col * resolution,
          row * resolution,
          grid(col)(row),
          resolution - 2
        )

    var x: Double = 500
    var y: Double = 0
    val stepLength = 10
    val steps = 300

    for step <- 0 until steps do
      drawPoint(x, y)
      val xOffset: Double = x - leftX
      val yOffset: Double = y - topY

      val columnIndex: Int = (xOffset / resolution).toInt
      val rowIndex: Int = (yOffset / resolution).toInt

      if (columnIndex > 0 && columnIndex < grid.length)
        && (rowIndex > 0 && rowIndex < grid(columnIndex).length)
      then
        val gridAngle: Double = grid(columnIndex)(rowIndex)

        val xStep: Double = stepLength * cos(gridAngle)
        val yStep: Double = stepLength * sin(gridAngle)

        x += xStep
        y += yStep

  end draw

  def drawPoint(x: Double, y: Double): Unit =
    val baseWeight = g.strokeWeight
    val baseStroke = g.strokeColor
    stroke(255, 0, 0)
    strokeWeight(10)
    point(x.toFloat, y.toFloat)
    strokeWeight(baseWeight)
    stroke(baseStroke)

  /** Given x and y coordinates we "GOTO" that part of the grid and then draw an
    * arrow showing the given angle.
    */
  def drawArrow(x: Double, y: Double, angle: Double, len: Double): Unit =
    pushMatrix()
    translate(x.toFloat, y.toFloat)
    rotate(angle.toFloat)

    val arrowSize = 2
    val lineLength = len.toFloat - arrowSize
    line(0, 0, lineLength, 0)

    triangle(
      lineLength,
      0,
      lineLength - arrowSize,
      (arrowSize / 2).toFloat,
      lineLength - arrowSize,
      (-arrowSize / 2).toFloat
    )

    popMatrix()

end Three

When the code for this is ran it will produce an image that looks like this:

Generated from Three.scala

There are a few different important things that you can change here to make quite a bit of difference in how your line is drawn. From the article:

We need to pick values for a few key parameters of how we draw the curves: the step_length, num_steps, and the starting position (x, y). The simplest is the step_length. Typically, this should be small enough that you don't see any sharp points on the curve. For me, that's usually around 0.1% to 0.5% of the image width. I go larger for quicker render speeds, and smaller if there are tight turns that need to be clean.

So in our example step_length (aka stepLength) is set to 10 resulting in the dots being pretty close together. If we were to change this to say, 30, you can tell the difference right away.

Notice how much farther apart the dots are now, which will end up resulting in a choppier line when traced.

The num_steps (steps) controls the amount of steps you'll take. So for example if you have a short step length and a short amount of steps it'll result in a short choppy line like this:

Finally, the starting positions x and y also make a difference since... that's where your line is starting from. In our example we're starting at 500 for the x axis and 0 for the y which makes the line start nice and center at the top of the grid. However, we could shift this down to be 300 for the y axis and you'll notice the original image then changes to this:

With these three values alone you start to see how you can control what each line looks like. Whether it's short, long, choppy, smooth, and where it's positioned can all make a pretty large difference in the end result. The original article gives some great illustrations of these differences at a larger scale.

Distorting the Vectors

One big design decision is how you want to distort the vectors in your field. The method you use for this will determine what shapes your curves take. It will determine whether there are loops, abrupt turns, and overlapping lines.

About 90% of the time, people use Perlin noise to initialize the vectors. This is handy and easy, because it gives you smooth, continuous values across the 2D plane.

Perlin noise is exactly what we're going to use here. It's so popular in fact that processing has built-in support for it via noise. The change we need to do here to actually implement this is shockingly small.

for col <- 0 until numCols do
  for row <- 0 until numRows do
+    val scaledX = col * 0.005
+    val scaledY = row * 0.005
+    val angle = noise(scaledX.toFloat, scaledY.toFloat) * Pi * 2
-    grid(col)(row) = ((row.toDouble / numRows.toDouble) * Pi)
+    grid(col)(row) = angle

When running with this change, you'll generate an image similar to this:

Generated from Four.scala

However, keep in mind now that even though your starting points may always be the same, the angles will not be due to the "controlled randomness" of the perlin noise.

Putting it all together

At this point we actually have most of what we need to really start generating some things. Now that we understand the grid behind what we're doing the first thing we can do is get rid of generating it. Just remember that it's always there doing the same thing, but we don't need to generate out the arrows. We can do this by removing the drawArrow method and also the for loop that calls it as we iterate through the columns and rows.

No more arrows!

Generated from Five.scala

Also currently we have a hard coded start x and y point which we don't really want. So instead what we can do is refactor this out to just define a steps, the stepLength and also a new lineCount for the amount of lines we'd like to draw. Then we can randomly create the starting x and y for every line. This sounds great, but it will introduce a new problem. Firstly, if we run it, it will look like this:

Generated from Six.scala

You'll notice that instead of just a single image, it will keep regenerating. We first notice this now because we are introducing randomness into the generation that changes on every "tick". We can prevent this by using noLoop. Also, all of the dots just look chaotic. It's hard to see any pattern here, so we'll instead switch to actually drawing a line.

At this point our full code looks like this:

import processing.core
import processing.core.*
import processing.core.PApplet
import scala.math.Pi
import scala.math.cos
import scala.math.sin
import scala.util.Random

class Seven extends PApplet:
  // The actual size of our canvas
  val Width = 1000
  val Height = 1000

  // The actual size we'll work with for our flows with extra margin
  val leftX = (Width * -0.5).toInt
  val rightX = (Width * 1.5).toInt
  val topY = (Height * -0.5).toInt
  val bottomY = (Height * 1.5).toInt

  // The resolution, which will impact how many cols and rows are on the canvas
  val resolution = (Width * 0.01).toInt

  val numCols = (rightX - leftX) / resolution
  val numRows = (bottomY - topY) / resolution
  val grid = Array.ofDim[Double](numCols, numRows)

  override def settings(): Unit =
    size(Width, Height)

  override def draw(): Unit =
    background(255)

    for col <- 0 until numCols do
      for row <- 0 until numRows do
        val scaledX = col * 0.005
        val scaledY = row * 0.005
        // We switch this here to return the angle with perlin noise
        val angle = noise(scaledX.toFloat, scaledY.toFloat) * Pi * 2
        grid(col)(row) = angle

    val stepLength = 100
    val steps = 1000
    val lineCount = 2000

    (0 until lineCount).foreach: count =>
      drawCurve(steps, count, stepLength)

    noLoop()

  end draw

  def drawCurve(steps: Int, count: Int, stepLength: Int) =
    var x: Double = Random().between(0, rightX)
    var y: Double = Random().between(0, bottomY)

    (0 until steps).foreach: _ =>
      val xOffset: Double = x - leftX
      val yOffset: Double = y - topY

      val columnIndex: Int = (xOffset / resolution).toInt
      val rowIndex: Int = (yOffset / resolution).toInt

      if (columnIndex > 0 && columnIndex < grid.length)
        && (rowIndex > 0 && rowIndex < grid(columnIndex).length)
      then
        val gridAngle: Double = grid(columnIndex)(rowIndex)

        val xStep: Double = stepLength * cos(gridAngle)
        val yStep: Double = stepLength * sin(gridAngle)

        val nextX = x + xStep
        val nextY = y + yStep

        line(x.toFloat, y.toFloat, nextX.toFloat, nextY.toFloat)

        x = nextX
        y = nextY

  end drawCurve
end Seven

And when ran, you'll start to see what we're actually going for.

Generated from Seven.scala

At this point you can really start playing around with the values of stepLength, steps, and lineCount, all which will pretty radically change the way the generated image looks. It's also at this point that you can start playing around with the way you generate your perlin noise. Chaning the * Pi * 2 will also start to radically change your image. Maybe you also want to start changing the stroke width in your drawCurve method, add some colors, etc. Here's a couple examples of doing different things.

Conclusion

The possibilities with generative art are endless, and this is just a drop in the bucket of techniques and ideas. My hope is that as I continue to explore different facets of this field, you'll see them here. Again, I want to give more focus to the art and blog by Tyler Hobbs, which is truly inspirational. Hopefully these examples will give you what you need to dive into flow fields and Scala. Until next time.

Lately - 03

Chris Kipp — Mon, 1 May 2023 00:00:00 +0200

Lately - 03

Lately

March and April were pretty busy for me. Looking back, it was a whirlwind, but a good one.

Working at the Scala Center

In the last months quite a bit of my time went into planning the last Scala Tooling Summit that happened in Lausanne, Switzerland. I wrote about this on the Scala site here.

The summit was incredible for me. For the last few years I've spent so much time working on tooling in the Scala ecosystem without ever really meeting many of those I rub shoulders with online. It was incredible to be around so many that have had such a large impact on the community, both past and present. It was one of the highlights of my "scala" career.

Some of the other work I've been focusing on is diving back into structured diagnostics in Dotty and working towards actionable diagnostics. Here's a few links related to this:

The Journey of a Dotty Diagnostic - A talk I gave at Scala.io about this in Paris.
The written version of the talk if that's more your thing
Large progress tracker for DiagnosticCode and DiagnosticRelatedInformation
Large progress tracker for actionable diagnostics
Contributor discussion about actionable diagnostics

In the coming month I'll be heavily focusing on moving these forward.

skan

This last month I spent a chunk of time on writing skan, which is a minimal terminal kanban board. I use it to track my ongoing work, and work I'd like to do. I wrote an article about it here.

Hardware

I finally got my Torn V3 build finished. It took longer than I expected, I ruined one PCB in the process, and I learned a ton. Overall a success. I still haven't nailed down the keymappings I want yet, so I'm constantly re-flashing. That's why the cover isn't on the left hand side. I also don't love my keys, so I plan on updating those and then taking a non-potato quality photo. For now, enjoy this!

Listening

Some fantastic music was released in the last couple months. So these two recommendations couldn't be further apart, but I've listened to both of these albums a ton this last month.

Did you know there's a tunnel under Ocean Blvd is wonderful. I wasn't a Lana Del Rey fan in the past, but this album is one of my favorites this year so far.

In a totally different vein I've also been listening to Still Life In Decay by FACS on repeat.

Still Life In Decay by FACS

While not a new album, I recently came across Maxime Denuc and was blown away. Specifically this album, Nachtorn. Here's a short blurb about it:

Nachthorn takes its name from one of the 78 stops that make up the main organ in St. Antonius Church in Düsseldorf. This instrument, equipped with a system developed by the German company Sinua, offers the possibility of controlling all of its keyboards and timbres via a computer. The organ thus becomes a powerful synthesiser. This set-up allowed me to fulfil an old dream of mine : to create an entirely acoustic dance music piece with the organ as sole actor. Oscillating between dub techno, harmonic locked grooves or after-hours pop, Nachhorn proposes a hypnotic music piece whose lines sketch the outline of an imaginary warehouse.

It's just so good.

Nachthorn by Maxime Denuc

Watching

I'm probably late to the party here, but I've been watching and enjoying Narcos for the first time.

In the movie realm I watched Come and See, which was a roller coaster ride of emotions. I didn't know much about it before watching, but it's not a film I'll ever forget.

Traveling

Since the previous lately post, I've found myself in Brussels, Newcastle, and Lausanne.

There's a building right out of the Brussels Central Station that always catches my eye.

I was in Newcastle for work. Work is looking at expanding there, and I did some presentations on our involvement in open source to a couple local universities.

The last night of the tooling summit we hiked up to a restaurant in Parc de l'Hermitage for some Fondue. The route was full of hills and beautiful buildings.

skan and building your own tools

Chris Kipp — Thu, 27 Apr 2023 00:00:00 +0200

skan and building your own tools

I've recently been spending some time building my own little tool to track my TODOs, skan. While building this, I've re-learned some lessons that I learned early only in my programming journey and some even played a large role in my decision to continue to pursue programming.

The lessons are actually quite simple. They might not all be applicable to you, and that's ok. We're all different, and we all learn differently. Hopefully you'll still enjoy this, and maybe get some insight into the tools I used for this project.

There's power in making your ideas reality

When you first start programming (at least in more modern times) you undoubtedly go through a phase where you're following tutorials, learning the basics of the stack you're using, and just exploring what's possible. This is a great way to get introduced to something, in fact I actually was in the process of doing this with go right before I started this project. I was following a video series on building a CLI kanban board with Bubble Tea, which is a really rad TUI library for go. However, there comes a time where you must move on from the tutorials and start building your own ideas, and when you do that, magical things happen. I've even found now that after you start building stuff on your own, finishing tutorials gets pretty difficult because you (well, at least I do) want to veer off immediately in your own direction and try your own things. The reason for that, again at least for me, is that it's an incredible feeling to build something from scratch -- to take an idea in your mind and make it a reality. Early on when I was learning to program I was part of a community called Merveilles and they embody this idea, that you can take your ideas and make them reality. Even if no one else cares, no one else will use them, you care, and that's enough. Simply the act of thinking of an idea and seeing it through fruition is a really powerful thing. I've built lots of little tools early on that made me feel so engaged and powerful when I was on my computer. Over the years I've gotten away from that, focused on building tools for other people, and forgot how good it feels to build something for myself. skan has reminded me of how good it feels to build something for me. Building this little tool has been probably the most fun I've had programming in quite some time.

If I was to summarize this, I'd say it's important to build your ideas, build things just for you, and surround yourself with people that encourage you to do it, even if it seems silly, because there's power in making your ideas reality.

You learn more when things don't work

I'll be pretty honest, I'm not a JVM guru. Scala was the first language I truly dove into and started to go beneath the surface, but I'm still in murky waters when I go too deep. Since skan is a CLI app, I didn't want to use the JVM, which originally had me considering other languages. However, I was intrigued when I saw oyvindberg/tui-scala, especially since the native image examples were quite snappy. scala-cli also makes it crazy simple to produce a native image with GraalVM, so I figured I'd give it a try. My very limited experience in the past with native images have been pretty terrible. Getting everything set up was a chore, and then when it was ready to go you almost always hit on cryptic initialization errors that really do force you to understand some stuff that I've never really cared about. I was pleasantly surprised when I was able to just run scala-cli package --native-image skan/ and it produced me a nice little executable. That was until I tried running it and got:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Native library libcrossterm.dylib (/libcrossterm.dylib) cannot be found on the classpath.
        at tui.crossterm.NativeLoader.loadPackaged(NativeLoader.java:29)
        at tui.crossterm.NativeLoader.load(NativeLoader.java:11)
        at tui.crossterm.CrosstermJni.<clinit>(CrosstermJni.java:9)
        at tui.withTerminal$.apply(withTerminal.scala:6)
        at skan.project$package$.run(project.scala:248)
        at skan.run.main(project.scala:18)

Thankfully, the author of tui-scala already had some of this documented and was willing to give a helping hand. Now I have this lovely hack that essentially does the following:

Uses coursier to fetch a jar that includes some files I need (necessary likely due to this)
Unpacks the jar, and based on the OS, grabs the necessary files and copies them to my resources folder.
Then when I package the app I include --graalvm-args -H:IncludeResources=libcrossterm.dylib as a flag.

Again, it's hacky, but it works. My command to actual make the image on MacOS looks like this (inside of a Makefile):

package-mac:
	make prepare-for-graal
	make generate-build-info
	scala-cli --power \
		package \
		--native-image \
		--graalvm-java-version 19 \
		--graalvm-version 22.3.1 \
		--graalvm-args --verbose \
		--graalvm-args --no-fallback \
		--graalvm-args -H:+ReportExceptionStackTraces \
		--graalvm-args --initialize-at-build-time=scala.runtime.Statics$$VM \
		--graalvm-args --initialize-at-build-time=scala.Symbol \
		--graalvm-args --initialize-at-build-time=scala.Symbol$$ \
		--graalvm-args --native-image-info \
		--graalvm-args -H:IncludeResources=libcrossterm.dylib \
		--graalvm-args -H:-UseServiceLoaderFeature \
		skan/ -o out/skan

Notice the make prepare-for-graal call before the rest of the stuff from packaging. That's what handles the setup. Again, much of this is copied from tui-scala, but because it didn't work at first it forced me to actually look into the flags, better understand them, better understand what crossterm even is, and better understand the inner-workings of tui-scala. Arguably, I wouldn't have learned any of this if it all just worked. Don't misunderstand me, I do wish it all just worked, but I also know that in software development many times things won't. The skills you acquire when they don't and the knowledge you gain while trying to understand why something doesn't work is valuable. Therefore, I think you learn more when things don't work.

You learn to appreciate things when you don't have them anymore

If you have keen eyes you may have noticed another call up above, make generate-build-info. This sort of relates to the above, but when things don't work, you have to get creative. I'll be honest, I don't often think about tools like sbt-buildinfo. I just use them and move on with my day. The whole idea of source generators can just be glossed over because they do what you want them to do, generate stuff so you can keep coding.

Since skan is a CLI app, you want a way to display the current version that is being used. The way I would typically do that is with a tool like I mentioned above, sbt-buildinfo. It's common to use, computes the version from git, and generates a BuildInfo.scala file that you can use. scala-cli has no ability to generate code like this. It's not until you realize you can't do something or that something isn't supported that you truly miss the ability to do that thing. To get around this I have another script that ends up being ran before any compilation is ran with scala-cli. The script is quite simple:

//> using scala "3.3.0-RC4"
//> using options "-deprecation", "-feature", "-explain", "-Wunused:all"
//> using lib "com.lihaoyi::os-lib:0.9.1"
//> using lib "com.outr::scribe:3.11.1"
//> using lib "com.github.sbt::dynver:5.0.0"

package skan.scripts

import java.util.Date

import sbtdynver.DynVer

@main def run() =
  val target = os.pwd / "skan" / ".scala-build" / "BuildInfo.scala"
  val version = DynVer.version(Date())
  scribe.info(s"Current version is ${version}")
  scribe.info(s"Generating BuildInfo.scala into ${target}")
  val buildInfo = s"""|package skan
                      |
                      |object BuildInfo:
                      |  val version = "${version}"
                      |""".stripMargin
  os.write.over(target = target, data = buildInfo, createFolders = true)

I uses the library portion of sbt-dynver to compute the version based off git tags, generates a BuildInfo.scala file with that version, and then copies that file into the main skan directory under the .scala-build/ directory that scala-cli creates. Then inside of my project.scala I have the following:

//> using file "../.scala-build/BuildInfo.scala"

This line just ensures that the generated file is "sourced" and can be used just like another file in my package.

You're probably thinking, this is brittle, hacky, and ugly. My response would be yes, yes it is. The lesson here is that we often overlook simple tools that cause us to not have to use hacks like this to accomplish things that other tools may easily offer you. The BuildInfo example is just a small one, there are other source generators that are much more involved and difficult to mimic, especially with 10 lines of code.

Build your own tools, or don't

I've said it multiple times, but skan was such a fun little project for me. It's not the most challenging software, most elegant, or even unique. There are other TUI kanban boards out there with a way larger feature set, but that's sort of the beauty of software development -- you can build whatever you want to fit your exact needs. I encourage you to build something for yourself. Whether it makes sense for other people, it doesn't matter. You'll probably hit on things that don't work and learn something new in the process. You'll probably wish you had something that X other tool had, and it'll cause you to add some hack that causes you to appreciate that other tool. At the end, you can look back at this tool you built for yourself and feel proud that you built something from scratch, and it's yours. The same concept probably applies to libraries, but I despise writing those. Maybe you despise the idea of writing your own tool, and that's ok too.

Lately - 02

Chris Kipp — Sat, 4 Mar 2023 00:00:00 +0100

Lately - 02

Lately

A lot of this past month was prepping for this next month. March will be a pretty busy month for me with traveling for work and a lot of events happening. Sometimes months like this are difficult for me since you look back at the work you've done and feel like you didn't accomplish much, until all that prep-work flourishes in the future.

Working at the Scala Center

As many of you may know, work has lent me out to the Scala Center and this last month was a lot of maintenance work around the ecosystem. The more I work in Scala tooling the more I recognize just how small that community of people are. The amount of work that the Scala Center gets done in pure maintenance alone across a handful of tools is miraculous.

Apart from maintenance work a large chunk of my time has been spent in preparation for a Scala Tooling Summit that will be happening in March. I look forward to give an update afterwards, but around 30 people from important tools in the ecosystem, the compiler team, and community contributors are all coming together to discuss the state of tooling, look at collective problems, and attempt to cement a clear roadmap moving forward focused on de-duplication of efforts between teams, unified approaches, and stabilisation. This has been an ongoing effort behind the scenes for the past few months. I'm quite excited about this, and expect to hear more after the meet.

Hardware

In part it could be because I'm now working on tooling full-time during the day, but also probably more attributed to a bit of burn out, but I've found myself coding much less in my free time and instead looking into new interests, or spending time re-exploring past ones. I don't have a traditional computer science or engineering degree like many of my colleagues seem to have, so much of the hardware / low-level things you'd explore in college I've never had the opportunity to. Lately for me that has manifested in my exploring circuitry, learning how to solder, and doing a few small projects. One project that I'm excited about is a Torn V3 build that I've started:

You can follow my progress on this on Mastodon.

Reading

Related to the above, I stumbled upon Code: The Hidden Language of Computer Hardware and Software by Charles Petzold, which is a fantastic overview of how code was born. Starting with an wide overview of braille, logic gates, and binary calculators you build you way up to a modern computer. It's a fascinating read that has been teaching me a lot of things I've never dove into before. It's been a great supplement for me as I've started diving into the hardware side of things to better understand what I'm actually doing.

Software

A crazy amount of my time is spent on my computer, especially in my terminal. So my software choices in this area aren't made lightly, and I often don't switch. However this past month has me exploring a few new pieces of software, a couple which seem to be sticking.

Arc is one of the first browsers that I've tried that really seem to challenge the way I use a browser. Challenging the way you use tabs, shortcuts, and layout. At first it just seemed silly, but the more I've gotten used to it and have been learning the shortcuts the more I'm convinced that there's been barely any advancement in the way we use browsers since almost as long as I can remember. While I'm for sure not using Arc to its fullest I'm enjoying slowing getting accustomed to it, and I will likely stick with it.

I've been a long-time iTerm2 user. Even though I barely use most of it's features, especially because I use tmux as a terminal multiplexer, I've valued the stability it offered and the ability to just display things the way I've wanted. However wezterm offers much of the same stability, is super snappy, and is configured via Lua. I've recently made the jump and I'll definitely be sticking with it as the easy configuration that can be stored in my dots repo along with nice features like instant reloading are just great.

Listening

This month I stumbled on a new to me band called Hermanos Gutiérrez that have been on repeat. Such a chill vibe that's easy to listen to while working, relaxing, or commuting.

El Bueno Y El Malo by Hermanos GutiérrezThey did an interesting interview where they spoke about some of their inspirations for their previous record and they listed some other fantastic artists like Adrian Quesada. He did a remake of El Muchacho De Los Ojos Tristes with Tita that has also been on repat for me this month.

Boleros Psicodélicos by Adrian Quesada

Watching

A movie I watched this past month that I really enjoyed was Bullet Train, which is a film adaption of a Japanese Book titled マリアビートル. Such an easy film to watch, some dark humour, and sprinkled with ridiculousness. Can we also take a second to appreciate the outfit Brad Pitt was wearing, which is one I'd happily wear.

Lately - 01

Chris Kipp — Mon, 23 Jan 2023 00:00:00 +0100

Lately - 01

Lately

I follow a few blogs that are in the habit of doing regular updates about what they are up to on a regular basis. A great example of this is the Recently posts by Tom MacWright. His blog is great by the way, check it out. In this new year I wanted to get in the habit of writing more frequently and sharing more of my interests than just programming. So you're likely to see one of these each month. It's also fitting as I'm normally pretty spread around with what I work on, so this allows me to simply share what I've been up to without needing to really dive deep into a topic to write about. I hope you enjoy.

Scala Center

Recently work has lent me out to the Scala Center to do some work on tooling in the Scala ecosystem. If you're not familiar, the Scala Center is a non-profit focused on Scala education and maintenance of some of the core parts of the Scala compiler and tooling ecosystem. Some of the first things I've dove into first when joining have been BSP related.

Firstly in an attempt to simplify the Bloop build, I started basically hacking away and removing things. Bloop is a build server for Scala/Java that focuses on speed. It's incredibly fast and has been pivotal to get Metals where it is now, but also suffered from an equally incredibly confusing build. I was able to rip out the Maven plugin, Gradle plugin, and the core config library into their own repositories to be developed and versions standalone. Bloop also had a complex meta-meta build that made contributing and getting started with Bloop difficult. That's all now gone. If you're curious, you can find some of this work in the following places:

The new home of bloop-config and the pr removing it from Bloop.
The new home of the bloop-maven-plugin and the pr removing it from Bloop.
The new home of the gradle-bloop plugin and the pr removing it.
The pr re-working and removing the meta-meta build in Bloop.

Apart from working on Bloop my main focus has been looking into the default choice in Metals, the Scala language server, to use Bloop as your build server. This last year more and more users have started defaulting to using the built-in BSP support in either sbt or Mill. Due to this I've written up a report on the differences between the various build servers here that give a pretty good overview and a plan to move forward. I've also then been working on small things related to this task in preparation to this move.

Neovim

This last month I did a handful of small quality of life improvements and updates to nvim-metals and have seen a bunch of newcomers in the new year. Must be that learning Vim was a new years resolution for a handful of Scala users!

More excitingly after a long time of inactivity on the Scala Treesitter grammar, a group of us from the Scala Community were given maintainer access and started hammering out improvements, especially to the Scala 3 grammar. Huge shout-out to Eugene and Anton for tackling the majority of the Scala 3 improvements.

Just check out the difference in syntax highlighting before and after the last month. Some of them are obvious, but some are more subtle. Overall, it's a huge improvement.

Before

After

This Site

Over the years this site has taken all sorts of forms. However, the focus has always been on a minimal self-made site. I really enjoy making my own site from scratch, and I enjoy looking at other sites that have done the same. I'll probably do a whole post on this in the future when I feel a bit happier with how everything is working together, but for now I'll outline a few changes:

I migrated this site from using Pandoc via a bash script to a small scala-cli project with Scala 3 using flexmark to turn my markdown into html.
I now have an RSS feed that you can follow here.
I've started to create lists to better keep track of things that I'd like to return to. I've always sort of done this, but I'd like to start doing it in one place and making it public. For now they're each a yaml file which gets read up and then turned into html via circe-yaml and Scalatags.
Apart from syntax highlighting and Pirsch analytics there is no JS necessary for my site.

Listening

Firstly, this year I'm doing a little experiment where I track all the new albums that I listen to fully and enjoy. You can see my progress on this under /lists/albums. This is an attempt to be a bit more attentive to the new music I'm discovering and also a fun way to track my favorites of the year. I'm excited to see what it looks like at the end of the year.

There's been two albums that have really stood out to me this month and have been on repeat.

Jonah Yano's new album portrait of a dog is a funky smooth project done with BADBADNOTGOOD. I love the crisp minimal drum lines, driving keys, and Jonah's magical voice. Often when listening I just get lost in it, my eyes close, and I just enjoy it. The whole album is great.

Funny enough, when I first started listening to this album I didn't really dig it. It took me until I was around half way through, and then I fell in love. Gigi's Recovery by The Murder Capital is the first album of theirs that I listened to, but I'll definitely be following them in the future. I get some Protomartyr vibes from them. They've got great lyrics, really catchy breaks, and throughout the album they know when to go hard and when to just keep it simple. By far my two favorites on the album are The Lie Becomes the Self and We Had to Disappear.

Reading

This last month I read The Way of Life, According to Lao Tzu. It's been a book I've wanted to read for some time, and I felt like it was maybe a good one to start the year off with. I read it slowly, just a few of the poems per day. While I enjoyed t, the book wasn't as impactful as I was expecting for some reason. I'll simply leave it as that.

I've also spent some time catching up on the articles of the sites I follow via RSS. You can find a handful of them here. Another article I often revisit that I did this month was the great article by Erin 'Folletto' Casali, The Impact of Toxic Influencers on Communities. If you're a part of the Scala community, I highly recommend the read in light of recent events and personalities.

Traveling

This month I spent a bit of time in Paris for work giving a talk called What I've learned in open source and why I think you should contribute.. It was for a semi-regularly hackday that we have at work where colleagues come together to give some talks, do workshops, and just hack on things together.

_Me in front of Boot Cafe.

Just a wall in Le Marais.

Statue on the wall in the Rodin Museum.

My Top Albums of 2022

Chris Kipp — Wed, 28 Dec 2022 00:00:00 +0100

My Top Albums of 2022

My Top albums of 2022

One of my goals this past year was to be a bit more attentive to the music I listen to. In the past I listened to so much music passively, partly due to me listening while programming. I used to be way more into classical music when I was younger and a good friend and I used to listen to an album a month closely. We would jot down notes and make sure we just sat and listened through the full album end to end. I learned so much in that exercise. I realized I had really gotten away from listening to music in that manner so this past year I wanted to be a bit more attentive. I'm happy to report that I feel like I was way more attentive, and I loved it. Even doing the simple Music of the Month on here was a ton of fun. So many people messaged me, engaged on Twitter, etc. It made it that much sweeter. Enjoying music with other people is just the best.

On that note, I wanted to conclude the year not with a December Music of the Month post, but instead share a collection of new albums that I really enjoyed this past year for different reasons and at different times. I won't offer in-depth reviews on them or really any thoughts at all. But I hope you enjoy some of these albums as much as I did. If Spotify is your thing, feel free to listen through the list here. I also encourage you to support your favorite artists and buy their music directly.

Here we go, in no particular order.

Benny The Butcher - Tana Talk 4

JID - The Forever Story

Kenny Beats - LOUIE

Messa - Close

Moncho y Su Banda - Que Bellas Son

Listen Here

Tomppabeats - Habbits

Vega Trails - Tremors in the Static

Wilma Vritra - Grotto

Yaya Bey - Remember Your North Star

Zack Fox - wood tip EP

Music of the Month - November 2022

Chris Kipp — Thu, 1 Dec 2022 00:00:00 +0100

Music of the Month - November 2022

November. November just flew by. It was here, and now it's gone already. This last month was a huge focus month for me with lots going on. Typically when I am in focus mode I'm headphones on, head down. This was one of those months. I was all over the place again with music, but here was an album and a song that stuck out to me this month.

Album of the Month

So this album was one that's been on my radar for a while, but I haven't really listened through it much. This month it just stuck for me. The album of the month for me was LOUIE by Kenny Beats. Killer album. It's his first solo project and the mans got skills to go around. Really well produced, dope beats, unique, and just pleasant to listen to. I listened to it while work, while commuting, and even hanging with friends. It just fits with whatever you're doing.

Song of the Month

The song of the month is in part because I was listening to Kenny Beats so much. Typically when I get stuck on someone for a while I branch out to other projects they've worked on or artists they've worked with. One of the many incredibly artists that Kenny has worked with has been Denzel Curry. On his album Melt My Eyez See Your Future he's got this song called Walkin which just gets in your head and stays there. It's the song of the month for me this last month.

Melt My Eyez See Your Future by Denzel Curry

See you next month

The Journey of a Dotty Diagnostic

Chris Kipp — Sun, 20 Nov 2022 00:00:00 +0100

The Journey of a Dotty Diagnostic

I was inspired by seeing the companion article Function reuse is just wishful thinking for Nicolas Rinaudo's talk, so I decided to do the same. This is meant to be a companion article for my talk, The Journey of a Dotty Diagnostic given at ScalaIO 2022 in Paris.

In this talk I dove into a few different areas exploring what actually makes up a diagnostic, how useful Scala diagnostics are, how a diagnostic actually gets from the compiler to your editor, and looked at what work is being done to improve this. Much of this was a learning experience for me, so come along for the ride.

What actually makes up a diagnostic?
The diagnostic usefulness scale
How does the diagnostic actually get to your editor?
Where is Scala on the scale?
What else is on the scale?
It's all about structure
Why did you even look into this?
What work is being done to improve this?
What we could have
What's keeping this from happening?
Conclusion

What actually makes up a diagnostic?

Normally this would be the place I break down the reason for even caring about diagnostics, but before I give you the actual use-case that spurred me to go down this rabbit hole we need to go over the basics first and come to a common understanding of what even makes up a diagnostic. Trust me, going through this will make what's to come much clearer, so bear with me.

Let's start with a very minimal code example.

val word: String = 1

Now you don't need to know Scala well to know this won't work. If you're in your editor you'd expect to see something like this:

Probably what you'd expect, but let's break this a part a bit and look at the various parts.

Firstly you have the message, which is what tells you what's actually wrong. This is typically front and center when you're viewing your diagnostic.

Secondly you have the severity level of your diagnostic. In our example here this an error, and that's indicated by the in your face red color. Typically you're message may be in this color along with something indicating where it is in your current file.

You're also probably familiar with the warning diagnostic, typically signified with a yellow/orange color as you can see here:

Now, some that you're probably less familiar with in Scala are info diagnostics. You can see a great example of this in the quill library where you can have a query and it will actually show you what's being generated. In this case it's indicated in blue.

This is an incredibly useful type of diagnostic that isn't used much in the Scala ecosystem. There's also hint diagnostics, but as far as I know, there aren't used anywhere in the Scala ecosystem.

Thirdly we have the position it occurs in. This is typically a combination of marking the line number and the specific character or span that the error appears on.

These three things more or less make up the core parts of a diagnostic. Without one of these the diagnostic becomes quite a bit less useful.

The diagnostic usefulness scale

So now that we know the core parts of a diagnostic, let's create what I call The Diagnostic Usefulness Scale.

On the far left we have essentially no help at all, and on the far right we have all of the help. Since what we covered make up the core parts of a diagnostic let's just stick them in the middle. Realistically no useful language is going to report less than this. Not reporting any of them, especially the position or the message pretty much makes the diagnostic useless. So it's fair to say that in reality our scale really looks something like this:

Since our core parts now move all the way over to the left you're hopefully focusing on the big area in the middle. I'd like to take a look at three questions based off this.

Where is Scala on this scale?
What else is on this scale?
Where are other languages on this scale?

How does the diagnostic actually get to your editor?

So before we can really dive into where Scala exactly is on this scale we need to take a deeper look at both the actual structure of a diagnostic inside of Dotty and also the journey it takes from the compiler all the way to your editor. It's also important to realize that the way a diagnostic is reported wildly depends on quite a few things:

Are you in the REPL?
Are you using X build tool?
Are you using a build server?
Are you using IntelliJ? Metals? Ensime?
Or some combination of the above?

For the purpose of this article we'll pretend that we have the below tool-chain:

We'll start in the compiler and we'll take a look at some places we actually do some reporting. First we'll look here in Checking.scala:

if (param.is(Erased))
  report.error("value class first parameter cannot be `erased`", param.srcPos)
else
  for (p <- params if !p.is(Erased))
    report.error("value class can only have one non `erased` parameter", p.srcPos)

Notice how it's just reporting a string and a position. That's it. It's a super quick way to report an error and looking around the Dotty code base I'd say this is the most common way I see errors reported. There are instances however that Messages are reported as well. You can see an example here in Checking.scala:

def fail(msg: Message) = report.error(msg, sym.srcPos)

def checkWithDeferred(flag: FlagSet) =
  if (sym.isOneOf(flag))
    fail(AbstractMemberMayNotHaveModifier(sym, flag))

You can see what AbstractMemberMayNotHaveModifier looks like, but it's one of many Messages that Scala has. Here's what the Message class looks like:

abstract class Message(val errorId: ErrorMessageID) { self =>

  protected def msg: String

  def kind: MessageKind

  protected def explain: String

  protected def msgSuffix: String = ""

  def canExplain: Boolean = explain.nonEmpty

  private var myMsg: String | Null = null
  private var myIsNonSensical: Boolean = false

  private def dropNonSensical(msg: String): String = {...}

  def rawMessage = message

  @threadUnsafe lazy val message: String = dropNonSensical(msg + msgSuffix)

  @threadUnsafe lazy val explanation: String = dropNonSensical(explain)

  def isNonSensical: Boolean = { message; myIsNonSensical }

  def persist: Message = {...}

  def append(suffix: => String): Message = mapMsg(_ ++ suffix)

  def mapMsg(f: String => String): Message = {...}

  def appendExplanation(suffix: => String): Message = {...}

  def showAlways = false

  override def toString = msg
}

I've taken out all the comments to shorten up the snippet, but you should get the idea. Pretty much everything inside of here is focused around the message of the diagnostic that you end up seeing as an end user.

Now let's take a look at how an actual Diagnostic is structured in Dotty.

class Diagnostic(
  val msg: Message,
  val pos: SourcePosition,
  val level: Int
) extends Exception with interfaces.Diagnostic:
  private var verbose: Boolean = false
  def isVerbose: Boolean = verbose
  def setVerbose(): this.type =
    verbose = true
    this

  override def position: Optional[interfaces.SourcePosition] =
    if (pos.exists && pos.source.exists) Optional.of(pos) else Optional.empty()
    
  override def message: String =
    msg.message.replaceAll("\u001B\\[[;\\d]*m", "")

  override def toString: String = s"$getClass at $pos: $message"
  override def getMessage(): String = message
end Diagnostic

Notice how the Diagnostic actually holds the Message and then you also see the other main parts that we talked about earlier with the pos and the level. So right here we again have the core parts of a diagnostic.

Now already at this point the way this move forwards depends on the reporter. Basically you could look at it in one of two ways:

Well why Zinc? Where does Zinc come into the picture? If you look into the Dotty codebase you'll notice a directory, sbt-bridge:

❯ tree -L 5 sbt-bridge/
sbt-bridge
├── resources
│  └── META-INF
│     └── services
│        └── xsbti.compile.CompilerInterface2
└── src
   ├── dotty
   │  └── tools
   │     └── xsbt
   │        ├── CompilerBridge.java
   │        ├── CompilerBridgeDriver.java
   │        ├── DelegatingReporter.java
   │        ├── DiagnosticCode.java
   │        ├── InterfaceCompileFailed.java
   │        ├── PositionBridge.java
   │        ├── Problem.java
   │        ├── ZincPlainFile.java
   │        └── ZincVirtualFile.java
   └── xsbt
      ├── CachedCompilerImpl.java
      ├── CompilerClassLoader.java
      ├── CompilerInterface.java
      ├── ConsoleInterface.java
      ├── DottydocRunner.java
      └── ScaladocInterface.java

Well what is this? If you look into the Zinc README.md you'll see the following note:

The compiler bridge classes are loaded using java.util.ServiceLoader. In other words, the class implementing xsbti.compile.CompilerInterface2 must be mentioned in a file named: /META-INF/services/xsbti.compile.CompilerInterface2.

And that's exactly what we're seeing here. The Dotty codebase actually implements ComplerInterface2 so that anything using Zinc can basically communicate with Dotty. The xsbt directory is for the legacy version of Zinc, but the dotty.tools.xsbt is the new one that we're concerned about. So if we have a Diagnostic in Dotty we need to look at how this is represented as it starts flowing and being reported through this bridge. To do that we need to take yet another step back and look at the interfaces where this all starts. These interfaces are actually a module in the sbt codebase called util-interface. In there you can find Problem.java. That Problem looks like this:

public interface Problem {
  String category();

  Severity severity();

  String message();

  Position position();

  // Default value to avoid breaking binary compatibility
  /**
   * If present, the string shown to the user when displaying this Problem. Otherwise, the Problem
   * will be shown in an implementation-defined way based on the values of its other fields.
   */
  default Optional<String> rendered() {
    return Optional.empty();
  }
}

Again, we'll notice the core parts of what we talked about earlier, the message, the position, and the severity level. We do have a couple extras here, the category and more importantly rendered(). Note that rendered here will be important when we talk about some extra information that Dotty shows when running your code in a REPL or through your build tool, but not through BSP. There is a difference. If we look back at the little flow chain graph that we said we're following we can actually label the arrows for each "protocol" or "interface" that is defining how these things are being transferred. When labeled it looks like this:

So we know what Problem looks like when it's passing through the sbt-bridge, which leaves us looking at what it looks like in the Language Server Protocol (LSP) and the Build Server Protocol (BSP). BSP was modeled after LSP so the Diagnostic in BSP should be a replica of what's in LSP. Here's what it looks like in LSP:

export interface Diagnostic {
	/** The range at which the message applies.*/
	range: Range;
	/**
	 * The diagnostic's severity. Can be omitted. If omitted it is up to the
	 * client to interpret diagnostics as error, warning, info or hint.
	 */
	severity?: DiagnosticSeverity;
	/** The diagnostic's code, which might appear in the user interface. */
	code?: integer | string;
	/** An optional property to describe the error code. */
	codeDescription?: CodeDescription;
	/**
	 * A human-readable string describing the source of this
	 * diagnostic, e.g. 'typescript' or 'super lint'.
	 */
	source?: string;
	/** The diagnostic's message. */
	message: string;
	/** Additional metadata about the diagnostic. */
	tags?: DiagnosticTag[];
	/**
	 * An array of related diagnostic information, e.g. when symbol-names within
	 * a scope collide all definitions can be marked via this property.
	 */
	relatedInformation?: DiagnosticRelatedInformation[];
	/**
	 * A data entry field that is preserved between a
	 * `textDocument/publishDiagnostics` notification and
	 * `textDocument/codeAction` request.
	 */
	data?: unknown;
}

At this point you should be able to assume exactly what I'm about to say. We again have the main things we looked at before with the range, severity, and the message. However we're now introduced to some new things we haven't really seen yet. We'll dig into these further later but take note of the code, relatedInformation and data fields.

Let's pretend we're using sbt now as our Build server. After the Dotty Diagnostic has been reported via the bridge and is being represented as a Problem internally for sbt it needs to then be reported over BSP. This means it again it gets translated, this time into the BSP Diagnostic. You can see this happen here in the BuildServerReporter.scala:

protected override def publishDiagnostic(problem: Problem): Unit = {
  for {
  id <- problem.position.sourcePath.toOption
  diagnostic <- toDiagnostic(problem)
  filePath <- toSafePath(VirtualFileRef.of(id))
} {
  problemsByFile(filePath) = problemsByFile.getOrElse(filePath, Vector.empty) :+ diagnostic
  val params = PublishDiagnosticsParams(
    TextDocumentIdentifier(filePath.toUri),
    buildTarget,
    originId = None,
    Vector(diagnostic),
    reset = false
  )
  exchange.notifyEvent("build/publishDiagnostics", params)
}

Notice the last line where the build/publishDiagnostics event is being emitted. This is how sbt as a build server is sending the diagnostics to whatever client is current attached. In our example this client is Metals. Once Metals receives it it will do a few things, like possibly some de-duplication, but you'll then see it turn the BSP diagnostic into an LSP one here:

private def publishDiagnostics(
    path: AbsolutePath,
    queue: ju.Queue[Diagnostic],
): Unit = {
  if (!path.isFile) return didDelete(path)
  val current = path.toInputFromBuffers(buffers)
  val snapshot = snapshots.getOrElse(path, current)
  val edit = TokenEditDistance(
    snapshot,
    current,
    trees,
    doNothingWhenUnchanged = false,
  )
  val uri = path.toURI.toString
  val all = new ju.ArrayList[Diagnostic](queue.size() + 1)
  for {
    diagnostic <- queue.asScala
    freshDiagnostic <- toFreshDiagnostic(edit, diagnostic, snapshot)
  } {
    all.add(freshDiagnostic)
  }
  // Removed some de-duplication stuff here to save room
  languageClient.publishDiagnostics(new PublishDiagnosticsParams(uri, all))
}

Finally, the last line you see Metals publishing this diagnostic to the language client, which in our case is your editor. So as you can see, the journey from the compiler actually reporting the error goes through quite a few places and representations before it actually hits your eyes.

Where is Scala on the Scale

Now that we've seen the structure of the actual Scala diagnostic coming from the compiler we can sort of gauge where Scala is at on our diagnostic usefulness scale. I'd probably put it around here:

But hold on a second. Maybe this isn't fair as there are a few extra things that Scala 3 does. Let's take a look at these. We'll take a simple piece of Scala code that as you know, won't work:

trait Foo:
  def hello(): Unit
  def goodBye(): Unit

class Greeting extends Foo

We'll use scala-cli and run this snippet. You'll see the following output:

Compiling project (Scala 3.2.0, JVM)
[error] ./demo.scala:5:7: class Greeting needs to be abstract, since:
[error] it has 2 unimplemented members.
[error] /** As seen from class Greeting, the missing signatures are as follows.
[error]  *  For convenience, these are usable as stub implementations.
[error]  */
[error]   def goodBye(): Unit = ???
[error]   def hello(): Unit = ???
[error] class Greeting extends Foo
[error]       ^
Error compiling project (Scala 3.2.0, JVM)

Notice that the error message actually has stub implementations that you could copy and paste into your code to make the code compile. This is a helpful little message.

Let's look at another one:

trait Foo[A]:
  def foo: Int

def foo =
  second[String]

inline def first[A]: Int =
  compiletime.summonInline[Foo[A]].foo

inline def second[A]: Int =
  first[A] + 1

This one is a bit more interesting and we'll show the error it produces in a few different ways. Firstly, in your editor you'll just see:

No given instance of type Foo[String] was found

Same if you run it with scala-cli:

Compiling project (Scala 3.2.0, JVM)
[error] ./demo.scala:5:3: No given instance of type Foo[String] was found
[error]   second[String]
[error]   ^^^^^^^^^^^^^^
Error compiling project (Scala 3.2.0, JVM)
Compilation failed

But now notice the difference if you run it with scala directly, sbt, or Mill:

-- Error: /Users/ckipp/Documents/scala-workspace/scripts/demo.scala:5:8 --------
 5 |  second[String]
   |  ^^^^^^^^^^^^^^
   |  No given instance of type Foo[String] was found
   |----------------------------------------------------------------------------
   |Inline stack trace
   |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   |This location contains code that was inlined from demo.scala:8
 8 |  compiletime.summonInline[Foo[A]].foo
   |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   |This location contains code that was inlined from demo.scala:8
11 |  first[A] + 1
   |  ^^^^^^^^
    ----------------------------------------------------------------------------
1 error found
Errors encountered during compilation

Now we get some extra information about the inlined code and where that inline code is located. This is a very simple example, but imagine a much larger code base. This could be incredibly useful! The difference here is whether or not the report is using the rendered() method to get the diagnostic message or just grabbing the actual message.

So with this new information maybe it's a bit fair to bump Scala over a bit on our Diagnostic Usefulness Scale:

What else is on the scale?

At this point you're probably starting to wonder what else is actually on this scale? What other languages have more to offer than what we're seeing. In order to explore this we'll again dive into a small code sample. We'll start with Scala:

object Main:
  val greeting = "hello"

  greeting = "hi"

I'm using this snippet both for its brevity but also because as minimal as it is, it'll show exactly what we want. Let's look at what we get using VS Code with Metals for this diagnostic:

This of course tells you the issue, but if you're like me you might be asking a few more questions when looking at this diagnostic:

Where is the original assignment?
Can I get a more detailed explanation of this error?
What are my alternatives to fix this?

None of these questions are really answered by this diagnostic. But let's take a look at another language. We'll use Rust with essentially the same snippet:

fn main() {

    let greeting = "hello";

    greeting = "hi";
    
}

In your editor (using VS Code again) you'll see the following:

Now let's revising those questions we had earlier. Firstly, Where is the original assignment? Notice that this is actually included in the Rust diagnostic:

That's pretty cool. Not only does it tell me, but I can just click and jump to it. Perfect. Again, this is a tiny example, but imagine if this was a huge code base. What about the next question, Can I get a more detailed explanation of this error?

So there is a short error message which is pretty clear, but notice the E0384 here. What is this? It's also at this point that I'm curious if you've ever noticed this in Scala. When running your code have you ever seen the following:

-- [E003] Syntax Deprecation Warning: examples/003_DeprecatedWithOperatorID.scala:4:18 
4 |  type A = Double with Int
  |                  ^
  |                with as a type operator has been deprecated; use & instead
1 warning found

Notice the [E003] here. I'm going to guess that most people just gloss right over this, but it's actually important... but unused. In Scala 3 some diagnostics have a unique ID attached to them. In this case usage of with is deprecated and the unique ID signifying that is 003. Well where is 003 documented? It's not. It's also not used literally anywhere at the moment in the Scala ecosystem (as far as I know). But let's look at how it is used in Rust. In the Rust diagnostic this is actually a clickable link that will bring you to the Rust error codes index. If we click on E0384 we're brought to a section with a more detailed explanation of the error message and a runnable code snippet to reproduce it. Again, this is a simple error that we've chosen, but imagine if this was a more involved error message that might be hard for users to understand. Think of newcomers to a language that might not be familiar with all the lingo and concepts a language has to offer. How incredibly useful this could be. There currently is no official thing like this for Scala.

The final question we asked was, What are my alternatives?

Notice that in this diagnostic it even includes a possible way to fix it. This is similar to what we saw up above with the stubs for Scala, but wait, there's a bonus here. If you click the link to go to where the first assignment of greeting is you'll notice there's a quickfix attatched to the position that will actually just fix it for you by adding a mut.

There is a common pattern you'll see in some languages like Scala where the compiler knows exactly a way to fix your problem, but doesn't expose this information in an structured way. You're always left with the information just in a string with other information. Rust seemingly does a great job at this by not only returning the information, but returning it in a structural way that tools and therefore users can benefit from.

Let's jump back to our scale. If we were to drop Rust on here I'd probably drop it much further to the right than Scala.

It's all about Structure

At this point hopefully you're wondering why this all is. Why doesn't Scala 3 have some of these niceties. The answer to this is sort of twofold. Firstly there's a mismatch in priority between compiler development speed (meaning those working on the compiler) vs the desired to have more structured output. It's all about structure, and I'll show you why.

Up above we went through the full flow and structure of a Dotty diagnostic. Let's now look at the structure of a Rust one and compare. The core Diagnostic can be found here in json.rs

struct Diagnostic {
    /// The primary error message.
    message: String,
    code: Option<DiagnosticCode>,
    /// "error: internal compiler error", "error", "warning", "note", "help".
    level: &'static str,
    spans: Vec<DiagnosticSpan>,
    /// Associated diagnostic messages.
    children: Vec<Diagnostic>,
    /// The message as rustc would render it.
    rendered: Option<String>,
}

This looks pretty familiar and more or less what you'd expect. Note that we do have a possible DiagnosticCode here that is getting forward, so let's look at what that looks like:

struct DiagnosticCode {
    /// The code itself.
    code: String,
    /// An explanation for the code.
    explanation: Option<&'static str>,
}

This looks strangely similar to the LSP one! I'll also again bring up the Rust Error index that has a full listing of the Diagnostics that are reported by Rust, with explanations, and runnable code samples.

But let's keep going because there are some more things in the Diagnostic which are new to use. The next one is the DiagnosticSpan:

struct DiagnosticSpan {
    file_name: String,
    byte_start: u32,
    byte_end: u32,
    /// 1-based.
    line_start: usize,
    line_end: usize,
    /// 1-based, character offset.
    column_start: usize,
    column_end: usize,
    /// Is this a "primary" span -- meaning the point, or one of the points,
    /// where the error occurred?
    is_primary: bool,
    /// Source text from the start of line_start to the end of line_end.
    text: Vec<DiagnosticSpanLine>,
    /// Label that should be placed at this location (if any)
    label: Option<String>,
    /// If we are suggesting a replacement, this will contain text
    /// that should be sliced in atop this span.
    suggested_replacement: Option<String>,
    /// If the suggestion is approximate
    suggestion_applicability: Option<Applicability>,
    /// Macro invocations that created the code at this span, if any.
    expansion: Option<Box<DiagnosticSpanMacroExpansion>>,
}

Some of this is familiar, but we do have some new interesting things. Most interesting to me is the suggested_replacement. Remember earlier when we talked about the compiler knowing how to actually fix this at times and maybe putting it into the message. Here's a great example of how this could be returned structurally. You'll also see there is a suggestion_applicability that explains the "confidence" in the correctness of the suggestion. I'll make a note here that this isn't just Rust doing this. This goes back way further. For example we can look at Fix-it hints from Clang. Here is a snippet from the Clang docs:

"Fix-it" hints provide advice for fixing small, localized problems in source code. When Clang produces a diagnostic about a particular problem that it can work around (e.g., non-standard or redundant syntax, missing keywords, common mistakes, etc.), it may also provide specific guidance in the form of a code transformation to correct the problem.

Note that in the DiagnosticSpan there is also is_primary in order to indicate if one span is the "primary" span or not. This is helpful when reporting multiple different diagnostic spans tied to one diagnostic.

Finally take note that in the Diagnostic there is another new field, children. This children shows that a single Diagnostic can also hold other diagnostics. Another thing that is currently unrepresentable in Dotty.

Why did you even look into this?

At this point you may be wondering why? What got you focused on this? So join me in going back a while and looking at a real-world tooling use-case that starting me down this path of exploring what diagnostics actually look like in Dotty, and then coming to the realization that we can do much better. Back in early 2021 Gabriele Petronella sent in a PR to Metals to take an error from the compiler about a missing import and create a Code Action out of it. This code action feature is the same type of feature you saw up above with Rust and the Quick Fix example. During this process this comment is what sparked it for me:

The main issue here is that in Metals in order to offer certain Code Actions we need to first recognize what type of diagnostic is being reported. As you saw earlier, there is a unique ID, but it's not forwarded at all. So we do things like this:

object TypeMismatch {
  private val regexStart = """type mismatch;""".r
  private val regexMiddle = """(F|f)ound\s*: (.*)""".r
  private val regexEnd = """(R|r)equired: (.*)""".r

  def unapply(d: l.Diagnostic): Option[(String, l.Diagnostic)] = {
    d.getMessage().split("\n").map(_.trim()) match {
      /* Scala 3:
       * Found:    ("" : String)
       * Required: Int
       */
      case Array(regexMiddle(_, toType), regexEnd(_, _)) =>
        Some((toType.trim(), d))
      /* Scala 2:
       * type mismatch;
       * found   : Int(122)
       * required: String
       */
      case Array(regexStart(), regexMiddle(_, toType), regexEnd(_, _)) =>
        Some((toType.trim(), d))
      case _ =>
        None
    }
  }
}

This recognizes situations where maybe you changed a method and now the explicit return type doesn't fit. So it offers you a code action to quickly change the return type to the new expected type. Now this probably seems a bit crazy, but when you have nothing but a string to work with, tooling is left to do gross things like this. Hacks like this are flaky and error-prone not only because they are regex, but because error messages can change. Imagine how much easier it'd be if we had access to a unique ID to know what type of diagnostic we were dealing with. We could rid tooling of these types of hacks. Now imagine if they even came with a suggested fix! Crazy talk.

What work is being done to improve this?

The good news is that there has been some progress made in this area. Up above we saw the structures that were used all the way from the Dotty compiler to your editor. These were a bit older since in recent months they've changed slightly. For example the new Problem.java got a couple new fields:

/**
 * The unique code attached to the diagnostic being reported.
 *
 * <p>NOTE: To avoid breaking compatibility we provide a default to account for older Scala
 * versions that do not have codes.
 */
default Optional<DiagnosticCode> diagnosticCode() {
  return Optional.empty();
}

/**
 * The possible releated information for the diagnostic being reported.
 *
 * <p>NOTE: To avoid breaking compatibility we provide a default to account for older Scala
 * versions that do not have the concept of "related information".
 */
default List<DiagnosticRelatedInformation> diagnosticRelatedInformation() {
  return Collections.emptyList();
}

We now have the ability to have a DiagnosticCode just like we saw in Rust and also the ability to nest diagnostics with the new DiagnosticRelatedInformation. This is great, but it also took a crazy amount of changes just to get the code forwarded all the way to the editor. Take a look:

In sbt (and zinc) - https://github.com/sbt/sbt/pull/6874
In Dotty - https://github.com/lampepfl/dotty/pull/14965
More Dotty - https://github.com/lampepfl/dotty/pull/15728
Changes in BSP - https://github.com/build-server-protocol/build-server-protocol/pull/320
In sbt (server) - https://github.com/sbt/sbt/pull/6998
In Bloop - https://github.com/scalacenter/bloop/pull/1750
In Mill - https://github.com/sbt/sbt/pull/6874
In Metals - https://github.com/scalameta/metals/pull/4239

All of this just to get a single thing reported. Here it is, in all of its glory:

"diagnostics": [
  {
    "range": {
      "start": {
        "line": 9,
        "character": 15
      },
      "end": {
        "line": 9,
        "character": 19
      }
    },
    "severity": 1,
    "code": "7",
    "source": "sbt",
    "message": "Found:    (\u001b[32m\"hi\"\u001b[0m : String)\nRequired: Int\n\nThe following import might make progress towards fixing the problem:\n\n  import sourcecode.Text.generate\n\n"
  }
],

There are some other tools in the ecosystem that are starting to jump on board with this idea of diagnostics that can just include a fix. For example scala-cli has one that was somewhat recently added to metals by @lwronski that given a using lib directive that is out of date, scala-cli will actually provide a diagnostic not only notifying the user that it's out of date, but offering a quick fix to update it to the latest:

This is a killer feature that I wish more tools would offer, especially the compiler. If you're curious under the hood what the diagnostic for this look like, here you go:

"diagnostics": [
    {
      "range": {
        "start": {
          "line": 0,
          "character": 15
        },
        "end": {
          "line": 0,
          "character": 40
        }
      },
      "severity": 4,
      "source": "scala-cli",
      "message": "com.lihaoyi::os-lib:0.7.8 is outdated, update to 0.8.1\n     com.lihaoyi::os-lib:0.7.8 -\u003e com.lihaoyi::os-lib:0.8.1",
      "data": {
        "range": {
          "start": {
            "line": 0,
            "character": 15
          },
          "end": {
            "line": 0,
            "character": 40
          }
        },
        "newText": "com.lihaoyi::os-lib:0.8.1"
      }
    }
  ],

What we could have

Let's revisit one of the example we had up above.

trait Foo:
  def hello(): Unit
  def goodBye(): Unit

class Greeting extends Foo

The actual diagnostic structure you end up getting from your build server about this looks like this:

"diagnostics": [
    {
      "range": {
        "start": {
          "line": 11,
          "character": 6
        },
        "end": {
          "line": 11,
          "character": 14
        }
      },
      "severity": 1,
      "source": "bloop",
      "message": "class Greeting needs to be abstract, since:\nit has 2 unimplemented members.\n/** As seen from class Greeting, the missing signatures are as follows.\n *  For convenience, these are usable as stub implementations.\n */\n  def goodBye(): Unit \u003d ???\n  def hello(): Unit \u003d ???\n"
    }
  ]

But what if instead of just the message containing all this information the error reported from the compiler looked like this:

"diagnostics": [
    {
      "range": {
        "start": { ... },
        "end": { ... }
      },
      "severity": 1,
      "code": "034",
      "source": "dotty",
      "message": "class Greeting needs to be abstract, since:\nit has 2 unimplemented members.\n/** As seen from class Greeting, the missing signatures are as follows.\n *  For convenience, these are usable as stub implementations.\n */\n",
      "data": {
        "range": {
          "start": { ... },
          "end": { ... }
        },
        "newText": "def goodBye(): Unit \u003d ???\n  def hello(): Unit \u003d ???\n"
      }
    }
  ]

Imagine just being able to click a quick fix and get these stubs automatically without needing to manually copy anything.

Remember the Rust error index? What if we had something like this for Scala? Something that was searchable, had examples, and was easy to use? With the recent obsession with newcomers in Scala 3 I'm quite surprised that this doesn't exist. If you're curious about what this could look like, I actually have a WIP version of one in ckipp01/dotty-error-index. Please feel free to contribute!

Also remember the diagnostic example we showed from Dotty that had inlined code in another box below the main message? What if we used diagnosticRelatedInformation for this instead. By doing that we'd automatically get extra navigation possibilities in editors like VS Code. Here is an example showing that:

What's keeping this from happening?

I slightly touched on this up above in the It's All About Structure section, but I believe the major things preventing some of this from happening are twofold.

Firstly, the compiler team has to actually buy into this idea. While I can say from experience that some totally have! But I can also say that some totally haven't. There was a big push like 6 years ago to add missing error message IDs and to have stable identifiers for each message. I commend this effort so early on. It made very slow progress for a while until in 2020 it was abruptly stopped:

The architecture for errors described in this issue proved to be heavy and impractical. The migration process frequently interferes with in-flight PRs and the benefits are marginal. We should find a more light-weight way to represent errors.

This same mindset is seemingly still around where the idea of forcing stable identifiers here is a chore and not worth the effort. You can see some conversations around this in various places like here.

While you can make the argument early on that pushing for stable identifiers like this may have slightly hindered the speed of getting things merged in, this mindset is also putting structured diagnostic reporting as a hindsight instead of something baked right into the language. It's a shame that while the error messages themselves have greatly improved in Scala 3, the structures is essentially identical to what it's been for as long Scala has been around. That's maybe alright for a research language, but it doesn't cut it in the industry where developers are expecting a level of help and structure that they get with languages like Rust.

I mentioned early in the articles that the most common way you'll see an error reported in Scala 3 is simply by taking a string and a position and reporting it like so:

if (param.is(Erased))
  report.error("value class first parameter cannot be `erased`", param.srcPos)
else
  for (p <- params if !p.is(Erased))
    report.error("value class can only have one non `erased` parameter", p.srcPos)

While this is by far the quickest way to report, it's only good for that. Under the hood it creates a -1 error code, which leaves us with an unusable error ID. There needs to be a push to force stable identifies and care taken to ensure unique error ids don't just disappear in favor of this style of reporting.

The second main blocker here is simply having someone focus and work on it. While it's easy to point to the community and say, "someone just needs to step up and do it", changes like introducing diagnosticRelatedInformation require a decent amount of refactoring, primarily due to this trying to be hacked in after the fact instead of being baked into reporting from the start. So knowledge of the actual compiler internals are necessary for this. Thankfully there are ongoing efforts like the Scala 3 Compiler Academy and also people like @smarter that seem behind this idea and are willing to help. This is incredibly important as without support from the actual compiler team, there's no way this will get into the compiler.

Conclusion

If you made it this far, bravo! I'm hoping that you now have a bit more clarity on what makes up a diagnostic, how that diagnostic gets to your editor when using the tooling chain we outlined above, and also an eagerness to see more structured and actionable diagnostics coming from the compiler. The entire ecosystem will benefit from this, from newcomers to experts alike. If you'd like to follow along with the ongoing work in this area, you can follow the Progress Issue about this in the Dotty repo.

Cheers, and thanks for following along.

Music of the Month - October 2022

Chris Kipp — Sun, 6 Nov 2022 00:00:00 +0100

Music of the Month - October 2022

I missed September. I was on holiday for most of, was traveling quite a bit, and then got sick. Life happens. On to October. This last month I was also all over for music. Honestly no one album really dominated, so I decided to pick one of my favorites that I recently got on vinyl. Enjoy this month.

Album of the Month

There are few bands out there that have had more of an impact on me than This Will Destroy You. I discovered them early on in college and was just floored by the amount of emotion they put into their music and the sheer energy you feel when listening to them. While this album isn't my favorite of theirs it was one I listened to a lot this past month since I ordered it a long time ago to my parents house, but haven't been back to the States to retrieve it for years. Since this last month brought me back to the US, I was finally able to bring it home with me. The album of the month this month is New Others - Part Two. It's definitely an album you'll want to listen to in its entirety and ideally front to back.

New Others Part Two by This Will Destroy You

Song of the Month

This months "Song of the Month" came by surprise. I came across a song on my weekly Spotify playlist that I really enjoyed. It went on to my seasonal playlist (I make one every season and have been since 2015), and I found myself playing it over an over. When I looked up the band, Casino Versus Japan, I found out he's located less than an hour away from where I grew up in Wisconsin. His song Marilyn Set Me Free is the "Song of the Month" for me this month. Super chill song that sort of makes you feel as if you're floating and grooving all at the same time. I want a full album of this vibe.

Hitori + Kaiso: 1998 - 2001 by Casino Versus Japan

See you next month

Scala Links

Chris Kipp — Fri, 21 Oct 2022 00:00:00 +0200

Scala Links

The amount of times I need to look this stuff up drives me mad. So here's a collection of Scala-related links to the various official pages, documentation sites, repos, etc. Useful if you need to periodically go to all these places like I do for contributing, reference, or desire for documentation. There a lot. Some would say too many.

Main Websites

API / Reference Docs

Repos

Music of the Month - August 2022

Chris Kipp — Wed, 31 Aug 2022 00:00:00 +0200

Music of the Month - August 2022

A very busy last month. I found myself gravitating toward a lot of my favorites, but also enjoying some new albums I stumbled across. I hope you enjoy what I have for you this month.

Album of the Month

For a while I listed to a ton of lofi. I've always enjoyed music without lyrics, and lofi often fit my mood when I wanted something chill to code or relax to. One of my favorite older lofi albums was the Harbor LP by tomppabeats. However, they haven't released a new full-length album in years. This was part of the reason I was so excited to see a new album from them drop this month. Habits is the album of the month for me this month.

Habits by tomppabeats

Song of the Month

Last month I listed a Moya by Godspeed You! Black Emperor as my song of the month. In part, that led me to listen to a ton more of GY!BE this past month. In general, I think Efrim Menuck is just a genius. If you're a GY!BE fan you may have also heard of A Silver Mt. Zion, which is project of Menucks with some of the same members of GY!BE. The song 13 Angels Standing Guard 'Round The Side Of Your Bed is one of my favorites of theirs and is the song of the month for me this month.

He Has Left Us Alone But Shafts Of Light Sometimes Grace The Corner Of Our Rooms... by Thee Silver Mt Zion Memorial Orchestra

See you next month

Music of the Month - July 2022

Chris Kipp — Sun, 31 Jul 2022 00:00:00 +0200

Music of the Month - July 2022

So I've finally stepped away from Jazz a bit this past month probably because it was a bit hectic for me. Typically when it's hectic I find myself floating back to more familiar records that bring familiarity and comfort.

Album of the Month

During this past month I watched Stranger Things for the very first time, and there are a handful of scenes that play Jim Croce. Growing up my Dad listened to a ton of "oldies" so a lot of these artists from the 60s and 70s remind me of good ol days on the farm where I grew up working with my Dad. Therefore, the album of the month for me is the classic You Don't Mess Around With Jim.

Song of the Month

It's hard to explain Godspeed You! Black Emperor, GY!BE, to someone that's not familiar with them. My "Music of the Month" series hasn't really shown it, but one of my most-listened to genres is post-rock. Every since I discovered Explosions in the Sky in high school, they genre has meant a lot to me. GY!BE has some incredible albums. They're slow burners, so some of them really take time to get into and maybe even more time to appreciate. 20 minute songs are the norm. This month one that I've found myself listening to over and over is Moya from the record Slow Riot from New Zero Kanada. Again, if you've never listened to them before, take a seat, make sure to listen to the whole thing, and enjoy.

Slow Riot for New Zero Kanada by Godspeed You! Black Emperor

See you next month

What I Learned in a Year of Podcasting

Chris Kipp — Mon, 18 Jul 2022 00:00:00 +0200

What I Learned in a Year of Podcasting

What I learned in the first year of starting a podcast, and commonly asked questions.

My hope is that this post will serve two purposes:

Be a reflection on starting and doing a podcast for a year on what I'd consider a pretty niche topic, Scala developer Tooling.
Be a collection of answers to commonly asked questions I get about starting a podcast, in hopes to help others on their journey.

Feel free to jump down to the section that interests you the most, or just keep reading.

How do you pick a topic?
How do you even make a podcast?
What do you need?
What does it cost?
How much time does it take?
How do you invite guests?
How do you control quality?

How do you pick a topic

When I first wanted to start a podcast, it was for a variety of reasons that probably differ a bit per individual, but one of the main reasons for me was that throughout Covid my interaction with people was almost entirely limited to online activity. It was really during that time for me that I got even more heavily involved in the Scala community, and starting really connecting with people that I was interacting with on a daily basis, digitally. With that being said, they were just drive-by interactions, short chats on Discord, or even just thumbs up on GitHib issues. I often heard others fondly talk about chatting with friends at conferences, skipping a session and hacking on something together, or just in general having conversations about the interests you share that not many of your direct in-person circles do. I longed for that experience, and thought maybe starting a podcast talking about something I deeply cared about and spent an exuberant amount of time on was a good idea.

The practical advice I pull out of that is that if you're going to start a podcast, start it about something you care about. Start it about something you don't mind spending hours of your weekend time researching about, or re-listening to recorded episodes as you edit, or search around for new topics to talk about. We'll get into this more later, but it also helps to be a part of the community of the thing you're going to be speaking about. This helps tremendously with finding new guests, hearing about new interesting topics you can cover, and to help spread the word around about this new show that you're putting effort into. You also want to make sure there is enough substance and options to talk about in this topic. This is all made easier when you're actively participating in that community.

It's also worth mentioning that when thinking about your topic, you should also put some thought into your audience. While it may seem obvious, if you pick a more niche topic you need to know that you're working with a much smaller pool of people that will probably engage in your podcast in a very different way. Here are some things I considered when thinking about my potential audience when I started Tooling Talks.

Tooling is a more niche topic than Scala in general, so I'm already limiting my audience compared to doing a more generic Scala podcast.
Tooling is something most new developers don't think or really care that much about if it's doing it's job. How can I make sure this podcast is also something a newcomer to the ecosystem could listen to and gain from?
Anecdotally, I see less male-identifying individuals in Scala tooling than other ecosystems, how can I ensure my podcast reaches a more diverse audience than if I otherwise wouldn't think about this.

These are still questions I often think about when I look at the trends in listeners, the amount of listeners I have, or when inviting new guests.

How do you even make a podcast

Thankfully, this is a topic pretty well covered out there, so I won't go too much into the technical details, but will give an overview of some of the choices I made here.

I find that most of the confusion about a podcast comes from distribution, not the actual "making" the podcast. Once you have a recording, that recording needs to somehow be distributed to the various services that people listen to. Your podcast is really just an rss feed that needs to be hosted so that people can subscribe to or that services like Spotify or Deezer can subscribe to in order to distribute your podcast. Apart form your actual feed you'd also need to actual host your audio files in a place the feed can access. You'd also then need to go to every service you want to distribute your podcast to and register your podcast with them so they know to follow your rss feed. While you can do all this manually, it's not something I'd recommend. There are plenty of services that offer to do all of this for you. Apart from just hosting your actual podcast and rss feed, they also make it much easier to distribute your podcast to other services, they offer embedded players for your podcast episodes, and some even offer you a simple landing page to act as a website. I personally use Buzzsprout for this, which as been fantastic. I even have a website with them even though I never link it, since I prefer my own minimal site. There are a bunch of options here, and the search that you should be doing is "what's the best place for me to host my podcast". Since there's abundant articles out there for this, I'll leave that research up to you, but instead leave you with a few things I considered when I chose to use Buzzsprout.

What's the cost? Most of these do cost a bit of money, so what's an actual affordable option.
Which offers a flexible embedded player? Like I mentioned, I wanted to host my own site, so this mattered to me.
Which offered the easiest way to distribute my podcast to the various places, like Spotify, Google podcasts, etc. If I could just click a button, that'd be awesome.
Which is commonly reviewed as one of the easiest to use? I don't want some no-name cheaper alternative that ends up sucking.

What do you need

If we wanted to completely simplify this section, you could just say:

A way to record yourself and your guest
A place to upload your recordings and your rss feed

However, that's pretty radically over-simplified. I'll break down what I chose to get and why.

Recording

Arguably the most important thing in making a podcast is audio quality. I have an entire section on this down below, but this often translates to three things:

Getting a better microphone to connect to your recording device
Using recording-specific software to help with noise reduction and quality
Using some type of post-production software to edit your podcast

Having an external microphone will almost always sound better than using your built-in mic. I think there is some confusion here where people assume that this is always very expensive, but there are mics in the under $100 range that will greatly improve your audio quality. The nice thing is that investing in a mic here doesn't just need to be for your podcast, but can also benefit you on work calls, recordings, or anything else you might want to use a mic for. I wouldn't recommend going out right away and dropping a ton of money here, but getting an affordable external mic here will help. Keep in mind that even with a nice mic, a crappy environment or incorrect usage can totally negate any gains here. Again, we'll talk about controlling quality down below.

There are multiple ways to record your podcast. Some of the most obvious are just to recording your Google Meet call, your Zoom audio, or whatever software you're choosing to use. This is for sure the cheapest, but not the highest quality. There is also software specifically made for recording which can do things like recording on separate audio tracks or local buffering that can really help the quality of your sound and ease post-production. I use Squadcast for this and have had a great experience.

For editing I use Audacity. There are a ton of other more-advanced tools out there for this, but if you want a tool that can normalize your audio, easily split and join clips, and make transitions with, then Audacity is great. It's also open source and free, which was a nice bonus for me. There is actually a ton of useful articles about using Audacity on the Buzzsprout blog.

Hosting and Distributing

I already covered this a bit up above, but I host my podcast on Buzzsprout. One of the main features I wanted was to be able to embed an episode. Here's what that looks like for me:

I also wanted to be able to style it to match my website, which was pretty easy to do. Since I wanted to host these, I also needed a website, which I was able to make myself(with Scala) and host on Vercel. Again, if you don't want to do this part, it's much easier to just use the built-in website that many podcast hosting sites will provide for you.

Since I use Buzzsprout, distribution was also pretty easy. Most of them were just button clicks, but there were a couple more that involved going to their respective sites and registering, and sometimes submitting the feed manually. I have my podcast being circulated on around 20 different site mostly thanks to how easy the process was.

I haven't mentioned it yet, but if you host your own site, you obviously need a domain name. Here is also the part you could talk some about branding, using a dedicated Twitter account, etc. I made some opinionated choices here to not really brand it too separately from myself. Mainly because I didn't want to have to deal with it feeling like a "brand" or separate project, but just an extension of myself. This is why I don't have a separate dedicated Twitter account for Tooling Talks, a separate way to support the show, and why the branding on the site looks very similar to my personal site.

What does it cost

So this section can be radically different depending on how much you want to invest in this. You could get by with a pretty minimal budget here if needed, but to give you a real idea, I've outlined the setup and regular costs for Tooling Talks down below.

Setup costs

Cost	Item
$14.16	Domain name and ICANN fee - Purchased for one year via Namecheap
$0	Hosting on Vercel - The free account is fine for this
$24.63	Logo for podcast - Ordered via Fiverr
€255	Creative Bundle (mic and interface) - Included a RØDE NT1-A and a Focusrite Scarlett Solo. Both of these I can use for streaming and for work calls.
€34.99	Mic arm - Not necessary but depends on your mic and greatly helps with placement.

Monthly costs

Cost	Item
€7	Avatar for guest - Again, from the same artist on Fiverr. Price fluctuates a bit due to tipping.
$12	Monthly cost to host on Buzzsprout - They do have a free account, but it only keeps your episodes for 90 days.
$20	Monthly cost for Squadcast - This is the cheapest tier they have.

One way to offset these costs is to provide a way for people to support the show financially. This can be through Patreon, ko-fi, or even GitHub Sponsors. I used ko-fi for a while, but after about 3/4 of the year of having it I decided to just close it down. I had gotten a couple one-off gifts there, but I also felt pressured to add regular updates their to justify it. Since I didn't want to do that, I just opted for my personal GitHub Sponsors page since that's where most of my activity can be seen anyways.

How much time does it take?

This is arguably as important or more important than the actual monthly financial cost to have a podcast. Depending on how much you value your time, this is where the cost really comes in, and also why it's important that you enjoy the topic you chose in the first place. While I've found that it can vary a bit on the guest, the quality of the recording, etc, here is how I end up breaking down the expected time an episode will take me. Note that some of these are hard to quantify since some of my guests I know really well and it's just a simple message, where others I've had to hunt down their contact info by reaching out to mutual contacts, cold emailing, Twitter DMing, etc. The same goes for preparing since if I'm going to interview someone about Metals there is way less prep needed than for example when I spoke to Rebecca about Unison. So consider all of the vales down below as averages.

Time	Activity
1 Hour	Figuring out how to contact and actually inviting, and fully planning the episode (average)
3 Hours	Playing around with the project we'll talk about or doing research on it
1.5 Hours	The actual recording of the episode
2 Hours	Editing the podcast. It typically takes me about 2 times the episode length to edit it
.5 Hour	Writing the intro, update the website, communicating with the avatar artist

8 hours per episode is probably a pretty accurate estimate of the average time I spend on a single episode. Originally, I was really surprised by this as I didn't really expect an episode to take this long. It hopefully puts it into perspective the amount of time that goes into making episodes. NOTE, my episodes are long. They don't need to be this long. If I'd aim for like a 45 min average, this would shave off hours of time.

How do you invite guests?

This is something I still feel like I haven't nailed down. In theory the ideal way this goes for me is:

Find someone I'd really like to do an episode with.
Shoot a message to the person asking them to be on the show by showing them who's been on previously, and link the process I follow so they know what to expect.
Once accepted I send them a Squadcast invite so they can actually put something on their calendar.
About a week before we interview I'll send them a list of questions likely to appear when we talk.

And that's it. The fist step of identifying a guest in my opinion is pretty easy, but the reaching them is not if they're outside of your network. The guests I've had on the show that I've already interacted with in the past were super easy to invite. I had their contact info, they recognized my name, and they were quick to respond with a yes. Then we go one layer out from that and we have people that I've never really spoken with, but we run in the same circles so they at least recognized my name, or maybe even have heard of the show before. Sometimes these were a bit harder since Twitter DMs are a terrible way to get a hold of someone. Most people you try to contact this way won't even see your message. The difficult part is that sometimes this is the only public way a person has to contact them. Sometimes this an oversight, but sometimes is intentional. This becomes extremely difficult when you're trying to reach someone outside of your network and in a different ecosystem for example. There are a handful of people I've had on my list to have on the show since I've started, and some of them I've never been able to figure out how to contact, as I don't even have a shared connection with them, and some of them have just ghosted me even when I found emails or other forms of contact. I think it's something you just sort of need to accept, that not everyone wants to be on a podcast, and that's ok. If I find the person's contact info and they don't respond, I give it a couple months and then try again. If they don't respond the second time, I move on unless I come across a mutual contact that is willing to make an introduction.

How do you control quality?

This is another one I'm still working on. Especially if you have guests that aren't local to you and you're doing your interviews remotely, there isn't a lot you can control on your guests end. With that being said, this is also a point that shouldn't dissuade your guest from being a guest. I don't want to lose out on a potentially great guest just because they don't have an external mic. There are other tips to help their quality as well. Here are some tips that I send ahead of time to my guests, and also some things I do to help maintain the quality of sound:

Investing in software like Squadcast. Squadcast for sure isn't cheap, but it offers a couple things that really help. Firstly, it buffers locally before upload so even if the internet gets crappy you're not losing sound quality. Secondly, the audio tracks are separated which makes it easier to edit a ticking noise on one side or loud bangs or stuff like that. Investing in some type of software that offers these things are a good way to help your own post-production and also help your guest not stress if they don't have a great internet connection.
Make sure your guest is wearing headphones. This will help with cutting out any echo that might be picked up from their speakers. Any headphones will do. I personally use a pair of Beyerdynamic DT-770 Pros.
Encourage your guest to not sit in a giant empty room. If you're basically sitting in an empty room, your sound will sound like it was recorded in a box. Having stuff around you to dampen the sound really helps.
Encourage your guest to be mindful of loud water bottles or things on their desk. This is an important one that can kill post-production time by having to go through and edit all these noises out. Metal water bottles with screw caps, clicky pens, and all sorts of things someone could play with on their desk can make noise and be picked up by their mic, especially if they are using their built-in mic or a cheap external mic.

I hope this all helps you on your podcast journey, or just gives you some insight into mine. I've really enjoyed talking with all my guests, and have also really learned to appreciate it when I hear quality podcasts since I know first-hand the effort it takes. If you're considering making a podcast, but still have unanswered questions, don't hesitate to reach out.

Thanks for stopping by.

Music of the Month - June 2022

Chris Kipp — Mon, 4 Jul 2022 00:00:00 +0200

Music of the Month - June 2022

I've been seemingly just living in the land of Jazz lately, because this month will again be filled with some incredible Jazzyish tunes. I've also been pleasantly surprised recently at the others in my tech circle that also seem to enjoy jazz. This has been a fun intersection for me.

Album of the Month

Like last month, I made a new discovery this month that blew my mind. There was an album I came across called Box Tree by Skúli Sverrisson and Óskar Guðjónsson. While it's not the focus of this month it's a beautiful album nonetheless, and it introduced me to the playing of Óskar Guðjónsson. While exploring his work I was introduced to ADHD, and was just blown away. Their album Adhd 7 is the album of the month for me this month.

They played a live show for KEXP that was also incredible. It's rare for me to just sit and watch a recorded show in its entirety, but I did for this.

Song of the Month

The song of the month for me comes off of the album Box Tree I mentioned up above by Skúli Sverrisson and Óskar Guðjónsson. The song is For Seconds at a Time, which is the first one I heard off the album. Starts off real simple, and then just coddles you into relaxation when the Sax comes in. Such a beautiful song.

The Box Tree by Skúli Sverrisson, Óskar Guðjónsson

See you next month

Music of the Month - May 2022

Chris Kipp — Fri, 3 Jun 2022 00:00:00 +0200

Music of the Month - May 2022

This month was one of those months were you find a new piece of music and you basically listen to it the entire month. Vega Trails, I love you.

Album of the Month

I discovered this duo this month by total happen chance. Vega Trails is made up of double-bassist and composer Milo Fitzpatrick from Portico Quartet and Jordan Smart from Mammal Hands. The music these two have made in their first album together, Tremors in the Static, is simply magical. I haven't been touched this much by an album in quite some time. The recording if phenomenal, the pieces are passionate, and the range and depth that is created just by the two of them surprises me every time I listen.

Tremors in the Static by Vega Trails

They did a filming of this in St Thomas's Church in London, and it equally as fantastic. Here is a video showing a few of the songs.

Song of the Month

A song that specifically stuck out to me this month was Moving by Backwards by SALES. I've been a fan of SALES for a while now and this is their latest single they've recently put out. It's just good indie dreamy pop.

Moving by Backwards by SALES

See you next month

Music of the Month - April 2022

Chris Kipp — Fri, 6 May 2022 00:00:00 +0200

Music of the Month - April 2022

I missed March for no other reason than I just forgot about it. However, this past month I had a few weeks off, and spent a good amount of time relaxing, thinking about the future, and of course listening a to a lot of music. I found my music all over the place this month, so I honestly could have picked a number of albums and songs. However, here are a couple that stood out to me this month.

Album of the Month

I listen to a fair amount of classical and 4 Seasons by Vivaldi has always been a favorite of mine as cliche as it sounds. It was one of my favorite live performances I've seen when visiting Paris in Saint-Chapelle, and the rendition I'm about to suggest is just incredible. Max Richter is such a gifted composer and beautiful human being. This past month was the ten year anniversary of his Recomposed by Max Richter: Vivaldi - The Four Seasons, which is my album of the month this month.

Due to it being the anniversary he released some new videos of the performance which are fantastic.

Song of the Month

I started to listen to quite a bit of lofi when I started coding, and then led to just me listing to a lot of lofi. I've found a lot of interesting artists out there, but I find Mishaal really interesting since in his songs you'll find a mix of Spanish, Arabic, and English due to his background. The song of the month for me this month is Kid Goku by Mishaal and Masked Man.

Kid Goku by Mishaal

See you next month... unless I forget again

The Debug Adapter Protocol and Scala

Chris Kipp — Sun, 13 Mar 2022 00:00:00 +0100

The Debug Adapter Protocol and Scala

The Debug Adapter Protocol and Scala. How it all works together.

In the last couple years in Scala we've seen a surge in usage of various tooling protocols. A growing number of developers are at least familiar with the Language Server Protocol (LSP), especially if you're a Metals user or were an early user of Dotty when it had a built-in LSP server. You've also more than likely been exposed to the Build Server Protocol (BSP) even if it was just in passing by seeing sbt create a .bsp/ directory in your Scala workspace. Another popular protocol is the Debug Adapter Protocol, which again you may have rubbed up against if you're using Metals. I've seen a lot of questions about the how DAP works with Metals and this one actually has a few more moving parts than the others to make it all work together. So, I wanted to jot down some notes both to ensure I understand all the moving parts, to serve as a detailed explanation of sorts of how it all works together, and to hopefully help you to get the full picture as well.

Keep in mind that this will differ a bit per client. Client here may also be a bit ambiguous since we'll be talking about Metals client extensions and DAP clients, so I'll try to always differentiate them by saying "Metals client" or "DAP client" even though they may be the same thing.

Also keep in mind that this won't really be a technical explanation of how things like expressions evaluation or breakpoints actually work, but more of an overview of all the moving pieces to better understand what happens between all of your tools when you trigger a run or debug.

What is DAP

If you're familiar with the goals of the LSP, then you're already familiar with some of the goals of DAP. Basically, don't re-implement all the debugger functionality over and over for every new language and tool that wants to implement debugging. Taken from the DAP website:

Adding a debugger for a new language to an IDE or editor is not only a significant effort, but it is also frustrating that this effort can not be easily amortized over multiple development tools, as each tool uses different APIs for implementing the same feature. The idea behind the Debug Adapter Protocol (DAP) is to abstract the way how the debugging support of development tools communicates with debuggers or runtimes into a protocol. Since it is unrealistic to assume that existing debuggers or runtimes adopt this protocol any time soon, we rather assume that an intermediary component - a so called Debug Adapter - adapts an existing debugger or runtime to the Debug Adapter Protocol.

The Debug Adapter Protocol makes it possible to implement a generic debugger for a development tool that can communicate with different debuggers via Debug Adapters. And Debug Adapters can be re-used across multiple development tools which significantly reduces the effort to support a new debugger in different tools.

This description also brings up an important point of the actual Debug Adapter since in many languages that do have existing debugger interfaces built right into the language or platform, like the Java Debug Interface, which is what java-debug uses, which is what scala-debug-adapter uses, which is what most build servers will be using, which is what Metals connects to... you get the idea. However, we'll get further into that down below. All that to say, the goal is that in a situation where a X editor user wants to transition to another editor, like Neovim, they can still have the same basic debugging experience as long as that new client has a DAP client implementation either built in or available as a plugin. The same can be said for the server side if a Metals users decides to use Bloop as their build server or sbt as their build server, they can share a common implementation of the server side of the protocol and not have to fully re-implement it twice. The server side example is exactly what the scala-debug-adapter is for.

Two different types of clients

I mentioned it up above, but also want to reiterate it here. Different language server clients may handle the DAP client part differently. Like in the case of VS Code, the DAP client is straight up just included in the editor. You can see an example of this in the scalameta/metals-vscode extension. Notice that the imports are coming right from vscode. This offers an extremely tight integration that is totally abstracted away for the user. There are other things that can then be built on top of that like the VS Code Test Explorer API that Metals recently added support for. Under the hood, the communication for these are still going through DAP. I like to think of these as "extensions" to DAP similar to LSP extensions that aren't necessarily part of the protocol, but follow the same pattern and even re-use parts that are part of the protocol. These then require more work for other clients to implement, and they aren't expected to work out of the box like other DAP features. For now, we won't focus on any of these, and we'll just stick to the core DAP features.

The other way this can look in a client is when your language server client doesn't natively implement a DAP client, but has you use an extension to implement this. You can see an example of this in scalameta/nvim-metals. Notice that in the setup_dap function the first thing we actually do is require mfussenegger/nvim-dap which is a fantastic plugin that implements the client portion of the protocol for Neovim.

So, whether you're using VS Code with a built-in DAP client integration or using Neovim and a plugin like nvim-dap, the core client functionality should be largely the same. Moving forward all of examples will assume the second setup using nvim-dap, since that's what I'm most familiar with.

How does everything get set up

I don't want this to necessarily be a "getting started with nvim-dap" guide, as there are already guides out there, plus the docs for nvim-dap are pretty detailed. Instead, I want to focus on how this all works together specifically when using nvim-metals. Most of this will be transferable to other Metals extensions that support DAP as well.

Let's start with a simple piece of code:

@main def dapExample() =
  println("hello people interested in DAP")

If you have nvim-dap installed and you open a Scala project with a main method you should see code lenses appear on your main method. In nvim-metals it will look like this:

The first question we need to answer is "how did these get here?" and then "how does this actually trigger a run or debug of my code?". Behind the scenes what actually happens is that Metals will have communicated with your build server and gotten any main methods in your build target via a buildTarget/scalaMainClasses request and cached those results. Then when the LSP request comes to Metals for the textDocument/codeLens Metals looks through the SemanticDB for the current document and looks for any main methods. If it finds them, it compares them to the cached ones that were retrieved earlier, and then creates code lenses for them with special commands attached to them.

Here are some illustrations of the above:

Example of the buildTarget/scalaMainClasses request and response with the build server

[Trace - 10:50:29 AM] Sending request 'buildTarget/scalaMainClasses - (7)'
Params: {
  "targets": [
    {
      "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/test/?id\u003dSanity.test"
    },
    {
      "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/?id\u003dSanity"
    }
  ]
}

[Trace - 10:50:29 AM] Received response 'buildTarget/scalaMainClasses - (7)' in 6ms
Result: {
  "items": [
    {
      "target": {
        "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/?id\u003dSanity"
      },
      "classes": [
        {
          "class": "dapExample",
          "arguments": [],
          "jvmOptions": [],
          "environmentVariables": []
        }
      ]
    },
    {
      "target": {
        "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/test/?id\u003dSanity.test"
      },
      "classes": []
    }
  ]
}

Example of what the SemanticDB for our code snippet looks like. Notice the first occurrence which is scala/main#. Once found we'd get the symbol for that occurrence and then check it against what was returned above.

Sanity/src/example/Hello.scala
------------------------------

Summary:
Schema => SemanticDB v4
Uri => Sanity/src/example/Hello.scala
Text => empty
Language => Scala
Symbols => 3 entries
Occurrences => 3 entries

Symbols:
_empty_/Hello$package. => final package object _empty_ extends Object { self: _empty_.type => +2 decls }
_empty_/Hello$package.dapExample(). => @main method dapExample(): Unit
_empty_/dapExample# => final class dapExample extends Object { self: dapExample => +2 decls }

Occurrences:
[0:1..0:5) => scala/main#
[0:10..0:20) <= _empty_/Hello$package.dapExample().
[1:2..1:9) => scala/Predef.println(+1).

Example of the code lens request and response.

[Trace - 07:22:25 PM] Received request 'textDocument/codeLens - (92)'
Params: {
  "textDocument": {
    "uri": "file:///Users/ckipp/Documents/scala-workspace/sanity/Sanity/src/example/Hello.scala"
  }
}

[Trace - 07:22:25 PM] Sending response 'textDocument/codeLens - (92)'. Processing request took 1ms
Result: [
  {
    "range": {
      "start": {
        "line": 0,
        "character": 1
      },
      "end": {
        "line": 0,
        "character": 5
      }
    },
    "command": {
      "title": "run",
      "command": "metals-run-session-start",
      "arguments": [
        {
          "targets": [
            {
              "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/?id\u003dSanity"
            }
          ],
          "dataKind": "scala-main-class",
          "data": {
            "class": "dapExample",
            "arguments": [],
            "jvmOptions": [],
            "environmentVariables": []
          }
        }
      ]
    }
  },
  {
    "range": {...},
    "command": {
      "title": "debug",
      "command": "metals-debug-session-start",
      "arguments": [...]
    }
  }
]

While the above is to generate the run and debug code lenses, more or less the same process happens for the test and test-debug lenses as well. The commands that are attached to the code lenses are LSP client commands that need to be implemented by the client. As you can probably guess, the two commands metals-run-session-start starts just a normal run and a metals-debug-session-start starts a debug session. How that's done however differs a bit by client. Since we're focusing on nvim-metals and nvim-dap I'll outline a bit of what is happening behind the scenes to tie everything together.

Both of the commands will end up calling this function:

local function debug_start_command(no_debug)
  return function(cmd, _)
    dap.run({
      type = "scala",
      request = "launch",
      name = "from_lens",
      noDebug = no_debug,
      metals = cmd.arguments,
    })
  end
end

With nvim-dap there are two main concepts around configuration that are important to grasp. The first is the adapter configuration. This is a table given to nvim-dap per language (although the keys are actually arbitrary, just think of it per language) that basically tells nvim-dap if it should launch a debug adapter and if so how, or if it should connect to a running debugger and if so where. This configuration can be a table with these details or a function that takes a callback and a configuration. In the case of nvim-metals we use the latter with the callback, which will be explained further below. The second configuration that is relevant here is the debuggee configuration which is the configuration for your application you'll be debugging. So if you're familiar with VS Code think of this as your launch.json. nvim-dap can actually work using a launch.json, but we won't focus on that here.

So in the above debug_start_command function the table being passed into dap.run() is your partially your debuggee configuration. The type is a reference to the adapter entry that matches this key, the request is either attach or launch indicating whether the debug-adapter should launch or attach to a debuggee, the name is a human readable name for the configuration (which we'll revisit), and the noDebug is whether or not debug mode should be enabled. If this is true breakpoints will be ignored. Finally, the metals key isn't part of the spec here and will actually be removed before being passed to nvim-dap when it fully becomes the debuggee configuration. However we use it to be able to forward the arguments from the code lens to the adapter configuration.

So where is the adapter configuration? nvim-metals fully handles the adapter configuration for you. The main reason for this is that before we can actually launch everything we need some information from Metals about how to connect to the debugger. In order to get this, we again utilize LSP to get this information before we actually start any DAP communication. Keep in mind we've already dove into quite a few things, but no DAP communication has even started yet. This is where the callback being part of the adapter configuration comes into play. When we setup the adapter configuration it looks something like this:

dap.adapters.scala = function(callback, config)
  local uri = vim.uri_from_bufnr(0)
  local arguments = {}

  if config.name == "from_lens" then
    arguments = config.metals
  else
    local metals_dap_settings = config.metals or {}

    arguments = {
      path = uri,
      runType = metals_dap_settings.runType or "run",
      args = metals_dap_settings.args,
      jvmOptions = metals_dap_settings.jvmOptions,
      env = metals_dap_settings.env,
      envFile = metals_dap_settings.envFile,
    }
  end

  execute_command({
    command = "metals.debug-adapter-start",
    arguments = arguments,
  }, function(_, _, res)
    if res then
      local port = util.split_on(res.uri, ":")[3]

      callback({
        type = "server",
        host = "127.0.0.1",
        port = port,
        enrich_config = function(_config, on_config)
          local final_config = vim.deepcopy(_config)
          final_config.metals = nil
          on_config(final_config)
        end,
      })
    end
  end)
end

Let's walk through this. We first check the config.name and if it's from_lens we know that this whole process was started from triggering a code lens, so we grab everything in the metals key and set that to arguments which will actually be sent to metals along with the metals.debug-adapter-start command. Let's ignore the else branch if the name isn't from_lens since we're focusing on the code lens example here. The execute_command function will send the LSP command to Metals which will then over BSP tell the build server to start the debug server. Your debug server is (or if it's not it should be) using scalacenter/scala-debug-adapter to start and manage the debug server. Then the res that is returned via BSP will have the information we need, mainly the host and port of the server that has already been started. This is then forwarded over LSP back to nvim-metals. The type here is now set to server since there is already a debugger running that we just want to connect to. The enrich_config takes in the debuggee configuration that we created before and strips the metals key out since it's no longer relevant for the actual run, and not part of DAP.

At this point is when DAP communication actually starts. However, there is a fair amount of stuff that has already happened. To recap all of this, here is a diagram showing what we've all covered.

The actual DAP communication

So after all of the set up above happens you're ready to actual have some DAP communication. At this point in nvim-metals nvim-dap pretty much fully takes over on the client side and communicates directly with the running debugger which was started by your build server. Given the code we had earlier the general flow of DAP communication isn't that interesting since we don't have any breakpoints set, we aren't doing expression evaluation, conditionals etc. To outline what the communication looks like between DAP client and DAP server, here is another diagram.

While the diagram above isn't actually exhaustive it contains the main events that get the point across of how the DAP server and client are communicating. The communication here can also become much more complicated when start adding in breakpoints. For example, let's pretend our code is slightly different than what we had above with a single breakpoint set:

@main def dapExample() =
  val greeting = "hello people interested in DAP"
  println(greeting) // breakpoint set on this line

If we set the breakpoint above where mentioned, it will pause at that point. In nvim-dap you can execute a .scopes command in the debug REPL and you'll see the variables in scope returned. The communication between the DAP server and client for this looks like so:

[Trace][03:21:33 PM] Sent request:
{
  "type": "request",
  "seq": 9,
  "command": "variables",
  "arguments": {
    "variablesReference": 3
  }
}
[Trace][03:21:33 PM] Received response:
{
  "type": "response",
  "seq": 14,
  "request_seq": 9,
  "command": "variables",
  "success": true,
  "body": {
    "variables": [
      {
        "name": "greeting",
        "value": "\"hello people interested in DAP\"",
        "type": "String",
        "variablesReference": 4,
        "namedVariables": 0,
        "indexedVariables": 0
      },
      {
        "name": "this",
        "value": "Hello$package$@183",
        "type": "Hello$package$",
        "variablesReference": 5,
        "namedVariables": 0,
        "indexedVariables": 0
      }
    ]
  }
}

You can see all sorts of examples of the communication that can take place if you look through the types of events in the specification documentation.

Without code lenses

So let's revisit the function from up above that we gave to dap.adapters.scala and now take a look at the else branch that we ignored before. One thing you may have noticed with the code lens is that it's fully handled by nvim-metals without really a great way to maybe set some arguments that you'd like to pass into your run or maybe some specific jvmOptions. The recommended way to do this with nvim-metals is to pre-define your debuggee configuration. Let's say you wanted to trigger a run with a specific argument and also a specific env file. You could defined a configuration like this:

dap.configurations.scala = {
  {
    type = "scala",
    request = "launch",
    name = "Run with arg and env file",
    metals = {
      runType = "runOrTestFile",
      args = { "myArg" },
      envFile = "path/to/.env",
    },
  },

NOTE: again, some of this is nvim-dap specific, so if you're trying to follow along, make sure to read through the docs and also take a look at the full example configuration for nvim-metals users.

So remembering the function from up above, the metals key here won't actually get passed into nvim-dap at all, but is instead used to send the correct stuff to Metals to ensure the debug server is set up correctly and the correct things like args are taken into account. So now when you'd trigger a dap.continue() (it's called continue but will start a session if none exists) you'll see this configuration as an option to use to start the process under Run with arg and env file. If you have 3 different ones defined, maybe one with args, one without, and one specifically for tests, you'd see those three to choose from. Here is an example of what mine looks like locally when I trigger it:

The communication here is slightly different than when using the code lens since at this point we're triggering the process to start, but we don't actually know for sure if we're in a file that even has a main method. So some of the same steps from above happen, but instead the order is a bit different. The runType key here is special functionality in Metals to search the current document you're in to see if there are any many methods or tests that it can run. If there is, it then goes ahead and runs it, or returns you a picker to select which you'd like if there are multiple mains or test suites to choose from. Here is a diagram showing the changed order of things when you trigger a run this way.

I'm amazed this all works

Even without going into the technical details of how each part of this works, it's incredible that it even does with the amount of moving parts, different protocols involved, and different tools being utilized. It speaks to the amount of work that goes on by many different people working in different projects to ensure when you click run in your editor, you can run your code. Hopefully it also explains a bit why when something may not be working exactly how you want it to, there is a lot that can go wrong in various places. There's a lot of extra things related to DAP that we haven't hit on in here that could each be a post of their own.

I hope this was insightful. Thanks for reading along.

Music of the Month - Feb 2022

Chris Kipp — Thu, 3 Mar 2022 00:00:00 +0100

Music of the Month - Feb 2022

February was a weird month for me. I tested positive for Covid, and it just knocked me out for a couple of weeks. I was amazed at how long I stayed sick. I wasn't able to do a whole lot, and staring at my computer wasn't really an option, so I spent a lot of time listening to new music, and listening intently without doing anything else. I always find this the best way to truly experience an album: sit down, put on some headphones, and listen through the entire album.

Album of the Month

My album of the month for February is Love by Amen Dunes. I've only briefly listened to Amen Dunes in the past, never his full albums. The only song of his that I would have recognized was Song of the Siren, but I randomly came across the song Love which lead me to the album which I immediately listened to in its entirety. This album brings you on a journey through joy, grief, heartbreak, and solace. It's a great album to listen to when you don't feel well and just want to immerse yourself in some low frills authentic good music. Fun fact, some of the members from Godspeed You! Black Emperor, one of my favorites, played on this album. If you know GY!BE you'll understand why this is notable seeing how radically different this albums is than anything they've put out. This is a fantastic album.

Love by Amen Dunes

Song of the Month

I listened to a lot of jazz this month. If it wasn't for the album above, the album this next song came off of could have also easily been the album of the month for me. The song of the month for me this month was Glad to be Unhappy by Paul Desmond. Such a smooth chill song. If you're unfamiliar with Paul Desmond this entire album, also named Glad to be Unhappy, is a great introduction.

See you next month

Music of the Month - Jan 2022

Chris Kipp — Tue, 1 Feb 2022 00:00:00 +0100

Music of the Month - Jan 2022

This is the first post of a monthly series I'd like to do where I share the music I've enjoyed over the past month. My music tastes are all over the place and I've found that it's often a good picture of my mood that month, what I've been going through, or just what I've been enjoying. These won't necessarily be new albums or songs, but whatever I find myself listening to a lot during the month. At the end of the month I'll pick an album I felt left a mark and a song that has been my "favorite". Give it a listen, and let's talk music.

Album of the Month

My album of the month for January is It's Nice Outside by Anti-Lilly & Phoniks. This album came out of nowhere for me. I wasn't aware of them at all before this month and I don't even remember how I got turned onto it. I heard Every Season first, and then I was hooked. The entire album just rattles through making it through tough times and coming out of the other side. The lyrics are raw, but balanced with Anti-Lilly's positive outlook. This is such a fantastic album.

It's Nice Outside by Anti-Lilly & Phoniks

Song of the Month

A song that I found made me pause almost every time it came on this month was A-001 by Corntuth off their album Music to Work To. January is also a pretty reflective month for me as I look back at the previous year and plan for the current. This song definitely facilitated some of that for me this month.

Music To Work To by Corntuth

See you next month

My Dos and Donts in sbt Build Definitions

Chris Kipp — Sun, 19 Dec 2021 00:00:00 +0100

My Dos and Donts in sbt Build Definitions

My Dos and Don'ts in sbt Build Definitions

Over the past couple of years I've been able to dig around a lot of sbt build files. Whether sbt is your Scala build tool of choice or not, you'll likely need to deal with it at some time or another. sbt baffled me when I first got started. The simple things were actually pretty simple, but anything else sort of forces you to get a better-than-surface-level understanding of sbt. Depending on that level of understanding, the team writing that build definition, or even the type of project your writing it for provides for a nice smörgåsbord of examples out there. All sorts of fun and crazy stuff. Because of this, I thought I'd make a list of Dos and Don'ts I typically try to follow when writing build definitions in sbt. These opinions were formed with a mixture of open-source and professional work, annoyances with incredibly complex builds, and probably some straight-up misunderstandings on my part since sbt still baffles me at times.

This list isn't exhaustive, it's highly opinionated, and is tainted by specific experiences and projects I've been a part of. More than anything, the goal of this is to hopefully spark a conversation in your team about build definitions, how you write them, and best practices to follow. Sometimes the best practice is simply having your team all on the same page and doing things the same way.

So without further ado, here are my personal Dos and Don'ts

Don't make separate build.sbt files for your different modules
Do stay away from bare settings
Don't let your settings leak into modules they don't need to be in
Do regularly go over your entire build definition
Don't shade things unless you absolutely have to
Do stick to lazy vals everywhere
Don't ever hardcode constants
Do stay away from crazy meta meta build stuff
Don't put stuff in project/ unless it is a collection of something or is "standalone"
Do have a conversation with your team about standards and consistency
Don't make your own way to do something if sbt already provides a way
Do document it when you do weird shit
Don't add stuff to your build definition you don't understand

Don't make separate `build.sbt` files for your different modules

Like everything, there seems to be trends in build definitions. One trend I saw for a while (and you may still use it) is to keep a separate build file for every project in your build. Meaning you'll likely have a structure like this:

.
├── a
│  ├── build.sbt
│  └── src
├── b
│  ├── build.sbt
│  └── src
├── build.sbt
├── c
│  ├── build.sbt
│  └── src
├── d
│  ├── build.sbt
│  └── src
└── project
   └── build.properties

My experience with this type of setup is that you end up jumping between your root build.sbt and your project-specific build.sbt and probably even some project/*.scala to figure out where your stuff is defined and where you need to change it. The projects I've seen use this style of multiple build.sbt files often make heavy use of project/*.scala|*.sbt files making the situation even worse. In theory I like this idea, but in practice I hate it. I actually don't see this nearly as often anymore with new projects. Just stick to a single build.sbt file in the root of your directory which contains all the settings for each separate project. It makes things easier to find, and then your entire build definition is basically contained in one place.

Taken from the sbt docs

We recommend putting all project declarations and settings in the root build.sbt file in order to keep all build definition under a single file. However, it’s up to you.

Do stay away from bare settings

My beef with bare settings can be illustrated with a very minimal build definition. Let's say we start a new project with an extremely minimal setup:

scalaVersion := "3.1.0"

lazy val root = (project in file("."))
  .settings(
    name := "bare-settings"
  )

Just to make sure we're all on the same page scalaVersion is the bare setting here. Now this works fine as long as root is your only project. However more than likely you'll eventually want to expand this to have another project. Let's pretend we'd like to add a domain module. So we do that like so:

scalaVersion := "3.1.0"

lazy val root = (project in file("."))
  .settings(
    name := "bare-settings"
  )

lazy val domain = project

Now, you might see the issue with the immediately if you're familiar with sbt and how it's various scopes work, but keep in mind that most newcomers to Scala won't, and they will immediately be met with confusion when they discover the following:

sbt:bare-settings> projects
[info] In file:/Users/ckipp/Documents/scala-workspace/bare-settings/
[info]     domain
[info]   * root
sbt:bare-settings> scalaVersion
[info] 3.1.0
sbt:bare-settings> domain/scalaVersion
[info] 2.12.14

If you don't understand why domain is on 2.12.14 when you set scalaVersion to 3.1.0, it's because you're getting bit by an assumption that a bare settings should be carried over to your subprojects. There is an interesting discussion here where Eugene Yokota proposes to deprecate and eventually remove bare settings in hope to avoid this confusion. It explains a bit of the history behind them, ideas to change the default behavior to be less confusing, and also contains some arguments for having them. It's worth a read.

The example up above is minimal, but you can see how once your project becomes larger and you have numerous bare settings how confusing this could become. The main argument for them is to help someone new to sbt since just using scalaVersion is easier than ThisBuild / scalaVersion. That's fair, since the latter also forces knowledge of ThisBuild. So the question really is, does the default behavior of bare settings make sense? Again, read the linked thread above for a much more detailed conversation on this.

My takeaway and personal experience is to avoid bare settings completely. The time spent explaining to your team why will be time better spent than a new teammate banging their head against the wall when they are bit by it later on if you don't and you use bare settings.

Don't let your settings leak into modules they don't need to be in

A common approach you'll see in a lot of builds is making use of common settings. You'll often see something like the following:

lazy val api = project
  .settings(commonSettings)
  .dependsOn(domain)

lazy val domain = project
  .settings(commonSettings)

lazy val commonSettings = Seq(
  scalaVersion := "3.1.0",
  libraryDependencies += "com.lihaoyi" %% "cask" % "0.8.0",
  scalacOptions += "-explain"
)

Most of the time having some sort of common settings is a good thing, but many times you'll end up with settings that are leaking into other projects that you don't actually want. In the above example there is no reason cask needs to be a dependency of domain, but it is. You might think this would never actually happen since it's so obvious you don't need a HTTP framework as a dependency in your domain module, but I've seen this happen time and time again. I've even guilty of it at times. This can lead to bloating your dependencies for a project causing you to rely on things you don't need, having invalid scalacOptions for projects, or just build settings being applied in places you don't want them, and maybe don't even realize they are being applied. Be extra careful with common settings and ensure that what you have in there truly are meant to be share every place that is using them. Again, this seems obvious in such a minimal example, but add hundreds of lines of sbt code to this and you'll quickly see the problem.

Do regularly go over your entire build definition

This actually relates to the above point about being careful with your common settings. I've been amazed at the amount of times I've been looking through a build file and thought "What is this doing in here, this was for an issue fixed long ago". It seems that we never refactor build definitions. We put them together, add some hacks, and then leave them until we need to add another hack. We forget that our build definition is also code, ripe for refactoring, and arguably will always have stuff left over that doesn't need to be in there. This also relates to other points I'll make about ensuring that when you add something to a build definition to avoid a bug, work around an issue, etc, you should also add a note as to what it's doing and why you're doing it.

When you change something in your build, take a bit of time, walk through it, make sure things are up to date, make sure exclusions that are there still need to be there, dependency overrides still need to be there, and other random settings you copied off Stack Overflow to overcome some issue still need to be there. This helps keep those build definitions nice and tidy and avoids the "Is this still relevant" question I so often ask when looking at build files.

Don't shade things unless you absolutely have to

I hesitated to put this in here since it's maybe more specific to just dependency management, but I thought it helped reinforce the idea of keeping things simple. Shading at times is necessary and even wise to ensure you don't force a transitive dependency on a downstream user, but in those situations you are adding a layer of complexity to your build definition that will cause others to have to spend the time understanding why it was done. An argument could be made that if this is clearly documented why then it's no issue, but I've often found that when shading is used you get a single line comment at most. Sometimes this is enough, but in other cases I've found hundreds of lines of code related to shading with no explanation. Understand that this can be almost impossible to decipher without the full backstory of why it was done.

My default rule is to avoid shading unless you absolutely have to do it and the benefit outweighs any complexity and confusion you may be adding to your build definition.

Do stick to `lazy val`s everywhere

This one may seem obvious to some, but it's also a commonly asked question. The most simple answer is that to avoid having to define everything before you use it, use lazy vals. The below example is something you'll see fairly often. Now if you start removing lazy from various places you will break stuff since if you have a project that isn't lazy and tries to access another project before it's defined or commons settings before it's defined you will run into issues.

ThisBuild / scalaVersion := "3.1.0"

lazy val V = new {
  val cask = "0.8.0"
}

lazy val api = project
  .settings(commonSettings)
  .settings(
    libraryDependencies += "com.lihaoyi" %% "cask" % V.cask
  )
  .dependsOn(domain)

lazy val domain = project
  .settings(commonSettings)

lazy val commonSettings = Seq(
  scalacOptions += "-explain"
)

To keep it simple for myself, every time I use a val at the top level in my sbt file, I make it lazy. One related thing I've seen in build files that isn't needed is nested lazy modifies like:

lazy val V = new {
   lazy val cask = "0.8.0"
}

Don't ever hardcode constants

This is actually a best practice taken right from the docs. Since it's in the docs I'll directly quote from there and use the same example.

Don’t hard code constants, like the output directory target/. This is especially important for plugins. A user might change the target setting to point to build/, for example, and the plugin needs to respect that. Instead, use the setting, like:

myDirectory := target.value / "sub-directory"

Do stay away from crazy meta meta build stuff

The amount of projects that actually need this is incredibly small, so maybe you'll never see this. However, I've come across projects with project/project/project/build.sbt which is ludicrous. The mental gymnastics that a user needs to go through to understand what's happening here can give them nightmares. Are there actual use cases for including logic in your sbt meta-build? Absolutely, but I question if you ever need to go another layer deeper. Most users won't understand what's happening or how you even interact or see settings defined there.

If you're a Metals user you may be thinking "Wait a second, Metals creates a project/project/metals.sbt... you hypocrite!". If you're using Metals and Bloop as your build server, Metals needs to export your build to Bloop. The only way to do that is to go one layer up and add the sbt-bloop plugin. So if you have stuff in your project/*.sbt we need to add a metals.sbt in your project/project/. If you have something in your project/project/*.sbt, we would also need to add the plugin in your project/project/project/ dir. You get the idea. This goes to show that in reality there are some use cases for this, but they are very few. If you can, avoid it.

Don't put stuff in `project/` unless it is a collection of something or is "standalone"

Another rule I like to follow is to not include things in project/ unless it's something that is "standalone" or a collection. A couple examples of these.

At work we use a private nexus, and when you publish sbt plugins maven style, it doesn't create a maven-metadata.xml file. NOTE: That it does create one for external plugins that you are mirroring, but not ones you publish yourself. This ends up being problematic when you run Scala Steward, since under the hood coursier needs a maven-metadata.xml or an index that it can scan, but in this scenario has neither. You can see the context for what I explained here. However, the way we got around this issue was to create a custom release step for sbt-release which created and/or updated the metadata file and was simply called during the release process. We stuck in in /project/CustomReleaseStep.scala. It's only referenced once from the build.sbt file, and apart from that never touched, never inherited from, and has no impact on the rest of the build definition. This is a great example of something that is "standalone" and belongs there.

The other time I like to put stuff in the project/ is when it's a collection of sorts. This can be a collection of dependencies that you're using in your build, a collection of maintainers, or something similar. These are also great things to stick in project/ to not take up a ton of space in your build.sbt.

Apart from those two things, I like to stick everything else in my build.sbt. I dislike when projects have a bunch of project/*.scala files with complex logic that conditionally set keys making it very difficult to understand where something is being set.

Do have a conversation with your team about standards and consistency

This might be the most important one on the list. Most of the time you don't work in isolation meaning that others will be touching your build definition. The benefit of having everyone on your team on the same page far outweighs all the other best practices since everyone hopefully understands the patterns you're following, the reasons you're doing them, and is more quickly able to edit and iterate on them. If you've never had a discussion with your team about your build definition, what standards you want to follow, and document those, you should. If you don't you'll end up with build definitions that are wildly different from one another, build definitions that have mixed ways of doing things, and ultimately creating technical debt the moment you add something to your build.

Don't make your own way to do something if sbt already provides a way

It might be tempting to try and come up with your own solutions when you don't like the "sbt" way of doing things. For the sake of an illustration let's pretend we hate the way it's currently done to have cross-versioned sources. You have some Scala 2 code and some Scala 3 code you want side by side. You don't like the default src/main/scala-<scala binary version>/ being the way it includes Scala-version specific sources so you decide to come up with your own "better" solution.

Compile / sources := {
  (Compile / sources).value.filter(_.getName().endsWith("2"))
}

That was easy, now it can quickly pick up all my Scala 2 sources, I just need to make sure they end with Scala 2, and then my Scala 3 ones can end with 3. You now just broke away from the expected norm for anyone familiar with sbt. What they expect to work won't, and you force the user to dig into what's going on to add an extra layer of complexity they must understand just to do something that sbt can already do. This can be incredibly frustrating for contributors to a project that just want to quickly contribute something, but are first forced to understand the intricacies of your Frankenstein build definition.

Are there exceptions to this rule? Probably.

Do document it when you do weird shit

Picture this, you've been struggling all week on getting something to work. You feel like you've read the entire sbt docs site, all the sbt code you could actually understand, and then you finally come up with a solution to your bespoke problem. It's wild, it's crazy, and it makes perfect sense to you in that moment. For the sake of anyone coming across that code in the future, if it's absolutely necessary to keep it, document it. Document what you tried, why it didn't work, why you chose to do whatever you did, and over explain it. I've seen projects do wild stuff, and that undocumented wild stuff was enough to make them not even attempt to work on the project, even if it desperately needs it.

Don't add stuff to your build definition you don't understand

At times it's tempting to copy some code into your build definition, confirm it works, and then call it a day. I encourage you not to do this. Mainly because having parts of your build definition that you don't understand is a liability if you need to change it but don't understand what you're changing. I've personally found the Scala tooling community incredibly helpful and open to questions of all levels. I've asked a lot of sbt questions in the sbt/sbt gitter chat, and they almost always got answered.

Conclusion

After reading this you may totally disagree with some of these rules, and that's absolutely fine. Think about them, discuss them with your team, and be consistent in the way you apply them while also understanding there are always exceptions to the rules. Thanks for reading.

An intro to the Scala presentation compiler

Chris Kipp — Thu, 15 Jul 2021 00:00:00 +0200

An intro to the Scala presentation compiler

An intro to the Scala Presentation Compiler

"Completions are powered by the presentation compiler". I remember the first time I read this phrase. As someone that was fairly new to Scala, new to working in tooling, and had never touched the internals of a compiler before, this phrase made absolutely no sense to me. I had no idea what the presentation compiler was, where it was located, and definitely no idea of how to use it. To be honest, I was also a bit afraid to even ask those questions. Often when we see phrases like "it's simple" or "it's easy", it really just means "I'm familiar with this".

A lot of topics in tooling seem to be less covered than those of hot libraries or general programming paradigms, which totally makes sense. There is a much smaller group of people working in this area. However, it does then make it a bit harder to get started. Thankfully, the Scala tooling ecosystem is packed full of friendly helpful people that have always been more than helpful whenever I've gotten stuck or had no clue where to start with something. Plus a lot of the Scala documentation is incredibly useful and underappreciated. Like this page on Trees, which will be relevant for what I show down below.

The goal for this post is so that hopefully when someone in the future hears a phrase about the presentation compiler and has no idea what it is, this can serve as an introduction to what it is, where to find it, and as an example of how to use it. When I first started searching around for these answers I came across a few useful sources that I'll reference since they helped me:

The Scala Presentation Compiler, which gives a brief overview of how the Eclipse Scala IDE used the Scala presentation compiler.
Building an IDE with the Scala Presentation Compiler: A Field Report, which gives a great overview of how Ensime used the Scala presentation compiler.
This stack overflow post, which gives a great minimal standalone example of how to use the presentation compiler.
Integrating IDEs with Dotty, The Experimental Scala Compiler by Guillaume Martres, where Guillaume does a great job at explaining how IDEs can interact with Dotty.

What is the presentation compiler

The above mentioned post by the Scala IDE teams defines the presentation compiler as follows:

The Scala IDE for Eclipse uses the Scala Presentation Compiler, a faster asynchronous version of the Scala Compiler. The presentation compiler only runs the phases up until and including the typer phase, that is, the first 4 of the 27 scala compilation phases.

This is a really great minimal explanation of what the Scala Presentation Compiler is. It's specifically made for usage from IDE-like tools that need a compiler that is:

Asynchronous
Interruptible at every point
Can do targeted typechecking
Stops after a specific point (type-checking) and provides a partial result

Martin Odersky actually explains this a bit way back in 2011 when giving a talk about what's coming in Scala 2.10. You may also be wondering where this thing is in the Scala code base.

For Scala 3 you can find it located here in the interactive package
For Scala 2 it's here in the interactive package

Also keep in mind that the compiler internals are completely different between 2 and 3, meaning that the presentation compiler API is completely different. So for tooling that interfaces with the presentation compiler, they'll need to account for this. For example that's why in the Metals mtags module, which is a module to abstract over compiler access, you'll see that Scala 3 has a completely different implementation.

What uses the presentation compiler

If you haven't already gotten this from looking at where these articles are coming from, you'll notice that many of the IDE-like tools or tools that provide IDE-like features in the Scala Ecosystem use or have used the Scala Presentation Compiler.

You may have noticed that IntelliJ is missing from that list. That's because IntelliJ actually uses its own implementation of the Scala Presentation Compiler. That's why it's possible to see subtle differences with diagnostics showing in IntelliJ that don't show when compiling with the actual Scala compiler.

Many of these projects use the presentation compiler to accomplish very similar goals, and that is to provide the common features that you'd expect from an editor. One of those common features is completions. For example you can see a lot of the functionality for this in the aptly named Completion.scala file in the Scala 3 codebase. The presentation compiler can do the majority of the things you'd expect to see in an editor like, get the hover information on a symbol, find the references of a symbol, or go to the definition site of a symbol. A really nice example of all this can be found in the DottyLanguageServer.scala, which was an embedded LSP server for Scala 3 as the team was developing Dotty. The presentation compiler can be used for a whole variety of things, and I'll give a very specific example of a minimal feature that was just added to Metals in 0.10.5 down below.

A minimal example of using the presentation compiler

One thing I've found in the past is that there isn't a lot of standalone examples out there of using the presentation compiler. It's not that surprising, since why would there be, but recently while working on a new feature for Metals, I thought it was a perfect small example of how the presentation compiler could be used to implement an editor feature that is maybe a little less common than the text-book examples of completions, definitions, or references. If you'd like to follow along or play with the code I'm about to show, there are examples for both Scala 2 and 3 located here. In the examples down below we'll focus on Scala 3.

The feature I wanted to implement in Metals was the LSP textDocument/selectionRange. This feature allows you to more intelligently expand a selection. This is a useful feature when you maybe want to quickly select everything inside of a parameter, select an entire method, or really just anything one level above where you are. Here is a quick gif showing what I mean:

If you were to look at the LSP communication for this request, you'd see the response from the server would look something like this for the above example:

[Trace - 00:07:29 AM] Sending response 'textDocument/selectionRange - (70)'. Processing request took 8ms
Result: [
  {
    "range": {
      "start": {
        "line": 2,
        "character": 14
      },
      "end": {
        "line": 2,
        "character": 28
      }
    },
    "parent": {
      "range": {
        "start": {
          "line": 2,
          "character": 6
        },
        "end": {
          "line": 2,
          "character": 29
        }
      },
      "parent": {
        "range": {
          "start": {
            "line": 1,
            "character": 2
          },
          "end": {
            "line": 2,
            "character": 29
          }
        },
        "parent": {
          "range": {
            "start": {
              "line": 0,
              "character": 0
            },
            "end": {
              "line": 2,
              "character": 29
            }
          }
        }
      }
    }
  }
]

Let's see how we can come up with these ranges using the presentation compiler.

We'll start with the very basic piece of code you saw in the gif. We can either read this in from a file, or just store it as a string. For now just storing it in a string is easiest, so we'll do that.

val ourScalaCode = """|object Main:
                      |  @main def hello: Unit =
                      |    println("Hello world!")
                      |""".stripMargin

I'll use Coursier to fetch the actual scala3-library to pass in when I create an instance of the InteracativeDriver. If you don't do this dotc (dotty compiler) will yell at you to ensure that the compiler core libraries are on the classpath. So we fetch them, and then we create the InteracativeDriver aka the Scala 3 presentation compiler like so:


val fetch = Fetch.create()

fetch.addDependencies(
  Dependency.of("org.scala-lang", "scala3-library_3", "3.0.0")
)

val extraLibraries = fetch
  .fetch()
  .asScala
  .map(_.toPath())
  .toSeq

val driver = new InteractiveDriver(
  List(
    "-color:never",
    "-classpath",
    extraLibraries.mkString(File.pathSeparator)
  )
)

Keep in mind that this is meant to be a very minimal example with everything in one place. Everything that we're doing is living in the same file and is only around 80 lines of code. So if this was a real world project we'd care a little more about where and how we are managing this driver, how we are setting up the context etc. For now, we're just creating it for a single run, so we'll literally keep it as simple as possible.

Next we'll take ourScalaCode from up above and create a virtual source file out of it. We'll then take it and then run it. We can also verify here that everything is valid with our code. For example we can assert that the run hasn't returned any diagnostics.

val filename = "Example.scala"
val uri = java.net.URI.create(s"file:///$filename")

val sourceFile = SourceFile.virtual(filename, ourScalaCode)

val diagnostics = driver.run(
  uri,
  sourceFile
)

assert(diagnostics.isEmpty)

At this point, we already have everything we need to dive into the compiler tree for the code that has just ran. We an access the entire untyped tree by doing the following:

val tree: Option[Tree[Untyped]] =
  driver.currentCtx.run.units.headOption.map(_.untpdTree)

Instead, remember we are trying to get the selection ranges starting at a given point in this code. So we'll pretend that the cursor is in the current position (the char surrounded by <<>>) when we are going to trigger the expand selection of the textDocument/selectionRange feature.

object Main:
  @main def hello: Unit =
    println("H<<e>>llo world!")

The offset for this position should be 55. So we'll create this position to use in a bit. For some of the future calls we are about to make we'll also need a context provided, so you'll also see that given. Often in the examples you'll see this done a bit different depending on how the context is created. Lots of times you'll start with your own fresh context and build it up to be what you want. This can be doing things like setting the classpath as we did in our example or setting the output to new VirtualDirectory("(memory)") so any generated classfiles don't actually get physically stored anywhere. There are a whole bunch of settings, but again, we're keeping it as simple as possible. I'll share this code snippet and then we'll dissect exactly what's going on.

val pos: SourcePosition = new SourcePosition(sourceFile, Spans.Span(55))

given ctx: Context = driver.currentCtx

val trees: List[dotty.tools.dotc.ast.tpd.Tree] =
  Interactive.pathTo(driver.openedTrees(uri), pos)

Firstly, the driver.openedTrees(uri) does a lookup and returns the full top level trees that were discovered in the file during the run. The pathTo method is an extremely useful method that returns the reverse path to the node in the tree that has the closest enclosing position on the given position. This list of trees is exactly what we are looking for to create our selection ranges. Note that this actual returns a typed tree. We don't actually need a typed tree for this, and if you look at the Scala 2 example, we do this same thing, but with an untyped tree. In Metals currently it's done this way mainly due to how easy it is, but it will probably change in the future work on an untyped tree.

Now that we have the trees we need, we'll map over them to collect the start and end positions of the trees, and then only take the unique ones. There are situations where multiple nodes may have the same range, but we discard any duplicates.

If you're curious about the duplicates, here is a minimal example. Given a small for comprehension like this:

 val total = for {
   a <- Some(1)
 } yield a

The AST for this would have multiple nodes that have the exact same range. Here are two different ones that have the exact same range.

// The range for this one
Apply(Ident(Some), List(Literal(Constant(1))))
// Is the exact same range as this one
Select(Apply(Ident(Some), List(Literal(Constant(1)))), map)

If the above looks foreign to you, the Reflection Overview Page in the docs is a great place to look.

So, once we have these positions, we'll zip them up with their index and print out the code snippets that correspond with the "ranges".

val ranges = trees.map { tree =>
  (tree.sourcePos.start, tree.sourcePos.end)
}.distinct

ranges.zipWithIndex
  .foreach { case ((start, end), index) =>
    val selection = ourScalaCode.slice(start, end)
    pprint
      .copy(colorLiteral = fansi.Color.Blue)
      .pprintln(s"Selection Range: ${index + 1}")
    pprint.pprintln(selection)
  }

And finally, we should have the selections that we've been working at getting! If you watch the gif up above again you should see selection ranges correspond starting at the smallest enclosing node going all the way up the tree until you have the entire source file covered.

"Selection Range: 1"
"\"Hello world!\""

"Selection Range: 2"
"println(\"Hello world!\")"

"Selection Range: 3"
"""@main def hello: Unit =
    println("Hello world!")"""

"Selection Range: 4"
"""object Main:
  @main def hello: Unit =
    println("Hello world!")"""

Conclusion

Hopefully this helps give a somewhat clear picture of a real-world use case of the presentation compiler. If you curious about more complex examples of how this would be set up in a project, or how presentation compiler features could be used in real applications, I'll drop a few links down below of places in code to dig through. As always, if you're interested in getting involved in Scala tooling but are unsure where to start, don't hesitate to reach out. Thanks for stopping by.

Some code to dig through

SelectionRanges.scala The runnable example code shown above. There is also a Scala 2 example in the same repo.
ScalaPresentationCompiler.scala An overview of everything Metals uses the Scala 3 presentation compiler for.
DottyLanguageServer.scala The built-in LSP server in the Scala 3 codebase.
Compiler.scala Shows how Ammonite accesses the Scala 3 compiler and uses various parts of the presentation compiler. This example is the most different than they others.
MarkdownCompiler.scala Shows how mdoc creates and uses the Scala 3 presentation compiler.
PresentationCompilation.scala Showing the usage of the presentation compiler in the Scala 2 REPL.

Scala and Vim Part 1

Chris Kipp — Thu, 2 Jul 2020 00:00:00 +0200

Scala and Vim Part 1

I was first introduced to Scala in the summer of 2018. Prior to Scala I worked in a couple languages with my main up to that point being Javascript. I had recently started getting into Vim and when I started working with Scala I had hopes of being able to start with it being my primary editor. At the time, it seemed like there was only one option for you if you wanted to use Vim with Scala and have some sort of help with intelligent completions and the like, Ensime. That paired with the very popular vim-scala plugin by Derek Wyatt seemed to be all you needed, but for a couple different reasons it was a nightmare for me to set up. I remember feeling defeated, upset, and dumb. I then followed my colleagues advice and just started using IntelliJ.

I used IntelliJ for about a year, and that was around the time in mid 2019 when there was starting to be a bit more buzz around the new kid on the block: Metals. At the time, the docs had instructions about using Coursier to bootstrap Metals to be used with vim-lsc. There was a some setup necessary, but out of the box I was was able to get a good chunk of things working. While there was definitely room for improvement, this initial small win for me was enough to start getting more interested in the tooling around Scala and Vim, and it's when I first started slowly contributing to Metals.

The Birth of coc-metals

Pretty soon after starting to contribute it was pretty evident that things like a simple automated installation process, an easy way to change configuration, or the ability to easily send commands to the server were necessary for wider adoption and just a nicety to have, especially if there was any desire to support some of the LSP extensions that Metals has. This naturally led to me wanting to create a plugin of sorts to aid in the install and generally make the setup easier. If you're familiar at all with the LSP and Vim world, you'll notice that there are a ton of options. At the time users were split between vim-lsc, vim-lsp, coc.nvim, deoplete, Ale, LanguageClient-neovim, and YouCompleteMe. The crazy part is, I'm probably forgetting some. There are a surprising amount of options for those looking for LSP support in Vim. I played around with a few of these, and to be honest, when I wanted to start the plugin, it would have been a toss up which I started with had it not been for the fact that coc.nvim provided a way to port over VS Code extensions to be used for Vim. Ultimately, if someone asks, why is the main Metals Vim plugin made for coc.nvim, the answer is because we can share a core with the VS Code plugin. You can even see this in the metals-languageclient library that Gabriele Petronella spearheaded that both coc-metals and metals-vscode use. The base of coc-metals was created pretty quickly thanks to how easy coc.nvim makes it to port over extensions. As things were coming together, I kept getting more excited, and I really wanted the experience for Vim Metals users to be both smooth and feature rich. I even made it a "resolution" for the new years, ha.

The current state of coc-metals

In the past 6 months there has been a ton of work done on coc-metals that I'm really proud of. With the help of multiple contributors every Metals LSP extension is supported and there is a seamless way to update your server version, restart the server, and continue to code, all without closing your buffer. With the latest release of coc-metals you can now debug, run, and test your code all within Vim (Nvim only at the moment) with the help of vimspector. You have a new project creator, worksheets that utilize virtual text (Nvim only), and if you want to use the Metals Tree View to explore the API of your dependencies, that's doable. It's great to see the full feature set of Metals supported in Vim if you choose to go the coc-metals route.

The double edged sword

Looking back at the past 6 months, usage of coc.nvim has also skyrocketed. When I first started looking to making a coc plugin, the project had around 6k stars. Less than a year before that, they were less than 1k. Right now they are sitting around almost 12k stars on GitHub. coc.nvim is a success story. They put in the work, created an ecosystem of plugins, and capitalized on the ability of users to port popular language plugins from VS Code over to Vim.

I can't find the quote now, but a while back I read a comment on Reddit about the worry of coc.nvim taking over your terminal and basically becoming a coc.nvim editor powered by Vim. You can see inklings of this when you realize they have their own way to manage plugins, an ecosystem that heavily relies on Node, which is a bit out of the ordinary for a terminal based editor, and when you use coc.nvim, you sort of get locked into the coc.nvim way of doing things.

This is where I'll become a bit opinionated, but in my opinion this all breaks a bit away from the experience I want with Vim, the ability to swap out components and choose how X, Y, or Z is done. I'm a big advocate of the Linux philosophy of Each Program Does One Thing Well, and I like to be able to choose that program. For example, coc.nvim has it's own fzf alternative, there is talk of a built-in symbol explorer, and they basically provide a full experience for everything you'd want in your LSP editing experience if you so choose to use it. This is huge strength, but also giant weakness for other users. For most developers that dabble in Javascript, they'll have Node installed. More than likely even if they don't they'll have Node installed, but it's still something I hear people wish didn't have to be a dependency to use coc-metals.

Overall, I'm very happy with coc-metals, and it's the way I recommend almost anyone to use Metals with Vim. However, there are some more recent things that get me excited and that I'll be shifting my focus and effort to this second half of the year.

nvim-metals

I'm a happy neovim user, and I encourage anyone to take a look at the Vision and Goals of the project. Even with coc-metals, I think it greatly enhances your experience with the way it handles messages, offers virtual text which unlocks the decoration protocol, and just offers an overall smoother experience. Apart from that, one of the main things that get me excited is that the Nvim v.0.5.0 release will have built-in LSP support. This is ideal and the way Nvim is approaching LSP is to have a solid core that offers a myriad of extensibility without providing all the shiny bells and whistles out of the gate. Much of the integration is call-back based which allows users and plugin authors to define custom callbacks for basically all of the functionality that you'd want in your LSP experience. This has already worked great with the emergence of new plugins like completion-nvim and diagnostic-nvim which greatly enhance the Nvim LSP experience. The future for native LSP support in Nvim looks really bright in my opinion with plugins written in Lua and everything being easily extendable or swapped out, and that's the experience I'm looking for. It feels much closer to the mindset of Each Program Does One Thing Well like I mentioned above without one taking over your entire system. It was for this reason that I started nvim-metals which at the moment serves and an example of how to setup Metals with the native LSP support in Nvim, and also provides some extra functionality that you'll need at the moment. Keep in mind that this is all changing rapidly, stuff is breaking, and there is no stable for really any of this. However, I'm very excited about it, and I envision myself fully switching from coc-metals to nvim-metals in the next 6 months. It will take more work to set up, especially in its current state, but eventually I believe it will provide a faster, more stable, and more extensible Metals + Nvim experience.

An Ode to Metals

I'd be amiss in this not to mention that Metals does a fantastic job of allowing editors of all kinds and all levels of support to have a solid editing experience. It's something I personally work hard at to ensure that whatever type of client is hooking up to Metals has a positive experience with the setup and configuration. It sort of goes unseen, but there is a lot of work that has gone into Metals to ensure that that all sorts of clients with all levels of support are taken care of. A few examples:

Does the client support snippets in completions?
Does the client have the ability to use virtual text or should it be given comments in worksheets?
Does the client have the ability to show Doctor results, or should Metals take care of launching it in the browser for you? You can choose html or json for this.
Plus many more...

It's because of this that an editor like Nvim with growing LSP support can still provide a great experience with Metals -- how well configuring the server works and also how open the team is to making necessary changes to ensure all editors are having a good experience. I couldn't be more proud to be part of the team.

The rest of the Ecosystem

I'd also like to touch a bit on the Vim and Scala ecosystem outside of Metals. I mentioned the vim-scala plugin earlier by Derek Wyatt. Every Scala developer that has used Vim to edit any Scala owes him a high five. The default syntax highlighting that actually ships with Vim is from that plugin. With that being said, I don't actually recommend usage of that plugin currently mainly because I think tooling in Scala has grown to a point where it offers everything that the plugin can do via Scalafmt or Scalafix. Organizing imports in an intelligent way was a huge piece of the missing puzzle for me as a Metals user until the recent scalafix-organize-imports rule by Cheng Lian was released. This is a godsend.

Another semi-related thing to this that excites me is the future of alternative syntax highlighting systems like tree-sitter. The great part is that a native tree-sitter module is also planed for the next major release of Nvim. I've recently been looking into the Scala grammar for tree-sitter, and I'm hopeful that the future holds the ability to use tree-sitter for syntax highlight for Scala in Nvim. I think this will be a game changer.

This article is radically incomplete since I know there are users out there that use Metals with some of the other Vim LSP clients, and maybe even some without any client and just plane old vanilla Vim. However, I wanted to give a bit of insight into the creation of coc-metals, the start of nvim-metals, and a bit of context to how I see the Scala and Vim experience. I hope to continually work at improving this, and hopefully by the end of the year I'll hear more people say, Scala support in Vim is fantastic. That's my goal anyways.

Cheers, and thanks for reading.

A Bloop Tour for Metals users

Chris Kipp — Sun, 24 May 2020 00:00:00 +0200

A Bloop Tour for Metals users

Many of the common questions I see around the Metals ecosystem have to do with Bloop. Questions like 'What is Bloop?', 'What's Bloop's role in Metals?', and 'What does the Bloop CLI do?' are somewhat commonplace. If we step outside of the Metals world, the mystery is even more enhanced. I've talked to seasoned Scala devs before that weren't aware of Bloop. All this makes it a perfect candidate for this mini-series on under-appreciated Scala tools after my last Coursier post. I don't pretend to be an expert in Bloop at all, but I've been lucky enough to work closely with those that are and have hopefully been able to pick up enough to help spread the word, especially in relation to Metals.

What is Bloop

First things first, taken in part from the Bloop website: Bloop is a build server and CLI tool for the Scala programming language built by the Scala Center and spearheaded by Jorge Vicente Cantero and Martin Duhem. Bloop has two main goals:

Compile, test, and run Scala code as fast as possible
Integrate easily with build tools, command-line applications, editors, and custom tooling

While those two points are quite clear, there are a couple other points worth noting to help with the general understanding of Bloop. Firstly, Bloop is a server, and like other servers, it responds to requests from clients. These clients can be something like Metals, communicating with Bloop via the Build Server Protocol (BSP), or Bloop CLI, which is communicating with Bloop via the Nailgun server protocol. They can be happening concurrently, caching compilations across different clients, and offering client isolation to avoid conflicts in a shared, stateful file system.

The What is Bloop portion of the website does a great job at outlining in more detail what I have listed above, along with some of the design goals of the project.

Metals and Bloop

If you've seen any of the presentations done about Metals, then you've probably seen a graphic similar to what I have below. However, the one below is a bit more simplified showing only one language client and one build tool.

This diagram is meant to show Bloop's part in the flow from build definition to your editor of choice. If you start on the left, you have your build definition in sbt for example. If you've used Metals before, you'll notice that when you first open up a project you'll be prompted to Import Your Build. What does this mean? The first time this happens, it means that Metals detects that you have no .bloop directory and therefore you need to import your build. When using sbt this means that Metals actually adds the sbt-bloop plugin to your build in project/metals.sbt. It then issues an sbt bloopInstall command which will dump out your build definition for all of your modules into JSON files. You can see this if you open up your .bloop directory in your workspace. These files contain things like your directory information, what dependencies your module has on other modules, all your classpath information, and more. Go ahead and take a look at everything in there. Once this information is gathered, Metals tells Bloop to compile your project.

At this point, if there are any errors during compilation, diagnostics are forwarded from Bloop to Metals and then to your client for you to see. If you fix the diagnostic, hit save, the save event is sent to Metals which then forwards that to Bloop to compile what has been changed. You can start to imagine the flow from the diagram above.

At this point, one of the questions you may have is how does Bloop start? Does Metals start it? Does Metals install Bloop? There can only be one Bloop server running on a machine. So when Metals is about to start, one of two things happens. Using Bloop Launcher either a Bloop server is detected and running, which Metals connects to, or the launcher starts one.

If you've ever used the debugging features in Metals, you have also utilized the DAP (Debug Adaptor Protocol) support that Bloop offers. You can find the entire debugging reference here.

Bloop CLI

Especially for Metals users, I highly recommend using Bloop CLI when you have a simple workflow of compiling, testing, compiling, etc. Up above I mentioned that compilations are cached for different clients. The power in this can be witnessed when you're in Metals, have your project compiled, and can run a test via the Bloop CLI and see it start basically immediately without another compilation happening. Personally, I've fully moved from running tests through sbt to running them through Bloop CLI simply because of how much faster it is for me to go from executing the bloop test <project> to seeing the test run. If you run the same test through sbt you normally have both a longer startup time and also compilation that needs to happen.

Many of the things that you would imagine using like, targeting a specific test suite, watching a test suite, passing in arguments, or testing upstream projects all exist. However, there are a few differences and nuances worth pointing out.

Pointers

You can't run bloop compile or bloop test without specifying a project. You need to use command substitution like below if you want to compile or test all projects. There is no implicit root project.

❯ bloop compile $(bloop projects)

By default compilation requires compilation of all downstream projects, however, to compile upstream projects (transitive projects) that depend on your project, use the --cascade flag.

❯ bloop compile --cascade root
Compiling root (1 Scala source)
Compiled root (982ms)
Compiling root-test (1 Scala source)
Compiled root-test (344ms)

If you want to clean your cache and ensure that you have all downstream projects cleaned as well, use the --propagate flag.

❯ bloop clean --propagate root

When running test suites, Bloop offers completions on your suite names. In order for this to work you need to make sure you have fully compiled before you'll see completions offered.
One thing to keep in mind is that the Bloop CLI doesn't fully replace the need for sbt shell or commands. More advanced custom workflows still need to be configured and done via sbt like publishing, packaging, and other integrations with things like Docker, native images, etc.
If you need a repl, bloop console will drop you into an Ammonite shell in the targeted project.

❯ bloop console root
Loading...
Welcome to the Ammonite Repl 2.1.4-2-ef9b0a0 (Scala 2.12.11 Java 1.8.0_242)
@

You can generate graphviz diagrams to view your dependencies with a command like the one found below.

❯ bloop projects --dot-graph | dot -Tsvg -o metals-diagram.svg && open metals-diagram.svg

Takeaways and Things to keep in mind

Whenever you run, test, or debug in Metals, it's all being powered by Bloop.
Compilations are cached. Consider using the Bloop CLI rather than your build tool to run your tests. You can actually witness this compile deduplication if you are watching a test with -w and also compiling a project with Metals. You'll see a message like this in your bloop output:

Deduplicating compilation of root from bsp client 'Metals 0.9.0+139-c169b4ce-SNAPSHOT' (since 39.439s)

The Bloop CLI portion of the website has a great tutorial that goes through usage of the tool, all the flags, etc. It's a great guide to get started with if you haven't used the Bloop CLI at all yet, or want to freshen up on all the available flags.
There are many integrations with Bloop including sbt, Mill, Maven, Gradle, and more. Take some time to check them all out, and appreciate the breadth of tools that rely on Bloop.
Keep in mind that if Metals starts Bloop, when you close Metals, it doesn't shut down the Bloop server. This is intentional. If you want to shut it down, you'll need to run a bloop exit.
Just to make the distinction, worksheets rely on mdoc and the recently merged in Ammonite support relies on Ammonite. Neither of these use Bloop. So if you have an issue with either of them, blowing away your .bloop directory won't help.

I use Bloop daily, and you may be using it as well without even realizing it. It's an incredible tool that has set the bar for how we run, compile, and test Scala code. If you haven't yet, take some time and head on over to the Bloop webiste, give the Bloop CLI a try, and tell a friend or colleague about it.

A special thanks to Ólafur for reading this over and always providing valuable feedback.

Thanks for stopping by.

Useful cs commands

Chris Kipp — Sat, 4 Apr 2020 00:00:00 +0200

Useful cs commands

Useful Coursier commands for every dev

The Scala ecosystem has a lot of incredible tools, and for some reason I feel like the we do a poor job at highlighting them as well as showing examples of how useful they can be to us on a daily basis. One of these tools that I believe deserves more attention than it does is Coursier.

Coursier

I often hear people mention Coursier mainly when they're talking about sbt, and how as of sbt 1.3, it ships with Coursier out of the box and utilizes it to fetch dependencies. While that's where the knowledge of Coursier ends for some, I want to show a few use cases of Coursier that I think are both simple and extremely useful for daily things we do as Scala devs.

Coursier CLI

Over the past couple of weeks I've noticed my usage of Coursier CLI spike as I've been either shown or come across new cs commands. cs is the native launcher for Coursier. You can find information on how to install it here along with a nice overview of the few commands I'll show in this post and some others I won't cover. Again, my goal for this isn't to do an in-depth dive into all the ways you can use cs but to give you some practical examples with a few commands that you should be able to start using right away. The following examples are all real scenarios that I came across in the last couple weeks.

resolve

While at work, I was in a conversation in a pull request about how my team didn't love the Java Api of Caffeine, which we were using for some caching. We came across Scaffeine, which is a thin Scala wrapper around the library. Great! However, we try to be mindful of how many dependencies we add, and the question came up of how many transitive dependencies we'd be bringing in by including this. So the question becomes: what is the easiest way to check this? Enter cs resolve. The resolve command is a quick way to print out transitive dependencies of one or more other dependencies. So, I did the following:

❯ cs resolve com.github.blemale:scaffeine_2.12:3.1.0
https://repo1.maven.org/maven2/com/github/blemale/scaffeine_2.12/3.1.0/scaffeine_2.12-3.1.0.pom
  100.0% [##########] 2.6 KiB (7.6 KiB / s)
https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.8/scala-library-2.12.8.pom
  100.0% [##########] 1.6 KiB (26.0 KiB / s)
https://repo1.maven.org/maven2/org/scala-lang/modules/scala-java8-compat_2.12/0.9.0/scala-java8-compat_2.12-0.9.0.pom
  100.0% [##########] 2.3 KiB (14.4 KiB / s)
https://repo1.maven.org/maven2/com/github/ben-manes/caffeine/caffeine/2.8.0/caffeine-2.8.0.pom
  100.0% [##########] 5.2 KiB (25.8 KiB / s)
https://repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.3.3/error_prone_annotations-2.3.3.pom
  100.0% [##########] 1.7 KiB (29.5 KiB / s)
https://repo1.maven.org/maven2/org/checkerframework/checker-qual/2.10.0/checker-qual-2.10.0.pom
  100.0% [##########] 2.3 KiB (39.7 KiB / s)
https://repo1.maven.org/maven2/com/google/errorprone/error_prone_parent/2.3.3/error_prone_parent-2.3.3.pom
  100.0% [##########] 5.2 KiB (62.5 KiB / s)
com.github.ben-manes.caffeine:caffeine:2.8.0:default
com.github.blemale:scaffeine_2.12:3.1.0:default
com.google.errorprone:error_prone_annotations:2.3.3:default
org.checkerframework:checker-qual:2.10.0:default
org.scala-lang:scala-library:2.12.8:default
org.scala-lang.modules:scala-java8-compat_2.12:0.9.0:default

As you can see, the full artifacts weren't fetched, only the metadata necessary to be able to list the transitive dependencies, which can be found at the bottom. You can also get a nice tree view of this by passing in the --tree or -t option.

❯ cs resolve -t com.github.blemale:scaffeine_2.12:3.1.0
  Result:
└─ com.github.blemale:scaffeine_2.12:3.1.0
   ├─ com.github.ben-manes.caffeine:caffeine:2.8.0
   │  ├─ com.google.errorprone:error_prone_annotations:2.3.3
   │  └─ org.checkerframework:checker-qual:2.10.0
   ├─ org.scala-lang:scala-library:2.12.8
   └─ org.scala-lang.modules:scala-java8-compat_2.12:0.9.0
      └─ org.scala-lang:scala-library:2.12.4 -> 2.12.8

This is such a quick way to be able to see the dependency tree of a dependency without adding it into your build or looking through build files.

install

Another useful command is the cs install command which allows you to install a launcher for a JVM-based application. The docs do a great job at explaining how this all works, but think of cs install as an apt install or brew install alternative for JVM-based apps. You can install, update, and uninstall apps that you want to use locally just like you would with <enter global package manager name here>. For my situation, I'm a big fan of the mdoc tool, but I normally use it when it's included in a project already. For this situation I just wanted to test something locally without including it in the project. I was reviewing an .adoc document and was curious if mdoc supported them the same way it does .md files. So I simply did the following:

❯ cs install mdoc
...
...
...
Wrote mdoc

This then allowed me full access to the mdoc cli commands, which I could use locally without having to include it in my project to test something out.

complete

Speaking of mdoc, I was using it in another project, and I was curious what the latest version was. I was actually using it in an Ammonite script (to create this website), so I didn't have access to the sbt dependencyUpdates command that I would typically use. I ended up looking on GitHub for the version, which lead me to the site, which lead me to the version. Ironically, I sent in a pr to add in a badge to the readme to make the latest version more discoverable when the author of mdoc, Ólafur Páll Geirsson, introduced me to this gem:

❯ cs complete org.scalameta:mdoc_2.13:
1.3.2
1.3.4
1.3.5
1.3.6
1.4.0-RC2
1.4.0-RC3
2.0.0
2.0.1
2.0.2
2.0.3
2.1.0
2.1.1
2.1.2
2.1.3
2.1.4
2.1.5

cs complete isn't actually on the Coursier website, but you can use it to give you all the options to complete the artifact that you are looking for. This provides such a simple way to explore what versions are cross published and available.

Tell your friends

I want re-iterate that I think there are a ton of great tools in the Scala ecosystem, but not everyone is aware of them. Hopefully, the few real-life examples above illustrate this in a small way. Next time someone mentions Coursier, ask them if they've used any of the cs commands, and show them a couple tips. Let's spread the word about the great tools we have access to.

Cheers.

How is this site built

Chris Kipp — Sun, 23 Feb 2020 00:00:00 +0100

How is this site built

How is this site built?

If you just want to know exactly how this site is built and don't care about the history, skip down to the current-process section. If not, I'll take this opportunity to actual outline the different sites that I've had along the way leading up to this one as they've radically changed along the way.

Version 1

I originally had a site that was fully JavaScript powered, and built from scratch. I even created a tiny micro-framework that I used to build it called gyul. I had a lot of fun with that build, but it was a bit slow on initial load and wouldn't work for visitors that had JS disabled. However I look back at it fondly since it was my first time truly building something that was mine, and it also got me away from using other blogging platforms. This also corresponded with a project I started called ándaga which was a way for me to track my own free-time to try and find patterns. Everything was integrated.

Version 2

As I got more into Scala, I wanted to see if I could generate my site using that instead of JS. This lead me to a collection of tools and a process that I'll outline below.

Create a log via ándaga-cli for a task that I was working on.
Log gets stored into MongoDB by ándaga-core, which are some serverless functions that were running on Vercel
Nightly, I had a script that ran which dumped my DB into a JSON file. This JSON got committed and pushed to the chronica repo, which was the repo of my website.
The push triggered GitHub Actions to run, which ran Ammonite, mdoc, and CommonMark to transform and enrich the markdown into html.
After building the created html pages got uploaded and hosted on Vercel
A similar process happened when I add a blog post and push it up manually

This was also a really fun project to set up and learn more about various tools. However, it was also a bit of a hassle to manage if I ever wanted to change anything. It was sort of like I built the entire system, and then if I wanted to change something, the entire system needed to change. During this time I also stopped logging my free time, which is a whole other story, and also just wanted to simplify the process further. That's what lead me to my current process.

The current process

The way that the site is currently built is quite simple. It just uses Pandoc to transform my markdown pages you see in the repo into html pages. I have some custom html partials that include the various parts of the site and the style. The entire thing is just built with this script via GitHub Actions, and then automatically deployed to Vercel. So far I've found that this site is incredibly fast and easy to manage. Pandoc is also a joy to work with. That's pretty much it, and that's all I really want.

Thanks for stopping by.

Chris

Why my own blog

Chris Kipp — Wed, 10 Jul 2019 00:00:00 +0200

Why my own blog

Why my own blog?

I've been meaning to start this for quite some time. Over the years I've had various forms of blogs. I was on Tumblr for some time and then I had a Medium page, but none of them stuck, and I only have theories of why that is. One of the main reasons I believe is that I didn't have full control over the content. My hope with the blog portion of chronica is that it's a place for me to dump my ramblings, talk about my experiments, and even share some stuff I've been working on.

A while back I read a blog post, that I regretfully can't find anymore, but it resonated more than any other blog post had for quite some time. It made a correlation between the current state of the internet and the current state of traveling. When you travel you are warned of the main streets. Stay away from those shops. It's well known that they are tourist traps. You will get lower quality for a higher price. However, if you put a bit more effort into the finding better places to stop that may be further out, you'll have a much better experience. The quality will better, it may even be cheaper, and it will be more authentic. You may even find other similar places that they recommend for you to check out.

I believe we are falling for the main street of the internet. The easily accessible "stores" and "shops" such as Facebook, Medium, and other common social and blog outlets are by far the easiest place to digest information, but at the cost of quality (albeit not always), and the many times the cost of privacy. I envision the internet experience returning back to a more decentralized platform where individuals have their own site and they own their data. Alternatives to common social platforms do exists, they just take a bit more effort. However, in my experience you're left with a richer experience. Taking the time to research and to find a place a bit farther off the main street is indeed more labor intensive, but rewarding.

My hope is the that my website is just that. A central place that I fully own and have control over where my projects, thoughts, and works can all be displayed. There is no overly invasive trackers, no advertisements, and no abstraction over what you are seeing. Everything is open sourced and transparent. This is an alternative to me using platforms like Medium which bombard you with advertisements and track as much information about you and your readings as possible, or Facebook that caters what you see further enforcing filter bubbles and essentially selling you as their product.

You can read more about how this site is build and works here.

Thanks for stopping by.

Chris