I am working on a library that involves access to graph data, informed by a schema or model. Just to get things rolling I started out with a convention of holding graph nodes in a map:

map_graph = %{"node1" =>
          %{"class_id" => "class1",
            "data" => %{ "testProp" => "value" }}
        %{"node2" =>
          %{"class_id" => "class1",
            "data" => %{ "testProp" => "value2" }}}
node = map_graph["node1"]
value = node["data"]["testProp"]

It’s quick, dirty, and gets the job done for proof-of-concept purposes. Clearly it’s inadequate for other data access patterns we’d like to be able to support. We can write transformation routines against this structure, but they won’t be able to work with anything else.

It’s time for some decoupling.

Behaviors

A Behavior defines an interface or contract that implementing modules must follow. It’s like saying “if you want to be a graph node accessor, you must implement these functions.”

defmodule Graph do
  @callback get_node(node :: any()) :: [any()]
end

defmodule Graph.MapImpl do
  @behaviour Graph

  defstruct [:data]

  @impl true
  def get_node(data, node_id) do
    data[node_id]
  end
end

Now we can do

MapImpl.get_node(map_graph, "node")

Now we have some added flexibility to define a completely different implementation that dishes up nodes from the filesystem based on a root path:

def Graph.FilesystemImpl
  @impl true
  def get_node(%{root: root_path}, node_id) do
    full_path = resolve_path(root_path, node_id)
    case File.stat(full_path) do
      {:ok, %File.Stat{type: :regular}} ->
        %{
          class: "file",
          data: file_data(full_path)
        }
      {:ok, %File.Stat{type: :directory}} ->
        %{
          class: "folder",
          data: folder_data(full_path)
        }
      {:error, _} -> nil
    end
  end

Notice here the data parameter on get_node/2 serves a completely different purpose, instead providing a pointer to where to access the data. This is the first strange smell about this approach.

Now it’s possible to write methods that work on either the map or the filesystem implementation. But you still need to know which one you’re in, and data needs to be formed correctly:

MapImpl.get_node(%{"node" => ...}, "node")
FileSystemImpl.get_node(%{"root" => "."}, "node")

You can make things more convenient with a wrapper struct that carries the implementation and the data, and delegates the functions:

defmodule Graph.Instance do

  @type t :: %__MODULE__{
          impl: module(),
          data: any()
        }

  def get_node(%__MODULE__{impl: impl, data: data}, node_id) do
    impl.get_node(data, node_id)
  end

Then you can write transformation routines that work on instances, and are really independent of the underlying representation.

Protocols

Protocols provide a different form of decoupling. Here’s the equivalent to the Behavior definition from above:

defprotocol Graph do
  def get_node(data, node_id)
end

Then the implementation comes in 2 parts:

defmodule Graph.MapGraph do
  defstruct [:nodes]

  def new() do
    %__MODULE__{
      nodes: %{}
    }
  end
end

defimpl Graph, for: Graph.MapGraph do

  @impl true
  def get_node(%{nodes: graph}, node_id) do
    graph[node_id]
  end
end

Now we don’t need to pass around the Instance wrapper with its impl. Our transforms are written for “anything that can do X” rather than “modules that promise to do X”. The data itself carries the behavior.

graph = Graph.MapGraph.new()
Graph.get_node(graph, "node")

The extra cool part is that since the defimpl is separate from the defmodule, you can use protocols to extend existing types. So imagine we have SomeLibrary.DataStructure that could be interpreted as a graph. With Behaviors we would need to write an entire wrapper module. But with protocols we write a defimpl Graph, for: SomeLibrary.DataStructure. Then when we pass that structure to some graph routine, the method calls will look exactly the same:

graph = SomeLibrary.DataStructure.new()
Graph.get_node(graph, node)

This is a level of decoupling and flexibility that’s hard to imagine in traditional OOP languages.

Summary

With behaviors, you call GraphImpl.get_node(graph, id). With protocols, you call Graph.get_node(graph, id) regardless of the graph’s actual type.

Behaviors are about modules promising to implement certain functions. The module author decides to follow the behavior contract. Complexity is pushed into the infrastructure.

Protocols are about data types being extended with new capabilities. You can implement a protocol for any existing type, even ones you didn’t create. Complexity is pushed into the type system.

I think there’s a nice categorical perspective as well: behaviors as functors in a slice category. Protocols as natural transformations, with the “legs” provided by the implementation. The naturality condition looks something like

graph |> Graph.get_node("node") == graph.nodes |> Map.get("node")

But I’d need to spend some time with pen and paper to convince myself on this one.

Why Elixir/OTP doesn’t need an Agent framework: Part 1

Why Elixir/OTP doesn’t need an Agent framework: Part 2

Sadly these are his only 2 articles. He makes a few short arguments and then backs it up with a razor-sharp tutorial demonstrating scalable LLM pipelines with minimal dependencies and highly readable code ready to extend with pattern-matching. Where else but Elixir are you going to find such ergonomic code compiling to such a battle-hardened VM?

I came to a similar conclusion implementing an “agentic” feature for a client app in Elixir, though my code admittedly wasn’t quite as clean. Others in the ecosystem are reaching similar conclusions: in an Elixir dev discussion on the topic, authors of the leading libraries admit struggling with whether their abstractions were worth it at all! When you’re working in a message-passing, actor-based execution model, you really get straight to the meat of what you want your “agent” to do.

In my experience there’s one key to interacting with LLM-based systems, whether in a chatbot or integrated in your app. That is:

Vigilant management of context.

I’ve done some informal over-the-shoulder mentoring of AI usage for several knowledge workers now. So far 100% of them fall into the same anti-patterns. Put up a sticky note: “Yes, really, you need to start a new chat already, this one is ruined.”

With that in mind I want to briefly survey the LLM dev tooling that I find useful, and suggest some ideas of what the next steps might look like.

Instructor -> Runtime Schemas

The only dependency used in the above tutorials is a port of the Instructor library. The goal of this library is to coerce the LLM response into a structured output. It’s essential for making tasks composable. Why? Because if the output of Task A has a known data model, then it can be used as input to a templated prompt for Task B.

There isn’t really any magic here. Instructor just pulls a subprocess under the hood. Imagine a short back-and-forth happening at the end of every prompt:

> Here's a JSON Schema. Make sure your response conforms.
> OK, here's my response: ___
> I couldn't parse that into JSON, please try again.
> Oops, here you go: ____
> This doesn't conform to the schema: The "name" attribute should be a string!
> Ooops, here you go: ___

and so on. Of course the initial prompt and the schema itself are the biggest factor on the quality of your outcome, but I think this abstraction really belongs in the core toolkit. Nobody wants to re-implement this in every task.

The shortcoming I’ve found working with this library is that your schemas (effectively) need to be defined at compile time. So imagine you’re doing data pipeline generation and want a workflow where one task defines a schema, and another to adhere to it. It’s a challenge with off-the-shelf tooling (to me, at least - YMMV and please let me know if it does).

I’ve done a proof-of-concept with my Paradigm library, which is designed for runtime schema manipulation. But I haven’t gotten around to a full Paradigm_Instructor implementation that handles the iterative coercion steps.

Zed -> Editor agents that don’t munch tokens

I’ve been watching the Zed text editor for a while, and recently made the switch from Sublime as my daily driver. Besides just being a nice modern and performant editor, it has LLM features baked in from the ground up. The two features I really love are

1. The “Text Thread” chat panel. There are shortcuts to add files and selections, and the entire thing just acts as a normal text buffer so you can rewrite history or delete tangents that would otherwise poison your context in a browser-based tool. This alone makes it worth using because (as I mentioned above), hawk-eyed management of context is key to successful use of LLMs.

2. The inline editor in your main text buffer. Basically your cursor (or selection) gives it the scope that it’s allowed to rewrite. Then, your Text Thread contents can be pulled in to provide extra context. To me, this is the perfect LLM workflow. It’s very efficient on tokens, too: I can use it all day long and spend pennies on API calls.

Their recent update leans heavily into Agentic editing. I’ve had mixed results. Sometimes it knocks through something viable, but sometimes it goes down terrible rabbit-holes where it gets nothing done. Either way it burns through tokens like crazy, so I tend to not reach for it. Even with minimal MCP handles, the baseline system prompt puts it at 5-6k tokens just to play ball. And that’s not something you can customize.

I’m looking for a middle ground between these two modes.

1. Completely manual context management with scoped edit permissions (my current preferred way to work).
2. Token-munching agentic mode with MCP handles for full codebase access and terminal (largely deranged and ineffective)

I’m not going to work on it, but I’ll know it when I see it! One sign I’m watching for is transparency in the MCP bootstrap process. Can we dispense with huge system prompts? Can we give it handles to expand tools, so that some MCP categories never even enter the context? It’s not just a matter of frugality of API calls. The workflow is critical to the quality of the outcome, and the full-scope editor apps are just not there yet.

Honorable Mention

For a really nice take on sandboxing + transparency (fish-tanking?) I have to shout out this claudebox project.