Combined Feed

Using use in Gleam

2024-07-27T00:00:00+00:00

Using `use` in Gleam

Recently, a colleague checked out Gleam’s language tour. They liked what they saw, but they were confused by Gleam’s use syntax. I like Gleam’s syntax a lotI even wrote an article about Gleam’s syntax., but I was also confused by use when I first encountered it. Here’s how I use use:The use expression was introduced in Gleam v0.25, as a more general replacement for the try keyword that preceded it.

What is `use` anyway?

use is a Gleam expression that allows me to write code that uses a callbackI’ve heard debate over whether a function argument to a higher order function is always a callback, but the language tour for Gleam uses “callback function”, so I’ve used callback here. function in an unindented style. Specifically, for functions whose last argument is a callback function. For example:

import gleam/list

fn catify_without_use(strings: List(String)) -> List(String) {
  list.map(strings, fn(string) {
    string <> " cat"
  })
}

Here, I take a list of strings and append " cat" to the end of each of them. list.map takes a list as the first argument, and a callback function that takes a list element and returns a new list element. Here’s what that same function looks like with a use expression:

import gleam/list

fn catify_with_use(strings: List(String)) -> List(String) {
  use string <- list.map(strings)
  string <> " cat"
}

Here the use expression has done three things:

First, we’ve moved the arguments for the callback function to the left of the arrow. fn(string) becomes use string <-.The name string is an identifier chosen by me, it could just as easily be use to_be_catted <- ...

Second, we’ve changed list.map into a function that takes one less argument than it usually does. It’s now list.map(strings) as though it only took one argument. list.map still takes two arguments, but use has changed how we write that second argument.This aspect of use makes it easier to see what the non callback arguments of a used function are, since the callback is often the bulkiest part of the outer function call signature.The use statement has exactly one identifier, because that’s how many arguments list.map’s callback function needs, see this note for more examples.

Third, the body of the callback function is now below the line of the use expression, indented in line with the use expression. Everything below the use expression until the end of catify_with_use becomes the body for the callback function.That is, everything in the same block as the use expression. See this note for more explanation.

So What?

I’ve introduced some syntax sugarA use expression is syntactic sugar for a regular call and an anonymous function. During compilation, a use expression expands into the fn(arg1, arg2, ...) { body() } form. It’s explicitly to allow a different way of writing expressions. that allows me to change how I write functions that take a callback function as their last argument. But in my example, it doesn’t help me much. In the example, it’s no longer clear that I’m writing a callback function, and I can’t do anything else after my list.map call, that doesn’t end up in the callback function. list.map can be used with a use expression, but it’s a poor choice over the default syntax in most cases.I chose list.map because it’s a commonly used Gleam function that is a poor fit for use. list.map is one of my favorite functions though, and I’ve used it a lot in one of my other Gleam articles.

So what is use useful for?

`result.unwrap` and early returns

Gleam has no exceptions, all errors must be returned as values from a function.Gleam also has no return keyword, the last expression in a block is returned as the value of that block. In a function, that’s the last thing in the function. Specifically, Gleam uses the convention of Result to capture this information. Success looks like Ok(value) and failure looks like Error(reason).

But what if I want to do something that might fail, and then continue to do something else in the same function.

import gleam/result

fn outer() -> Result(success, failure) {
  let id = parse_id() |> result.unwrap(0)
  ... // More code that uses the id
}

parse_id might fail, and therefore returns a Result. To use the inner value, we need to unwrap it somehow. On a successful parse, the wrapped value will look like Ok(id), and this code uses result.unwrap to pull out the id on an Ok case, or set the id to 0 in the Error case.

But 0 as an id is made up, and likely has no reliable meaning in our system. If we wanted to convey that the id failed to parse, we already had an Error that we could have returned directly.

Instead of using result.unwrap, we can use result.mapBoth list.map and result.map are named map, because they are examples of the higher order function map that applies a function to every element of collection.In the case of result.map the “collection” is only the contents of the Ok, Errors are simply returned without invoking the callback function., which takes a Result and a callback function, where the callback function is only invoked when the Result is Ok. If the Result is an Error then it returns the Error.This is how Gleam can do an “early” return, result.map only evaluates the callback function on success, so in the failure case the value of the result.map expression (and therefore the block) is the Error, and it doesn’t evaluate any more code. The callback function gets the unwrapped, inner value as its one argument.

So we can do:

import gleam/result

fn outer() -> Result(success, failure) {
  result.map(parse_id(), fn(id) {
    ... // More code that uses the id
  })
}

The problem with invoking result.map this way, is that now all of the internals of outer are indented inside the callback function. Here, we can use a use expression to eliminate the extra indentation and focus our function on the success case:

import gleam/result

fn outer() -> Result(success, failure) {
  use id <- result.map(parse_id())
  ... // More code that uses the id
}

We have still accomplished what we wanted to accomplish, which is that if parse_id fails, it returns early with the Error.If you’re familiar with Rust’s ? operator, the expression use id <- result.map(parse_id()) is equivalent to Rust’s let id = parse_id()?; Now, we can focus our codeand limited attention on the unwrapped id in the success case.

Avoiding boilerplate with `result.map`

A use expression can also allow you to avoid a lot of boilerplate. For example, reading from a file is an operation that can fail, so it returns a Result. If I want to read lines from a file and then transform them, I can use result.map:

import gleam/result
import gleam/list

fn transform_lines() {
  read_file_lines()
  |> result.map(list.filter(...))
  |> result.map(list.map(...))
  |> result.map(list.sort(...))
  |> result.map(something_else())
}

but because result.map returns a new Result, I have to continue chaining result.map calls until I’ve finished all of the transformations that I need.

A use expression allows us to gracefully handle the failure case, while removing the need for chained calls to result.map:

import gleam/result
import gleam/list

fn transform_lines() {
  use lines <- result.map(read_file_lines())

  lines
  |> list.filter(...)
  |> list.map(...)
  |> list.sort(...)
  |> something_else()
}

These two functions are equivalent, but the use expression allows us to focus on the transformations we care about.Since use is only syntax sugar, all of the piped transformations could be written inside result.map’s callback function without a use expression:fn(lines) { ... }.

Chaining `result.try`

use expressions are especially helpful when doing multiple different things that might fail. result.try takes a Result and callback function that itself returns a Result.try the first thing, then try a second thing if the first one succeeds.I’ve heard this concept of chaining result.try calls together referred to as railroad-oriented design. If any operation fails, it switches to the Error “track” and returns that Error, otherwise it continues along the Ok track until the next operation that could fail, which has another “railroad switch”. If the first argument is an Error it returns that error. Otherwise, it evaluates the callback function and returns whatever Result that callback function does. For example:

import gleam/result

fn handle_form(form_data: RegistrationForm) {
  result.try(
    unique_username(form_data.username), 
    fn(username) {
      result.try(
        validate_password(form_data.password), 
        fn(password) {
          result.map(
            register_user(username, password), 
            fn(user) {
              "welcome " <> user.username <> "!"
            }
          )
        }
      )
    }
  )
}

Because each operation can separately fail, we can’t chain these together like I did in the result.map boilerplate example. This creates a cascade of indented callback functions that makes it hard to keep track of what’s going on and what the final return value in the success case is.I’ve heard this referred to as callback hell. With use expressions the meaning is much clearer:

import gleam/result

fn handle_form(form_data: RegistrationForm) {
  use username <- result.try(unique_username(form_data.username))
  use password <- result.try(validate_password(form_data.password))
  use user <- result.map(register_user(username, password))
  
  "welcome " <> user.username <> "!"
}

Here it’s much easier to tell which operations we’re doing, and what the final return value is.The last operation uses result.map because we’re finally returning something that can’t fail. It was the same in the first example, did you notice?

Context Management

Another place where use expressions shine is with context management: functions that do setup, cleanup, or both. In Gleam, there’s no special way of handling context management, and so these functions use a callback function as an argument to wrap the user behavior they’re managing. For example, opening a database connection with sqlight:

import sqlight

fn get_data() {
  use conn <- sqlight.with_connection("my_database.db")
  
  ... // query the database
}

sqlight.with_connection will open a connection to the database, execute code, and then close the connection afterwards. The database connection is available as conn for the rest of the function. Technically, all of the user code is wrapped in a function:

import sqlight

fn get_data() {
  sqlight.with_connection("my_database.db", fn(conn) {
       ... // query the database
  })
}

but the use expression allows us to focus on the query we want to write, not on database management.

Another example of this kind of callback function wrapper comes from wisp, Gleam’s web framework:This example comes from wisp’s logging example.

import wisp

pub fn middleware(
  req: wisp.Request,
  handle_request: fn(wisp.Request) -> wisp.Response,
) -> wisp.Response {
  let req = wisp.method_override(req)
  use <- wisp.log_request(req)
  use <- wisp.rescue_crashes
  use req <- wisp.handle_head(req)

  handle_request(req)
}

Here wisp.log_request uses a callback function to allow logging to take place after a request has been handled, regardless of how the user of wisp chooses to do that handling.

The other wisp functions use a similar pattern allowing for customization to the application while still handling core web application concerns.

This is also an example of use expressions preventing cascading nested callbacks, without use:

import wisp

pub fn middleware(
  req: wisp.Request,
  handle_request: fn(wisp.Request) -> wisp.Response,
) -> wisp.Response {
  let req = wisp.method_override(req)
  wisp.log_request(req, fn() {
    wisp.rescue_crashes(fn() {
      wisp.handle_head(req, fn(req) { 
        handle_request(req)
      })
    })
  })
}

Without use, how the request is being handled is obscured by the nested callback functions used to manage wisp request context.This example has three of these context manager functions, but I’ve seen wisp applications with 8 or more. You could write that without use, but the use expressions allow the custom logic to remain readable.

Takeaways

I’ve shared a number of examples where use expressions can add clarity to code, both with error handling and context management. As I showed in the list.map example, use expressions aren’t always helpful. The key is to use use when it allows you to highlight the happy path of your code, while handling concerns like failure and logging.Thank you to Nicole, Jeff Miller, and Mark, as well as the Gleam discord for helping me write and edit this article.

A use expression is syntax sugar, and it’s always possible to write Gleam code without it, though maybe not as clearly.Why use, specifically, as a keyword? This issue highlighted the need for a general syntax sugar. with statements in koka were identified as a similar solution, but with was already a well-used special form in Elixir, and Gleam didn’t want to confuse BEAM programmers coming from Elixir.After a lot of discussion use was chosen, in part because it’s the same length as let and it wasn’t already in use anywhere.

Additional Notes

A note of `use` scope

Everything below the use expression comprises the body of the callback function until the end of the current block. By default, this will be the end of the function, but we can use {} to create a smaller block:From the language tour: “use is an expression like everything else in Gleam, so it can be placed within blocks.”

import gleam/result

fn example() {
  let smaller_block = {
    use value <- result.try(thing_that_might_fail())
    ... // do something with the value
  }

  no_longer_in_use_callback(smaller_block)
}

A example of this comes from the new decode library:This example comes from the README for decode.

let decoder =
  decode.into({
    use name <- decode.parameter
    use email <- decode.parameter
    use is_admin <- decode.parameter
    User(name, email, is_admin)
  })
  |> decode.field("name", decode.string)
  |> decode.field("email", decode.string)
  |> decode.field("is-admin", decode.bool)

decoder
|> decode.from(data)

Here, decode.into is using a {} block to succinctly create a decoder function using a combination of use and decode.parameter.This is a clever use of use as syntax sugar, since decode.parameter simply returns its argument. So the above block translates to:fn(name) { fn(email) { fn(is_admin) { User(name, email, is_admin) }}}Which is much harder to read, especially since the ordering of fields matters.

A note on `use` arguments

The number of arguments in the use expression are exactly the same as the arguments required for the callback function being replaced.

`bool.guard`

bool.guard takes a function that requires no arguments:fn() -> a

// without `use`
bool.guard(condition, "early return", fn() {
  ...
  "late return"
})

// with `use`
use <- bool.guard(condition, "early return")
...
"late return"

`list.fold`

list.fold takes a functions that requires two arguments.fn(a, a) -> a

// without `use`
list.fold(numbers, 1, fn(accumulator, element) {
  accumulator * element
})

// with `use`
use accumulator, element <- list.fold(numbers, 1)
accumulator * element

Tools I Use: firefox

2024-07-15T00:00:00+00:00

Tools I Use: firefox

Firefox is my daily use browser. I’ve tried a number of browsers over the years, but I keep coming back to Firefox. Here’s a bunch of tools and features I use in Firefox:

Bookmark Keywords

Firefox allows me to add keywords to bookmarks. I can type these abbreviated shortcuts into the address bar instead of the full link. I can add keywords to bookmarks from the bookmark manager.I learned about bookmark keywords from this excellent blog post.Here is a brief note in the documentation on bookmark keywords.

For example, I’ve set the shortcut e for my website (erikarow.land), since I use my own website as reference notes.

Now I can type e in the address bar and go directly to my website.I often use the ctrl+l shortcut to quickly access the address bar. On MacOS the shortcut is cmd+l.

Text substitution

In addition to setting a keyword for a static bookmark, I can also use %s in the bookmark url for text substitution.

For example, I can set a bookmark for

https://bookshop.org/search?keywords=%s

with a keyword of bk. Then I can type bk data feminism in the address bar to search for that book.I own Data Feminism in physical copy already. You can read it online for free here.And download an open access pdf here.Check out the research the Data Feminism lab is doing now at their website.

While working on a project, I could use bookmark keywords and text substitution to search through tickets or code forge issues.

Address bar search (tab search)

The Firefox address bar can be used to quickly access a search engine. But Firefox also has alternate search shortcuts that can be used to narrow the suggestions that the address bar gives as autocompletions.I first heard of these from this article.

The autocomplete search I use most often is tab search. If I open the address bar and type % followed by a space, it autocompletes on only my open tabs. It’s often easier for me to search for an open tab than to manually scan for it, especially if I have a large number of open tabs.

There are a number of these built in:List retrieved from the firefox documentation.

Add ^ to show only matches in your browsing history.
Add * to show only matches in your bookmarks.
Add + to show only matches in bookmarks you’ve tagged.
Add % to show only matches in your currently open tabs.
Add # to show only matches where every search term is part of the title or part of a tag.
Add $ to show only matches where every search term is part of the web address (URL). The text “https://” or “http://” in the URL is ignored, but not “file:///”.
Add ? to show only search suggestions.

Screenshot Tool

Firefox has a built in screenshot tool which I wrote about here.

Process Manager

Firefox has a built in process manager that allows me to see the CPU and Memory usage of every tab that I have open. With the browser acting as application platform, this is quite useful.

The process manager gives me the ability to unload tabs and kill the process for a specific website. When I want to do something memory intensive on my computer, I can unload a bunch of expensive browser applications.

The process manager can be opened with shift+escape

Developer Tools

Firefox has a suite of web developer tools. There’s way more than I know how to use, but I use some of the tools often:

Mobile View

Firefox has a responsive design mode, which allows me to see what my website would look like at different resolutions. I use the response design mode to see what my website would look and act like on a mobile screen. My sidenotes wouldn’t work unchanged on a mobile screen, so they turn into expandable notes.

Inspector

I often use the inspector tool to see how a specific element of a screen was formatted or styled. What HTML did they use? What CSS styles are active on that component?

Developer Tool Keyboard shortcuts

Developer Tools | F12 or ctrl+shift+iA list of all the developer tool shortcuts can be found here.Non developer shortcuts can be found here.
Mobile View | ctrl+shift+m
Inspector | ctrl+shift+c
Console | ctrl+shift+k

Pinned Tabs

I like to pin tabs that I refer to frequently, typically social mediaMastodon, etc. and chat applicationsCommunity Slacks, etc. that I use from the browser.I tend to use the browser clients for these chat applications because of how easy it is to find them with the pinned tab feature.

A pinned tab shows only an icon instead of a title, stays to the left of the browser window it’s pinned to, and does not get closed when using ctrl+f4 or ctrl+w to close a tab.A pinned tab must be unpinned before it can be closed by keyboard shortcuts. Right clicking the tab and selecting “Close Tab” will still work.

I can pin a tab by right clicking that tab, and selecting “Pin Tab” from the menu.

Always Show Scroll Bars

By default, Firefox will autohide scroll bars after a short while. There is a setting to have scrollbars always show. I wrote an article about that here.

Tab Settings

I have Firefox set to “Open previous windows and tabs” on startup. This restore makes it easier to close Firefox when I need to reboot or free up the memory firefox uses for something else.

I also turn off the setting for “Ctrl+Tab cycles through tabs in recently used order”. I want my tab cycling to be predictable.Both of these tab settings are found under Settings -> General.Learn more about restoring sessions here.

Extensions

The extensions I use are:

uBlock Origin

uBlock Origin is a memory efficient ad blocker. It works, no complaints.It works so well that I forget how many ads there are the internet, only to be rudely reminded when I use other computers or browsers without uBlock Origin.

ClearURLs and Privacy Badger

ClearURLs automatically removes tracking elements from URLs. When I click links that have tracking components, this extension will automatically strip them out.

Privacy Badger detects any remaining third-party trackers that follow me from site to site, and stops them.

Both of these extensions work to improve my privacy, but when they’re working correctly I don’t notice them.

DF YouTube (Distraction Free)

DF Youtube strips nearly everything from the YouTube UI. No comments. No sidebar. No recommended videos.

I’m left with a search bar and the playing video and description, and my focus.

I use DF YouTube to view videos on YouTube without getting distracted or stuck in a YouTube rabbit hole.

OneTab

OneTab will compress open tabs into a simple list with much smaller memory usage. It allows me to archive what I was working on without using all of my computer’s memory.

All tabs get saved to the same list, separated into groups based on when they were added to the “One Tab”. This list format is convenient for text search.

Previously, I would “archive” things I wanted to read by bookmarking them to come back to later, now I use OneTab for this purpose.Like my bookmarks, I tend to add things to the list much quicker than I remove things. At time of writing, I have 11,000+ “tabs” listed in OneTab.

Block Site

Block Site allows me to restrict access to specific domains. When navigating to a blocked domain, instead of seeing that site, I see a specific “Restricted Access” screen created by the extension.

I use this to circuit break websites that I check compulsively. I can temporarily disable the extension at any time to navigate to those websites, but having the restricted screen is enough of a mental block that I usually stop checking those websites as often.

`about:robots`

Firefox has a hidden about:robots page with references to pop culture.Thanks to someone in the Gleam discord for sharing this with me.

Things I like about Gleam's Syntax

2024-07-06T00:00:00+00:00

Things I like about Gleam’s Syntax

Gleam is a Type-Safe programming language that transpiles to ErlangAnd javascript., so you can run its code on the BEAM. I’ve slept on Gleam for a while because its syntax. It was too much Rust and not enough Elixir for me.

Recently, I was reminded of Gleam when someone mentioned that Gleam doesn’t have an if statement, but instead uses case for everything. As a consequence, I ended up re-reading the Gleam language tour.

As another reminder to check my assumptions from time to time, Gleam’s syntax has some goodies that I hadn’t noticed before, It’s definitely grown on me now that I’ve used Rust for a while.Gleam’s syntax draws a lot from Rust’s syntax. Gleam’s transpiler is written in Rust.

Some examples:

Handy pipe operator

Gleam adopts a pipe operator similar to ElixirGleam’s list.at function returns a Result type, which allows me to know whether I’ve received a value from the list or used an index outside the bounds of the list. The former will be Ok(value) and the latter will return Err(Nil) per the type signature of list.at.This is in contrast to Elixir’s Enum.at/2 function, which returns value or nil with no indication of whether that nil came from the list or from an out-of-bounds index.:

import gleam/io
import gleam/list

pub fn main() {
  [1, 2, 3]
  |> list.append([4,5,6])
  |> list.reverse
  |> list.at(2) 
  |> io.debug() // prints Ok(4)
}

Labeled arguments

Labeled arguments is how Gleam does keyword arguments which are notably missing from both Elixir and Rust. So this is a great addition. The main benefit of labeled arguments is that you can pass them into the function in any order, since they’re identified by name.Note that I had to use name twice in greet. The first time is the label, the second time is the variable I use in the body of the function. Using one name by itself raises a compiler error.Elixir can simulate keyword arguments with keyword lists, but those require a Keyword.get call to retrieve keys inside the function body, and can’t be passed in positionally.

import gleam/io

fn greet(name name: String, in place: String) {
  "Hello " <> name <> " from " <> place
}

pub fn main() {
  greet(name: "Christine", in: "NASA")
  |> io.debug() // "Hello Christine from NASA"
}

Elegant function capturing

Function capturing allows me to quickly create a one-argument anonymous function from a named function. The one argument will go wherever I place the _ in the function capture. It only seems to work for creating one-argument anonymous functions, putting multiple _ in a capture will compile errorBy contrast, in Elixir you can have arbitrarily many arguments in a captured function with the numbered & syntax: &Enum.reduce([1, 2, 3], &1, &2)(At least, I think it’s arbitrarily many, I stopped checking after &11 worked).

import gleam/io
import gleam/list
import gleam/string

pub fn main() {
  let prepend = string.append("other ", _)

  list.map(["hello", "world"], prepend) 
  |> io.debug() // ["other hello", "other world"]
}

Function capturing is also nice for arranging functions for the pipe operator, since Gleam lets me pipe directly into an anonymous functionElixir does not let you pipe directly into an anonymous function. In Elixir 1.12.0, then/2 was added to remedy this problem.:

import gleam/io
import gleam/string

pub fn main() {
  "hello"
  |> string.append("some text ", _)
  |> io.debug()
}

Only case statement, no if

Gleam doesn’t have an if statement, it uses case for that insteadGleam does use if as a keyword, but only in guard expressions.:

case if_boolean {
  True -> "case statements only"
  False -> "you didn't get a choice"
}

I like this syntax better than an if statement. It shines a spotlight on the branch, makes it clear which path belongs to which condition, and Gleam will do exhaustiveness checking to force me to handle both cases.

It also makes it easier to refactor into a sum type:

type CaseExample {
  CaseGood
  CaseAnyway
  ThirdOption
}

case case_example {
  CaseGood -> "case statements only"
  CaseAnyway -> "you didn't get a choice"
  ThirdOption -> "more choices!"
}

Sub pattern names using `as`

Gleam lets me assign names to sub-patterns using as in pattern matching expressions. Elixir solved this problem by making = expressions return their value, so you could use them in pattern matching, but I always found that somewhat awkward. as makes it clear that I’m naming a sub-pattern for reuse.

case list {
  [[_, ..] as inner_list] -> inner_list
  other -> []  
}

Note that Gleam makes it easy to destructure types with custom variables, avoiding the need for as:

// example, use gleam/option.Option instead
type OptionalString {
  Some(string: String)
  None
} 

fn unpack_string(optional_string) {
  case optional_string {
    Some(chosen_variable) -> chosen_variable
    None -> ""
  }
}

Succinct alternative clause patterns using `|`

Gleam lets you succinctly specify alternate patterns in a case statement using |:

type NumberKind {
  Prime
  Composite
}

case number {
  2 | 3 | 5 | 7 -> Prime
  _ -> Composite
}

Support for `todo` to mark uncompleted work

Gleam includes a todo keyword to indicate that some code isn’t finished. That code will type check, but will raise a warning on compilation:

fn fancy(something: SomeComplexType) {
  todo
}

The warning:

This code will crash if it is run. Be sure to finish it before running your program.

Gleam lets me add a description to the todo using as:

import gleam/io

pub fn main() {
  let incomplete = fn(x) {
    case x {
      2 | 3 | 5 | 7 -> "Prime"
      _ -> todo as "Something about composite numbers"
    }
  }
  incomplete(7)
  |> io.debug()
}

Takeaways

I really like what Gleam has done with its syntax. Despite my initial misgivings, I’m finding it hard not to play with Gleam as I write this article.

If Gleam’s syntax initially put you off, consider giving it another look. I know I like what I found.

How to Embed an SVG in CSS

2024-05-15T00:00:00+00:00

How to Embed an SVG in CSS

Recently, I needed to use an SVG as a background image in CSS. I didn’t want to host the SVG somewhere, so I couldn’t just link to it. I did have access to the SVG source, so I wondered if I could embed it. As I quickly found out, you can use a data URL.I use CSS data URLs to embed the fonts I use for this website. I didn’t know they could be used for SVGs too. Here’s an example on MDN doing just that.I initially learned how to do this from this stack overflow post.

Embedding a specific SVG

Let’s work through an example with this SVG triangle:

The SVG source looks like:

<svg xmlns="http://www.w3.org/2000/svg" 
  width="150" height="150" 
  viewBox="0 0 203 203" 
  fill="none">
    <polyline points="100,3 200,200 3,200 100,3" 
      fill="none" stroke="pink" stroke-width="3"/>
</svg>

SVG Data URL

The general format for an SVG data url is:

data:image/svg+xml;utf8,[some actual SVG here]

So in theory we could paste our SVG from above like this:If you paste this into the browser bar in Firefox, it renders the pink triangle.

data:image/svg+xml;utf8,<svg xmlns="http://www.w3.org/2000/svg" width="150" height="150" viewBox="0 0 203 203" fill="none"><polyline points="100,3 200,200 3,200 100,3" fill="none" stroke="pink" stroke-width="3"/></svg>

However, to use the CSS url() function, we need to enclose the data URL in double quotes "", which means that our SVG needs to use single quotes:

background-image: url(
  "data:image/svg+xml;utf8,<svg xmlns='http://www.w3.org/2000/svg' width='150' height='150' viewBox='0 0 203 203' fill='none'><polyline points='100,3 200,200 3,200 100,3' fill='none' stroke='pink' stroke-width='3'/></svg>"
);

Additionally, any SVG content needs to be URL escaped, for example # used for SVG colors need to be replaced with %23.My example triangle doesn’t have any, but this replacement proved vital for my actual use case.

To be safe, you could URL escape the entire SVG, which is what MDN does in this example.In practice, I didn’t need to escape the SVG tags for my use case rendered in Firefox.

With that, we now have the ability to inline SVGs in CSS.

How to Count Unique Lines Using Unix Pipes

2024-04-26T00:00:00+00:00

How to Count Unique Lines Using Unix Pipes

Sometimes I want to take a command line output and count the unique lines from that output. I can do this with a combination of standard unix tools and pipes:Read more about sort, uniq, and head in the linked GNU docs.Alternatively, use man sort, etc. to see what the available flags are on your system of choice.I always find new things when I read the docs. sort for example, has a --batch-size flag that limits the maximum number of inputs to be merged at once.Hillel Wayne advocates for this kind of “browsing” documentation instead of “searching” in his newsletter.

<output> | sort | uniq -c | sort -nr | head -n 5

By using the long flag names, we can get a better sense of what’s going on:I generally prefer long flags, because I tend to forget which shortflag goes to what. Especially if I haven’t used a particular command in a while. On my system, bash provides tab-completion for these long-flags, which makes them easier to discover and type.

<output> | sort \ 
  | uniq --count \ 
  | sort --numeric-sort --reverse \
  | head --lines 5

Step by step:

First, we sort the output alphabetically with sort.

Then, uniq --count merges matching lines while prepending the count to each line.We needed to sort first because this merge only happens on identical lines next to each other in the output.

Then, sort --numeric-sort --reverse sorts these lines by the prepended count, in descending order.

Finally, we use head --lines 5 to only get the top 5 counts.I see this as the unix equivalent to the SQL: select count(field) from table group by field order by count(field) desc limit 5.

Applications

I wanted to share a few places where I’ve found this useful recently. This technique combines well with other command line “power tools”, such as awk and jq:

Cut - Aggregate a CSV Column

If I have a csv like this:

hi,1,a
hello,2,a
world,3,b

I can use cut to select just a single column:Read more about cut here.I was originally going to use awk for this example. Thanks to PgSuper for suggesting to use the simpler cut.The original awk is: awk 'BEGIN { FS = "," } ; { print $3 }'Alternatively, cut -d',' -f3 using the shortflags.

cat my.csv \
  | cut --delimiter ',' --field 3

Then, I can combine this output with our aggregation:

<previous>  | sort | uniq -c | sort -nr | head -n 5

To get our aggregated count:

      2 a
      1 b

jq - Aggregating the Web

Our count unique aggregation works well with jq for counting API output.I tend to prefer jaq, a clone of jq, for performance reasons, but jq is the more popular tool. Both work with the same syntax from this example.

For example, Codeberg’s Forgejo instance exposes a public API with information about the repos hosted there:By default /repos/search returns 25 items, at time of writing.

curl "https://codeberg.org/api/v1/repos/search"

I can use jq to turn the resulting JSON into a list of primary languages used on each repo:Learn more about jq from the tutorial.

<previous> | jq '.data[].language'

Here, I use jq to access the data key, operate over a list of objects, and then select the language key from each one.

Finally, we can aggregate to see what languages are most used:

<previous>  | sort | uniq -c | sort -nr | head -n 15

For me that looks like:

      9 ""
      4 "Shell"
      4 "Lua"
      2 "Python"
      2 "Markdown"
      1 "Kotlin"
      1 "HTML"
      1 "GDScript"
      1 "Emacs Lisp"

It looks like a fair number of repos don’t have their language field set in the Forgejo API response.

I created a little webring in Gleam

2024-03-31T00:00:00+00:00

I created a little webring in Gleam

Recently, my friends have been building cool web 1.0 features for their websites, such as this hit counter. Inspired by web 1.0, I decided to create a web ring for me and my friends.

What is a webring?

A webring is a way of adding discoverability for sites in the ring.

On a technical level, it’s a doubly linked list of websitesDiagram made with mermaid.live:

Each website links to the next and previous website in the linked list, creating a “ring” of websites that are all connected.

In theory, it’s possible to literally link to the next site in the ring, but this quickly becomes impractical as member sites join or leave the ring. The solution is to write a little webring service:

How do you run a webring service?

Instead of having member sites link directly to each other, a webring service can handle routing each “next” and “previous” link to the appropriate place.

At its core, this service consists of three things:

A way to identify where someone accessing the service is in the ring.
A /previous endpoint to route people to the previous ring member.
A /next endpoint to route people to the next ring member.

The endpoints can be powered by a simple HTTP 303 “See Other” redirectRead more about that here on the MDN documentation., so the hard part is figuring out where you are in the ring and what next and previous member sites are.

Where are you ringing from?

I’ve seen a few different ways to keep track of which member site the webring is being accessed fromAnother thing I considered was a from=<website> query parameter, but then I would have to worry about appropriately escaping web domains.:

An integer member number
A short hash id
EmojisHere’s a blog post from the person who runs the Indieweb Ring about why Emoji IDs are problematic in practice.

Of these, I decided to use short hash ids, unique to each member site. I chose to use a SHA256 hash of the members URL, converted to padded, URL-safe base64 with a known saltTherefore, each member site only has to keep track of its own hash_id, and it’s designed not to change as members join or leave the ring.:

fn hash(string) {
  {string <> "salty salt"}
  |> bit_array.from_string()
  |> glesha.hash(glesha.Sha256)
  |> bit_array.base64_url_encode(True)
  |> string.slice(0, 16)
}

I’ll come back to how I used these hashes in a moment. First, we need to know who our neighbors in the ring are!

Ringing my neighbors

Starting from a list of member URLs, how do I turn them into a ring?It turns out that neither of the popular webrings that first come to my mind solve this problem.An Indieweb Webring just picks a random link every time as detailed at the bottom of this blog post.Devine’s webring just links to the index. This is nice because it means you only need to have one symbol/link on your website instead of previous/next links.

One thing I considered was using ZippersThanks to my friend Hayleigh for suggesting I think about Zippers., a functional programming technique for traversing data structures such as lists and trees.Sourceror, an AST manipulation library for Elixir uses Zippers, and has an excellent guide to them.

However, I don’t care so much about traversing the list quickly, so much as finding the immediate neighbors of each member. This reminded me of the list.window function in the Gleam standard library.

In particular, this example looks close to what I need:

window([1,2,3,4,5], 3)
// -> [[1, 2, 3], [2, 3, 4], [3, 4, 5]]

This function is missing the neighbors for 1 and 5, but otherwise is doing what I need. Looking at the implementation gives us this:

fn do_window(
  acc: List(List(a)), 
  l: List(a), 
  n: Int
) -> List(List(a)) {
  let window = take(l, n)

  case length(window) == n {
    True -> 
      do_window([window, ..acc], drop(l, 1), n)
    False -> acc
  }
}

This function is asking: “Starting at the current element, what are the next n-1 elements after it?” Returning the list if there aren’t enough elements. But I tack on the first few elements when length(window) < n I’ll get what I need. With some optimizations to avoid length (which is an O(n) check in Gleam’s Erlang target), I get:

fn do_window(acc, l, original) {
  let window = list.take(l, 3)

  case window {
    [a, b, c, ..] ->
      do_window(
        [
          Row(
            b, 
            hash(b), 
            previous: a, 
            next: c
          ), 
        ..acc
        ], 
        list.drop(l, 1), 
        original
      )
    ... // omitted
    [] -> acc
  }
}

Since I’m already traversing the list, I go ahead and save the hash of each member while I’m there.

Now that I’ve computed all the neighbors, I can cache them into a Sqlite database. Member links are going to be read far more often than there will be changes in membership, so this caching is useful. It also allows us to index the field with the hash, giving us a faster than O(n) read time for links.

Constellation

All together I’ve separated this caching logic into its own project: Constellation. This is just a webring for me and my friends, so I’m okay with manually running Constellation when I need to change membership.

Constellation reads from a members.txt that contains one member URL per line, computes the hashes and neighbors, and then creates a members.db which I can rsync to my server for use with the web service.

Now how does that web service work?

A Gleaming Web(ring) Service

Gleam’s web framework is Wisp which proved quite capable for my little webring.

There are a number of examples in the Wisp repo. My webring is only a little more than a fancy router, so I started from the routing example.Literally, you can see in my second commit that I forgot to change the readme.

Gleam doesn’t have any macros, which makes routing exactly pattern matching. Or as the comment in the example puts it:

Wisp doesn’t have a special router abstraction, instead we recommend using regular old pattern matching. This is faster than a router, is type safe, and means you don’t have to learn or be limited by a special DSL.

That gives my webring a router that looks like thisSince routing is just pattern matching, even complex web applications in Gleam use nested case statements or functions that contain fine grained pattern matching.:

pub fn handle_request(req: Request) -> Response {
  use req <- web.middleware(req)

  case wisp.path_segments(req) {
    [hash, "previous"] -> handle_previous(hash)
    [hash, "next"] -> handle_next(hash)

    _ -> wisp.not_found()
  }
}

handle_previous looks like this:

fn handle_previous(hash) {
  let db_path = "members.db"
  use conn <- sqlight.with_connection(db_path)

  let previous =
    sqlight.query(
      "select previous from ring where hash = ?",
      on: conn,
      with: [sqlight.text(hash)],
      expecting: dynamic.element(0, dynamic.string),
    )

  case previous {
    Ok([previous_link]) -> 
      wisp.redirect(previous_link)
    _ -> wisp.not_found()
  }
}

There’s a lot going on in this function.

We have a use statement, which allows the function to close the database connection when it’s done.Here’s the language tour section on use sugar for more explanation.

Sqlite doesn’t know anything about Gleam types, so I use sqlight.text to let Sqlite know what type I’m passing in.

Gleam is statically typed, but we don’t know what type we’re going to get from the databaseEspecially with Sqlite, the authors of The Advantages of Flexible Typing.. The dynamic module from the standard library allows us to write a simple parser to convert what the database returns into types that Gleam understands.I initially wrote more boilerplate than I needed, but I realized that I could simplify the parsers.

Wisp has a built in function wisp.redirect that handles the HTTP 303 redirects that I need.

Up And Running

That’s it, after a similar handle_next function, I put up the service behind Caddy on a small server. If everything is working correctly, then it should be linked at the bottom of this article.For Desktop users, it should also be under my intro blurb at the top-left of the article.

The source code for the web service is here. Special thanks to Joe and Sara for inspiring me to write a webring. Here’s to making the internet a little cozier.

Extra Goodies

At this point in the explanation, the webring does everything it needs in order to be a ring, but we added a few extra features to make it shine:

Random Endpoint

Shortly after I got the webring up and running, I got a feature (and pull!) request to add a “random” feature.

Adding it to the router was simple:

pub fn handle_request(req: Request) -> Response {
  use req <- web.middleware(req)

  case wisp.path_segments(req) {
    [] -> home_page(req)

    ["random"] -> handle_random() // One line!

    [hash, "previous"] -> handle_previous(hash)
    [hash, "next"] -> handle_next(hash)

    _ -> wisp.not_found()
  }
}

And the handle_random function itself takes advantage of Sqlite’s ORDER BY RANDOM():

select next from ring order by random() limit 1

Otherwise the redirection works exactly the same as handle_previous.

A Lustrous Home Page

I wanted to have something for people to see when they navigated to https://webring.club, the root domain of my webring.

Getting the members was a simple select member from ring SQL query:

let db_path = "members.db"
use conn <- sqlight.with_connection(db_path)
let members =
  sqlight.query(
    "select member from ring order by member asc",
    on: conn,
    with: [],
    expecting: dynamic.element(0, dynamic.string),
  )

I chose to use Lustre for html templating.I wrote about Lustre for interactive frontend before in Embedding Gleam in my blog with Vite. Lustre also offers backend templating. Lustre’s elements are just functions, so I can use them in a list.map over the members:

let members =
  result.unwrap(members, [])
  |> list.map(fn(member) {
    html.li(
      [], 
      [html.a(
          [attribute.href(member)], 
          [html.text(member)]
       )]
    )
  })

let html =
  html.body([], [
    html.h1(
      [], 
      [html.text("The Constellation Webring")]
    ),
    html.ul([], members),
  ])
  |> element.to_document_string_builder()

My Favorite Gleam Feature

2024-01-08T00:00:00+00:00

My Favorite Gleam Feature

I found myself liking Gleam’s syntax more than any other language that I’ve used.With the specific exception of Elixir, where many of these insights took root. I chose to focus this article on Gleam, but both Elixir and Gleam share this feature. This article follows the path of logic as I tried to unravel why.

My journey to understanding started with a simple question:

Why are Gleam’s functions backwards compared to Erlang and Python? Specifically, why are the arguments to Gleam’s list.map in reverse order as compared Erlang’s lists.map?

Reverse Order

Gleam’s list.map has the following signature:Slightly simplified. The actual signature has a with label on the function argument, which allows for list.map([1,2,3], with: fn(x) { x + 1}).

pub fn map(list: List(a), fun: fn(a) -> b) -> List(b)

Gleam takes a List as its first argument, and a function to map over the list, second.

By contrast, here is the signature of Erlang’s lists.map:

map(Fun, List1) -> List2

Erlang takes a function as the first argument and a list as the second argument.

Similarly, Python’s map() looks the same way:Technically, the return type hint in Python3 would look different, but for the sake of expediency, I’m using -> iterator.

map(function, iterator) -> iterator

In order to understand why Gleam is backwards, we need to understand why Erlang and Python are forwards. Why are Erlang’s and Python’s functions arranged that way?

Partial Application

Haskell shows a justification for why they’re created that way.And it is the other way, Haskell’s map is arranged like Erlang’s.

In Haskell, all functions are implicitly curried. That is, they’re secretly all functions that take one argument and return other functions.

This makes it easy to partially apply a function:

mapPlusOne = map (+1)

Here, I’ve created a mapPlusOne function that takes a list of numbers and returns a list of those numbers with 1 added to them. Haskell’s currying means that I can accomplish this by simply putting the two functions next to each other.

In this way, map is “partially applied” and instead of getting back a list, I’ve created a useful function that I can reuse.

Partial application is most useful when the most generic part of the function is set in the first argument, getting more specific as you go to the right.

This makes it easy to “cut off” a function in a partially applied but generally useful state.

By contrast, if map was arranged like Gleam, then the partial application would be less useful:

mapOnSpecificList = map [1,2,3] --insert function here

It’s far more likely that I’ll want to vary the input to a specific modification function, than to vary the modification function to a specific input. Data often changes more quickly than logic.

Python is following in these footsteps and provides functools.partial to accomplish similar things to Haskell:I’m editorializing a bit, functools.partial wasn’t added to Python until version 2.5 in 2006.Python’s map() has always been explicitly for functional programming, as noted in the release notes for Python 1.0.0: “New built-in functions map(), filter() and reduce() perform standard functional programming operations”.Interestingly, Lisp introduced a mapList function in 1959, but it took the arguments in Gleam’s order: mapList(L,f).Nearly every other language uses the map(function, list) order, as shown on Wikipedia.From this table, only Elixir, Haxe, R, and XQuery have non-method functions in Gleam’s order.

from functools import partial
map_plus_one = partial(map, lambda x: x + 1)

Erlang seems to have been influenced by the same ideas.

Okay, now we understand why Erlang’s functions are the way they are, why are Gleam’s functions backwards?

Pipe Operator

Gleam has a pipe operator |> that will take the value on the left, and try to pass it in as the first argument of the function on the right. It looks like this:

[1,2,3] |> list.map(fn(x) { x + 1})

Gleam’s list.map takes two arguments, in the example above we’re able to write list.map(fn(x) { x + 1}) because the pipe operator is silently filling the first argument with the list on the left side. This becomes a bit clearer if we use Gleam’s capture operator syntax:As noted in the language tour, this is such a common operation that the former is simply a special shorthand for the latter.In Elixir, the pipe operator is literally just a macro that takes [1,2,3] |> a_function(other, args) and replaces it with a_function([1,2,3], other, args) during compilation.

[1,2,3] |> list.map(_, fn(x) { x + 1})

This has the same effect as calling the function directly with that value:

list.map([1,2,3], fn(x) { x + 1})

The pipe operator has the benefit of making data transformations more “sentence”-like. Since the operator moves left to right, a series of pipe operators can transform data in the same direction as native English speakers read language:

[1,2,3] 
|> list.map(fn(x) { x + 1}) 
|> list.filter(fn(x) { x > 2 }) 
|> list.at(1) // Ok(4)

Haskell, by contrast (often)It’s worth noting that Haskell has a left-to-right composition operator >>> and a reverse application operator which works like a pipe &, but in practice I don’t see it being used anywhere near as pervasively as the pipe operator in Gleam. ends up having a right to left flow to its logic. The specific details are finalized on the right, and then transformed by functions as the calls are made to the left:Haskell’s equivalent to the list.at function is the !! operator. By default all operators in Haskell are infix, but they can be turned into prefix form by wrapping them in parentheses. In this case, I also partially apply !! with index 1.The !! operator is different from list.at in one way: Gleam’s version returns a result type, whereas Haskell’s version raises an error on an index missing from the list. I did find maybeAt which returns a Maybe, Haskell’s version of result, but it’s in a library in not in Haskell’s standard library.

(!! 1)
  . filter (>2)
  . map (+1)
  $ [1,2,3]
-- 4

While I do love the pipe operator, we still haven’t answered the question: Why are Gleam’s functions backwards? To understand that, we need to explore another property of Gleam.

“method”-like functions

There are other languages that don’t have the pipe operator that do have this left to right directionality to data transformations.

Method chaining gives us a similar ability to read transformations left to right. For example in Javascript:Here’s the documentation for the Array map method. The other two methods are similar.

[1,2,3]
  .map(x => { return x + 1})
  .filter(x => { return x > 2})
  .at(1);

So before, when I said that Gleam’s functions are backwards compared to Python, that was technically true. But in practice, methods are far more common in Python than functions. Python method signatures all begin with self, similarly Rust’s methods also begin with self as their first given argument.

For example, Rust’s Iterator.map method has this signature:This signature has been simplified for clarity, you can confirm it against the real signature here.

fn map<B, F>(self, f: F) -> Map<Self, F>
where
  F: Fn(Self::Item) -> B

It returns type Map because it’s lazy, but otherwise the signature looks a lot like Gleam’s list.map.

Looking more closely at Gleam, we see a similar pattern: all of the functions in gleam/list take a value of type List as their first argument. And nearly all them return a List allowing pipe operator “chaining” to happen. In this way, Gleam’s functions are “method”-like, in that their signatures line up with what we would expect from method signatures in Python or Rust.The lack of this design in command line applications is part of why unix’s xargs exists, to “fix” pipelines with the arguments the wrong way around.

However, there is a substantial difference between Gleam and these method chaining languages: Gleam doesn’t have methods.

Lack of methods

GleamAnd Elixir. doesn’t have methods. All functions are associated with modules, and not with specific data. This comes as a consequence of Gleam’s default immutability.Gleam’s Erlang target allows you to manage state with living processes, and its Javascript target has support for mutable arrays. When I pass a list to list.map, it returns a new list without mutating the original.

This makes it easier to write functions that have referential transparency, which means that a function given the same inputs will always have the same output. This makes it easy to reason about Gleam code, since seeing the inputs and logic of the function is enough to see what the function is going to do.

The net result of this lack of methods, is that the primary way to compose code in Gleam is to use the pipe operator.There’s also the use expression, but that’s primarily syntax sugar for working with callbacks.This lack of options is seems to be by design. Gleam has one happy path for doing things and not much room for alternate conventions. This ends up working in favor of my favorite feature.

So now Gleam has a pipe operator, and makes it the primary way to compose code, there’s one more quality of Gleam that enables my favorite feature:

A culture of qualified imports

By default an import declaration in a Gleam module is qualified. That means that instead of dumping the functions and types from the imported module into the the current module’s namespace, they’re accessible by prefixing the imported modules name:

import other_module

pub fn main() {
  other_module.useful_function(...)
}

To prevent this prefixing from getting unwieldy, Gleam will use the last part of the import path as the namespace:In Elixir, all modules are always available at their fully qualified name Module.Path.Here, and you need to use the alias macro in order to shorten that prefix.For example: alias Module.Path.Here will allow that module to be used as Here.I prefer Gleam’s design here.In fact, Gleam forces you to use the shortened prefix: gleam/list.map will raise an error. It does allow the as keyword to rename this qualified import if there’s a conflict.

import gleam/list

pub fn main() {
  // instead of gleam/list.map
  [1,2,3] |> list.map(...)
}

Gleam does allow for unqualified imports, where functions can be used directly the current modules namespace, but they must be enumerated explicitly:

import gleam/list.{map, filter}

pub fn main() {
  [1,2,3]
  |> map(fn(x) { x + 1})
  |> filter(fn(x) { x > 2})
}

However, Gleam warns against unqualified imports in the language tour:

This may be useful for values that are used frequently in a module, but generally qualified imports are preferred as it makes it clearer where the value is defined.Emphasis mine.

It is the combination of these three factors: a language designed around pipe operator, a lack of methods in the language, and a culture of qualified imports that leads to my favorite feature of Gleam:

My Favorite Feature: Discoverability

As a consequence of these three factors working in combination, I can visibly see which modules each function in a data transformation pipeline come from. For example in:

import gleam/list

[1,2,3]
|> list.map(...)
|> list.filter(...)
|> list.at(...)

I know where map and filter and at come from, and I can easily look up the documentation for gleam/list to understand more about each function and what other functions are available. The implementation details of a function in Gleam are plain to see and easy to learn more about.

So my favorite feature of Gleam, is that it’s discoverable. I can read someone else’s code and understand each every module that they chose to use, and where every function comes from.In a language with method chaining, a Language Server grants you write-time discoverability. You can add a . and hit tab, and discover all of the currently available methods.However, at read time, this discoverability is lost, making it hard to understand which trait or type or object or class implements the method that you’re currently dealing with.Often classes will reuse method names, overloading a given term, and making harder to search for where that method is implemented.

This discoverability makes it a joy to read source code written in Gleam. I’ve improved as a software engineer as a consequence of being able to learn from the core libraries of my favorite language.

Thank you to John, Theodore, Nicole, Jeff, Miccah, and Jake for helping me write and edit this article. And thank you to the Gleam discord for helping me validate some of these concepts.

Embedding Gleam in my blog with Vite

2024-01-06T00:00:00+00:00

Embedding Gleam in my blog with Vite

My interest in Gleam came primarily from the idea of having Erlang with Static types. Gleam is able to transpile to Javascript, so I wanted to try embedding it in my blog. I might want to add interactive explanations to my blog in the future, and it would be neat to be able to write them in Gleam.Update 2024-01-06: I now recommend using esgleam instead of vite-gleam, which I’ve written about here. All the Gleam explanation still lives here, but the simplified deployment instructions are in the new blog post.

Step 1: Create Lustre Project

Lustre is a frontend web framework for Gleam. It has an Elm-inspired runtime that makes it easy to create dynamic components with Gleam.Lustre can also be used on the server side as a HTML templating DSL.

I can create a new Gleam project with the CLI:

gleam new shiny_counter

I can then enter the newly created project folder and add Lustre:

cd shiny_counter
gleam add lustre

Now I can start building my Lustre application.

A simple Lustre application consists of an initialization function, an update function, and a view function.

Init

The initialization function is responsible for setting the initial state of the Lustre model.

In my shiny_counter it looks like this:

type Model =
  Int

fn init(_) -> Model {
  0
}

Here, I define the Model to be just an integer, and initialize that integer to 0.

Update

The update function takes a model and a message as arguments and returns an updated model based on the contents of that message.

In the case of shiny_counter I only need an increment and decrement message:Since the model of shiny_counter is an Integer, it’s signed, so if I send a Decrement message enough times, I will get a negative number. If I wanted a model where I couldn’t produce a negative number, I would want to add extra cases into my update function.

pub type Msg {
  Increment
  Decrement
}

fn update(model: Model, msg: Msg) -> Model {
  case msg {
    Increment -> model + 1
    Decrement -> model - 1
  }
}

View

The view function takes a model and returns a Lustre Element constructor that handles both communicating with the update function as well as rendering the contents of the view to HTML.

In the case of shiny_counter, I convert the current state of the model to a string, then I render that count as well as provide two buttons which send defined messages to update the model:

view(model: Model) -> Element(Msg) {
  let count = int.to_string(model)

  html.div(
    [],
    [
      html.p([], [element.text(count)]),
      html.p(
        [],
        [
          html.button([event.on_click(Decrement)], [element.text("-")]),
          html.button([event.on_click(Increment)], [element.text("+")]),
        ],
      ),
    ],
  )
}

From here, I can add styling to make it live up to its shiny_counter name. Here, I’ve added styling to center the text and added the all important ✨ emoji:

fn view(model: Model) -> Element(Msg) {
  let count = int.to_string(model)

  html.div(
    [],
    [
      html.p(
        [attribute.style([#("text-align", "center")])],
        [element.text(count <> " ✨")],
      ),
      html.p(
        [attribute.style([#("text-align", "center")])],
        [
          html.button([event.on_click(Decrement)], [element.text("-")]),
          html.button([event.on_click(Increment)], [element.text("+")]),
        ],
      ),
    ],
  )
}

Start the application

Now that I’ve defined the key parts of a simple Lustre application, I need to combine them into an application. I do that with lustre.simple in this case, since shiny_counter doesn’t have complex interactive behavior. Once I have the application, I can simply start it:

pub fn main() {
  let app = lustre.simple(init, update, view)
  let assert Ok(dispatch) = lustre.start(app, "[gleam_example]", Nil)

  dispatch
}

lustre.start starts an application anchored on an HTML element specified by a CSS selector. In this case [gleam_example] finds an element that has the gleam_example attribute:

<div gleam_example>
</div>

Lustre will replace the selected element entirely with the running application after it loads.When I initially tried Lustre, the example I borrowed from used "body" as the CSS selector, and completely wiped out the template of my blog, making it awfully hard to write this article.I could tell that I misconfigured Lustre, because my blog was briefly visible until the Javascript loaded. This blog post is about embedding Gleam in my blog, not replacing my blog with Gleam. In this case, I chose an attribute since it’s unlikely that I’ll have anything else on the page with that custom attribute.

Now that I have a Lustre application that does what I want, I need to embed it in my blog.

While Gleam does transpile to Javascript it generates a number of different modules that are tedious to use in my static site generator. Enter Vite as a way to bundle the Javascript into a single file:

Step 2: Install and configure Vite

Vite is a Javascript bundling tool, it’s explicitly designed to transform Node libraries or other Javascript modules and make them easy to use in a browser.Here’s a link to the shell.nix that I used for this project.

I can install Vite with npm:

npm install vite

Vite is designed for Javascript and not for Gleam, and so I need to rely on a plugin to show Vite where Gleam’s transpiled Javascript is. I can add that plugin with npm:Thanks to a member of the Gleam community for creating the vite-gleam plugin.

npm add vite-gleam

Now that I’ve added vite-gleam I can create a config file to use it:The rollupOptions.input setting allows me to change the entry file for Vite. By default this is index.html, but since I’m going to be using the bundled Javascript in this blog post, I don’t need a generated index.html file. This setting is described in the Backend Integration instructions for Vite.The output.entryFileNames setting allows me to precisely specify the name of the generated Javascript file. By default, the generated Javascript has a randomly generated name suitable for cache busting. Since I’m copying the generated Javascript file from my Gleam project into my static site generator, I wanted a specific filename to reference in the embedded <script> tag.

import gleam from "vite-gleam";

export default {
  plugins: [gleam()],
  build: {
    rollupOptions: {
      input: 'main.js',
      output: {
        dir: './dist',
        entryFileNames: 'assets/gleam_vite_example.js',
      }
    }
  }
}

Then I create a file called main.js to act as the entry point for Vite:

import { main } from './src/shiny_counter.gleam'

document.addEventListener("DOMContentLoaded", () => {
  const dispatch = main({});
});

Now I can finally call vite buildBy default, Gleam transpiles to Erlang, but vite-gleam will use --target javascript to generate the Javascript code before bundling.If you would like to transpile to Javascript without Vite, then set target = "javascript" in the project’s Gleam.toml., this will use the vite-gleam plugin to first transpile the Gleam code to Javascript, then it will use the main.js entry point to bundle all of the modules into a single file named gleam_vite_example.js.

Step 3: Use the bundled Javascript in my Static Site

Now I can copy the Javascript from my Gleam project into my static site generator. Once there, I can add markup to my site to use it.

Djot, the markup language that I use for my blog allows me to do an HTML passthrough block:

```=html
<div style="border: 1px solid">
  <div gleam_example>
    <script type="module" src="/assets/gleam_vite_example.js"></script>
  </div>
</div>
```

Here, I’ve specified a <div> tag with the gleam_example attribute, added a script tag to use the bundled Gleam Javascript, and then wrapped both in a stylized div.

As mentioned in Step 1, Lustre will replace the attributed div with the running Lustre application after it starts up, leaving me with an embedded interactive component.

Step 4: Publish

It works!You can see the source code for shiny_counter here.

It’s really neat that I can take Gleam and embed it directly into my blog posts. Thank you to everyone in the Gleam discord for helping me figure all of this out.

Embedding Gleam in my blog with esgleam

2024-01-06T00:00:00+00:00

Embedding Gleam in my blog with esgleam

Previously in Embedding Gleam in my blog with Vite, I used the vite-gleam plugin to bundle my compiled Gleam JS and embed it into my blog. Since then, the creator of the vite-gleam plugin has created esgleamAs of the time of writing, the most recent version of esgleam is 0.3.0. which allows me to bundle my Gleam JS without needing Node installed.

esgleam downloads esbuild as a pre-built executable which acts as a bundler for my project’s javascript.

I explained in detail how the Gleam application works in my previous article, so I will focus on how esgleam makes it easier for me to deploy here.

`esgleam`

Adding esgleam to my shiny_counter project was simple:

gleam add esgleam

I could then run esgleam/install to download the esbuild executable:

gleam run -m esgleam/install

And then bundling the shiny_counter Javascript was simple:

gleam run -m esgleam/bundle

By default esgleam finds my src/shiny_counter.gleam and turns it into dist/shiny_counter.js.I found that the defaults were perfect for my needs, but esgleam has an Advanced Usage which allows for specific customization of the bundling process.

That was all I needed to do to bundle the Javascript for my project. Then all I needed were a few more steps to embed the Javascript in my project:

Embedding the Javascript

I then copied the shiny_counter.js to my static sites assets folder. Unlike when I bundled with vite, I didn’t have a main.js that enabled the listener for the project, so I need to that manually here:As a reminder, the gleam_example is a custom attribute which shiny_counter is using a CSS selector to find and mount the Lustre application to.

```=html
<div style="border: 1px solid">
  <div gleam_example>
  <script type="module"> 
  import { main } from '/assets/shiny_counter.js'

  document.addEventListener("DOMContentLoaded", () => {
    const dispatch = main({});
  });
  </script>
  </div>
</div>
```

Which works!Thanks to Enderchief for creating esgleam to simplify the deployment story for Gleam browser Javascript!

Using Regex in Erlang

2023-12-07T00:00:00+00:00

Using Regex in Erlang

Recently, I was helping someone port some code to Erlang that involved a Regex. I hadn’t worked with Regex in Erlang before, so here are some notes on what I learned.

`re` - Perl-like regular expressions for Erlang

The Erlang standard library has the re module, which supports regular expression matching for Erlang strings and binaries.Both Gleam and Elixir use String to mean an Erlang binary. The default string type in Erlang is sugar over a linked list of code points.

The re module provides three main functions: replace, run, and split, which all operate on string-like inputErlang uses the iodata type to represent a generalization on binaries that can be more efficiently worked with, all three of these functions take an iodata or charlist. and a regex pattern.

Regex matching can be done with run like this:

> re:run("my string", "[myexpression]").
{match,[{0,1}]}

By default run returns all captured parts of the input, as a list of {Offset, Length} pairs. It also only returns the first match, not all matches.

To return the captures as strings, I can use the {capture, all, list} option:

> re:run("my string", 
    "[myexpression]",
    [{capture, all, list}]
  ).
{match,["m"]}

To get all of the matches, I can use the global option:

> re:run("my string", 
    "[myexpression]",
    [global,{capture, all, list}]
  ).
{match,[["m"],["y"],["s"],["r"],["i"],["n"]]}

Named Captures

re supports Perl-style named captures, which look like this:

Pattern1 = "(?<myname>capture)".

It can used by changing the ValueSpec component of the capture option:There isn’t a built in way to associate names with captures. Elixir provides a named_captures function to easily do this.From reading the source code of the Elixir implementation, it should be possible to combine re:inspect with lists:zip to get a list of {Name, Capture} pairs.

> re:run(
    "my capture of my capture", 
    Pattern1, 
    [
      global,
      {capture, ["myname"], list}
    ]
  ).
{match,[["capture"],["capture"]]}

Compiling An Expression

Erlang provides a compile function to compile a regular expression for re-use throughout the lifetime of a program:

Compiling the regular expression before matching is useful if the same expression is to be used in matching against multiple subjects during the lifetime of the program. Compiling once and executing many times is far more efficient than compiling each time one wants to match.

A regex can be compiled into a pattern for use with compile:

{ok, Pattern2} = re:compile("[myexpression]").

The used like any other regex pattern:

> re:run(
    "my string",
    Pattern2,
    [{capture, all, list}]
  ).
{match,["m"]}

Takeaways

The Erlang regular expression module is a bit difficult to use, but it will be nice if I have need of a zero-dependency regex module when using Erlang.

Tools I Use: codespell

2023-12-04T00:00:00+00:00

Tools I Use: codespell

codespell is a command line spellcheck tool that is code-aware. Codespell allows me to spellcheck my software projects without getting stuck on all the jargon that software projects use.

Usage and Tips

I often use codespell to spellcheck the markup for this blog. It’s nice to have a tool that doesn’t get stuck on the code blocks I use in my articles.

It does occasionally get caught on rarer words or words that have more common English spellings for example, running it on my site’s markup folder gives me this:Unconfigured. My configuration later in the article fixes this.

content/notes/crate-reqwest.dj:1: Crate ==> Create
content/notes/crate-reqwest.dj:4: Crate ==> Create
content/notes/crate-surf.dj:1: Crate ==> Create
content/notes/crate-surf.dj:4: Crate ==> Create
content/notes/bookmarks/2023-07-20.dj:149: simpy ==> simply

Both “Crate” and “simpy” are correct, the former is a legitimate English word, and the latter is a part of a URL.As confirmed by Wikitionary: crate. Which specifically has a definition for the Rust usage.This catch on “crate” is because of the default dictionary option, which is set to clear,rare. This behavior can be modified with the --builtin clear option to only show errors on unambiguous errors.

As such, while codespell has a --write-changes flag, I would caution against its use in a non-interactive mode.

Interactive

codespell does allow for interactive updates with a combination of flags:

codespell <optional_folder> --interactive 1 --write-changes

Ignore words

Another option is to use the --ignore-words-list option, which takes a comma separated list of words to ignore:

codespell <optional_folder> --ignore-words-list crate,simpy

If you have enough words to make passing them as a comma separated list unwieldy, then you can use --ignore-words to pass a file with each word on its own line.

Config

codespell will look for a .codespellrc file in the local directory for configuration.Despite mentioning toml in the README, the configuration format is not toml. Which took me a bit to figure out. toml only seems to apply if you configure it with pyproject.toml.The default configuration format seems to be closer to an ini format.

I’ve set my config for this site like this:

[codespell]
builtin = clear
ignore-words-list = simpy

Takeaways

codespell is a neat little tool for spellchecking in a code heavy environment. I use it regularly to spellcheck these articles. I use few of its additional features or options, just a simple, effective command-line spellchecking tool.

Serve a Static Directory with Erlang

2023-11-06T00:00:00+00:00

Serve a Static Directory with Erlang

For a long time, I’ve been serving static files to locally test this website using python3 -m http.server.Actually, I used a custom script that overrode the Handler class to set the mimetype for files without an extension to "text/html", since I use that to get clean urls without .html on the end. I’ve wanted to do this with Erlang for a long time, after I found out that Erlang has a built in httpd server.

Recently, I found outThank you to someone on the Gleam discord who pointed this out to me! that a short command like the python one will be added in OTP 27Investigating the origin of the pull request, I found this list of “serve this directory of static files” commands across a variety of languages. If Erlang isn’t your jam, enjoy!. Soon you will be able to invoke erl -S httpd to serve static files locally. Looking at the diff I was able to write an Elixir script that serves a static directory for me now.

The Script

The core of the script is powered by :httpd.start_service/1 which takes a proplist of settings that configure the httpd server. I always found the list of configuration options intimidating, so I was glad to have a place to start with from the diff.

Required Property

Four properties are required:

[
  {:port, 8008},
  {:server_name, 'localhost'},
  {:server_root, absolute_path},
  {:document_root, absolute_path}
]

A few notes:

All string properties need to be single-quoted Elixir charlists, because :httpd is expecting Erlang strings, not Elixir binaries.From the Elixir documentation: “The rationale behind this behaviour is to better support Erlang libraries which may return text as charlists instead of Elixir strings.”
I had to set :server_name to exactly 'localhost', in order to get my websites relative links to work correctly. So far I can tell, I can run multiple versions of :httpd on different ports with the same name without issue.
absolute_path, like the diff, I used an absolute_path path generated using some Erlang functions instead of a local path, but I believe a relative path would work.
:server_root is the root directory that all other properties are relative to.
:document_root is what I actually want to serve my static files.

Additional Properties

I also set some additional properties for my needs:

`:bind_address`

The default value is set to :any, I specifically set mine to {127, 0, 0, 1}. I think this would be the same as the :any behavior, but better safe than sorry.

`:mime_types`

Like I did with Python, I was able to set the mimetype for no extension to text/html by adding {'', 'text/html'} to my list of mimetypes, which I otherwise took from the diff.

`:default_type`

This undocumentedThe documentation says to use mime_type, but I found that when I set this to 'text/html', my extensionless files still were rendered as plain text. After investigation, I found that mime_type does nothing and hasn’t since at least OTP-17.0.The default_type property does do what mime_type claims, but isn’t mentioned in the documentation. property sets the default mimetype for files that don’t match the existing :mime_types.

`:directory_index`

Idiosyncratically, I use home instead of index.html for my folder indexes on my website, Erlang easily let me set this with {:directory_index, ['home']}.I don’t remember where I picked this up from, but I’ve used in my last 3 static site generators.

`:modules`

:httpd comes with a number of modulesThere’s a mod_trace module that enables support for the TRACE request type. I didn’t know these existed, but apparently they’re included in HTTP/1.1. It’s even listed on MDN!? to extend its behavior. For my script, I enabled the same three as the diff:

:mod_alias, I believe this enables the directory index behavior.
:mod_dir, will generate an Apache style file listing, if a directory index isn’t found, though this shouldn’t apply for my website.
:mod_get, which enables GET requests for regular files, which is what I want to a static directory being served.

Other Details

`:inets.start/0`

The :inets service must be started before :httpd can work. So my script includes a call to :inets.start() before :httpd.start_service/1.

`no-halt`

The :inets Getting Started suggests that after starting :inets, you can start an :httpd server by calling :inets.start(:httpd, config). This does start a httpd server, but in my script it immediately ended because the main process no longer had any work.

I can avoid this behavior by passing --no-halt when calling my script, but I chose to mimic the diff, by setting up a receive do which blocks until it receives a message that the server is down. This gives the same effect, without needing to pass additional arguments on invocation.

Takeaways

I’m glad to hear that Erlang is soon going to get the ability to quickly and easily host a static directory like Python. If you’re interested in this ability now, I’ve put my script here.If you would prefer the escript, I’ve created that here. I prefer the Elixir syntax, but the escript version has the benefit of only needing Erlang to run.

Always Show Scrollbars in Firefox (and Thunderbird)

2023-11-05T00:00:00+00:00

Always Show Scrollbars in Firefox (and Thunderbird)

I use the scrollbar to gauge how far I’ve read through an article or webpage. But the default of hiding the scrollbar after a few moments makes this frustrating.Prior to figuring out how to do this, I would wiggle the mouse over the bar every time I wanted to check. This quickly became frustrating. Firefox has a setting for this, which makes the scrollbar always visible.Credit to this Stack Overflow answer for pointing me to the setting.

In Firefox 116, I can find this setting in Settings -> General -> Browsing -> “Always show scrollbars”

After enabling this setting, the scrollbar is always visible on pages long enough to need one, with a backdrop that visually distinguishes it from the webpage.Note that this blog post might not be long enough to show you a scrollbar.

Thunderbird

Thunderbird (version 102) has an equivalent setting:
Settings -> General -> Language & Appearance -> Scrolling -> “Always show scrollbars”

Similar to Firefox, when this setting is enabled it will always show a scrollbar on anything long enough to need one, with a backdrop to visually distinguish it.

When is an Erlang process a shell?

2023-10-31T00:00:00+00:00

When is an Erlang process a shell?

Recently, I was trying to port an Erlang function to Elixir, with the goal of answering one question: “When is an Erlang process a shell?”

Answering that question involved source diving Erlang, learning new things about git, and reading mailing lists.

The Original Code

The code I wanted to port was this functionErlang’s syntax might seem a bit weird if you haven’t been exposed to it before. Erlang was originally implemented in prolog, which inspired much of the syntax.Variables in Erlang are UpperCase, which makes them easy to pick out of code once you’re used to it, but opposite of the norm in Algol inspired languages.The Prolog inspiration also led to a “sentence-like” structure, with commas, semicolons, and periods. Note that this function end with a . character.:

is_shell(ProcessId) ->
  case erlang:process_info(ProcessId, group_leader) of
    undefined -> false; %% process is dead
    {group_leader, Leader} ->
      case lists:keyfind(shell, 1, group:interfaces(Leader)) of
        {shell, ProcessId} -> true;
        {shell, Shell} ->
          case erlang:process_info(Shell, dictionary) of
            {dictionary, Dict} ->
              proplists:get_value(evaluator, Dict) =:= ProcessId;
            undefined -> false %% process is dead
          end;
        false -> false
      end
  end.

I can follow along here, we’re retrieving something called group_leaderErlang and Elixir have the concept of atoms. Atoms are equal to only themselves, and their value is exactly their name. This seems pretty useless until you consider pattern matching.I can use a pattern like {:my_atom, value} to match on the :my_atom as a key and assign the second tuple item to the variable value.Note that in Elixir atoms are typically denoted with a prefixed colon, like :this. In Erlang, atoms are lower case with no other markings, like this, which works because variables are upper case in Erlang. from the process’s metadata. The equivalent function call in Elixir would be Process.info(pid, :group_leader).

What is a group_leader? There was sparse information about group leaders in the Erlang documentation, but I found a mailing thread response that said this:Note, the original source uses outdated terminology for origin and peer, the peer module superseded the previous module in OTP 25.0. Original source here.

Group leaders let you redirect I/O to the right endpoint. It is one of the few properties that is inherited by one process to the other, creating a chain.

By default, an Erlang node has one group leader called ‘user’. This process owns communication with the stdio channels, and all input requests and output messages transit through that one.

Then, each shell you start becomes its own group leader. This means that any function you run from the shell will send all its IO data to that shell process.

A group leader gives us a sense of where each process belongs to, and since each shell has its own group leader is_shell/1 needs to unpack this information to find whether it’s a shell.

This answers the question about group leaders, the next curiosity in the function is group:interfaces which is called on the group leader process id.

`group:interfaces`!?

So the first thing I did was to extract the group leader of my Elixir IEx shell, and try to call :group.interfacesIn this story I’m switching back and forth between Erlang and Elixir syntax. Know that module:function/arity in Erlang is equivalent to :module.function/arity in Elixir.Elixir has first class support for calling Erlang functions, and this manifests as similar but subtly different syntax. on that process id.

Just one problem, IEx raises an UndefinedFunctionError, telling me that it doesn’t exist. That’s okay, it’s been 11 years since the Erlang code was written, perhaps that function was taken out.

So the next thing I try is to repeat the same experiment directly in the Erlang shell erl. I open a new terminal, invoke erl, get the erlang:process_info(self(), group_leader). I pass this process id to group:interfaces(GroupLeaderPid)., and I get a valid list of interfaces!?

I’m confused for a short while, before remembering that I use nix-shell to manage my Elixir version. So when I ran the :group.interfaces invocation in IEx, it was in OTP 26, erts-14.1. However, when I ran group:interfaces in erl, it was in OTP 25, erts-13.2.2. Something must have changed in the group module between OTP 25 and OTP 26.erts stands for the Erlang Run-Time System Application. Erlang versions its applications separately from the OTP as a whole, leading to the two version numbers I shared.

Git Spelunking

In order to figure out what what was going with group I needed to turn back time.Thanks to Jeff for the turn of phrase. group is an internal module in Erlang, and lacks external documentation. So I needed to explore the source code directly.

Thankfully, git gives us an easy way to checkout old code. Running git checkout OTP-25.0 on the otp repository drops me into the OTP code as it was at that release.

A quick ripgrep for interfaces/1 leads me to lib/kernel/src/group.erl and interfaces/1:In the middle of this process, I confused user_drv:interfaces with group:interfaces and convinced myself that it had mysteriously changed from a function that didn’t do what I needed to a function that worked between OTP 25.0 and OTP 25.3. I had no basis for why this change might have happened or why it happened so late after the function was created, when it’s clear that 11 years ago in the function that starts this article, it was used in the same way as OTP 25.3.Specifically, I think I got lost because I had backgrounded my text editor in user_drv.erl, and when I checked out OTP 25.0 the interfaces function appeared.

interfaces(Group) ->
    case process_info(Group, dictionary) of
	{dictionary,Dict} ->
	    get_pids(Dict, [], false);
	_ ->
	    []
    end.

get_pids([Drv = {user_drv,_} | Rest], Found, _) ->
    get_pids(Rest, [Drv | Found], true);
get_pids([Sh = {shell,_} | Rest], Found, Active) ->
    get_pids(Rest, [Sh | Found], Active);
get_pids([_ | Rest], Found, Active) ->
    get_pids(Rest, Found, Active);
get_pids([], Found, true) ->
    Found;
get_pids([], _Found, false) ->
    [].

In interfaces it first extracts the process dictionaryJoe Armstrong in his book, “Programming Erlang, 2nd Edition” says this: “Each process in Erlang has its own private data store called the process dictionary. The process dictionary is an associative array (in other languages this might be called a map, hashmap, or hash table) composed of a collection of keys and values. Each key has only one value.”He continues: “Note: I rarely use the process dictionary. Using the process dictionary can introduce subtle bugs into your program and make it difficult to debug.” then it tail recursively extract tuples that contain user_drv or shell as their first element.Erlang has ad-hoc polymorphism, which it uses here in get_pids to succinctly pattern match on the shape of the arguments passed in.

In our original function, we only care about the shell, but otherwise I now knew enough to port is_shell/1 into Elixir:

Port of function into Elixir

In this version, I skipped past implementing interfaces and simply pulled the :shell key out of the process dictionary of the group leader:In Elixir it’s convention to add a ? to the name of a function that returns a boolean.Joe Armstrong’s comment about the process dictionary implies that it is a keyword list, but most of the Erlang functions I see interacting with it treat it as a proplist. These two data structures are similar, both are built on top of the Erlang list.A keyword list, is a list of 2-tuples, with the first element being an atom key, and the second being a value for the key. A keyword list is a valid proplist, but a proplist has a shortcut where an atom key by itself stands in for {atom, true}, this breaks all of Elixir’s Keyword functions, that presume a well-formed keyword list.Keyword List Example: [{:key1, "value"}, {:key2, 5}, {:key3, true}]Proplist Example: [{:key1, "value"}, {:key2, 5}, :key3]In this code snippet, I use the pin operator ^. In Erlang, variables can only be assigned once, so after their first assignment, you can use them to pattern match.Elixir allows variable shadowing, meaning that you can reuse a variable name. In order to do pattern matching on a variable’s contents, instead of reassigning it, Elixir added the pin operator ^ to “pin” the value of the variable for the pattern match. Learn more here.

def is_shell?(pid) do
  case Process.info(pid, :group_leader) do
    :undefined ->
      false

    {:group_leader, leader} ->
      {:dictionary, leader_dict} = Process.info(leader, :dictionary)

      case :proplists.get_value(:shell, leader_dict) do
        ^pid ->
          true

        shell when is_pid(shell) ->
          case Process.info(shell, :dictionary) do
            {:dictionary, dict} ->
              :proplists.get_value(:evaluator, dict) === pid

            :undefined ->
              false
          end

        :undefined ->
          false
      end
  end
end

Refactor to use `with`

The nested case statements in the previous function can be refactored using Elixir’s with statement for a cleaner function:

def is_shell?(pid) do
  with {:group_leader, leader} <- Process.info(pid, :group_leader),
       {:dictionary, leader_dict} <- Process.info(leader, :dictionary),
       {:shell, ^pid} <- :lists.keyfind(:shell, 1, leader_dict) do
    true
  else
    {:shell, shell} ->
      with {:dictionary, dict} <- Process.info(shell, :dictionary) do
        :proplists.get_value(:evaluator, dict) === pid
      else
        _ -> false
      end

    _ ->
      false
  end
end

Shell improvements OTP 26

Earlier, I found that group:interfaces/1 was missing from OTP 26. This seems to be connected to the OTP 26 shell improvements.

It’s worth looking at group:whereis_shell/0 as a comparison with my is_shell?/1 function:

`group:whereis_shell`

The OTP 26 implementation of group:whereis_shell contains this snippet:

GroupPid ->
  {dictionary, Dict} = 
    erlang:process_info(GroupPid, dictionary),
  proplists:get_value(shell, Dict)

Which is nearly identical to my pre-with refactor implementation in Elixir. Which means I’m probably on the right track.

`group:start_shell/1`

I also wanted to call attention to this comment above group:start_shell/1:

%% start_shell(Shell)
%%  Spawn a shell with its group_leader from the beginning set to ourselves.
%%  If Shell [is] a pid the[n] set its group_leader.

Which confirms what we learned earlier about group leaders and how they relate to shells.

That would have been the end of the story, until I read the comment above the is_shell/1 function in the original codebase:

%% Theoretically pman_process:is_system_process/1 should say true for
%% the shell.  Well, it doesn't, so this is a workaround until it
%% does.

What is pman_process? Unlike :group.interfaces/1, the function isn’t missing, the module is gone. This led me down a rabbit hole that required learning how to bisect a git repo:

What is `:pman_process`?

A search for “Erlang pman” turns up a process viewing tool from an ancient version of Erlang. The screenshots of the interface look like they’re right out of Tab Window Manager.I briefly used twm when I used OpenBSD on the desktop, but I preferred cwm.

After a lot of digging, I found that PMan was a process viewing application built into Erlang, whose behavior was absorbed by Observer. PMan was written on the Graphics System (GS) application. GS was superseded by wxwx is an Erlang port of wxWidgets a cross-platform GUI library. for graphical applications for Erlang.

PMan and the GS related backends were removed in OTP 17.0. If I checkout a version of OTP prior to OTP 17, I can finally find what pman_process is doing. Prior to its removal, it was located in lib/pman/src/pman_process.erl.

The module includes a purpose comment:

A front-end to the erlang:process_info() functions, that can handle processes on different nodes in a transparent way.

Also some convenience functions for process info, as well as some application specific functions for process classification.

This purpose explains why it might contain an is_system_process/1 function.

One Last Mystery

pman_process contains a few -define calls that declare module constants that are lists of process names and function signatures that mark something as a “system” process.

Where does this list come from? Why is hard coded? How do you determine what needs to be on these lists?

git blame gives me no answers, the code for pman_process was added to the OTP git repo in its initial commit in 2009. And it remained untouched until it was removed prior to OTP 17.0.

Git Spelunking with Bisect

2023-10-29T00:00:00+00:00

Git Spelunking with Bisect

Today, I continued on the git spelunking that prompted the post from yesterday. I’ve been diving deeper in the Erlang codebase today, and so I needed some new tools.

`git bisect`

git-bisect lets you search for a commit using binary search. I wanted to find the last commit that contained a particular Erlang function in the otp codebase. My process looked something like this:Thank you to Yannick for helping me learn how to use this.You can read more about git-bisect here.

First I start the bisect

git bisect start

Then I specify a commit that didn’t have the function, in this case HEAD:

git bisect bad

Then I specify a commit that does have the function. In this case I knew that the function was in OTP_R14A. git-bisect lets me specify a tag here:

git bisect good OTP_R14A

At this point git-bisect took me automatically to a commit that was roughly halfway between those two commits. While I’m here, I can run whatever checks I want to on the command line:

rg '<function-to-find>'

If the function is there, I mark git bisect good. If it’s not, then I mark git bisect bad. Either way, git-bisect moves me to the next pivot commit in the search. otp has tens of thousands of commits, but binary search meant that it took around 15 steps to get the commit I wanted.

Once I was done, I could run:

git bisect reset

To get back to where I started.

It’s great to know that I can run whatever arbitrary manual commands I need at each manual step, but in this case I was running the same command each time.

In this case, that’s where git bisect run comes in:

`git bisect run`

git bisect run lets you run a script to check for good/bad at each step of the binary search, instead of having to do it manually.Thank you to Mikkel for suggesting this to me.You can read more about git-bisect run here.

The initial suggestion I got was to use git grep as in:

git bisect start
git bisect bad
git bisect good "[tag]"
git bisect run git grep "[function name]"

But I knew that there were two versions of is_system_process/1 at various times in the Erlang codebase. I only wanted to know about the older one, so I used this modified version instead:

git bisect start
git bisect bad
git bisect good OTP_R14A
git bisect run git grep "[function name]" "path/to/folder/*"

Where path/to/folder/* was pointing the folder I knew the function was in.I initially tried to use the exact filename, but in failing commits the file didn’t exist throwing an error that git bisect run didn’t know how to deal with. The glob seemed to avoid this problem.

With this, the binary search took a few seconds and quickly returned the same commit that I found manually.

Now that I had the commit in hand, I wanted to know what the next tagged release that contained that commit was. Enter git-describe:

`git describe`

git-describe is purpose built for finding tags from commits. By default, it finds the tag that immediately predates the commit. But if you use the --contains option, it will find the tag that “contains” the commit, that is the commit I want to find.Credit to Stack Overflow for helping me find this one.

If the tag is exactly the commit, then it will only return the tag. Otherwise, it will have a suffix that shows:Source where you can read more about git-describe.

the number of additional commits on top of the tagged object and the abbreviated object name of the most recent commit.

I can use sed to strip out the suffix, since I only want the tag name:

git describe --contains "<commit>" | sed 's/~.*//'

Another way to accomplish this is with git tag --contains:Thanks to Miccah for this suggestion.Read more about git-tag here.

git tag --contains "<commit>" --sort=creatordate

Which will return all of the tags that contain the commit, sorted by their creation date.

Now I have the tag I want, but when was it created? I found a couple techniques that work:

Technique 1: `git log -1`

git log will display information about the parents of a given commit. But if you use the -1 option, it will limit it to 1 commit, only the one we pass to it. And we can pass a tag to it:Credit to Stack Overflow for this one too.Read more about git-log here.

git log -1 "<tag>"

This will show the default information about the commit behind the tag.

If I only wanted the date, I could use:

git log -1 --format=%ai "<tag>"

Technique 2: `git for-each-ref`

git for-each-ref will iterate over all refs that match a given pattern. It also lets you format information from that ref.I found information on this one from two Stack Overflow answers: one and two. The latter from a comment on the accepted answer.Read more about git for-each-ref here.

That pattern can be as specific as a single tag:

git for-each-ref \
  --format="%(refname:short) | %(creatordate)" \
  "refs/tags/OTP_17.0-rc1"

I like this one because I can easy generalize a solution to ask other questions. When was every tag released?

git for-each-ref \
  --format="%(refname:short) | %(creatordate)" \
  "refs/tags"

By default, it seems to sort alphabetically by refname. If I want them in chronological order, I can add --sort=creatordate on the end.

Takeaways

I came out of this expedition with a rich collection of tools that I can use in future spelunking. All of these tools are built into git and have excellent documentation in the reference manual.

I often read source code in order to get a better understanding of the tools, libraries, and software that I use. These git tools allow me to explore that source code in specific historical context, and understand how codebases evolve and change over time.

Bonus: A Simpler Search Method

When I asked about how to find the commit I needed in a Recursers chat, I got more answers after I had already found the commit I was looking for. Here is one that was simpler than git-bisect:Thanks to Nathan and Benjamin for this suggestion.

`git log -S`

You can use git log -S "<name of function>" to find commits that touch the string <name of function>. I found this didn’t take much longer than git bisect run on the Erlang codebase. And, I didn’t have to find an early “good” commit before searching.

Git Checkout Tag

2023-10-27T00:00:00+00:00

Git Checkout Tag

You can checkout a tag in a git repo by usingThis is pretty useful for exploring historical version of code bases. “What was this codebase like at release X?” I needed this in order to find a removed function in Erlang.:

$ git checkout <tag>

Simple as that.

Note that running this command without a branch will put you in “detached HEAD” state, meaning that any commits you make will be reachable only by their exact hash. If you want to make changes with a tag as your starting point, use -b <branch name> like this:

$ git checkout -b <branch name> <tag>

Pathspecs

When I initially looked up how to do this, I found instructions to use git checkout tags/<tag>.

This is contrary to git’s own documentation, but it worked!

This seems to work because tags are a ref in the refs/tags namespace, and git correctly identifies the right tag, and thus the right commit from there. I believe this is the same mechanism as when you give a short hash and git finds the correct commit.

Firefox Has a Built-In Screenshot Tool

2023-10-25T00:00:00+00:00

Firefox Has a Built-In Screenshot Tool

Today I learned that Firefox has a built-in screenshot tool. It’s bound to ctrl+shift+S and allows you to easily select an element of the screen or a selection of the screen. Firefox then gives you the option to copy that selection to clipboard or save it as an image.

ExampleHere I took a screenshot multiple times to get the recursive effect.:

Firefox documents this shortcut in its section on tools.

Paper: Four Concepts for Resilience Engineering

2023-10-06T00:00:00+00:00

Paper: Four Concepts for Resilience Engineering

Today’s paper is by David Woods: Four concepts for resilience and the implications for the future of resilience engineering. I found this paper through Fred Hebert’s excellent summary.

Resilience has been used a number of different ways in Resilience Engineering literature. This conflation of different definitions makes it difficult to parse and understand what that literature is arguing for and why.

Woods’ paper attempts to codify these various definitions into four categories. The categories clarify which properties of a system we’re talking about, and which aspects of a system are worth studying.

Resilience as Rebound

Rebound is about returning to normal functioning after a disruption.

This ability to rebound seems to depend directly on the conditions before the disrupting event. How was the system prepared before the chaos began?

While literature on rebound tends to focus on individual disruptions or traumas, the more interesting thing to study is the idea of surprises. A surprise is, to quote Woods:

the event is a surprise when it falls outside the scope of variations and disturbances that the system in question is capable of handling

A surprise is an event that challenges an existing model and forces the system to learn or adapt the model.

Resilience as Robustness

Robustness is the ability of a system to resist disruptions. However, robustness is specifically defined as resistance against a known set of disruptions. From Rebound, we know that a system will often deal with surprises, that by definition, fall outside the robustness a model.

Thus resilience is more interested in adapting to surprise than preparing against known threats. However, this distinction is lost when conflating resilience with robustness.

Another problem is that expanding robustness in one way, often opens additional vectors of failure in the event of surprises. For example, an umbrella gives you some shelter from rain, but the mechanism can jam or strong winds can drag you into unwanted positions.

Resilience as Graceful Extensibility

Graceful Extensibility sees resilience as the opposite of brittleness. Where brittleness is how a system performs when it’s pushed to near or beyond its limits.

This is a definition of resilience that specifically cares about the kinds of surprise that I talked about in Rebound. While surprise falls outside of our systems limits, it has regular characteristics because many classes of challenge re-occur.

Graceful extensibility is play with the idea of graceful degradation. Graceful extensibility suggests the ability to grow from surprises which stands in contrast to robustness or graceful degradation, which only allow us to mitigate breakdowns.

Another aspect of concern is decompensation where exhaustion of a system under sustained disruption reduces the capacity of the system to adapt to new disruptions. Think on-call fatigue in an operations teams or deformation of a material under stress that changes its properties.An effective way to “cut” paper without scissors is to fold it back and forth across a joint until the paper gives way with little force. Allowing you to tear the paper straight by hand. As Woods writes:

When the time to recovery increases and/or the level recovered to decreases, this pattern indicates that a system is exhausting its ability to handle growing or repeated challenges, in other words, the system is nearing saturation of its range of adaptive behavior.

Resilience as Sustained Adaptability

Sustained adaptability asks three questions of a resilience engineer:

What governance or architectural characteristics explain the difference between systems that have sustained adapt- ability that don’t?
What design principles and techniques allow you to engineer sustained adaptability?
How would you know if you succeeded in engineering sustained adaptability in a system?

Predictable challenges that test a system include:

Surprises will continue to challenge boundaries of a system.
Conditions will change over time, shifting the boundaries of a system.
When, not if, the system fails to adapt, people will need to bear that burden.
How the system needs to gracefully extend, and what allows you to gracefully extend will change over time.
The system will need to be able to benefit from surprises, as discussed in graceful extensibility.

Architecting a system to have sustained adaptability relies on understanding that all adaptive systems are constrained by trade-offs, and that certain architectures allow for adjustment of those trade-offs.

Woods argues that definitions 1 and 2 have led to fewer developments and are less useful overall than definitions 3 and 4. Though he exercises caution in his judgment given the youth of the field.Thanks to Jeff for reading and editing with me for these notes.

Configure IEx to Show Lists as Lists

2023-10-02T00:00:00+00:00

Configure IEx to Show Lists as Lists

Charlists are a holdover from Erlang as an alternate form of string. In Erlang, strings are syntax sugar on a list of binary characters that make up that string. In Elixir, strings are syntax sugar over UTF-8 encoded binaries.Erlang also has the concept of binaries (called a bitstring), they just aren’t the default string implementation. Elixir chose to implement strings directly on top of bitstrings instead of lists. Since Elixir has first class support for Erlang code, it needs to easy to convert Elixir strings into the charlists that Erlang functions expect.If you’re interested in reading more, here is a note about charlists in Elixir’s documentation.

Elixir’s support for charlists manifests in interesting ways. By default, Elixir’s IEx renders lists that contain printable characters as charlists.

Thus:

iex()> Enum.chunk_every(1..10, 2)
[[1, 2], [3, 4], [5, 6], '\a\b', '\t\n']

The single-quoted ' characters on the right are charlists. However, it’s a bit annoying to have to translate between charlists and the actual lists. In this case: [7, 8] and [9, 10] that those charlists represent.

The pretty printing that IEx does when it returns a value is powered by Elixir’s inspect. inspect takes some configuration, as detailed in Inspect.OptsRead the rest of the options available here.:infer is why our example above is partially charlists, it only recognises the list starting with 7 to be entirely printable characters and opportunistically transforms it.:

:charlists - when :as_charlists all lists will be printed as charlists, non-printable elements will be escaped. When :as_lists all lists will be printed as lists.

When the default :infer, the list will be printed as a charlist if it is printable, otherwise as list. See List.ascii_printable?/1 to learn when a charlist is printable.

Kernel.inspect/2 accepts a list of options that are internally translated to an Inspect.Opts struct:

iex()> inspect(Enum.chunk_every(1..10, 2), charlists: :as_lists)
"[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]"

Now I have a string that looks correct, but I would like IEx to return the value, but print it correctly. And I would like to not have to invoke inspect with my settings every time.

Thankfully, IEx gives us IEx.configure that allow me to configure my IEx session. IEx.configure takes a keyword list of options to set, and offers inspect as a possible key:

iex()> IEx.configure(inspect: [charlists: :as_lists])
:ok
iex()> Enum.chunk_every(1..10, 2)
[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]

Success!

If I want to make this happen every time, I can add the IEx.configure call to ~/.iex.exs.You can set things on a per-project basis too, read more here.

Just Add Water

2023-10-02T00:00:00+00:00

Just Add Water

Or, Using Partial Application In Elixir for Better Errors.

Today, I was adapting the generator for a recipe website written by Hundred Rabbits into Elixir. While adapting their idiomatic C code into Elixir, I needed a way to pass a predefined list of ingredients to the builder API for Recipes.

To do this, I defined a function that returned a keyword listThe cheese: cheese syntax looks a bit funny, but it’s Elixir syntax sugar for {:cheese, cheese} where the second cheese is the bound variable.:

def ingredients do
  cheese = create_ingredient(...)

  salt = create_ingredient(...)

  [cheese: cheese, salt: salt]
end

And then in my builder API, I called that function inside the builder scope:

def recipes do
  ingredients = Ingredients.ingredients()

  ...
end

In order to add an ingredient to a recipe, I used a function add_ingredient_quantity/3, used something like this:

def recipes do 
  ingredients = Ingredients.ingredients()

  create_recipe("salted cheese")
  |> add_ingredient_quantity(ingredient[:cheese], "2 oz")
  |> add_ingredient_quantity(ingredient[:salt], "1 teaspoon")
end

This worked great for ingredients that I had already defined in my keyword list, but the AccessRead more about Access here. behaviour that powers Elixir’s [] syntax returns nil if a key isn’t found in the accessed object.

I wanted a way to detect that I needed to define a new ingredient in a recipe, so that every ingredient gets its own page, like Grim Grains. My naive solution was to put a guard clause on add_ingredient_quantity that checked if the ingredient was nil:

def add_ingredient_quantity(recipe, ingredient, quantity) when not is_nil(ingredient) do
  ...
end

This would certainly complain, because of a pattern match error. However, the Access had already returned a nil, so the missing ingredient’s atom wasn’t available for an error message.

This became evident when I tried to add waterYou would want to drink some water if you were eating that much salted cheese, after all. to my recipe:

def recipes do 
  ingredients = Ingredients.ingredients()

  create_recipe("salted cheese")
  |> add_ingredient_quantity(ingredient[:cheese], "2 oz")
  |> add_ingredient_quantity(ingredient[:salt], "1 teaspoon")
  |> add_ingredient_quantity(ingredient[:water], "1 quart") 
end

This resulted in a pattern match error, but the error only knew that it was passed a nil with no context about where that nil came from.

Partial Application in Elixir

To solve this, I replaced the ingredient[:water] callI first thought about overriding the Access behaviour to return something other than nil, but I decided against it. with a partial application.

In Elixir I can use an anonymous function to partially apply a named Elixir function. Here, I wanted to get a value out of a specific Keyword List, so I chose Keyword.fetch. Keyword.fetch takes two arguments: a keyword list and a key to getRead more about Keyword.fetch/2 here.. By partially applying Keyword.fetch with ingredients and wrapping it in a with statement I can raise a helpful error:

def recipes do
  ingredients = Ingredients.ingredients()
  get_ingredient = fn key -> 
    with {:ok, value} <- Keyword.fetch(ingredients, key) do
      value
    else
      :error -> raise "#{key} not found in ingredients"
    end
  end

  create_recipe("salted cheese")
  |> add_ingredient_quantity(get_ingredient.(:cheese), "2 oz")
  |> add_ingredient_quantity(get_ingredient.(:salt), "1 teaspoon")
  |> add_ingredient_quantity(get_ingredient.(:water), "1 quart") 
end

Note that we’re using an anonymous function, so get_ingredient needs to be applied with the . operatorI initially tried to use the capture operator on Keyword.get/3, passing raise as the default as shown here. This successfully raised a helpful error on a missing key, but it eagerly raised on every key!.

Now, when I’m missing an ingredient I’ll get a helpful error message that reminds me to just add it.

Parsing TSVs in Go

2023-10-02T00:00:00+00:00

Parsing TSVs in Go

I’m building a terminal application to learn Go, and I wanted to persist data using Tab Separated Values. Here are some of the things I learned while figuring out how to do that.

CSV Reader in Standard Library

Go has a CSV reader in their standard library. You can import it like this:

import (
	"encoding/csv"
)

Once it’s imported, you can create a CSV Reader by calling csv.NewReader on an io.ReaderI learned about reading CSVs in Go from Go Samples.:

file, err := os.Open("my.tsv")
if err != nil {
	log.Fatal(err)
}

// remember to close the file at the end of the program
defer file.Close()

csvReader := csv.NewReader(file)

Since I want to read Tab-Separated Values instead of Comma-Separated Values, I can set the Comma used by my CSV Reader to be a tab character:

csvReader.Comma = '\t'

I don’t expect my CSV to be memory intensive, so I can read it all into memory with csvReader.ReadAll():

data, err := csvReader.ReadAll()
if err != nil {
	log.Fatal(err)
}

Higher Order Functions in Go… Kinda

Now I have TSV loaded in memory, but csvReader.ReadAll() returns a slice of records of type []string. If I want to convert these record into structs, I’ll need to parse the records myself.

I’m used to working with functional languages, where I would take the list of records and use map to transform them into structs.

Generics

Go doesn’t have a builtin map function, but we can build one using generics:I learned how to build a generic map function from ZetCode.Note we have to call the function map2 because map is reserved for the hashmap in Go.

func map2[InputType, OutputType any](
	data []InputType, f func(InputType) OutputType) []OutputType {

	result := make([]OutputType, 0, len(data))

	for _, element := range data {
		result = append(result, f(element))
	}

	return result
}

This function takes a slice of InputType and a function that takes InputType and returns any OutputType, iterates over the slice building a return slice by calling the function on each element before appending.

Because Go supports Generics, we can vaguely specify that the input must be of a type, but not worry about which type that is until we see the input data. Same goes for the output type, it’s entirely determined by the specific function that’s passed in as the second argument.

Prior to learning about this version of map2, I didn’t know that Go had an any type. It’s perfect for a situation like this, where we don’t care what type the function returns, as long as it’s consistent.

Slices

One thing that was new to me coming to go is how Go handles slices. Elixir uses immutable lists, but Go seems to use allocated arrays called slices.Question: Why are they called slices? Does Go have something called arrays?

The make function from before takes a type, a starting size, and a capacity, and allocates and returns an initialized object of that type. In the case of map2, that object is a slice of type OutputType.

Then we call append, which will take an allocated slice of a type, and items of that type, and append those items to the end of the slice.

If that slice still has sufficient capacity to accommodate the new items, it will reslice to include the new items. Otherwise, a new array will be allocated.The documentation for append refers to arrays, which implies that arrays are distinct from slices. Based on the way this function is written, I would guess that arrays refer to the physical memory that has been allocated, and that slices are a reference to that allocated array, which may not be the whole array. I’ll need to do further reading to confirm.

Parsing at last

Now that we have our map2 function we can invoke it like this:

result := map2(data, func(s []string) Bookmark {
	return Bookmark{Name: s[0], LastRead: s[1], Chapter: s[2]}
})

Here we define an anonymous function that takes a record of type []string, and returns a struct of type Bookmark.

The result here will be a slice of type Bookmark, with each Bookmark containing the data from the relevant row from the TSV.

Customizing Fonts in Various Places

2023-10-02T00:00:00+00:00

Customizing Fonts in Various Places

Notes on how I’ve customized fonts in various places.

I like to use Atkinson HyperlegibleYou’re reading it now, unless you’ve overridden this websites fonts. where I can for the ability to distinguish characters.

Firefox

Firefox uses the system font by default.

Adding the following to my NixOS configuration.nix did the trick:

fonts.fontconfig.defaultFonts = {
  sansSerif = [
    "Atkinson Hyperlegible"
     # ...
  ];
};

One quick nixos-rebuild switch later, and the new font was in place.

Note that I needed to restart Firefox in order for the font to take effect.

This left my fontsize in my address bar too small, so I modified:

layout.css.devPixelsPerPx = 1.25

In the about:config for FirefoxCheck out the instructions in this answer.. Note this resizes everything in Firefox, but overall the effect is worth it for me.

Mastodon

I use the standard Mastodon web client, which includes:

Settings -> Appearance -> Animations and Accessibility -> "Use system's default font"

Checking the box and hitting save loaded my font without issue.

Slack (browser)

Updating the font in Slack is easy asI learned how to do this from this verge article.:

/slackfont Atkinson Hyperlegible

If I ever want to reset the font I can use the slash command without an argument:

/slackfont

Installing fonts on NixOS

Setting the system font on NixOS was easy, but first I needed to install the fontRead more in the wiki for NixOS..

My fonts path for my user is:

$HOME/.local/share/fonts

I copied the Atkinson Hyperlegible .otf files there, then ran:

fc-cache

A Joy to Work With

2023-10-02T00:00:00+00:00

A Joy to Work With

Today, I’ve been playing Joy, a purely functional concatenative programming language.Thanks to Shae for reminding me of Joy, and for suggesting that it might be useful for systems programming.

Everything in Joy is a function that takes a stack and returns a stack.

Hello world looks like thisSourced from here:

"Hello, world!\n" putchars .

Piece by piece, "Hello, world!\n" puts a literal string on the stack. putchars takes a stack consisting of a single string, and prints it to the console. The . character tells joy to execute the sequence of functions.

Command Line Arguments

My interest in Joy is for short executable programs, so I wanted to know how to learn how to use command line arguments in Joy.

Here’s what I came up withI’m using a joy executable compiled from here:

#!/usr/bin/env joy
argv 1 drop [" " concat] map [putchars] step .

Piece by piece:

argv takes all arguments passed in with the program, including the program’s name, and returns an aggregate (here a list of strings).

1 is a pure function that takes a stack and adds itself to that stack.

drop takes an aggregate and a number, and drops number of elements from the aggregate. In this case, the name of the calling program.

[" " concat] is a quote, which allows me to pass a function unexecuted onto the stack. I believe in this case the concat is partially applied with " " as the second argument.

concat takes two sequences and concatenates them.

map takes an aggregate and a quoted function, and executes the function on each element of the aggregate, returning a stack containing a new aggregate with elements transformed by the function. Here it will take each element from our modified argv aggregate and concatenates them with " ".

step takes an aggregate and a quoted function, and executes the function once for each element of the aggregate. In this case, I call putchars for each string.

Thus:

$ ./hello.joy hello world
hello world

I’ve been using the resources found here which have been most useful at making this short program work.

Writing From a Learning Firehose

2023-10-02T00:00:00+00:00

Writing From a Learning Firehose

I’m attending the Recurse Center (remotely).

I value writing things down because I don’t trust my memory to retain the information that I’ve learned. Writing everything down from the diversity of people and the pace of learning from the recurse center is difficult.

I had a great conversation about how to write about the things that you’re learning or working on.Special thank you to Aditya, Mike, and Rachel for their generous ideas! These are some of my takeaways from that conversation.

Spool Brain to Disk

Or, take notes as you go.

Aditya uses git commit messages to record his thoughts in the moment. Anything that would want to refer back to later, ends up in the git messages. It acts an append-only buffer, timestamped with when specific notes were written.

I plan to use jrnl to do the same, with a custom journal for a writing project I’m working on.

Lower the Bar

Write less and be okay with it.

Rachel suggested writing for 5 minutes, 1 minute. Make it a tiny habit to start writing.

Mike uses writing as accountability. A short summary of what he did posted in a public blog post.

In both cases, the lesson is to write less, and to be okay with hitting publish sooner rather than later.

Conclusion

Spool your brain
Lower the bar
Pair and share

I plan to use these suggestions to write about what I learn.

Percent Encoding URLs

2023-09-19T00:00:00+00:00

Percent Encoding URLs

I use djot for the markup on my static site generator. In djot, the link syntax looks like this:

[link text](http://example.com)

This works fine for most links, but certain links that include parenthesis, it breaks. This is mostly a problem for Wikipedia, which uses parenthesis for disambiguation of topics. For example:

[Variety](https://en.wikipedia.org/wiki/Variety_(cybernetics)#Law_of_requisite_variety)

As written, this will break the link, since djot will parse the first closing ) as the end of the URL, which is not the complete link. This is because, by design, djot parses markup in linear time without backtracking.

Percent Encoding Parenthesis

This is where Percent Encoding comes in. Percent encoding is a method specified in the URI specification that allows for the encoding of arbitrary data in a URI.If you would like to read the details, they seem to be specified here in RFC 3986 Uniform Resource Identifier (URI): Generic Syntax.

Specifically, I needed to encode ) as %29.Technically, ( is also a reserved character and should be escaped, but I don’t run into any issues with djot with that character, so I leave it out here.

Using my link from before, that looks like this:

[Variety](https://en.wikipedia.org/wiki/Variety_(cybernetics%29#Law_of_requisite_variety)

If I use this markup: Variety, I correctly get the link to the Wikipedia article.

Elixir URI encoding

When I first learned about Percent Encoding, I quickly found the URI.encode/1 function in Elixir. To my dismay, it didn’t correctly escape parenthesis:

iex()> URI.encode("(hello)")
"(hello)"

Frustrated, I continued escaping my links for djot by hand.

Today, I re-read the documentation of URI.encode and found a line that I previously missedFull documentation here:

This function also accepts a predicate function as an optional argument. If passed, this function will be called with each byte in string as its argument and should return a truthy value (anything other than false or nil) if the given byte should be left as is, or return a falsy value (false or nil) if the character should be escaped. Defaults to URI.char_unescaped?/1.

The documentation for URI.char_unescaped?/1 explains that it is deliberately escaping the minimum required and purposely leaving reserved characters unescaped:

Checks if character is allowed unescaped in a URI.

This is the default used by URI.encode/2 where both reserved (char_reserved?/1) and unreserved characters (char_unreserved?/1) are kept unescaped.

It also hints at a better predicate function for URI.encode, URI.char_unreserved?/1, which escapes the parenthesis like I need:

iex()> URI.encode("(hello)", &URI.char_unreserved?/1)
"%28hello%29"

Takeaways

djot’s linear parsing forces it to assume that the first ) it encounters is the end of a URL.

Using percent encoding, I can encode ) as %29, which solves the djot issue while allowing the browser to route the URL correctly.

Elixir’s URI.encode doesn’t escape parenthesis by default, but does allow an alternate predicate function that does.I wrote a short script that I can call using kakoune’s pipe command. Since it has no dependencies, the startup time is minimal, but I may rewrite it in a faster startup time language later.

Include Diff in Git Commit Editor

2023-09-10T00:00:00+00:00

Include Diff in Git Commit Editor

Running git commit with the --verbose flag allows me to include the diff that’s being applied by the commit inside the editor where I’m writing my commit message. This makes it easier for me to describe the changes that were being made, since I can scroll down in my editor to see themRead more in the documentation for git-commit.

Even though the lines aren’t prefixed with #, they are ignored by git-commit when submitting my commit message.

For example:

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
#
# On branch main
#
# Initial commit
#
# Changes to be committed:
#	new file:   hello.txt
#
# ------------------------ >8 ------------------------
# Do not modify or remove the line above.
# Everything below it will be ignored.
diff --git a/hello.txt b/hello.txt
new file mode 100644
index 0000000..3b18e51
--- /dev/null
+++ b/hello.txt
@@ -0,0 +1 @@
+hello world

This can be set to on by default in the config using commit.verbose = trueRead (a little) more in the docs here.

Or as it shows up in my global config:

[commit]
	verbose = true

Temporary Short Names

2023-08-31T00:00:00+00:00

Temporary Short Names

When writing code, I prefer to use the full name of something to retain context and design decisions established by that name. However, there are cases where using a name many times in rapid succession becomes tedious, bordering on line noise. In those situations, it may make sense to use a temporary alias or short name to keep code legible.

Here are a couple examples, one in Elixir and one in Rust, of how you can do that:

Rust - Enum Alias

If you have a long Enum name like thisCredit to Logan Smith for introducing me the following trick. Go watch his video to hear more about the reasoning to prefer an alias over a * import, as well as other opinions on Rust. (This Enum suggestion is point 2 in the video).:

enum DecidedlyLongEnumNameHere {
  VariantX,
  VariantY,
  VariantZ,
}

Doing an exhaustive pattern match on that Enum becomes tedious:

match enum {
  DecidedlyLongEnumNameHere::VariantX => x(),
  DecidedlyLongEnumNameHere::VariantY => y(),
  DecidedlyLongEnumNameHere::VariantZ => z(),
}

Rust lets you do a use declaration inside of a function, so you can create a short alias like this:

use DecidedlyLongEnumNameHere as D;
match enum {
  D::VariantX => x(),
  D::VariantY => y(),
  D::VariantZ => z(),
}

The alias has the same scope as where you’ve used it, and is declared right next to its use, so context from the name is retained.

Elixir - Scoped Aliases

In Elixir, you may have a similar problem with long Module or Struct namesOr both, since structs are named after their creating module, and that module often defines functions useful to that struct.:

def my_fun do
  [1, 2, 3]
  |> OverlyLongModuleName.fun1()
  |> OverlyLongModuleName.fun2()
  |> OverlyLongModuleName.fun3()
  ...
end

I prefer not to use a module wide short name, since that obscures the calling code by disconnecting that code from information important to understand its purpose.

Elixir supports lexical scope on alias and importCheck out hexdocs for more information., so you can do this:

def my_fun do
  alias OverlyLongModuleName, as: O

  [1, 2, 3]
  |> O.fun1()
  |> O.fun2()
  |> O.fun3()
  ...
end

This allows for code to be shortened while retaining proximal knowledge about module context.

I haven’t tried either of these in collaborative codebases, but it’s nice to know that both are possible.

Legible Timeouts with Erlangs timer

2023-08-28T00:00:00+00:00

Legible Timeouts with Erlang’s `timer`

By convention, timeouts in Erlang (and Elixir) are set in units of milliseconds. However, this isn’t type checked by either language, so you end up with magic integers for timeouts throughout a codebase.

While both Erlang and Elixir support underscores in numbers for readability: 5_000, you’re still expected to know that timeouts are in milliseconds.

Enter `timer`

Erlang’s timer provides legible helper functions for expressing timeouts:

timer:seconds/1

timer:minutes/1

timer:hours/1

Each takes an integer and converts it from the stated time to milliseconds.

timer also provides timer:hms/3 for hours, minutes, and seconds, returning their sum in milliseconds.

Using `:timer` in Elixir

Elixir supports seamless integration of Erlang standard library functions, though the syntax is a bit different:

:timer.seconds(5)

So you can replaceI’ve considered using an opaque type to force the use of legible functions, but that would involve wrapping every function in Erlang and Elixir that uses a timeout, which isn’t practical.:

Process.sleep(5000)

with:

Process.sleep(:timer.seconds(5))

Takeaway

Next time you need to set a timeout in Erlang or Elixir, consider using timer to embed your design decisions and communicate without magic numbers.Thanks to Zach for pointing me to timer!

Set null Placeholder in PostgreSQL psql

2023-08-26T00:00:00+00:00

Set null Placeholder in PostgreSQL psql

By default, psql outputs null as nothing, which is easily mistaken for an empty string.

You can useRead more about the available psql commands here:

\pset null '<chosen alternative>'

This seems to be tied to a particular session of psql as it was reset when I restarted the psql session.

Unicode tangent

The impetus for finding this command came from seeing ¤ used as a null placeholder in “The Art of PostgreSQL”.

It is U+00A4 Currency SignRead more about the unicode here. used to denote an unspecified currency. I’ve never seen the symbol before, but apparently it is common enough to be in the International Reference Variant of ASCII.Thanks to Jeff for exploring PostgreSQL with me!

My Favorite Elixir Discovery - Access

2023-08-23T00:00:00+00:00

My Favorite Elixir Discovery - Access

A while ago, I read Hillel Wayne’ssource suggestion to “Search less, browse more”. Specifically to systemically go through the the information available for your chosen tools, rather than searching for specific answers to questions as they arise. This is the story of what I found when I started browsing through Elixir’s documentation.As found on HexDocs

But first, a diversion into:

GraphQL queries

One of the advantages of GraphQL is that you specify the structure that you’re querying with your query. You specify the edges and nodes of the data that you want, and it’s delivered in JSON with exactly that schema. However, this can get unwieldy if the accessible starting point is far away in the graph from the data you need.

For example, here is a query accessing <meetup.com>’s API searching for Online Elixir Events hosted by GroupsA lot of Elixir meetups went online during the pandemic, but Meetup’s search made them hard to find.Latitude: -48.876667 and Longitude: -123.393333 points to Point Nemo the most distant point of ocean from land.:

{
  keywordSearch(
    input: { first: 1000 }
    filter: {
      query: "elixir"
      lat: -48.876667
      lon: -123.393333
      source: GROUPS
      radius: 100
      eventType: ONLINE
    }
  ) {
    edges {
      node {
        result {
          ... on Group {
            name
            unifiedEvents {
              count
              edges {
                node {
                  title
                  isOnline
                  eventUrl
                  description
                  dateTime
                  timezone
                  group {
                    name
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Notice the nested curly braces at the end. I can get the data I need, and I know the structure that it’s in, but the data I care about is hidden by a nest of indirection.

Here’s how I first attempted to unpack the data I needed:

Unpacking GraphQL in Elixir

def main() do
  Neuron.Config.set(url: "https://api.meetup.com/gql")
  {:ok, query_results} = File.read!("elixir.gql") |> Neuron.query()

   results = query_results.body["data"]["keywordSearch"]["edges"] 
     |> Enum.filter(fn elem -> elem["node"]["result"]["unifiedEvents"]["count"] > 0 end) 
     |> Enum.map(fn elem -> elem["node"]["result"]["unifiedEvents"]["edges"] end) 
     |> Enum.concat()
     |> Enum.map(fn elem -> elem["node"] end)
end

This is only getting the maps of data that I care about, it doesn’t include additional work to do filtering that the API didn’t provide sufficient granularity to do before the queryNeuron is an Elixir GraphQL client: https://hexdocs.pm/neuron/readme.html.

Part of what makes it awkward is that we alternate between nodes, which are parsed as maps, and edges, which deserialize as lists.

Access, a better alternative?

This brings me back to browsing the Elixir documentation. My first inkling that there might be a better way arrived with Kernel.get_in/2Read more here. get_in/2 gets a value from a nested structure, I have one of those! It takes a data structure and a list of keys, and uses the Access behaviourI wondered why Access was implemented as a Behaviour and not a Protocol. It turns out that it comes down to performance, Access is invoked far more than any protocol functionality in Elixir, and it was becoming a bottleneck. So they switched it to a Behaviour, which is built right into the Erlang/OTP ecosystem, and is optimized with first-class support. to extract a value.

Access is what powers the thing["squareBracket"] syntax, so we were already using it. The real power of the module comes from special functions in the Access moduleread more about Access here. The get_in/2 function takes more than just atom or string keys, it can also take functions that can process our GraphQL Edges. Let’s talk about three:

`Access.all/0`

Access.all/0 accesses all elements of a list.

For example:

iex()> books = [
         %{
           name: "Naming Things", 
           url: "https://www.namingthings.co/"
         },
         %{
           name: "Essence of Software", 
           url: "https://essenceofsoftware.com/"
          }
        ]

iex()> get_in(books, [Access.all(), :name])
["Naming Things", "Essence of Software"]

`Access.filter/1`

Access.filter/1 access all elements of a list that match a provided predicate function.

For example:

iex()> books = [
         %{
           name: "Naming Things", 
           url: "https://www.namingthings.co/"
         },
         %{
           name: "Essence of Software", 
           url: "https://essenceofsoftware.com/"
          }
        ]

iex()> get_in(
  books, 
  [
    Access.filter(
      &String.starts_with?(&1.name, "N")
    ), 
    :name
  ]
)
["Naming Things"]

`Access.key/2`

Access.key/2 allows you to access the given key in a map or struct. This seems superfluous, but in Elixir, Structs don’t implement the Access behaviour by default. This means that you access keys with the . syntax, but not the [] syntax:

defmodule MyStruct do
  defstruct [:name]
end

iex()> my_struct = %MyStruct{name: "A name"}
iex()> my_struct.name
"A name"
iex()> my_struct[:name]
** (UndefinedFunctionError) function 
MyStruct.fetch/2 is undefined (MyStruct does not 
implement the Access behaviour. If you are using 
get_in/put_in/update_in, you can specify the 
field to be accessed using Access.key!/1)

The error message hints at the usefulness of Access.key/2, though it recommends the version that can throw an error: Access.key!/1

iex() get_in(my_struct, [Access.key(:name)])
"A name"

This is extremely useful for structs generated by other Elixir librariesI personally found this useful for access nested structures preloaded from Ecto.

Returning to GraphQL

With Access in hand, I can refactor my nested access call from earlierI can skip my earlier use of filter since Enum.concat/1 behaves nicely with empty lists.:

def main() do
  Neuron.Config.set(url: "https://api.meetup.com/gql")
  {:ok, query_results} = File.read!("elixir.gql") |> Neuron.query()

  results = get_in(
    query_results, 
    [
      Access.key(:body), 
      "data", 
      "keywordSearch", 
      "edges", 
      Access.all(), 
      "node", 
      "result", 
      "unifiedEvents", 
      "edges", 
      Access.all(), 
      "node"
     ]
  ) |> Enum.concat()
end

This shows us exactly how indirect the original data was. Now the structure is easy to see and modify.

Takeaways

Access provides a clean way to access data in nested data structures. Access.key provides a way to access structs. Access.all and Access.filter/1 provide ways to handle lists in the middle of a data structureI’ve heard that this style of accessors is referred to as Functional Lenses, and other programming languages have implementations or libraries supporting this kind access..

Here, I only needed to access the nested data, but Elixir’s Kernel provides functions for get_and_update_in/3, pop_in/2, put_in/3 for changing the data in that nested structure. Check them out: https://hexdocs.pm/elixir/1.15.0/Kernel.html

i3 Window Manager Space Mode

2023-08-22T00:00:00+00:00

i3 Window Manager Space Mode

When I tried Spacemacs, the one thing I really liked was the “leader” key setup they had around the spacebar.

i3 window manager supports arbitrary modes, which can have their own keybinds. This is how I currently have a small “space-mode” setup in my i3 configThe $reset is vital here, otherwise you’ll be stuck in “space-mode” forever.

set $reset mode "default"

bindsym $mod+space mode "space-mode"

mode "space-mode" {
    bindsym space exec --no-startup-id dmenu_run; $reset
    bindsym $mod+space exec --no-startup-id dmenu_run; $reset
    bindsym Escape $reset
    bindym Return exec alacritty; $reset
}

Adding RSS Feed Link Tag

2023-08-11T00:00:00+00:00

Adding RSS Feed Link Tag

Recently, I added some Atom (RSS)https://stackoverflow.com/questions/3489578/which-is-most-used-rss-or-atom-and-which-one-is-better-to-build-from feeds to my website. I wanted to make sure that I appropriately linked them in my website’s header.

The solution is to use a <link>Link documentation: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/link element:

<link 
   rel="alternate" 
   title="Erika Rowland Combined Feed" 
   type="application/atom+xml" 
   href="https://erikarow.land/combined.xml"
>

I have multiple feeds so I wanted to make sure I knew how to appropriately link all of them. It seems that it’s valid to simply add a <link> element for each feed:

https://webmasters.stackexchange.com/questions/102139/can-a-website-have-multiple-rss-feeds-how-would-the-link-and-channel-elemen

Edit the Clipboard

2023-08-08T00:00:00+00:00

Edit the Clipboard

I recently wanted to write some code in a Github issue, but while Github Flavored Markdown supports code snippets, the web editor has a variable spaced font that makes writing formatted code difficult.

Enter `vipe`

vipe is a tool from moreutils that allows you insert an editor into a unix pipe.

vipe will create a temporary file with anything passed from standard input (empty if invoked directly) and open $EDITOR on that file. Anything that saved in that file when the editor closes is piped to standard output.

To solve my Github problem, I realized that I could invoke:

$ vipe | xclip -selection clipboard

which let me type my code snippet in my editor of choice, then copied that snippet onto my clipboard.

xclip -out can be used to output the current state of the clipboard, so:

$ xclip -out | vipe | xclip -selection clipboard

can be used to modify the current clipboard in an editor.

An Unlinked Adventure

2023-08-06T00:00:00+00:00

An Unlinked Adventure

I’ve been working on my new tool maru, to get it polished for release. To that end, I decided to look at cargo-dist a tool that aims to help with release engineering for Rust projects. Since maru is a Rust CLI, it seemed like a great fit.

This is the story of how the interaction between cross-compilation and NixOS led to a rabbit hole of dynamic linking on linux.Thank you to Jeff for pairing with me as we ventured into this rabbit hole.

`cargo-dist`

cargo dist init will generate a github action to handle automatic release creation on Github when tags update. For maru, I was missing repository = ... in my Cargo.toml, which cargo-dist noticed and prompted me to fix.

After adding:

repository = "https://github.com/erikareads/maru"

to my Cargo.toml, cargo dist init helpfully generated a github action workflow and updated the Cargo.toml with configuration metadata. I chose to use the shell installer, to see how it worked for maru.

Previously, I had run

cargo install --git "https://github.com/erikareads/maru"

which compiled maru locally, which worked on my NixOS machine.

After generating the Github action yaml with cargo-dist, I committed the changes and pushed a tag. To my delight, the Github action automatically compiled maru for three different architectures and generated a release.

However, when I downloaded the maru-x86_64-unknown-linux-gnu.tar.xz, unpacked it, and ran ./maru I got this error:

$ ./maru
bash: ./maru: cannot execute: required file not found

Uh oh, what required file isn’t working? This is where the linking adventure really began.

The adventure begins - `strings`

Jeff suggested using strings to see the difference between my working locally compiled version and the broken downloaded version.

diff <(strings ~/.cargo/bin/maru) <(strings ~/Downloads/maru-x86_64-unknown-linux-gnu/maru ) | less

From which we found:

< /nix/store/yaz7pyf0ah88g2v505l38n0f3wg2vzdj-glibc-2.37-8/lib/ld-linux-x86-64.so.2
---
> /lib64/ld-linux-x86-64.so.2
> libgcc_s.so.1
...

Oh, non-standard NixOS strikes again. I don’t yet understand what that difference means, but it’s clear that it’s at the heart of what’s breaking the binary.

Now that we know that an interaction with NixOS is probably the source of the issue. A searchI used Duck Duck Go for my searches during this adventure. of

nixos rust cannot execute: required file not found

Led to a matklad article about ld on NixOS. There he suggests readelf as a path forward:

$ readelf -d ~/Downloads/maru-x86_64-unknown-linux-gnu/maru
Dynamic section at offset 0x16b560 contains 29 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 ...

Comparing that with the locally built working version:

$ readelf -d ~/.cargo/bin/maru
Dynamic section at offset 0x175ea8 contains 32 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
 0x000000000000001d (RUNPATH)            Library runpath: [/nix/store/...]
 ...

The locally built one has an extra shared library -> the ld linker that we found different in the strings output. In addition, NixOS sets a RUNPATH that allows the binary to find the shared libraries on NixOS.

Knowing that ld is the problem on NixOS, I searched:

compiled binary doesn't work on nixos ld

Which led to an article on nix-ld. Binaries assume the ld is a fixed location for glibc, but NixOS breaks that convention. nix-ld addresses the problem by overriding the RUNPATH that was set when we compiled locally, even on binaries that where we can’t recompile them ourselves.

Which is great, but I have the source code available, maru is my project! Is there a way that I can compile maru such that this linking path problem isn’t an issue?

How does ripgrep do it?

My first thought was to look at ripgrep, a known well-distributed Rust CLI. I installed ripgrep using NixOS, so maybe that handled this issue for me, but I wanted to check. Looking in ripgrep‘s releases, I see that it *doesn’t* compile for unknown-linux-gnu, but instead for unknown-linux-musl.

Downloading this binary of ripgrep, the invocation works without issue.

musl?

This led me to search for:

rust compile without dynamic linking

Which led to this stack overflow question which led me to this answer:

Rust statically links everything but glibc (and libgcc, iirc) by default.

If you want to get a 100% statically linked binary, you can use MUSL with 1.1. https://github.com/rust-lang/rust/pull/24777 is the initial support, we hope to make it much easier to use in the future.

EDIT: It’s distributed via rustup now, so you can add x86_64-unknown-linux-musl as a target : rustup target add x86_64-unknown-linux-musl

And then build to this target : cargo build --target=x86_64-unknown-linux-musl

Linking against musl will allow maru to be statically compiled, which should avoid the dynamic linking problem on NixOS.

After running the relevant rustup and cargo build commands, I confirmed that a locally musl compiled binary worked on my NixOS machine.

Patching the generated github actions

Now, I don’t want to locally compile every binary of maru for my project, so I needed to update the Github actions in order to do the musl compile.

An initial test, just switching out the build target failed, because it didn’t have the target environment installed via rustup.

The fix was to add a conditional to my Github action workflow:

- name: Maybe install target
  if: ${{ contains(matrix.dist-args, 'musl') }}
  run: rustup target add x86_64-unknown-linux-musl

The Github actions API exposes a function contains, which allows us to only run this command on the musl compilation in the matrix of operating systems. rustup was already being used to install Rust in the Continuous Integration, so we expected that the rustup target would work.

After a few minutes of eagerly waiting for the new version to compile, we confirmed that the musl binary worked both on NixOS and on Debian.

This was great, but the shell installer provided by cargo-dist no longer worked, since it was looking for the gnu binary.Turns out this was a known problem and was mentioned in the limitations/caveats in the documentation. Check out this issue for discussion on the problem. The shell installer checks to see whether glibc exists on the installing system, which isn’t sufficient for NixOS to avoid the problem, but it doesn’t switch to musl if the gnu binary isn’t found in the release.

Results

Thanks to this adventure, maru is now statically linked for linux systems, and runs without issues straight after download on both NixOS and Debian.

Jeff and I learned about excellent debugging tools such as readelf.Also strace and file which I didn’t mention in the story, since readelf provided the correct information. file did share: interpreter /lib64/ld-linux-x86-64.so.2, which succinctly showed the path.

Playing with SVG

2023-08-01T00:00:00+00:00

Playing with SVG

This week I had a chance to play with SVGs. Previously, I’ve felt intimidated by generatin graphics with code, but I found working with SVGs to be delightful.

Basics - Path

<svg width="100" height="100" xmlns="http://www.w3.org/2000/svg">

  <path d="M 10 10 H 90 V 90 H 10 Z" fill="transparent" stroke="black"/>

  <!-- Points -->
  <circle cx="10" cy="10" r="5" fill="red"/>
  <circle cx="90" cy="90" r="5" fill="red"/>
  <circle cx="90" cy="10" r="5" fill="red"/>
  <circle cx="10" cy="90" r="5" fill="red"/>

</svg>

One of the more powerful tools that SVG provides is the <path>. A path can have attributes, but it’s primary driver is d for data, which takes a list of Commands, where each command is a composed of a letter and numbers that are that command’s parameters.

Capital Letters signify absolute commands.

lower case letters signify relative commands.

In the example above:

M stands for MoveTo, moving the cursor to the specified point.

H stands for Horizontal, drawing a line from the current point to the end x coordinate.

V stands for Vertical, drawing a line from the current point to the end y coordinate.

Z stands for ClosePath, closes the current path drawing a line to the initial point.

<path> allows me to chain these together to draw things, in this case a square.

Space vs Comma

<svg viewBox="0 0 200 100" xmlns="http://www.w3.org/2000/svg">
  <!-- Example of a polyline with commas between point pairs-->
  <polyline points="0,100 50,25 50,75 100,0" />
  <!-- now without commas--> 
  <polyline points="0 100 50 25 50 75 100 0" transform="translate(100,0)" />
  <!-- translated so they're not on top of each other-->
</svg>

SVG Cat

.Emoji designed by OpenMoji – the open-source emoji and icon project. License: CC BY-SA 4.0 This one in particular was designed by Sofie Ascherl: https://openmoji.org/library/emoji-1F408/Yes, that makes this a copy cat.

<svg viewBox="0 0 72 72" xmlns="http://www.w3.org/2000/svg">
<g xmlns="http://www.w3.org/2000/svg" id="line" stroke-linecap="round" stroke-linejoin="round" stroke-miterlimit="10" stroke-width="1" fill="none" stroke="#000">
    
    <path d="m12.46 35.74c-2.333 1-4.917 0.8333-4.917 0.8333-1.677 0.1458-3.115-4.01-2.485-4.733l3.318-5.1-1.75-3.417s5.008-1.415 7.883 2.09c0.3444 0.42 0.7943 0.7429 1.279 0.9871 0.0298 0.015 0.0602 0.0302 0.0912 0.0456 2.593 1.289 5.546 1.571 8.385 0.9981 7.222-1.458 14.07-1.37 21.7 2.212 7.625 3.583 14.53-2.25 13.64-7.5-0.793-4.647 3.562-7.583 6.75-5"/>
    <path d="m16.05 48.82c0.6006-2.206 8.491-3.648 8.491-3.648s3.228-1.201 1.426-4.504"/>
    <path d="m18.3 33.24c-1.543 1.834-3.893 4.803-0.44 9.158 0 0-6.756 2.853-6.006 8.033 0 0 0.3624 2.476 2.402 2.402"/>
    <path d="m23.5 50.03c-1.156 7.254 2.386 6.055 3.017 5.661 1.148-0.7173 1.848-9.854 3.952-11.31 1.592-1.104 8.167-0.3021 8.167-0.3021"/>
    <path d="m38.44 41.33c0.0911 1.742 0.7529 3.402 1.734 4.845 0.6616 0.9727 1.803 2.32 1.453 2.985-4.479 8.5 0.6224 7.022 1.083 6.167 3.188-5.917 6.125-4.104 4.647-10.52 0 0 5.27-1.81 5.52-7.977"/>
    <path d="m48.15 45.59s2.367 3.204 7.758 2.693c0 0-3.326 6.762 0 7.62 1.917 0.4941 4.722-11.16 4.722-11.16s-1.839-0.7937-3.951-4.182"/>
  </g>
</svg>

Takeaways:

You don’t need spaces as long as you can tell that numbers or commands are distinct.
You can continue adding additional sets of parameters to a command. For example: l says this:

Any subsequent coordinate pair(s) are interpreted as parameter(s) for implicit relative LineTo (l) command(s) (see below).
SVG doesn’t do math, so 1-2 is two points: 1 and -2.
c is a cubic bezier curveMDN tutorial on Path Cubic Bezier Curves:

Cubic Béziers take in two control points for each point.
In the original, the stroke details: stroke-linecap, etc., were attributes of each element, weThanks Jeff for pairing with me on this! refactored this into the group’s attributes.

Mix Completions

2023-07-13T00:00:00+00:00

Mix Completions

I’ve released mix_completions, a mix task that allows you to add shell completions for mix. mix_completions supports completions for bash and zsh.fish already had community contributions adding completions for mix: fish completions

How it works

mix_completions adds three tasks to work with shell completions:

`mix complete`

mix complete caches the available completions from mix help. If mix complete detects a mix.exs file, it will cache these completions in the project in .mix_complete.cache. Otherwise it will store the cache in $XDG_CACHE_HOME/.mix_complete.cache.

This means that you can have a separate set of completions for a project with custom tasks than what is generally available to mix globally.

`mix complete.bash`

mix complete.bash will refresh the cache a la mix complete, and outputs bash that when sourced enables shell completions for mix.

`mix complete.zsh`

mix complete.zsh, like its bash counterpart, refreshes the cache and generates output that can be loaded by zsh. Instructions on where to put this output can be found in the README.

Why Cache Completions

With shell completions there seem to be two ways of sourcing the completions:

Ask the program for the completions available. This is the approach that cobra and click take. The shell completions they output are simply a parser for completions output by the program itself.
Caching completions outside of the program. This is the approach that tools like completely take. Saving the completions in plain text or directly in the shell output.

I chose to use option 2, because Elixir has a non-trivial startup time. Recently work has reduced the startup time of the BEAM for Elixir, but mix still takes a few seconds to run on my machine. Instead of waiting multiple seconds for completions to come back, I chose to cache the completions in a TSV that can parsed and handled by the shell completions directly.

Where to store the cache?

This project led me to learn about the XDG Base Directory Specification. I’ve known about these environment variables for a while, but I didn’t know how to use them as a software author.

The XDG specification gives guidance on where to put configuration or cache files on unix systems. The recommendations were simple to use: XDG_CACHE_HOME is configurable by a user, and defaults to $HOME/.local/cache if unset.

I can either put my cache as a file in that directory or create a subdirectory if I have multiple cached things to store. mix_completions only stores one TSV for its cache, so it’s stored directly in the XDG_CACHE_HOME directory.

Finding all the tasks

Initially, when I wanted to find all of the available mix tasks, I used System.shell to call mix help and then processed the output with awk. This got the job done, but felt fragile.

While explaining this to a friend, they pointed out that mix help --names does this work for me.Thanks Jeff! Though I did have to filter iex from the list, and I was still invoking System.shell.

Then I realized that if mix had an option for this, it must know how to find (and print) all of the available tasks. Now, mix_completions uses vendored versions of the same functions that mix help uses. This proved to be useful when I switched from a simple list of tasks to a tab-separated values file of both tasks and their shortsdocs, since Mix.Task provides helpers to fetch things like a task’s shortdoc.

Limitations

Since I chose to work in a project agnostic way, I have no way of knowing what arguments and flag are available for subcommands. Thus I chose to only provide completions for available tasks, since I can ask mix for these directly. The fish community completions provide more granularity, at the cost of pre-assuming which tasks you have available.

I chose to use caching over hard-coding so that you can detect new tasks and add completions for them, if only at the surface level.

Try It Now

If you would like to try mix_completions run:

mix archive.install hex mix_completions

Instructions on how to source the completions for your chosen shell are in the README.

Please report any issues you find. :)

zsh Completion

2023-07-03T00:00:00+00:00

zsh Completion

Today I continued my exploration of shell completion with zsh completion. And it’s a lot simpler than bash completion.

Enabling zsh Completion

I learned that you can enable zsh completion to autostart by adding these two lines to your .zshrc:

autoload -Uz compinit
compinit

Both were added by the zsh configuration wizard when I opened the shell for the first time. The first line loads compinit so that it’s available to zsh. The second line calls compinit to enable completions.

Custom completion

A minimal example of zsh completion looks like this:

#compdef my_echo

_my_echo()
{
  completions=("hello" "hear" "today")
  _describe "completions" completions
}

That’s it.

In order to enable this completion, I needed to name the file _my_echo, and add it to my zsh fpath in my .zshrc:

fpath=(~/.zsh/functions $fpath)
autoload -Uz compinit
...

Let’s break down the minimal example:

`#compdef`

When compinit runs, zsh will look at the first line of each file it finds on the fpath. #compdef is a tag that defines the function to be called for name, in our case my_echo. Since the filename matches the function name, it loads _my_echo.

`_describe`

_describe is a utility function provided by zsh that takes a tag, in this case "completions", and an array of completions and loads them.

Unlike bash, there’s no further work needed. zsh has all the information it needs, and dutifully provides tab completion for my_echo.

The simplicity of zsh after bash is refreshing.

Bash Completion

2023-06-26T00:00:00+00:00

Bash Completion

This week, I’ve been exploring how tab completion works in Bash. I knew that it was possible to generate bash completions for command line utilities, but I never knew how it worked.

So I cracked open the source code for click, a python framework for command line applications that had shell completion as a feature. Helpfully, the shell completions were contained in shell_completion.py.The way that click does shell completions is interesting, it uses the application itself to provide completions. I may revisit how this works in the future.

The most helpful line was one that looked like this:

complete -o nosort -F complete_function program_name

With both precise completion_function and program_name determined by the installation of the click application.

complete is a bash builtin, but help complete provides little detail as to how it works. In particular, there’s no mention of the -F flag that click uses.

Info to the Rescue

While help proved to be unhelpful, info a tool I knew about, but hadn’t previously explored, had exactly what I needed.

Section 8.6 Programmable Completion and Section 8.7 Programmable Completion Builtins proved to share exactly what I wanted to know.I’m linking to the web versions, but this information is available through info bash on the command line.

Let’s walk through our line of bash from before:

`complete program_name`

complete works by registering completions for a particular name, in most cases this is the builtin or program that we want completions for. Once those completions are registered, the particular method defined will work every time completion is invoked, typically by the the tab key.

`-o nosort`

-o sets a comp-option with specifies extra behavior for complete.

In this case, -o nosort tells complete not to sort the completions array that it receives. In this case, it allows click to precisely control the order of completions that it shows.

`-F complete_function`

This is most interesting part of complete’s behavior. -F specifies a bash function that will be invoked when completions are asked for on the registered name.

This function receives three arguments from complete:

$1 is set to the the name of command which is receiving completions.
$2 is set to the word being completed.
$3 is set to the word preceding the word being completed.

$3 allows for the invoked function to handle nested commands, such as git’s branch which will complete on available branches for the repo.

When the function finishes, the completions are retrieved from the value of the COMPREPLY array variable.

Putting it together

I now had a mental model of how the completions worked, but I wanted to solidify my knowledge by creating a minimal example.

My named command will be my_echo, a shell wrapper around echo.

#!/usr/bin/env bash
echo $1

With that in mind, I wrote the following bash:

_my_echo_complete() {
  local options=("hello" "hear" "world" "today")
  for option in ${options[*]};
  do
    COMPREPLY+=($(echo "$option" | grep $2));
  done
}

_my_echo_setup() {
  complete -F _my_echo_complete my_echo
}

_my_echo_setup

I hadn’t worked with bash arrays before, so it took some experimentation to learn about () to create an array. Note that the parenthesis are used both for options and next to the += on COMPREPLY.

The [*] operator inside ${options[*]} unpacks the array so that for can iterate through it.

Once I wrote this bash, I added it to a file called complete_function which I then sourced:

$ source complete_function

This version still has issues (hitting tab with no input makes grep throw error once for every item in the array), but it’s a working version of tab completion for a program I “wrote”.

I look forward to trying to add tab completion in bash to future tools.Special thanks to Jeff and Kathrin who joined me in my exploration of Bash completion.

YouTube RSS Feeds

2023-06-18T00:00:00+00:00

YouTube RSS Feeds

I’m trying to get back into using RSS feeds. YouTube used to make it easy to follow channels by RSS, but they’ve removed that ease in favor of what seems to be channeling you into creating a Google account.

YouTube feeds do still generate an RSS feed though. It can be found at:

https://www.youtube.com/feeds/videos.xml?channel_id=<channel_id>

Where <channel_id> is replaced with the identifier for the YouTube channel you want to follow. However, YouTube no longer uses these channel ids in the channel URL, phasing them out in favor of @ style handles for each channel. This leads to easier to remember links to channels, but that @-handle doesn’t work in the feeds link above.

I found that this channel id was still available under the “share” arrow on the “About” page for a channel.

I was talking to my friend Jeff, and we decided to see if we could do something about this manual process of navigating to the “About” page and clicking the share arrow.

The Search is On

My first thought was to open the browser DevTools on the About page. Here I used the inspector to find the HTML for the share arrow. This didn’t lead far, as the arrow is powered by javascript. I didn’t know how to search the javascript for the HTML class, since javascript is often split up across multiple files.

After a web search I found a ChromeI usually browse with Firefox, but Jeff was more familiar with the Chrome DevTools. I haven’t determined if there is an equivalent “search everything” for Firefox DevTools. DevTools help article: Search: Find text across all loaded resources.

Here, I meant to search for the HTML class, but I accidentally pasted the clipboarded channel-id.

Turns out it was right there in the HTMLThis is the channel-id for the @charmcli youtube channel.:

<meta itemprop="identifier" content="UCTVHQ9GMFYLalQDynyXNTKQ">

Parsing HTML

I wanted to create a script that would take a channel’s @-handle, and copy the resulting RSS link to my clipboard so I could use it in my RSS reader.

Fetching the HTML was easy enough:

$ curl https://www.youtube.com/@charmcli/about
<wall of html>

Now I needed a way to select out the channel-id from the HTML tags. Enter pup.

pup is a command line HTML parser. I could pipe the HTML from curl into pup to select all meta tags:

$ curl https://www.youtube.com/@charmcli/about \ 
> | pup 'meta'
<only meta tags>

I wanted to then select a specific tag based on a property:

$ curl https://www.youtube.com/@charmcli/about \
> | pup 'meta[itemprop="identifier"]'
<meta itemprop="identifier" content="UCTVHQ9GMFYLalQDynyXNTKQ">

Finally, pup provides an attr function to select from an attribute, in this case content:

$ curl https://www.youtube.com/@charmcli/about \
> | pup 'meta[itemprop="identifier"] attr{content}'
UCTVHQ9GMFYLalQDynyXNTKQ

All together

The final script looks like this:

channel_id=$(curl https://www.youtube.com/$1/about | pup 'meta[itemprop="identifier"] attr{content}')
echo -n "https://www.youtube.com/feeds/videos.xml?channel_id=$channel_id" | xclip -selection clipboard

Now I can follow my favorite YouTube channels from my RSS reader!

Does Go Have Enums?

2023-06-18T00:00:00+00:00

Does Go Have Enums?

As I continue to explore Go for my new project, I wondered if there was a way to do Enums in Go.

Enums are important for the way I write code, because I try to avoid overloading identifiers. That is, I want each identifier to capture as much of its own context as possible.

Booleans are the worstOkay, that’s not fair nil or null is also semantically anemic, and provide less information than a boolean. But only slightly. for this. They are semantically anemic, in that if I see one in a code base, I have to find what generated it to get context about what it does.

For example:

do_something("hello", false)

What does that false do? I’ll need to look up the function signature of do_something to find out.

In Elixir, booleans are a union of two atoms:

@type bool :: true | false

So I can easily replace the boolean with an “enum”:

@type text_state :: :capitalize | :lower_case

Now I can get a better sense for my functiondo_something is still a bad name, but at least the arguments are more readable. from before:

do_something("hello", :capitalize)

I lose the ability to easily use conditionalsElixir lets me pattern match, which means that I can avoid conditionals in a lot of places where, say, python would require them., but I gain significant readability in my functions, even at the call site.

Go Enums

Go does have EnumsThanks to this awesome guide for an introduction to Go enums., though they behave a bit differently than Elixir’s “alternative atoms for matching” approach.

They look like this:

type TextState int64

const (
	Capitalize TextState = iota
	LowerCase
)

Go Enums are simply constants that are statically newtyped so they only type to themselves.

iota auto-increments an untyped integer so that you don’t have to specify a value for each element of the constant.

It was weird to me that Enums in Go tend to be backed by integers, but since Go is statically typed, you get strong guarantees that your value is Capitalize and not 1.

I look forward to using Go Enums to decrease the likelihood of bugs in my code.

Combined Feed

Using use in Gleam

Using use in Gleam

What is use anyway?

So What?

result.unwrap and early returns

Avoiding boilerplate with result.map

Chaining result.try

Context Management

Takeaways

Additional Notes

A note of use scope

A note on use arguments

bool.guard

list.fold

Tools I Use: firefox

Tools I Use: firefox

Bookmark Keywords

Text substitution

Address bar search (tab search)

Screenshot Tool

Process Manager

Developer Tools

Mobile View

Inspector

Developer Tool Keyboard shortcuts

Pinned Tabs

Always Show Scroll Bars

Tab Settings

Extensions

uBlock Origin

ClearURLs and Privacy Badger

DF YouTube (Distraction Free)

OneTab

Block Site

about:robots

Things I like about Gleam's Syntax

Things I like about Gleam’s Syntax

Handy pipe operator

Labeled arguments

Elegant function capturing

Only case statement, no if

Sub pattern names using as

Succinct alternative clause patterns using |

Support for todo to mark uncompleted work

Takeaways

How to Embed an SVG in CSS

How to Embed an SVG in CSS

Embedding a specific SVG

SVG Data URL

How to Count Unique Lines Using Unix Pipes

How to Count Unique Lines Using Unix Pipes

Applications

Cut - Aggregate a CSV Column

jq - Aggregating the Web

I created a little webring in Gleam

I created a little webring in Gleam

What is a webring?

How do you run a webring service?

Where are you ringing from?

Ringing my neighbors

Constellation

A Gleaming Web(ring) Service

Up And Running

Extra Goodies

Random Endpoint

A Lustrous Home Page

My Favorite Gleam Feature

My Favorite Gleam Feature

Reverse Order

Partial Application

Pipe Operator

“method”-like functions

Lack of methods

A culture of qualified imports

My Favorite Feature: Discoverability

Embedding Gleam in my blog with Vite

Embedding Gleam in my blog with Vite

Step 1: Create Lustre Project

Init

Using `use` in Gleam

What is `use` anyway?

`result.unwrap` and early returns

Avoiding boilerplate with `result.map`

Chaining `result.try`

A note of `use` scope

A note on `use` arguments

`bool.guard`

`list.fold`

`about:robots`

Sub pattern names using `as`

Succinct alternative clause patterns using `|`

Support for `todo` to mark uncompleted work

`esgleam`

`re` - Perl-like regular expressions for Erlang

`:bind_address`

`:mime_types`

`:default_type`

`:directory_index`

`:modules`

`:inets.start/0`

`no-halt`

`group:interfaces`!?

Refactor to use `with`

`group:whereis_shell`

`group:start_shell/1`

What is `:pman_process`?

`git bisect`

`git bisect run`

`git describe`

Technique 1: `git log -1`

Technique 2: `git for-each-ref`

`git log -S`