Hi, I’m Erika Rowland (a.k.a. erikareads). Hi, I’m Erika. I’m an Ops-shaped Software Engineer, Toolmaker, and Resilience Engineering fan. I like Elixir and Gleam, Reading, and Design. She/Her. If you're looking for experienced talent, I would love to chat. Published on Modified on

Elixir Library - Ecto

Note on Ecto.

Ecto reflection

Ecto.Schema has built in reflection. Each schema macro generates a number of hidden functions for the module that allow me to introspect details about my schemas at runtime. Some of my favorites:Using Ecto version 3.11.1 documentation here.

iex()> MySchema.__schema__(:fields)
[:field1, :field2]

iex()> MySchema.__schema__(:type, :field1)
:string

I use this reflection to avoid duplication, as in an all_fields function:

def all_fields do
  __MODULE__.__schema__(:fields)
end

Which can be used in the Changeset or anywhere I need to know all of the fields.

I also use this reflection for Schema validation testing:Thanks to my friend Jeffrey Matthias for sharing this gem with me.

defmacro validate_schema_fields_and_types(
  schema,
  expected_schemas_and_types
  ) do
  quote do
    test "#{unquote(schema)}: it has the correct fields and types" do
      schema = unquote(schema)
      expected_schemas_and_types = 
        unquote(expected_schemas_and_types)

      actual_fields_with_types =
        for field <- schema.__schema__(:fields) do
          type = field_type(schema, field)

          {field, type}
        end

      assert Enum.sort(actual_fields_with_types) ==
               Enum.sort(expected_schemas_and_types)
    end
  end
end

def field_type(module, field) do
  case module.__schema__(:type, field) do
    {
      :parameterized, 
      Ecto.Embedded, 
      %Ecto.Embedded{related: embedded_type}
    } ->
      {:embedded_schema, embedded_type}

    {:parameterized, Ecto.Enum, enum_data} ->
      {Ecto.Enum, Keyword.keys(enum_data.mappings)}

    anything_else ->
      anything_else
  end
end

This macro takes a module that contains an Ecto.Schema, and a list of {key, type} tuples, and compares them using Ecto’s built in reflection.

For example with a Schema like this:

defmodule Bread do
  use Ecto.Schema

  @primary_key {
    :id, 
    :binary_part, 
    autogenerate: true
  }
  schema "bread" do
    field :name, :string
    field :kind, 
      Ecto.Enum, 
      values: [:yeast, :sweet, :flat]
    field :gluten_free, :boolean
  end
end

I can validate the fields and types like this:

@expected_fields_and_types [
  {:id, :binary_id},
  {:name, :string},
  {:kind, {Ecto.Enum, [:yeast, :sweet, :flat]}},
  {:gluten_content, {
    Ecto.Enum, 
    [:contains_gluten, :gluten_free]
    }
  }
  ]

validate_schema_fields_and_types(
  Bread, 
  @expected_fields_and_types
)

Here, I should get an error on the mismatch between :gluten_free and :gluten_content, my schema hasn’t been updated from boolean to enum yet.

Embedded Schema as Parser

Since Ecto schemas are separate from SQL, they can be used in places where SQL doesn’t make sense. For example, an embedded_schema can be used as a low-power parser.Using Ecto version 3.11.1 documentation here.While Ecto can parse dates, it can only do iso8601 formatted strings. Anything outside that format will come back as “is invalid” with no helpful pointers.

defmodule MyParser do
  use Ecto.Schema

  @primary_key false
  embedded_schema do
    field :arrival_date, :date
    field :thread_count, :integer
    field :screw_type, Ecto.Enum, 
      values: [:wood, :drywall, :machine]
  end

  def all_fields do
    __MODULE__.__schema__(:fields)
  end
end

The schema enables two things:

  1. I can define the field names and types.
  2. I can use Ecto’s built in reflection to avoid duplication.
import Ecto.Changeset

def parse(parameter_map) do
  %MyParser{}
  |> cast(parameter_map, MyParser.all_fields())
  |> validate_number(:thread_count, greater_than: 0)
  |> apply_action(:insert)
end

The schema can then be used with the Changeset API to parse and validate an input map. The input map needs to have homogenous keys, either all atoms or all strings. Mixed keys are not allowed, as specified in the cast documentation.

Big Integer field types in Postgres with Ecto

As I wrote in a sidenote below, Ecto seems to pass types set in migrations through as the stringified version of the atom. So setting a bigint field in Ecto simply involves :bigint:Using Ecto version 3.11.1 documentation here and Ecto SQL 3.11.1 documentation here.So far as I can tell, this isn’t documented. I found a few references to :bigint in the source code of the Postgres adapter in Ecto.Adapters.Postgres, but nothing about how to set it.

create table(:beaches) do
  add :grain_count, :bigint
end

In Ecto schemas, Elixir has arbitrarily sized integers so a bigint field uses :integer:

defmodule Beaches do
  use Ecto.Schema

  schema "beaches" do
    field :grain_count, :integer
  end
end

UUID Keys in Ecto

In order to set UUID fields in Ecto, they must be configured in both the migrations for the tables and in the Schemas that live in Elixir modules.Using Ecto version 3.11.1 documentation here and Ecto SQL 3.11.1 documentation here.

These can be configured both manually on an ad-hoc basis, and globally for an entire project.

Manually setting UUID keys

Migrations

If you’re using pure SQL to do your migrations, no change is needed, you can use the UUID SQL type in your migration.

If you’re using the Ecto migration helpers then table accepts a primary: false option:

create table(:pastas, primary: false) do
  add :id, :uuid, primary: true
  add :name, :string, null: false
  add :spiraliness, :integer, null: false

  timestamps()
end

Postgres supports a :uuid type, but if you want to generate the UUIDs from Elixir, use :binary_id.As far as I can tell, Postgres will understand any atom that directly corresponds to the name of a Postgres type. So the :uuid here isn’t an Ecto thing, so much as a Postgres thing. The atom is getting transformed into a string that Postgres understands.

Schemas

The type of a schemas Primary key can be set with the @primary_key module attribute, the foreign key type can be set with the @foreign_key_type module attribute:

defmodule MySchema do
  use Ecto.Schema

  @primary_key {:id, :binary_id, autogenerate: true}
  @foreign_key_type :binary_id
  schema "my_table" do
    ...
  end
end

The @primary_key attribute takes the name of the id field, the type of that field, and whether to autogenerate the ids on the Elixir side. The type of a id field in Ecto can only be :id or :binary_id as noted here. Where :id is any integer type, and :binary_id supports any binary format such as created by Ecto.UUID.

The @foreign_key_type follows the same typing, and needs to be set otherwise Ecto will assume any references field will have an integer key type.

Globally setting UUID keys

Migrations

For migrations, you can configure the :migration_primary_key and :migration_foreign_key. In config.exsThese could be under the same config :app setting, but I’ve separated them here for visual clarity.Note that :app and App are your application name and should be replaced.:

config :app, App.Repo, 
  migration_primary_key: [type: :uuid]

config :app, App.Repo, 
  migration_foreign_key: [type: :uuid]

Schemas

It’s bit tedious to set the module attributes for every single schema in a project. The documentation for Ecto suggests creating a Schema module that automatically sets the appropriate module attributes:

# A wrapper module that 
# sets the module attributes that we want.
defmodule App.Schema do
  defmacro __using__(_) do
    quote do
      use Ecto.Schema
      @primary_key {
        :id, 
        :binary_id, 
        autogenerate: true
      }
      @foreign_key_type :binary_id
    end
  end
end

# Here I use App.Schema instead of Ecto.Schema
defmodule App.MySchema do
  use App.Schema
  ...
end

Double nil from Ecto.Repo.one/2

Repo.one/2 has this API:Repo.get/3 has a similar API that returns nil on no results.

one(
  queryable :: Ecto.Queryable.t(),
  opts :: Keyword.t()
) :: Ecto.Schema.t() | term() | nil

Specifically, it returns nil to indicate no result.

However, if you write a queryable that selects a nullable field, that field can also return nil with no way to distinguish between the two cases.

query = from p in Post,
  join: c in assoc(p, :comments), 
  where: p.id == ^post_id, 
  select: p.nullable_field
Repo.one(query)
# `nil` but is it the value or a lack of one?

An alternate API could look like this:Using Repo.one/2 might be a code-smell that might lead to N+1 queries, but in the cases where it does make sense, I would like an unambiguous API.

def try_one(queryable, opts) do
  case Repo.all(queryable) do
    [one] -> {:ok, one}
    [] -> {:error, :ecto_no_results}
    other -> {:error, :ecto_multiple_results}
  end
end

This gives us a result type (non-raising) alternative to Repo.one!/2:This is the snippet from Repo.Queryable.one!/3 implemented here.

def one!(name, queryable, tuplet) do
  case all(name, queryable, tuplet) do
    [one] -> one
    [] -> raise Ecto.NoResultsError, queryable: queryable
    other -> raise Ecto.MultipleResultsError, queryable: queryable, count: length(other)
  end
end