Problem: Emails and usernames usually need to be unique and case insensitive. Typically, this gets handled one of two ways: lowercasing the value when saving, or less frequently, creating a unique index for email/username that uses the lowercase version, like LOWER(username)

Both of these have issues though. Lowering the values loses data and can be annoying later on, and also puts responsibility for data integrity on the developer. Using a LOWER() index doesn’t lose data, but it requires lowering values when querying on that field.

Solution: Postgres has a case insensitive text extension, called citext (big surprise). Citext works exactly like a text column, except it keeps the submitted value and indexes it how we want. This means we never have to worry about case sensitivity in our application code.

Support: Phoenix (Ecto) and Rails (ActiveRecord) both support citext, each with only a tiny tweak necessary. Additionally, because Postgres doesn’t enable the Citext extension by default, we need to enable it via a migration.

Example: Using fresh Phoenix and Rails apps, let’s add a users table that has a single column, email. Using the built in generators will help show how minimal the tweaking required is.

# Phoenix
mix phx.gen.html Accounts User users email:string
# Phoenix doesn't accept :citext as a valid type for the generator, so we will use :string here. 

# Rails
bundle exec rails generate model User email:citext

After running the generators, here’s the tweaks needed for both Phoenix and Rails:

# priv/repo/migrations/20190416235625_create_users.exs

  use Ecto.Migration

  def change do
+   execute "CREATE EXTENSION IF NOT EXISTS citext",  "DROP EXTENSION IF EXISTS citext"
    create table(:users) do
-     add :email, :string
+     add :email, :citext

      timestamps()
    end
+   create index(:users, [:email], unique: true)

  end

# db/migrate/20190416235153_create_users.rb

  def change
+   enable_extension(:citext)
    create_table :users do |t|
      t.citext :email
+     t.index :email, unique: true

      t.timestamps
    end

Now it’s ready to go! Let’s run the migrations and add some tests to verify it works.

# Phoenix
mix ecto.migrate

# Rails
bundle exec rails db.migrate

Couple quick tests to double check.

# test/my_app/accounts/accounts_test.exs

defmodule MyApp.AccountsTest do
  # ...
  describe "users" do
    # ...
    test "can't duplicate case insensitive emails" do
      email = "ASDF@example.com"
      user = Accounts.create_user(%{email: email})
      assert {:error, %Ecto.Changeset{errors: [email: {"has already been taken", _}]} } = Accounts.create_user(%{email: String.downcase(email)})
    end
    # ...
end

# test/models/user_test.rb

class UserTest < ActiveSupport::TestCase
  setup do
    @email = "ASDF@example.com"
    @user = User.create(email: @email)
  end
  test "looks up case insensitive email" do
    assert_equal @user, User.find_by(email: @email.downcase)
  end
  test "can't duplicate case insensitive emails" do
    # DB constraint will raise an error
    assert_raise(ActiveRecord::RecordNotUnique) { User.create(email: @email.downcase) }

    # validates_uniqueness_of will return a user object which is not persisted to the DB, and has an error on it
    assert_equal ["has already been taken"], User.create(email: @email.downcase).errors.messages[:email]
  end
end

Note about the Rails tests: When using validates_uniqueness_of, ActiveSupport will return an unpersisted model object, which includes the validation errors. Therefore, we use one assertion when relying on Postgres and a different assertion when using the validation helper. Because there are varying opinions on using validates_uniqueness_of, I’ve included both examples.

Summary: Using citext makes things easier for us. We don’t have to worry about character case or data integrity, it’s just automatically handled for us. Actual code for the Phoenix example is available on the citext branch of my phoenix_examples repo on Github.

Plug.Test.init_test_session(conn, current_user_id: 1)

That’s it!

There are a couple ways to utilize this. Let’s take an autogenerated controller test:

describe "index" do
  test "lists all users", %{conn: conn} do
    conn = get conn, admin_user_path(conn, :index)
    assert html_response(conn, 200) =~ "Listing Users"
  end
end

To set session just for this test, we just have to add one line:

describe "index" do
  test "lists all users", %{conn: conn} do
    # this could be streamlined using pipes |>
    conn = Plug.Test.init_test_session(conn, current_user_id: 1)
    conn = get conn, admin_user_path(conn, :index)
    assert html_response(conn, 200) =~ "Listing Users"
  end
end

If we wanted to set a session value for an entire set of controller tests, that’s simple too. We just need to add one block in the test:

defmodule MyAppWeb.UserControllerTest do
  use MyAppWeb.ConnCase
  alias MyApp.User

  # ... add the following block:

  setup %{conn: conn} do
    conn = conn
      |> Plug.Test.init_test_session(current_user_id: 1)
    {:ok, conn: conn}
  end
  # ... rest of tests will now have that session value
end

Now, the conn struct used for all the tests will have current_user_id: 1 set in session, as expected. This is a huge improvement, since it removes the complexity and ambiguity of previous ways to accomplish testing session data.

Older approaches

There are quite a few older approaches, most of which are more complex approaches which may not work in all situations. Typically, they require either customizing the conn used in tests or by using tags paired with custom connection setup. Sometimes, this includes bypassing pipelines. There are other ways which include tweaking your app’s pipelines(1, 2) and setting current_user directly.

To be clear, all of these approaches work, and have been necessary in the past. But luckily, Plug 1.5 added made this all much simpler.

Requirement: I need to create a schema that includes a PostGIS Point columm.

Problem: Phoenix generators are great, but they don’t support special types, such as PostGIS columns.

The docs for geo_postgis are sufficient for figuring out how to work with geographic data types, but they don’t cover how to make it work with Phoenix generators, changesets, or tests.

Goal: Use Phoenix generators (since contexts are too useful to ignore), and have the generated changesets, views, and tests work with only minor tweaks.

How: Use virtual attributes in the Ecto schema to be the bridge to the custom type.

1. First step is to plan out what type of data you will have, and what primitives it is composed of.

   In this case, a Point has 2 floats, representing latitude and longitude. Most interaction will be with the float values instead of the Geo.Point struct. These 2 floats will become virtual attributes to our schema struct.

2. Use the virtual attributes in our generator.

   For a Admin context and a locations table, our generator command will look like this:

   mix phx.gen.html Admin Location locations lng:float lat:float # ... other columns
   

   There are still a few steps before this works and tests are passing. The migration, schema, and tests need updating.

3. Fix the migration to create the proper column (coordinates).

   defmodule MyApp.Repo.Migrations.CreateLocations do
     use Ecto.Migration
   
     def change do
       create table(:locations) do
         # remove the lines for lat/lng. They are commented here to show which lines
         # add :lat, :float
         # add :lng, :float
       end
   
   # add the next line, which creates a column `coordinates` on table `locations`
       execute("SELECT AddGeometryColumn ('locations','coordinates',4326,'POINT',2);")
   # add an index if necessary
       create index(:locations, [:coordinates], using: :gist)
     end
   end
   

4. Update the generated schema:

   Find lib/my_app/admin/locations.ex to make the lat/lng columns virtual, add the coordinates column, and cast lat/lng to coordinates

   defmodule MyApp.Admin.Location do
     use Ecto.Schema
     import Ecto.Changeset
     alias MyApp.Admin.Location
   
     schema "locations" do
       field :coordinates, Geo.Point     # add the actual column
       field :lng, :float, virtual: true # add the virtual flag here and below
       field :lat, :float, virtual: true
     end
   
     @doc false
     def changeset(%Location{} = location, attrs) do
       location
       |> cast(attrs, [:lng, :lat])
       |> validate_required([:lng, :lat])
       |> cast_coordinates()        # remember to cast the coordinates!
     end
   
     # something like this to cast the virtual columns to actual column
     # it would be good to add validation but I've left it out for brevity
     def cast_coordinates(changeset) do
       lat = get_change(changeset, :lat)
       lng = get_change(changeset, :lng)
       geo = %Geo.Point{coordinates: {lng, lat}, srid: 4326}
       changeset |> put_change(:coordinates, geo)
     end
   end
   

5. Update the generated tests:

   Find test/my_app/admin/admin_test.exs and test/my_app/controllers/location_controller_test.exs. These tests must be updated to check the coordinates attribute instead of lat/lng attributes.

   # test/my_app/controllers/location_controller_test.exs
      
   defmodule MyAppWeb.LocationControllerTest do
     use MyAppWeb.ConnCase
      
     alias MyApp.Admin
      
     @create_attrs %{lat: 60.5, lng: 70.5} # ensure that valid values are used
     @update_attrs %{lat: 45.7, lng: 56.7} # ensure that valid values are used
     @invalid_attrs %{lat: 200, lng: 200}  # use invalid values here
      
     # ...
   end
   

   # test/my_app/admin/admin_test.exs
      
   defmodule MyApp.AdminTest do
     use MyApp.DataCase
      
     alias MyApp.Admin
      
     describe "locations" do
       alias MyApp.Admin.Location
      
       @create_attrs %{lat: 60.5, lng: 70.5} # ensure that valid values are used
       @update_attrs %{lat: 45.7, lng: 56.7} # ensure that valid values are used
       @invalid_attrs %{lat: 200, lng: 200}  # use invalid values here
      
     # any test checking equality will need to reset the lat/lng attributes
     # generated test
       test "list_locations/0 returns all locations" do
         location = location_fixture()
         assert Admin.list_locations() == [location]
       end
   
     # updated test
       test "list_locations/0 returns all locations" do
         location = %{ location_fixture() | lat: nil, lng: nil}
         assert Admin.list_locations() == [location]
       end
      
     # for tests checking values, remove the assertions for lat/lng and add one for `coordinates`
     # generated test
       test "create_location/1 with valid data creates a location" do
         assert {:ok, %Location{} = location} = Admin.create_location(@valid_attrs)
         assert location.lat == 60.5
         assert location.lng == 70.5
       end
   
     # updated test
       test "create_location/1 with valid data creates a location" do
         assert {:ok, %Location{} = location} = Admin.create_location(@valid_attrs)
         assert location.coordinates == %Geo.Point{coordinates: {70.5, 60.5}, srid: 4326}
       end
     end
   

6. Run tests. Everything should be passing!

   Now go celebrate by adding some real functionality!

$ git status
...
    new file:   presentation.key/Data/st-ED415AB1-F3E3-40E7-AE3C-62C8F48D61F2-1769.jpg
    new file:   presentation.key/Data/st-FA2A3A03-7617-4164-9E65-1304161A58D5-2058.jpg
    new file:   presentation.key/Metadata/BuildVersionHistory.plist
    new file:   presentation.key/Metadata/DocumentIdentifier
    new file:   presentation.key/Metadata/Properties.plist
    new file:   presentation.key/preview-micro.jpg
    new file:   presentation.key/preview-web.jpg
    new file:   presentation.key/preview.jpg
...

I just want the files there as binary, so that anyone can download them if they like.

I tried a few different things, including a .gitattributes file with *.key binary, using different versions of git, and even different versions of OS X (el Capitan and Sierra).

In the end, the problem was from when I had saved the presentations. The hide file extension option in Keynote’s save dialog was the problem. De-selecting that option is all it took for it to work as I wanted (and as I think most people would expect).

A graceful method of upgrading Ruby is to update the Gemfile version of did_you_mean to 0.10.0 if it wasn’t already. That version was released 4 months before Ruby 2.3, so some apps may already be using it. Version 0.10.0 works with both 2.3 and previous versions.

Fix came from this Github issue

Let’s assume that you have a Ghost blog working locally already, and now you’re just trying to deploy to GH pages.

There’s a fork of Buster that fixes this problem. Install Buster from that fork (use the –update flag if you already have Buster installed):

pip install https://github.com/thinkamabob/buster/archive/master.zip --upgrade  
buster generate --domain=https://code.phoffer.com/  
Now you can host your blog on Github Pages. Congratulations!