Articles

Persisting your entities in Erlang

Tweet

An Erlang Persistence layer for Erlang

In this article, I'll describe epers, which is a small project I've been working on. It's currently a "small" project, but quite ambitious, and the idea is to try some "new" ideas and concepts in the erlang we do every day.

NOTE: epers has now grown, and is now called sumo_db, and can now be found at the Inaka github repository, so if you're interested, you can still read the article, and work using the sumo_db repo instead of the epers one.

epers stands for "erlang persistence". As the name suggests, it tries to make it easy to use databases in erlang programs, to make it a little more agile, and (humbly) offer a nice adapter for several databases, hiding their implementation details (and the api of the library/framework/driver used to communicate with them).

To achieve this, it aims to offer a somewhat consistant api to define and work with your model, while at the same, not coupling your code too tightly to it.

Getting it

You can find epers at GitHub.

Current status

Epers is a young project, and it's open to, (and looking for) more contributors and users :)

Currently, only the MongoDB and MySQL repositories are implemented, and SQLite, Redis are in progress.

Planned to have are: Mnesia.

Full example

For the impatient, there's a complete example (a blog website) here.

General overview: Decoupling your business logic from your persistence layer in your Erlang Applications

The "programming model" that epers offers, forces you to write code that completely decouples state from the code that handles behavior, and the code that accesses a database. And maybe even make your code portable across different database engines. This has a few advantages, to name a few:

• Your code will be less coupled to the database in use.
• You might be able to easily change and combine different databases in your project without changing your code much.
• Your code will be more testable.
• It will be easier to modify your code for new features.
• Some domain events are natively supported, allowing your app to react to them. This is actually covered in this article.

So let's start by defining some concepts used in epers.

A preview: How to use the persistence framework

Your main entry point to epers is the epers module. So if you want to skip some reading and go straight to the code, remember that you should have everything needed in this module, and should not need to call any other modules by yourself.

From the epers module, you can call:

• create_schema/0: Creates the schema needed for your entities in the databases you have configured (more details below).
• find/2: Fetches an entity by id.
• find_by/2: Fetches entities by certain conditions.
• find_by/4: As find_by/2, but accepts limit and offset for pagination.
• find_one/2: Finds the first entity that matches the given criteria.
• persist/2: Persists an entity, creating it or updating it. Returns the new entity state (with an updated id if it was generated).
• delete/2: Deletes an entity of the given type.
• delete_all/1: Deletes all entities of the given type.

Basic usage of your erlang repositories

Once you have your entities and your repositories properly configured (see below), you can use them right away. Let's say we want to create a new user:

Let's delete it:

Getting by id:

Getting by age:

Same as above but paginated:

Finding one by email:

Concepts

Let's go over a few key concepts of epers.

Entities: The key to decouple your repositories from your business logic

epers works with "entities". An entity is, conceptually speaking, "something" in your software that needs to be persisted, has its own business rules, has state, and has behavior. In plain epers-ish, this means erlang modules that implements the epers_doc behavior.

Example

Users in a website, can be represented and implemented into the erlang module mywebsite_user:

Don't worry about the epers_ functions, we'll see more about them later.

State

The state in your entities is the actual data that needs to be persisted. Since you completely manage the details for it, it can be anything you want: a record, a proplist, or anything else. We'll see the details in a moment. For now, let's say it's a proplist:

Example

Behavior

The behavior of your entites is represented by code. With epers, the functions you write in the modules (that represent your entities), are the behavior. The behavior will usually need the current state to actually work, and it will receive the state so it can act upon it.

Usually, if the behavior has a side effect (like changing the state of the entity), it will return a new "state" that you can save for later use (this is actually because of the referential transparency feature in erlang, and of course, that there are no objects in erlang that can encapsulate both behavior and state at the same time).

Example

Let's say you have a story that states something like:

As a user, I want to change my email address, so I can...

You would write (in the mywebsite_user module):

The set/3 function can be something like:

Business Rules for your business logic in your Erlang application

The business rules are what your entities can or can't do. The behavior is shaped by the business rules, and they should be enforced by throwing errors or reporting them somehow. You will usually have these as "requirements", "specs", "stories", etc.

Example

If A user can't change his/her email if age > 40, you could rewrite change_email/2 like this:

The get/2 function can be something like:

The epers_doc behavior

Note that, in the example code shown so far, there's almost nothing that couples it to epers. So, if you regret using epers at some particular point of your development, it would not be *that* hard to do it :)

Anyway, the epers_doc behavior states that you have to implement the following functions, (which are basically adapters, translators, or bridges, needed to translate from and to your state representation into #epers_doc{}:

epers_schema/0

Is called when epers:create_schema/0 is called. This should return the schema for your entity. Example:

Defining a database schema for your entities

To create an #epers_schema{}, call epers:new_schema/2. The 1st argument is the name of the entity (will directly translate into tables, collections, etc), and the 2nd argument is a list of #epers_field{}, which are created by calling epers:new_field/2 and epers:new_field/3. The 1st argument is the name of the field (automatically translated into columns, fields, etc), and the 2nd and 3rd arguments are the type and extra attributes for the field.

Different database implementations might support different types and attributes, the default types for epers are:

• integer
• string
• text
• binary

And the default attributes are:

• auto_increment: mysql supports this, but other drivers might not.
• not_null
• {length, Length}
• id: This is required in 1 of the fields of your entities. It will tell epers that this particular field can be used to identify your entities.
• unique
• index

This level of abstraction is what kind of makes possible to isolate your code from the database details (except for when you use custom repos or when using types or attributes that are only available to one of the database engines and not all).

epers_wakeup/1

Called when epers fetches an entity from the database and needs your code to generate the proper entity state from that raw data. You get a proplist and you must return your own representation for the entity state (a record, a proplist, etc).

epers_sleep/1

Is the reverse operation of epers_wakeup/1. It will be called when epers is about to persist your entity. You receive your own representation of a state, and you must return a proplist with the fields and records that you want to persist.

Repositories

Repositores in epers are in charge of dealing with the details of how entities are persisted or fetched. Epers come with a default implementation of a repo that lets you persist (insert/update) and do basic queries.

Repositories are configured somewhere in your application configuration, and then associated to entities like this:

This is needed so epers knows which repo to call when persisting or querying entities. This also allows you to change repo implementations easily, and hopefully, without needing to change code.

Example

Let's say we want to setup a mysql repository (with the default implementation) for our ebsite_user entity:

It's possible that if you are using the default implementation of a repo (any of the modules epers_repo_* modules) you can change implementations without changing your code, and only reconfiguring your application. Let's see how would the configuration look like to use mongodb:

Which is pretty much the same as mysql :)

Default repositories for specific or complex queries

The default repositories support the following api:

• create_schema/2: Creates the schema for the given entity type.
• persist/2: Persist (create or update) the given entity.
• find_by/3: Finds all entities with a given criteria.
• find_by/5: Finds all entities that match a given criteria, paginated.
• delete/3: Delete entities of a given type and that match the given criteria.
• delete_all/2: Delete all entities of a type in this repo.
• call/4: If you have a custom repo implementation, this is the way to call a custom function.

Custom repositories for all your database queries

Sooner or later you will need to do some more complex queries to your database, and the default repo implementations wont help you much. For these special situations, you can write your own repo implementation.

Since epers does not offer a query language of its own, you wont be able to use any abstraction over the database. Thus, you will lock that repo features to one of the already existant drivers.

Let's say that you want to extend the default repo implementation for mywebsite_user, to add a count, and you choose to use mysql for this. You can write the following mywebsite_user_repo module:

And you can access it by calling:

Epers will locate the repo implementation for the entity mywebsite_user and call the total_users function.

Because of the extends directive, your repo will contain all the default repo features, like persist, find, etc.

Passing arguments

You can all:

This will invoke the function crazy_query(Arg1, Arg2, Arg3) on your custom repo.

Built-in domain events for your business logic

Since this article is a little too long already, this subject is covered in depth here.

What's next?

Lots of improvements are possible, like an own Query Language, that can be used to code queries portable across different database engines, more repositories, etc.

Now give it a try, and stay tuned for the future versions :) All feedback is welcome!