It’s already over a month I’ve been working on my Google Summer of Code project - providing Ecto with NoSQL adapters. Before my work began we decided together with José Valim - my mentor and creator of the Elixir programming language - to start with MongoDB. There were couple of reasons, among them the fact that it’s one of the most popular NoSQL databases, and it seemed that even though it’s not a SQL database the semantics are not that distant.

In this post I would like to report my journey up to today, and share my experience.

All the code is available on Github in the project’s repo: mongodb_ecto.

What the hell is this Ecto thingy?

Ecto is a “database wrapper and language integrated query” for Elixir.

“Database wrapper” means that Ecto provides unified API for interacting with different databases - as of now Ecto supports 4 databases - PostgreSQL, MySQL, MSSQL and SQLite, the support for MongoDB is coming soon. Ecto also provides things that might be familiar to ORM users in object-oriented languages, despite Elixir being an immutable, functional and declarative language - Ecto is able to provide models, callbacks, extensive type-casting, and migrations. It is used as the basis for the Phoenix web framework.

“Language integrated query” means it provides a very natural way of interaction with the database, you can write your queries in Elixir, and they will be translated to your database native dialect - be it SQL, object notation or something else. It may be familiar to users of C#, as it was inspired by the LINQ extantion, but thanks to Elixir supporting true macros no language changes were required to make it work. In result you can write your queries like:

{% highlight elixir %} from c in Comment, join: p in Post, on: c.post_id == p.id, where: p.published, limit: 10, select: {p.title, c.text} {% endhighlight %}

My work so far

I was lucky enough to be qualified to this year’s Google Summer of Code project to work on extending Ecto to NoSQL databases. I was posting weekly reports to Ecto and Beam Community (my host organization) mailing lists about the progress and future plans, you can look in the archives for them, if you’re interested. But 5 of 13 weeks has already passed and I feel like a bigger update is in order.

Current state of MongoDB adapter

It’s already couple of weeks that MongoDB adapter passes all tests from Ecto’s integration suite. All the basic functionality is there - you can insert your documents to the database, query, update and remove them. Thanks to some changes to Ecto itself there is support for map field type that enables harnessing the power of Mongo’s dynamic schemaless features. It’s possible to query documents using inline JavaScript functions and regular expressions. For the operators that are not directly supported you can use expressive fragments using Elixir’s keyword syntax. Support for logging is the most recent addition.

I am working on providing support for embedded models, querying on nested maps (and embedded models), providing easy way to use aggregate pipelines and other things that may be specific to MongoDB.

Last couple of days I worked hard on improving the documentation, which is currently rather extensive. There is also an example application you can try out in a matter of minutes.

Thanks to the work on the adapter some changes to Ecto happened (mostly by the work of José) - better support for binary id types such as UUID or ObjectID, the abstract map type that benefits not only MongoDB, but is translated to jsonb column in PostgreSQL and regular JSON in text fields in other SQL databases. Recetnly Ecto gained new update syntax, that is more expressive, and helped to change the only update_all macro callback in Repo back to a regular function.

I’d say that the adapter can be used for non-production projects. In fact having a real project that would test various features would be really appreciated. If you’re willing to try - don’t hesitate to contact me, I’ll be happy to help you.

Conclusion

I have to admit, I am really happy I got a chance to work with such an exceptional language with so many wonderful people. Elixir community is brilliant and extremely welcoming. The IRC channel #elixir-lang on freenode is never empty, and you can always count on help from competent people.

If you haven’t tried Elixir before - give it a try - it’s a great experience.

I have to thank my awesome mentor José Valim for his support and help on pursuing the project. I coudn’t have a better one. I also have to thank Eric Meadows-Jönsson, member of the Elixir core team, who was helping greatly with my project. Thank you!