Sunday, June 22, 2014

Why Not Erlang? The Lack of Onramps

Garret Smith gave a talk titled "Why the Cool Kids Don't Use Erlang" at the 2014 Erlang User Conference.  From free-form surveys, Garret identified several key factors limiting Erlang adoption versus more popular alternatives such as Go, Clojure, and Scala:

  • Lack of developers
  • Considered hard to learn
  • Concern over current level of adoption (risk, cost, reliance on scarce resources)
  • Convincing managers is hard
  • Dissatisfaction with libraries, documentation, and tooling -- often cited as barriers to adoption

In light of Garret's informative work, I wanted to reflect on my personal reasons for using languages other than the Erlang/Elixir stack.  To clarify, I love Erlang.  I think the programming model is brilliant.  Implementing objects as actors that run concurrently makes a lot of sense to me.   Due to the wonderful syntax and implementation, the overhead for using the actors is very low -- little more complicated than calling a method in an object-oriented language.

Erlang's approach to structuring code offers a nice balance between functional and object-oriented programming paradigms. The use of actors provides instances which encapsulate their own state and local namespaces like objects.  However, the combination of dynamic typing and code organized into functions and modules makes it easy to reuse code in a manner similar to mixins.  Erlang completely avoids the class hierarchy of inheritance-baseed object-oriented languages and separates the concept of identity and code organization.  As a result, Erlang completely avoids the brittle and complex design patterns required by objected-oriented languages, while avoiding the complexities and frustrations of purer functional languages.

So why do I rarely ever use the Erlang/Elixir ecosystem? Simply put, the lack of onramps.

Ruby became popular because of Rails.  Node.js used web developers' pre-existing comfort with Javascript as an onramp.  Python is easy to get started with, scales, and has a huge ecosystems of specialized libararies, especially due to its ability to easily integrate with C.  Scala and Clojure are attractive to users already using Java and the JVM and can tap into the wide range of existing libraries.  And lastly, Go is backed by Google, providing great advertising and immediate exposure to a ton of internal developers who can battle test and evangelize the language.

Building onramps is key to language adoption.  Elixir/Erlang, D, Ceylon, and Rust have all struggled with this, resulting in slower adoption than their peers.

My work for the last few years has spanned a few different areas:

  • Modeling & Simulation -- development and implementation of mathematical models for studying the physics of molecules
  • Data Science -- analysis of large data sets in physics, chemistry, and bioinformatics
  • Data storage and processing systems for scientific data
  • RPG game

By and large, most of my work has been in the form of contributions to existing projects which don't use Erlang/Elxir.  Since changing languages for an existing project isn't an option (or even a smart choice in most cases), none of these projects could serve as onramps.

What about for new projects, from scratch?  All of my new projects have been data science projects.  Converting data from one file format to another.  Combining datasets.  Visualization of datasets.  Computing statistics.  None of these are strong areas for Erlang/Elixir, but they are for Python.

Python makes it really easy to start with small scripts or an interactive session (especially with iPython Notebook) since there is no project setup required.  Python also scales to large applications -- as a result, I don't have to worry about changing languages down the road since my project won't outgrow Python.  As a strong, general-purpose language with a large, high-quality ecosystem, Python is suitable for almost any task or project, no matter how specialized.  As a result, I rarely ever have a need to leave the Python ecosystem.

Erlang/Elixir projects require a bit more overhead to setup.  The Erlang/Elixir language is not ideal for projects requiring lots of data manipulation, math, graphics, or low-latency performance (games).  Erlang/Elixir certainly doesn't have the necessary breadth of libraries to support general-purpose tasks. They are good at one thing and one thing only: distributed, fault-tolerant systems.

I'm starting to become interested in Scala.  Why? My new job at Red Hat involves working with and hacking on software built in Java and Scala -- existing projects are serving as onramps.

I'm also curious about Julia.  Why?  Julia has the iJulia Notebook similar to iPython Notebook, making it really easy to get started with.  Like Python, Julia also has a large set of libraries for data analysis and visualization, made possible through easy integration with C and Fortran, which makes it well-suited to many of the projects I work on.  The appropriateness for my work provides an onramp.

In my mind, if Erlang and Elixir want to grow, the community needs to identify ways to expand the scope of Erlang and Elixir to other potential use cases so that there are a larger number of natural onramps.