MAKE working with Ipython notebooks more convenient

I really enjoy using the ipython notebook for coding. It gives me the possibilty to have code, plots and documentation in the same place all wrapped up in a nice format.

annoying

There are only two things that started to annoy me:

1. Until now it is not possible to use pythons from notebook1 import function syntax to split and reuse notebooks
2. I often find myself wanting to take my notebooks to meetings but there is no convenient way to turn all notebooks into html files which are easily shared and viewed on every computer.

MAKE to the rescue

Make is a command line tool for automatization. By storing rules in a file called makefile in our notebook directory we can run a sequence of commands we need often to save us all the typing.

make is lazy (= fast?) by design as it will not run commands if their input is unchanged.

There are plenty of tutorials out there and the documentation of make is very good, so I won't go into the workings of make.

Lets see how we can use it for out notebooks:

Importing from other notebooks

Since we can't import from notebook.ipynb we can't reuse its code. One work aroud is to convert the notebook from .ipynb to a regular .py script which can then be imported regularly.

# find all notebooks in the directory
# and store their names in a variable
notebooks = $(wildcard *.ipynb)

# we wan't to have '.py's for all our notebooks
scripts : $(notebooks:.ipynb=.py)

# and this is how we are going to get them
%.py : %.ipynb
    ipython nbconvert --to python $<

Converting to html

Similarly to the example above we can convert our notebookt to .html by adding the following targets and rules to our makefile.

docs : $(notebooks:.ipynb=.html)
%.html : %.ipynb
    ipython nbconvert --to html $<

Wrapping it all up

# file: makefile
notebooks = $(wildcard *.ipynb)

# all in one target
all : docs scripts

scripts : $(notebooks:.ipynb=.py)
%.py : %.ipynb
    ipython nbconvert --to python $<

docs : $(notebooks:.ipynb=.html)
%.html : %.ipynb
    ipython nbconvert --to html $<

# easily removing all the clutter we generated
clean :
    -rm $(notebooks:.ipynb=.py) $(notebooks:.ipynb=.html)

We can now invoce make in the command-line by simply typing:

make

It will generate all scripts and docs (run make scripts or make docs if you don't need both) for all our notebooks.

To remove all the files we generated we can run make clean

Importing and documentation is now only a single make command away

More automatization?

To automatically re-run make whenever a file has changed check out this stackoverflow conversation.