Emacs Ipython Notebook

Posted on | Comments:

The Emacs IPython Notebook (EIN) package provides a Jupyter Notebook client and integrated REPL (like SLIME) in Emacs. EIN improves notebook editing by allowing you to use Emacs. It also expose IPython features such as code evaluation, object inspection and code completion. These features can be accessed anywhere in Emacs and improve Python code editing and reading in general in Emacs.

Highlighted features:

  • Copy/paste cells in and between notebooks.
  • Console integration: You can easily connect to a kernel via a console application. This enables you to start debugging in the same kernel. It is even possible to connect a console over ssh 1.
  • An IPython kernel can be “connected” to a buffer. This enables you to evaluate buffer/region using same kernel as notebook. Notebook goodies such as tooltip help, help browser and code completion are available in these buffers. 2
  • Jump to definition (go to the definition by executing M-. over an object).
  • Execute code from an org-mode source block in a running kernel.

Other notebook features:

  • Inline images
  • Auto/manual-completion
  • Popup (tooltip) help
  • Syntax highlighting in each cell type (Python/Markdown/ReST/HTML)
  • Help browser (opens when executing function?)
  • Traceback viewer
  • Integration with the emacs debugger

Links:

Quick try

The fastest way to get EIN running in this modern age is to download from MELPA or, if you are a spacemacs user, through installing the ipython-notebook layer. Using zeroein is no longer supported, though in theory it should still work.

If you are installing from MELPA and have issues with some functions not being available after emacs starts, try adding the following to your emacs init file:

1
2
3
4
(package-initialize)
(require 'ein)
(require 'ein-notebook)
(require 'ein-subpackages)

Requirements

  • EMACS 25.3, 26.x, or 27

  • Jupyter Notebook 4.x or higher

  • IPython 5.8 or higher.

  • Tornado 4.0.2 or higher.

  • websocket.el >= 1.7

  • request.el >= 0.3

  • request-deferred.el >= 0.2

  • dash >= 2.13

  • s >= 1.11

  • auto-complete.el >= 1.4: You need to configure subpackage ein-ac to enable this feature.

  • skewer-mode >= 1.6.2: Skewer mode gives EIN the ability to execute dynamic javascript in the note book.

  • (optional) Jupyterhub_ 0.8 or higher: EIN supports logging in to Jupyterhub servers using PAM authentication, though this only works with v0.8, which currently is the development version of Jupyterhub.

  • (optional) markdown-mode

  • (optional) python-mode: It should work with either python.el or python-mode.el. python.el is required to use the ein:console-open command.

  • (optional) smartrep.el: This package enables you to omit typing prefix keys (e.g., C-c C-n C-n C-n ... instead of C-cC-n C-c C-n C-c C-n ...). You need to configure subpackage ein-smartrep to enable this feature.

  • (optional) jedi.el: Python auto-completion for emacs using jedi. In your emacs initialization file add

    (setq ein:completion-backend 'ein:use-ac-jedi-backend)

Also, EIN heavily relies on standard Emacs libraries including EWOC, EIEIO and json.el.

Install

Warning

As EIN relies on many packages and it will not work properly with outdated versions, installing it using el-get or MELPA is highly recommended.

Using el-get

If you use developmental version of el-get installation is simple. Emacs IPython Notebook is registered as package ein. See the el-get website for more information.

Note

If you get error “Cannot open load file: request” that means you have an older version of el-get. You can fix this problem by either (1) installing request.el manually, (2) using the latest recipe, or (3) updating el-get to its master.

You can get the latest recipe here:

See issue 98 for more information.

Using package.el (MELPA)

You can install EIN using package.el when the MELPA package repository is added to its setting. See MELPA website for more information.

Manual install

Put Emacs lisp ein*.el files and the Python file ein_remote_safe.py in a directory defined in your load-path.

You should byte compile EIN, especially when using MuMaMo, otherwise editing large notebook will be very slow. You can use the following command to compile EIN. If you don’t specify all the optional packages, there will be compiler warning but that is OK as long as you don’t use that optional package.

1
2
3
4
5
6
7
8
9
10
emacs -Q -batch -L .          \  # don't forget the dot!
    -L PATH/TO/websocket/     \
    -L PATH/TO/requests/      \
    -L PATH/TO/nxhtml/util/   \  # optional (for MuMaMo)
    -L PATH/TO/auto-complete/ \  # optional
    -L PATH/TO/popup/         \  # optional (for auto-complete)
    -L PATH/TO/fuzzy/         \  # optional (for auto-complete)
    -L PATH/TO/smartrep/      \  # optional
    -L PATH/TO/rst-mode/      \  # optional
    -f batch-byte-compile *.el

Setup

Here is the minimal configuration. See customization_ for more details.

1
2
3
(require 'ein)
(require 'ein-notebook)
(require 'ein-subpackages)

Usage

  1. Start the Jupyter notebook server from the terminal or call M-x ein:run or M-x ein:jupyter-server-start from emacs. Note starting the notebook server from emacs will automatically call ein:jupyter-server-login-and-open, making steps 2 and 3 below unnecessary!
  2. If you have token or password authentication enabled then you will need to call M-x ein:login and enter the appropriate password. Assuming authentication works the notebooklist buffer will automatically open.
  3. In the notebook list buffer, you can open notebooks by selecting the [Open] buttons. See the notebook section for operations and commands available in the notebook buffer.

Commands/Keybinds

Running a Jupyter Notebook Server from Emacs

Using the commands below you start a jupyter notebook session from within emacs (i.e. no need to drop to the terminal shell and call jupyter notebook). EIN will also try to determine the access url and token authentication for the running server and automatically log you in.

Note that the below work best with current (> v4.3.1) versions of jupyter.

  • function (ein:jupyter-server-startserver-cmd-path notebook-directory &optional no-login-p login-callback**port)

    Start SERVER-CMD_PATH with ‘–notebook-dir’ NOTEBOOK-DIRECTORY. Login after connection established unless NO-LOGIN-P is set. LOGIN-CALLBACK takes two arguments, the buffer created by ein:notebooklist-open–finish, and the url-or-port argument of ein:notebooklist-open.This command opens an asynchronous process running the jupyter notebook server and then tries to detect the url and password to generate automatic calls to ‘ein:notebooklist-login’ and ‘ein:notebooklist-open’.With C-u prefix arg, it will prompt the user for the path to the jupyter executable first. Else, it will try to use the value of ‘ein:last-jupyter-command*’ or the value of the customizable variable ‘ein:jupyter-default-server-command’.Then it prompts the user for the path of the root directory containing the notebooks the user wants to access.The buffer named by ‘ein:jupyter-server-buffer-name’ will contain the log of the running jupyter server.

  • function (ein:runserver-cmd-path notebook-directory &optional no-login-p login-callback port)

    Start SERVER-CMD_PATH with ‘–notebook-dir’ NOTEBOOK-DIRECTORY. Login after connection established unless NO-LOGIN-P is set. LOGIN-CALLBACK takes two arguments, the buffer created by ein:notebooklist-open–finish, and the url-or-port argument of ein:notebooklist-open.This command opens an asynchronous process running the jupyter notebook server and then tries to detect the url and password to generate automatic calls to ‘ein:notebooklist-login’ and ‘ein:notebooklist-open’.With C-u prefix arg, it will prompt the user for the path to the jupyter executable first. Else, it will try to use the value of ‘ein:last-jupyter-command*’ or the value of the customizable variable ‘ein:jupyter-default-server-command’.Then it prompts the user for the path of the root directory containing the notebooks the user wants to access.The buffer named by ‘ein:jupyter-server-buffer-name’ will contain the log of the running jupyter server.

  • function (ein:jupyter-server-stop&optional force log)

  • function (ein:jupyter-server-login-and-open&optional callback)

    Log in and open a notebooklist buffer for a running jupyter notebook server.Determine if there is a running jupyter server (started via a call to ‘ein:jupyter-server-start’) and then try to guess if token authentication is enabled. If a token is found use it to generate a call to ‘ein:notebooklist-login’ and once authenticated open the notebooklist buffer via a call to ‘ein:notebooklist-open’.

  • variable ein:jupyter-default-server-command

    The default command to start a jupyter notebook server.Changing this to jupyter-notebook requires customizing ein:jupyter-server-use-subcommand to nil.

  • variable ein:jupyter-server-use-subcommand

    Users of “jupyter-notebook” (as opposed to “jupyter notebook”) need to Omit.

  • variable ein:jupyter-default-notebook-directory

    If you are tired of always being queried for the location of the notebook directory, you can set it here for future calls toein:jupyter-server-start

  • variable ein:jupyter-server-args

    Add any additional command line options you wish to include with the call to the jupyter notebook.

  • variable ein:jupyter-server-buffer-name

    The name of the buffer for the jupyter notebook server session.

Notebook list

You can start notebook by M-x ein:notebooklist-open and enter the port or URL of the IPython notebook server.

  • function (ein:notebooklist-openurl-or-port callback)

    This is now an alias for ein:notebooklist-login

  • function (ein:notebooklist-new-notebookurl-or-port kernelspec &optional callback no-pop retry)

  • function ein:notebooklist-open-notebook-global

  • function (ein:notebooklist-loginurl-or-port callback &optional cookie-plist)

    Deal with security before main entry of ein:notebooklist-open.CALLBACK takes two arguments, the buffer created by ein:notebooklist-open–success and the url-or-port argument of ein:notebooklist-open.

  • function ein:junk-new

  • function (ein:notebooklist-enable-keepalive&optional url-or-port)

    Enable periodic calls to the notebook server to keep long running sessions from expiring. By long running we mean sessions to last days, or weeks. The frequency of the refresh (which is very similar to a call to ‘ein:notebooklist-open‘) is controlled by ‘ein:notebooklist-keepalive-refresh-time‘, and is measured in terms of hours. If ‘ein:enable-keepalive’ is non-nil this will automatically be called during calls to ‘ein:notebooklist-open‘.

  • function ein:notebooklist-disable-keepalive

    Disable the notebooklist keepalive calls to the jupyter notebook server.

Keymap for ein:notebooklist-mode.

  • <remap> <self-insert-command>` undefined`

  • -` negative-argument`

  • 0` digit-argument`

  • 1` digit-argument`

  • 2` digit-argument`

  • 3` digit-argument`

  • 4` digit-argument`

  • 5` digit-argument`

  • 6` digit-argument`

  • 7` digit-argument`

  • 8` digit-argument`

  • 9` digit-argument`

  • q` quit-window`

  • SPC` scroll-up-command`

  • S-SPC` scroll-down-command`

  • DEL` scroll-down-command`

  • ?` describe-mode`

  • h` describe-mode`

  • >` end-of-buffer`

  • <` beginning-of-buffer`

  • g` revert-buffer`

  • TAB` widget-forward`

  • C-M-i` widget-backward`

  • <S-tab>` widget-backward`

  • <backtab>` widget-backward`

  • C-c C-r` ein:notebooklist-reload`

Reload current Notebook list.

  • C-c C-f` ein:file-open`

  • C-c C-o` ein:notebook-open`

  • p` ein:notebooklist-prev-item`

  • n` ein:notebooklist-next-item`

Notebook

The following keybindings are available in notebook buffers. Modified notebooks are saved automatically with a frequency dependenant on the setting of ein:notebook-autosave-frequency. If ein:notebook-create-checkpoint-on-save is True than a checkpoint will also be generated in the Jupyter server every time the notebook is saved. A notebook can be returned to a previous checkpoint via ein:notebook-restore-to-checkpoint. Checkpoints can also be manually created via ein:notebook-create-checkpoint.

  • C-c i` ein:inspect-object`

  • C-c '` ein:edit-cell-contents`

  • C-c C-c` ein:worksheet-execute-cell`

  • C-c C-'` ein:worksheet-turn-on-autoexec`

  • C-c C-e` ein:worksheet-toggle-output`

  • C-c C-v` ein:worksheet-set-output-visibility-all`

  • C-c C-l` ein:worksheet-clear-output`

  • C-c C-S-l` ein:worksheet-clear-all-output`

  • C-c C-;` ein:shared-output-show-code-cell-at-point`

  • C-c C-k` ein:worksheet-kill-cell`

  • C-c M-w` ein:worksheet-copy-cell`

  • C-c M-{` ein:notebook-worksheet-move-prev`

  • C-c M-}` ein:notebook-worksheet-move-next`

  • C-c M-+` ein:notebook-worksheet-insert-prev`

  • C-c C-w` ein:worksheet-copy-cell`

  • C-c C-y` ein:worksheet-yank-cell`

  • C-c C-a` ein:worksheet-insert-cell-above`

    • C-c C-b` ein:worksheet-insert-cell-below`

  • C-c C-t` ein:worksheet-toggle-cell-type`

  • C-c S` ein:worksheet-toggle-slide-type`

  • C-c C-u` ein:worksheet-change-cell-type`

  • C-c C-s` ein:worksheet-split-cell-at-point`

  • C-c C-m` ein:worksheet-merge-cell`

  • C-c C-n` ein:worksheet-goto-next-input`

  • C-c C-p` ein:worksheet-goto-prev-input`

  • C-c <up>` ein:worksheet-move-cell-up`

  • C-c <down>` ein:worksheet-move-cell-down`

  • C-c C-h` ein:pytools-request-tooltip-or-help`

  • C-c C-i` ein:completer-complete`

  • C-c C-$` ein:tb-show`

  • C-c C-x C-l` ein:notebook-toggle-latex-fragment`

  • C-c C-x C-r` ein:notebook-restart-session-command`

  • C-c C-r` ein:notebook-reconnect-session-command`

  • C-c C-z` ein:notebook-kernel-interrupt-command`

  • C-c C-q` ein:notebook-kill-kernel-then-close-command`

  • C-c C-#` ein:notebook-close`

  • C-c C-f` ein:file-open`

  • C-c C-o` ein:notebook-open`

  • C-c C-.` ein:pytools-jump-to-source-command`

Jump to the source code of the object at point. When the prefix argument ‘‘C-u‘‘ is given, open the source code in the other window. You can explicitly specify the object by selecting it.

  • C-c C-,` ein:pytools-jump-back-command`

Go back to the point where ‘ein:pytools-jump-to-source-command’ is executed last time. When the prefix argument ‘‘C-u‘‘ is given, open the last point in the other window.

  • C-c C-/` ein:notebook-scratchsheet-open`

  • C-c !` ein:worksheet-rename-sheet`

  • C-c {` ein:notebook-worksheet-open-prev-or-last`

  • C-c }` ein:notebook-worksheet-open-next-or-first`

  • C-c +` ein:notebook-worksheet-insert-next`

  • C-c -` ein:notebook-worksheet-delete`

  • C-c 1` ein:notebook-worksheet-open-1th`

  • C-c 2` ein:notebook-worksheet-open-2th`

  • C-c 3` ein:notebook-worksheet-open-3th`

  • C-c 4` ein:notebook-worksheet-open-4th`

  • C-c 5` ein:notebook-worksheet-open-5th`

  • C-c 6` ein:notebook-worksheet-open-6th`

  • C-c 7` ein:notebook-worksheet-open-7th`

  • C-c 8` ein:notebook-worksheet-open-8th`

  • C-c 9` ein:notebook-worksheet-open-last`

  • M-RET` ein:worksheet-execute-cell-and-goto-next`

  • M-.` ein:pytools-jump-to-source-command`

Jump to the source code of the object at point. When the prefix argument ‘‘C-u‘‘ is given, open the source code in the other window. You can explicitly specify the object by selecting it.

  • M-,` ein:pytools-jump-back-command`

Go back to the point where ‘ein:pytools-jump-to-source-command’ is executed last time. When the prefix argument ‘‘C-u‘‘ is given, open the last point in the other window.

  • M-p` ein:worksheet-previous-input-history`

  • M-n` ein:worksheet-next-input-history`

  • <M-S-return>` ein:worksheet-execute-cell-and-insert-below`

  • <C-up>` ein:worksheet-goto-prev-input`

  • <C-down>` ein:worksheet-goto-next-input`

  • <M-up>` ein:worksheet-move-cell-up`

  • <M-down>` ein:worksheet-move-cell-down`

  • C-:` ein:shared-output-eval-string`

  • C-x C-s` ein:notebook-save-notebook-command`

  • C-x C-w` ein:notebook-rename-command`

  • function (ein:worksheet-execute-all-cellws)

    Execute all cells in the current worksheet buffer.

  • function (ein:worksheet-delete-cellws cell &optional focus)

    Delete a cell. (WARNING: no undo!) This command has no key binding because there is no way to undo deletion. Use kill to play on the safe side.If you really want use this command, you can do something like this (but be careful when using it!):(define-key ein:notebook-mode-map "\C-c\C-d" ’ein:worksheet-delete-cell)

  • function ein:junk-rename

  • function (ein:iexec-mode&optional arg)

    Instant cell execution minor mode. Code cell at point will be automatically executed after any change in its input area.

  • function (ein:notebook-create-checkpointnotebook)

    Create checkpoint for current notebook based on most recent save.

  • function (ein:notebook-restore-to-checkpointnotebook checkpoint)

    Restore notebook to previous checkpoint saved on the Jupyter server. Note that if there are multiple checkpoints the user will be prompted on which one to use.

  • function (ein:notebook-enable-autosavesnotebook)

    Enable automatic, periodic saving for notebook.

  • function (ein:notebook-disable-autosavesnotebook)

    Disable automatic, periodic saving for current notebook.

Polymode in the Notebook

EIN now provides proper multi-major mode support in notebook buffers using polymode_. To use simply set ein:polymodeto t and restart emacs.

  • variable ein:polymode

    When enabled ein will use polymode to provide multi-major mode support in a notebook buffer, otherwise ein’s custom and outdated multi-major mode support will be used. Emacs must be restarted after changing this setting!

Advanced Editing

Worksheet cells can be edited in a manner similar to source blocks in Org buffers. Use C-c ' to edit the contents of the current cell. You can execute the contents of the buffer and the results will be sent to the output of the cell being edited.

  • C-c '` ein:edit-cell-exit`

Close the EIN source edit buffer, saving contents back to the original notebook cell, unless being called via ‘ein:edit-cell-abort’.

  • C-c C-k` ein:edit-cell-abort`

Abort editing the current cell, contents will revert to previous value.

  • C-c C-c` ein:edit-cell-save-and-execute`

Save, then execute the countents of the EIN source edit buffer and place results (if any) in output of original notebook cell.

  • C-c C-x` ein:edit-cell-view-traceback`

Jump to traceback, if there is one, for current edit.

  • C-x C-s` ein:edit-cell-save`

Save contents of EIN source edit buffer back to original notebook cell.

  • function ein:edit-cell-contents

  • function ein:edit-cell-exit

    Close the EIN source edit buffer, saving contents back to the original notebook cell, unless being called via ‘ein:edit-cell-abort’.

  • function ein:edit-cell-abort

    Abort editing the current cell, contents will revert to previous value.

  • function ein:edit-cell-save

    Save contents of EIN source edit buffer back to original notebook cell.

  • function ein:edit-cell-save-and-execute

    Save, then execute the countents of the EIN source edit buffer and place results (if any) in output of original notebook cell.

Connected buffer

You can connect any buffer (though typically a buffer that contains a Python file) to an opened notebook and use the kernel of that notebook to execute code, inspect objects, auto-complete code, jump to the other source, etc. Once the buffer is connected to the notebook, minor mode ein:connect-mode is enabled and the following keybinds are available.

  • C-c C-c` ein:connect-run-or-eval-buffer`

Run buffer using the ‘‘%run‘‘ magic command or eval whole buffer if the prefix ‘‘C-u‘‘ is given. Variable ‘ein:connect-run-command’ sets the command to run. You can change the command and/or set the options. See also: ‘ein:connect-run-buffer’, ‘ein:connect-eval-buffer’.

  • C-c C-l` ein:connect-reload-buffer`

Reload buffer using the command set by ‘ein:connect-reload-command’.

  • C-c C-r` ein:connect-eval-region`

  • C-c C-h` ein:pytools-request-tooltip-or-help`

  • C-c C-i` ein:completer-complete`

  • C-c C-z` ein:connect-pop-to-notebook`

  • C-c C-a` ein:connect-toggle-autoexec`

Toggle auto-execution mode of the current connected buffer.When auto-execution mode is on, cells in connected notebook will be automatically executed whenever run, eval or reload command 3 is called in this buffer.4Namely, one of‘ein:connect-run-buffer’‘ein:connect-eval-buffer’‘ein:connect-run-or-eval-buffer’‘ein:connect-reload-buffer’Note that you need to set cells to run in the connecting buffer or no cell will be executed. Use the ‘ein:worksheet-turn-on-autoexec’ command in notebook to change the cells to run.

  • C-c C-o` ein:console-open`

Open IPython console. To use this function, ‘ein:console-security-dir’ and ‘ein:console-args’ must be set properly. This function works best with the new python.el which is shipped with Emacs 24.2 or later. If you don’t have it, this function opens a “plain” command line interpreter (comint) buffer where you cannot use fancy stuff such as TAB completion. It should be possible to support python-mode.el. Patches are welcome!

  • C-c C-x` ein:tb-show`

  • C-c C-.` ein:pytools-jump-to-source-command`

Jump to the source code of the object at point. When the prefix argument ‘‘C-u‘‘ is given, open the source code in the other window. You can explicitly specify the object by selecting it.

  • C-c C-,` ein:pytools-jump-back-command`

Go back to the point where ‘ein:pytools-jump-to-source-command’ is executed last time. When the prefix argument ‘‘C-u‘‘ is given, open the last point in the other window.

  • C-c C-/` ein:notebook-scratchsheet-open`

  • C-:` ein:shared-output-eval-string`

  • M-.` ein:pytools-jump-to-source-command`

Jump to the source code of the object at point. When the prefix argument ‘‘C-u‘‘ is given, open the source code in the other window. You can explicitly specify the object by selecting it.

  • M-,` ein:pytools-jump-back-command`

Go back to the point where ‘ein:pytools-jump-to-source-command’ is executed last time. When the prefix argument ‘‘C-u‘‘ is given, open the last point in the other window.

Other useful commands:

  • function (ein:connect-to-notebook-command&optional not-yet-opened)

    Connect to notebook. When the prefix argument is given, you can choose any notebook on your server including the ones not yet opened. Otherwise, already chose from already opened notebooks.

  • function ein:connect-eval-buffer

    Evaluate the whole buffer. Note that this will run the code inside the ‘‘if name == “main”:‘‘ block.

  • function (ein:connect-run-buffer&optional ask-command)

    Run buffer using ‘‘%run‘‘. Ask for command if the prefix ‘‘C-u‘‘ is given. Variable ‘ein:connect-run-command’ sets the default command.

Shared output buffer

  • function ein:shared-output-pop-to-buffer

    Open shared output buffer.

The map for ein:shared-output-mode-map.

  • C-c C-x` ein:tb-show`

  • C-c C-.` ein:pytools-jump-to-source-command`

Jump to the source code of the object at point. When the prefix argument ‘‘C-u‘‘ is given, open the source code in the other window. You can explicitly specify the object by selecting it.

  • M-.` ein:pytools-jump-to-source-command`

Jump to the source code of the object at point. When the prefix argument ‘‘C-u‘‘ is given, open the source code in the other window. You can explicitly specify the object by selecting it.

Traceback viewer

Tracebacks from the notebook buffer can be difficult to understand. You can open a Traceback viewer by calling ein:notebook-view-traceback.

In the Traceback viewer, following keybinds are available.

Keymap for ein:traceback-mode.

  • RET` ein:tb-jump-to-source-at-point-command`

  • p` ein:tb-prev-item`

  • n` ein:tb-next-item`

PyTools

These commands can be used in the notebook buffer and the connected buffer.

  • function ein:pytools-doctest

    Do the doctest of the object at point.

  • function ein:pytools-whos

    Execute ‘‘%whos‘‘ magic command and popup the result.

  • function (ein:pytools-hierarchy&optional ask)

    Draw inheritance graph of the class at point. hierarchymagic extension is needed to be installed. You can explicitly specify the object by selecting it.

  • function (ein:pytools-pandas-to-sesdataframe)

    View pandas DataFrame in SES (Simple Emacs Spreadsheet). Open a ‘ses-mode’ buffer and import DataFrame object into it.SES is distributed with Emacs since Emacs 22, so you don’t need to install it if you are using newer Emacs.

  • function (ein:pytools-export-bufferbuffer format)

    Export contents of notebook using nbconvert to user-specified format (options will depend on the version of nbconvert available) to a new buffer.Currently EIN/IPython supports exporting to the following formats:HTMLJSON (this is basically the same as opening the ipynb file in a buffer).LatexMarkdownPythonRSTSlides

Misc

  • function helm-ein-kernel-history

    Search kernel execution history then insert the selected one.

  • function helm-ein-notebook-buffers

    Choose opened notebook using helm interface.

  • function anything-ein-kernel-history

    Search kernel execution history then insert the selected one.

  • function anything-ein-notebook-buffers

    Choose opened notebook using anything.el interface.

Org-mode Integration (ob-ein)

Configuration:

1
2
3
4
M-x customize-group RET org-babel
Org Babel Load Languages:
  Insert (ein . t)
  For example, '((emacs-lisp . t) (ein . t))

Snippet:

1
2
3
4
5
6
#BEGIN_SRC *language* :session localhost :results raw drawer
  import numpy, math, matplotlib.pyplot as plt
  %matplotlib inline
  x = numpy.linspace(0, 2*math.pi)
  plt.plot(x, numpy.sin(x))
#+END_SRC

  • Language can be ein-python, ein-r, or ein-julia. The relevant jupyter kernel must be installed before use. Additional languages can be

    configured via:M-x customize-group RET ein Ob Ein Languages

The format for the :session header argument is {url-or-port}/{path-to-notebook}. Just specifying {url-or-port}executes your source block in a single anonymous notebook (this effects an ipython repl in org). You should also specify :results raw drawer for proper rendering inside the org buffer. For example:

1
2
3
4
5
6
7
#+BEGIN_SRC ein-python :session localhost :results raw drawer
  import sys

  a = 14500
  b = a+1000
  sys.version
#+END_SRC

If your code block generates an image, like from an matplotlib plot, ein will automatically save to a file in the directory specified by ein:org-inline-image-directory and generate an appropriate inline link. You can also specify the file to save the image to using by the :image argument as in the example below:

1
2
3
4
5
6
7
8
9
#BEGIN_SRC ein :session localhost :results raw drawer :image output.png
  import matplotlib.pyplot as plt
  import numpy as np

  %matplotlib inline
  x = np.linspace(0, 1, 100)
  y = np.random.rand(100,1)
  plt.plot(x,y)
#+END_SRC

To get proper syntax highlighting for non-Python kernels, use the function ein:org-register-lang-mode to define a new ein-based org source language. For example, to get proper syntax highlighting for an R kernel, first call

1
(ein:org-register-lang-mode "ein-R" 'R)

Then org SRC blocks with language “ein-R” will use R syntax highlighting:

1
2
3
#BEGIN_SRC ein-R :session localhost :results raw drawer :image output.png
  plot(1:10, 1:10)
#+END_SRC

You can also link to an IPython notebook from org-mode files.

  1. Call org-mode function org-store-link 5 in notebook buffer. You can select a region to specify a position in the notebook.
  2. Go to org-mode file and type C-c C-l (org-insert-link). This will insert a link to the notebook.
  3. Type C-c C-o (org-open-at-point) to open the link at the point of cursor.

Customization

  • variable ein:org-async-p
  • variable ein:org-inline-image-directory
  • function ein:org-register-lang-mode

Support for The Hy Language (EXPERIMENTAL)

New in v0.14, ein has limited support for executing hy code in a notebook running an ipython kernel. Before trying this feature you will need to install hy per the quickstart instructions.

Once you have set up hy in your ipython kernel, open or create a notebook running this kernel and create or go to an empty cell. You will need to change the cell type to hy using ein:worksheet-change-cell-type. Once that is done you can enter hy expressions and they will be correctly evaluated by the kernel!

If you are running a hy kernel you can, of course, write hy expressions in code cells and have the expected results.

Customization

You can customize EIN by typing M-x customize-group RET ein RET. All the configurable variables are listed below.

Subpackages

  • variable ein:completion-backend

    EIN defaults to your individual company-mode or auto-complete-mode configuration. Change this setting to gather completions from the jupyter server:* ein:use-none-backend: local completions only (configured outside EIN) * ein:use-company-backend: company-style remote completions (elpy takes precedence) * ein:use-ac-backend: deprecated auto-complete remote completions

  • variable ein:use-smartrep

    Set to t to use preset smartrep configuration.WarningWhen used with MuMaMo (see ein:notebook-modes), keyboard macro which manipulates cell (add, remove, move, etc.) may start infinite loop (you need to stop it with C-g). Please be careful using this option if you are a heavy keyboard macro user. Using keyboard macro for other commands is fine.

  • variable ein:load-dev

Notebook list

  • variable ein:url-or-port

    List of default url-or-port values. This will be used for completion. So put your IPython servers. You can connect to servers not in this list (but you will need to type every time).

  • variable ein:default-url-or-port

    Default URL or port. This should be your main IPython Notebook server.

  • function (ein:notebooklist-load&optional url-or-port)

    Load notebook list but do not pop-up the notebook list buffer.For example, if you want to load notebook list when Emacs starts, add this in the Emacs initialization file:(add-to-hook ’after-init-hook ’ein:notebooklist-load)or even this (if you want fast Emacs start-up):;; load notebook list if Emacs is idle for 3 sec after start-up (run-with-idle-timer 3 nil #’ein:notebooklist-load)You should setup ‘ein:url-or-port’ or ‘ein:default-url-or-port’ in order to make this code work.See also: ‘ein:connect-to-default-notebook’, ‘ein:connect-default-notebook’.

Notebook

  • variable ein:worksheet-enable-undo

    When non-nil, allow undo of cell inputs only (as opposed towhole-cell operations such as killing, moving, executing cells).Changes to this variable only take effect for newly opened worksheets.

  • variable ein:polymode

    When enabled ein will use polymode to provide multi-major mode support in a notebook buffer, otherwise ein’s custom and outdated multi-major mode support will be used. Emacs must be restarted after changing this setting!

  • variable ein:notebook-modes

    Notebook modes to use (in order of preference).When the notebook is opened, mode in this value is checked one by one and the first usable mode is used.Available modes:ein:notebook-multilang-mode`ein:notebook-mumamo-modeein:notebook-python-modeein:notebook-plain-modeExamples:Use MuMaMo if it is installed. Otherwise, use plain mode. This is the old default setting:(setq ein:notebook-modes ‘(ein:notebook-mumamo-mode ein:notebook-plain-mode)) Avoid using MuMaMo even when it is installed:(setq ein:notebook-modes ‘(ein:notebook-plain-mode)) Use simplepython-modebased notebook mode when MuMaMo is not installed:(setq ein:notebook-modes ‘(ein:notebook-mumamo-mode ein:notebook-python-mode)) `

  • variable ein:notebook-querty-timeout-open

  • variable ein:notebook-querty-timeout-save

    Query timeout for saving notebook. Similar to ein:notebook-querty-timeout-open, but for saving notebook. For global setting and more information, see ein:query-timeout.

  • variable ein:cell-traceback-level

    Number of traceback stack to show. Hidden tracebacks are not discarded. You can view them using [ein:tb-show].

  • variable ein:cell-autoexec-prompt

    String shown in the cell prompt when the auto-execution flag is on. See also ein:connect-aotoexec-lighter.

  • variable ein:junk-notebook-name-template

  • variable ein:iexec-delay

    Delay before executing cell after change in second.

  • variable ein:complete-on-dot

  • variable ein:helm-kernel-history-search-key

    Bind helm-ein-kernel-history to this key in notebook mode.Example:(setq ein:helm-kernel-history-search-key "\M-r")This key will be installed in the ein:notebook-mode-map.

  • variable ein:anything-kernel-history-search-key

    Bind anything-ein-kernel-history to this key in notebook mode.Example:(setq ein:anything-kernel-history-search-key "\M-r")This key will be installed in the ein:notebook-mode-map.

  • variable ein:helm-kernel-history-search-auto-pattern

    Automatically construct search pattern when non-nil.Single space is converted to “”.A backslash followed by a space is converted to a single space.A “” is added at the beginning and end of the pattern.This variable applies to both helm-ein-kernel-history and anything-ein-kernel-history.

  • variable ein:output-type-preference

    Output types to be used in notebook. First output-type found in this list will be used. This variable can be a list or a function returning a list given DATA plist. See also ein:output-type-prefer-pretty-text-over-html.Example: If you prefer HTML type over text type, you can set it as:(setq ein:output-type-preference '(emacs-lisp svg png jpeg html text latex javascript))Note that html comes before text.

  • variable ein:shr-env

    Variables let-bound while calling shr-insert-document.To use default shr setting:(setq ein:shr-env nil)Draw boundaries for table (default):(setq ein:shr-env '((shr-table-horizontal-line ?-) (shr-table-vertical-line ?|) (shr-table-corner ?+)))

  • variable ein:notebook-autosave-frequency

    Sets the frequency (in seconds) at which the notebook is automatically saved, per IPEP15. Set to 0 to disable this feature.Autosaves are automatically enabled when a notebook is opened, but can be controlled manually via ein:notebook-enable-autosave and ein:notebook-disable-autosave.If you wish to change the autosave frequency for the current notebook call ein:notebook-update-autosave-freqency.

  • variable ein:notebook-create-checkpoint-on-save

    If non-nil a checkpoint will be created every time the notebook is saved. Otherwise checkpoints must be created manually via ein:notebook-create-checkpoint.

Console

  • variable ein:console-security-dir

    Security directory setting.Following types are valid:stringUse this value as a path to security directory. Handy when you have only one IPython server.alistAn alist whose element is “(URL-OR-PORT . DIR)”. Key (URL-OR-PORT) can be string (URL), integer (port), ordefault (symbol). The value of default is used when other key does not much. Normally you should have this entry.functionCalled with an argument URL-OR-PORT (integer or string). You can have complex setting using this.

  • variable ein:console-executable

    IPython executable used for console.Example: "/user/bin/ipython". Types same as ein:console-security-dir are valid.

  • variable ein:console-args

    Additional argument when using console.WarningSpace-separated string is obsolete now. Use a list of string as value now.Setting to use IPython profile named “YOUR-IPYTHON-PROFILE”:(setq ein:console-args '("--profile" "YOUR-IPYTHON-PROFILE"))Together with ein:console-security-dir, you can open IPython console connecting to a remote kernel.:(setq ein:console-args '("--ssh" "HOSTNAME")) (setq ein:console-security-dir "PATH/TO/SECURITY/DIR")You can setup ein:console-args per server basis using alist form:(setq ein:console-args '((8888 . '("--profile" "PROFILE")) (8889 . '("--ssh" "HOSTNAME")) (default . '("--profile" "default"))))If you want to use more complex setting, you can set a function to it:(setq ein:console-args (lambda (url-or-port) '("--ssh" "HOSTNAME")))See also: ein:console-security-dir.

Connect

  • variable ein:connect-run-command

    %run magic command used for ein:connect-run-buffer. Types same as ein:console-security-dir are valid.

  • variable ein:connect-reload-command

    Setting for ein:connect-reload-buffer. Same as ein:connect-run-command.

  • variable ein:connect-save-before-run

    Whether the buffer should be saved before ein:connect-run-buffer.

  • variable ein:propagate-connect

    Set to t to connect to the notebook after jumping to a buffer.

  • variable ein:connect-aotoexec-lighter

    String appended to the lighter of ein:connect-mode (ein:c) when auto-execution mode is on. When nil, use the same string as ein:cell-autoexec-prompt.

  • variable ein:connect-default-notebook

    Notebook to be connect when ein:connect-to-default-notebook is called.Example setting to connect to “My_Notebook” in the server at port 8888 when opening any buffer in python-mode:(setq ein:connect-default-notebook "8888/My_Notebook") (add-hook 'python-mode-hook 'ein:connect-to-default-notebook)ein:connect-default-notebook can also be a function without any argument. This function must return a string (notebook path of the form “URL-OR-PORT/NOTEBOOK-NAME”).As ein:connect-to-default-notebook requires notebook list to be loaded, consider using ein:notebooklist-loadto load notebook list if you want to connect to notebook without manually opening notebook list.

  • function ein:connect-to-default-notebook

    Connect to the default notebook specified by ‘ein:connect-default-notebook’. Set this to ‘python-mode-hook’ to automatically connect any python-mode buffer to the notebook.

Misc

  • variable ein:filename-translations

    Convert file paths between Emacs and Python process.This value can take these form:alistIts key specifies URL-OR-PORT and value must be a list of two functions: (TO-PYTHON FROM-PYTHON). Key (URL-OR-PORT) can be string (URL), integer (port), or default (symbol). The value of default is used when other key does not much.functionCalled with an argument URL-OR-PORT (integer or string). This function must return a list of two functions: (TO-PYTHON FROM-PYTHON).Here, the functions TO-PYTHON and FROM-PYTHON are defined as:TO-PYTHONA function which converts a file name (returned by buffer-file-name) to the one Python understands.FROM-PYTHONA function which converts a file path returned by Python process to the one Emacs understands.Use ein:tramp-create-filename-translator to easily generate the pair of TO-PYTHON and FROM-PYTHON.

  • function (ein:tramp-create-filename-translatorremote-host &optional username)

    Generate a pair of TO-PYTHON and FROM-PYTHON for ‘ein:filename-translations’.Usage:(setq ein:filename-translations ‘((8888 . ,(ein:tramp-create-filename-translator "MY-HOSTNAME")))) ;; Equivalently: (setq ein:filename-translations (lambda (url-or-port) (when (equal url-or-port 8888) (ein:tramp-create-filename-translator "MY-HOSTNAME"))))This setting assumes that the IPython server which can be connected using the port 8888 in localhost is actually running in the host named MY-HOSTNAME.Adapted from ‘slime-create-filename-translator’.

  • variable ein:query-timeout

    Default query timeout for HTTP access in millisecond.Setting this to nil means no timeout. If you have curl command line program, it is automatically set to nil as curl is reliable than url-retrieve therefore no need for a workaround (see below).If you do the same operation before the timeout, old operation will NO LONGER be canceled (as it the cookie jar gets clobbered when curl aborts). Instead you will see Race! in debug messages.NoteThis value exists because it looks like url-retrieve occasionally fails to finish (start?) querying. Timeout is used to let user notice that their operation is not finished. It also prevent opening a lot of useless process buffers. You will see them when closing Emacs if there is no timeout.If you know how to fix the problem with url-retrieve, please let me know or send pull request at github! (Related bug report in Emacs bug tracker: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11469)

  • function (ein:notebook-jump-to-opened-notebooknotebook)

    List all opened notebook buffers and switch to one that the user selects.

Gotchas and caveats

Although EIN mostly works fine, there are some deficits I noticed but have not fixed yet. It seems that they originate from some upstream bugs so there is little I can do in EIN (but I’m not sure – it’s possible that I am misusing the libraries!).

If you know how to fix/workaround them, patches are very welcome.

url-retrieve

While using EIN, probably most of the error messages are about server connections. It looks like the problem is in url-retrieve. But in those cases you don’t lose any notebook data and your IPython kernel is fine. You can just type the command again and it will go fine most of the time. For saving notebook, I implemented code to retry when there is an error comes from url-retrieve to make it even safer.

MuMaMo

When using MuMaMo based notebook mode, you will notice that highlighting outside of the cell input is turned off while you are in the input area. It seems there is a bug in MuMaMo 6.

If you are using smartrep and MuMaMo together, see also the warning in ein:use-smartrep document.

python-mode.el

Using python-mode.el without fisrt byte-compiling can cause a segfault with the MuMaMo based notebook mode.

Also, mumamo-idle-set-major-mode generates the error message (wrong-type-argument listp python-saved-check-command) time to time, making minibuffer bit noisy while editing notebook. See Tips to fix this problem.

Advanced

By telling IPython a little bit about Emacs Lisp, you can execute Emacs Lisp from IPython, just like you can execute Javascript in the web client. See emacslisp.py for more details.

1
2
3
4
5
6
7
In [1]:
%run PATH/TO/emacslisp.py

In [2]:
EmacsLisp('(+ 1 2 3)')
Out [2]:
6

Reporting issues

Please use M-x ein:dev-bug-report-template to write a bug report. It pops up a buffer containing some system information and instruction for bug report.

Logging

Sometime more information than provided in the *Message* buffer is needed to debug.

  1. Execute (ein:log-set-level 'debug) (e.g., M-: (ein:log-set-level 'debug) RET).

  2. Then do some operation which cause the problem.

  3. Go to the log buffer _*ein:log-all* (it starts with a space) and paste the whole buffer to the issue tracker.

    Please enclose the log with three backquotes to make the snippet as a code block, like this:

    1

    [verbose] Start logging. @#<buffer ein: 8888/NAME>
    [info] Notebook NAME is ready @#<buffer ein: 8888/NAME>
    [info] Kernel started: 5e4f74d1-ce91-4e7e-9575-9646adea5172 @#

    1

    See also: GitHub Flavored Markdown - Introduction

    If it is too long, you can use paste bin service such as gist.

websocket.el

websocket.el has its own logging buffer. Sometime it is useful to see this log. To do this:

  1. (require 'ein-dev)
  2. (setq websocket-debug t) or call ein:dev-start-debug.
  3. Then do the operation which causes the problem.
  4. Go to log buffer using ein:dev-pop-to-debug-shell and ein:dev-pop-to-debug-iopub. These command must be called in the notebook buffer.

Debugging

If you are interested in debugging EIN, you should start by calling the command ein:dev-start-debug. If the bug is websocket related, you may need to run it with a prefix key like this: C-u M-x ein:dev-start-debug RET to get a backtrace. This command sets debug-on-error to t and does some patching to the debugger. This patching is required because printing EWOC objects freezes Emacs otherwise. It also changes log level to report everything the log buffer. You can reset the patch and log level with ein:dev-stop-debug.