Skip to content

Commit f396334

Browse files
authored
Merge pull request #1579 from parente/bundler-api
Bundler extension API
2 parents a249c9c + 2473efc commit f396334

25 files changed

+1336
-15
lines changed

docs/source/examples/Notebook/Distributing Jupyter Extensions as Python Packages.ipynb

+98-3
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,9 @@
2121
" [AMD modules](https://en.wikipedia.org/wiki/Asynchronous_module_definition)\n",
2222
" that exports a function `load_ipython_extension`\n",
2323
"- server extension: an importable Python module\n",
24-
" - that implements `load_jupyter_server_extension`"
24+
" - that implements `load_jupyter_server_extension`\n",
25+
"- bundler extension: an importable Python module with generated File -> Download as / Deploy as menu item trigger\n",
26+
" - that implements `bundle`"
2527
]
2628
},
2729
{
@@ -105,11 +107,12 @@
105107
"metadata": {},
106108
"source": [
107109
"## Did it work? Check by listing Jupyter Extensions.\n",
108-
"After running one or more extension installation steps, you can list what is presently known about nbextensions or server extension. The following commands will list which extensions are available, whether they are enabled, and other extension details:\n",
110+
"After running one or more extension installation steps, you can list what is presently known about nbextensions, server extensions, or bundler extensions. The following commands will list which extensions are available, whether they are enabled, and other extension details:\n",
109111
"\n",
110112
"```shell\n",
111113
"jupyter nbextension list\n",
112114
"jupyter serverextension list\n",
115+
"jupyter bundlerextension list\n",
113116
"```"
114117
]
115118
},
@@ -255,6 +258,98 @@
255258
"jupyter serverextension enable --py my_fancy_module [--sys-prefix|--system]\n",
256259
"```"
257260
]
261+
},
262+
{
263+
"cell_type": "markdown",
264+
"metadata": {},
265+
"source": [
266+
"## Example - Bundler extension"
267+
]
268+
},
269+
{
270+
"cell_type": "markdown",
271+
"metadata": {},
272+
"source": [
273+
"### Creating a Python package with a bundlerextension\n",
274+
"\n",
275+
"Here is a bundler extension that adds a *Download as -> Notebook Tarball (tar.gz)* option to the notebook *File* menu. It assumes this directory structure:\n",
276+
"\n",
277+
"```\n",
278+
"- setup.py\n",
279+
"- MANIFEST.in\n",
280+
"- my_tarball_bundler/\n",
281+
" - __init__.py\n",
282+
"```"
283+
]
284+
},
285+
{
286+
"cell_type": "markdown",
287+
"metadata": {},
288+
"source": [
289+
"### Defining the bundler extension\n",
290+
"\n",
291+
"This example shows that the bundler extension and its `bundle` function are defined in the `__init__.py` file.\n",
292+
"\n",
293+
"#### `my_tarball_bundler/__init__.py`\n",
294+
"\n",
295+
"```python\n",
296+
"import tarfile\n",
297+
"import io\n",
298+
"import os\n",
299+
"import nbformat\n",
300+
"\n",
301+
"def _jupyter_bundlerextension_paths():\n",
302+
" \"\"\"Declare bundler extensions provided by this package.\"\"\"\n",
303+
" return [{\n",
304+
" # unique bundler name\n",
305+
" \"name\": \"tarball_bundler\",\n",
306+
" # module containing bundle function\n",
307+
" \"module_name\": \"my_tarball_bundler\",\n",
308+
" # human-redable menu item label\n",
309+
" \"label\" : \"Notebook Tarball (tar.gz)\",\n",
310+
" # group under 'deploy' or 'download' menu\n",
311+
" \"group\" : \"download\",\n",
312+
" }]\n",
313+
"\n",
314+
"\n",
315+
"def bundle(handler, model):\n",
316+
" \"\"\"Create a compressed tarball containing the notebook document.\n",
317+
" \n",
318+
" Parameters\n",
319+
" ----------\n",
320+
" handler : tornado.web.RequestHandler\n",
321+
" Handler that serviced the bundle request\n",
322+
" model : dict\n",
323+
" Notebook model from the configured ContentManager\n",
324+
" \"\"\"\n",
325+
" notebook_filename = model['name']\n",
326+
" notebook_content = nbformat.writes(model['content']).encode('utf-8')\n",
327+
" notebook_name = os.path.splitext(notebook_filename)[0]\n",
328+
" tar_filename = '{}.tar.gz'.format(notebook_name)\n",
329+
" \n",
330+
" info = tarfile.TarInfo(notebook_filename)\n",
331+
" info.size = len(notebook_content)\n",
332+
"\n",
333+
" with io.BytesIO() as tar_buffer:\n",
334+
" with tarfile.open(tar_filename, \"w:gz\", fileobj=tar_buffer) as tar:\n",
335+
" tar.addfile(info, io.BytesIO(notebook_content))\n",
336+
" \n",
337+
" # Set headers to trigger browser download\n",
338+
" handler.set_header('Content-Disposition',\n",
339+
" 'attachment; filename=\"{}\"'.format(tar_filename))\n",
340+
" handler.set_header('Content-Type', 'application/gzip')\n",
341+
" \n",
342+
" # Return the buffer value as the response\n",
343+
" handler.finish(tar_buffer.getvalue())\n",
344+
"```"
345+
]
346+
},
347+
{
348+
"cell_type": "markdown",
349+
"metadata": {},
350+
"source": [
351+
"See [Extending the Notebook](../../extending) for more documentation about writing nbextensions, server extensions, and bundler extensions."
352+
]
258353
}
259354
],
260355
"metadata": {
@@ -277,5 +372,5 @@
277372
}
278373
},
279374
"nbformat": 4,
280-
"nbformat_minor": 0
375+
"nbformat_minor": 1
281376
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
Custom bundler extensions
2+
=========================
3+
4+
The notebook server supports the writing of *bundler extensions* that transform, package, and download/deploy notebook files. As a developer, you need only write a single Python function to implement a bundler. The notebook server automatically generates a *File -> Download as* or *File -> Deploy as* menu item in the notebook front-end to trigger your bundler.
5+
6+
Here are some examples of what you can implement using bundler extensions:
7+
8+
* Convert a notebook file to a HTML document and publish it as a post on a blog site
9+
* Create a snapshot of the current notebook environment and bundle that definition plus notebook into a zip download
10+
* `Deploy a notebook as a standalone, interactive dashboard <https://github.com/jupyter-incubator/dashboards_bundlers>`_
11+
12+
To implement a bundler extension, you must do all of the following:
13+
14+
* Declare bundler extension metadata in your Python package
15+
* Write a `bundle` function that responds to bundle requests
16+
* Instruct your users on how to enable/disable your bundler extension
17+
18+
The following sections describe these steps in detail.
19+
20+
Declaring bundler metadata
21+
--------------------------
22+
23+
You must provide information about the bundler extension(s) your package provides by implementing a `_jupyter_bundlerextensions_paths` function. This function can reside anywhere in your package so long as it can be imported when enabling the bundler extension. (See :ref:`enabling-bundlers`.)
24+
25+
.. code:: python
26+
27+
# in mypackage.hello_bundler
28+
29+
def _jupyter_bundlerextension_paths():
30+
"""Example "hello world" bundler extension"""
31+
return [{
32+
'name': 'hello_bundler', # unique bundler name
33+
'label': 'Hello Bundler', # human-redable menu item label
34+
'module_name': 'mypackage.hello_bundler', # module containing bundle()
35+
'group': 'deploy' # group under 'deploy' or 'download' menu
36+
}]
37+
38+
Note that the return value is a list. By returning multiple dictionaries in the list, you allow users to enable/disable sets of bundlers all at once.
39+
40+
Writing the `bundle` function
41+
-----------------------------
42+
43+
At runtime, a menu item with the given label appears either in the *File -> Deploy as* or *File -> Download as* menu depending on the `group` value in your metadata. When a user clicks the menu item, a new browser tab opens and notebook server invokes a `bundle` function in the `module_name` specified in the metadata.
44+
45+
You must implement a `bundle` function that matches the signature of the following example:
46+
47+
.. code:: python
48+
49+
# in mypackage.hello_bundler
50+
51+
def bundle(handler, model):
52+
"""Transform, convert, bundle, etc. the notebook referenced by the given
53+
model.
54+
55+
Then issue a Tornado web response using the `handler` to redirect
56+
the user's browser, download a file, show a HTML page, etc. This function
57+
must finish the handler response before returning either explicitly or by
58+
raising an exception.
59+
60+
Parameters
61+
----------
62+
handler : tornado.web.RequestHandler
63+
Handler that serviced the bundle request
64+
model : dict
65+
Notebook model from the configured ContentManager
66+
"""
67+
handler.finish('I bundled {}!'.format(model['path']))
68+
69+
Your `bundle` function is free to do whatever it wants with the request and respond in any manner. For example, it may read additional query parameters from the request, issue a redirect to another site, run a local process (e.g., `nbconvert`), make a HTTP request to another service, etc.
70+
71+
The caller of the `bundle` function is `@tornado.gen.coroutine` decorated and wraps its call with `torando.gen.maybe_future`. This behavior means you may handle the web request synchronously, as in the example above, or asynchronously using `@tornado.gen.coroutine` and `yield`, as in the example below.
72+
73+
.. code:: python
74+
75+
from tornado import gen
76+
77+
@gen.coroutine
78+
def bundle(handler, model):
79+
# simulate a long running IO op (e.g., deploying to a remote host)
80+
yield gen.sleep(10)
81+
82+
# now respond
83+
handler.finish('I spent 10 seconds bundling {}!'.format(model['path']))
84+
85+
You should prefer the second, asynchronous approach when your bundle operation is long-running and would otherwise block the notebook server main loop if handled synchronously.
86+
87+
For more details about the data flow from menu item click to bundle function invocation, see :ref:`bundler-details`.
88+
89+
.. _enabling-bundlers:
90+
91+
Enabling/disabling bundler extensions
92+
-------------------------------------
93+
94+
The notebook server includes a command line interface (CLI) for enabling and disabling bundler extensions.
95+
96+
You should document the basic commands for enabling and disabling your bundler. One possible command for enabling the `hello_bundler` example is the following:
97+
98+
.. code:: bash
99+
100+
jupyter bundlerextension enable --py mypackage.hello_bundler --sys-prefix
101+
102+
The above updates the notebook configuration file in the current conda/virtualenv environment (`--sys-prefix`) with the metadata returned by the `mypackage.hellow_bundler._jupyter_bundlerextension_paths` function.
103+
104+
The corresponding command to later disable the bundler extension is the following:
105+
106+
.. code:: bash
107+
108+
jupyter bundlerextension disable --py mypackage.hello_bundler --sys-prefix
109+
110+
For more help using the `bundlerextension` subcommand, run the following.
111+
112+
.. code:: bash
113+
114+
jupyter bundlerextension --help
115+
116+
The output describes options for listing enabled bundlers, configuring bundlers for single users, configuring bundlers system-wide, etc.
117+
118+
Example: IPython Notebook bundle (.zip)
119+
---------------------------------------
120+
121+
The `hello_bundler` example in this documentation is simplisitic in the name of brevity. For more meaningful examples, see `notebook/bundler/zip_bundler.py` and `notebook/bundler/tarball_bundler.py`. You can enable them to try them like so:
122+
123+
.. code:: bash
124+
125+
jupyter bundlerextension enable --py notebook.bundler.zip_bundler --sys-prefix
126+
jupyter bundlerextension enable --py notebook.bundler.tarball_bundler --sys-prefix
127+
128+
.. _bundler-details:
129+
130+
Bundler invocation details
131+
--------------------------
132+
133+
Support for bundler extensions comes from Python modules in `notebook/bundler` and JavaScript in `notebook/static/notebook/js/menubar.js`. The flow of data between the various components proceeds roughly as follows:
134+
135+
1. User opens a notebook document
136+
2. Notebook front-end JavaScript loads notebook configuration
137+
3. Bundler front-end JS creates menu items for all bundler extensions in the config
138+
4. User clicks a bundler menu item
139+
5. JS click handler opens a new browser window/tab to `<notebook base_url>/bundle/<path/to/notebook>?bundler=<name>` (i.e., a HTTP GET request)
140+
6. Bundle handler validates the notebook path and bundler `name`
141+
7. Bundle handler delegates the request to the `bundle` function in the bundler's `module_name`
142+
8. `bundle` function finishes the HTTP request

docs/source/extending/handlers.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -124,4 +124,4 @@ following:
124124
125125
References:
126126
1. `Peter Parente's
127-
Mindtrove <http://mindtrove.info/#nb-server-exts>`__
127+
Mindtrove <http://mindtrove.info/4-ways-to-extend-jupyter-notebook/#nb-server-exts>`__

docs/source/extending/index.rst

+1
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,4 @@ override the notebook's defaults with your own custom behavior.
1414
handlers
1515
frontend_extensions
1616
keymaps
17+
bundler_extensions

notebook/bundler/__init__.py

Whitespace-only changes.

notebook/bundler/__main__.py

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Copyright (c) Jupyter Development Team.
2+
# Distributed under the terms of the Modified BSD License.
3+
4+
from .bundlerextensions import main
5+
6+
if __name__ == '__main__':
7+
main()

0 commit comments

Comments
 (0)