Skip to content

Commit 9765c31

Browse files
author
lezwon
committed
added docs
1 parent f2c4fc3 commit 9765c31

File tree

1 file changed

+26
-0
lines changed

1 file changed

+26
-0
lines changed

docs/source/production_inference.rst

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
Inference in Production
2+
=======================
3+
PyTorch Lightning eases the process of deploying models into production.
4+
5+
6+
Exporting to ONNX
7+
-----------------
8+
PyTorch Lightning provides a handy function to quickly export your model to ONNX format, which allows the model to be independent of PyTorch and run on an ONNX Runtime.
9+
10+
To export your model to ONNX format call the `to_onnx` function on your Lightning Module with the filepath and input_sample.
11+
12+
.. code-block:: python
13+
filepath = 'model.onnx'
14+
model = SimpleModel()
15+
input_sample = torch.randn((1, 64))
16+
model.to_onnx(filepath, input_sample, export_params=True)
17+
18+
You can also skip passing the input sample if the `example_input_array` property is specified in your LightningModule.
19+
20+
Once you have the exported model, you can run it on your ONNX runtime in the following way:
21+
22+
.. code-block:: python
23+
ort_session = onnxruntime.InferenceSession(filepath)
24+
input_name = ort_session.get_inputs()[0].name
25+
ort_inputs = {input_name: np.random.randn(1, 64).astype(np.float32)}
26+
ort_outs = ort_session.run(None, ort_inputs)

0 commit comments

Comments
 (0)