Episode 5
Inference mode
What's inference mode? Why doesn't my code run fast if I use no_grad or make sure requires_grad=False? How come inference mode is safe but AutoNonVariableTypeMode is not?
May 10, 202114m 26s
Audio is streamed directly from the publisher (cdn.simplecast.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
What's inference mode? Why doesn't my code run fast if I use no_grad or make sure requires_grad=False? How come inference mode is safe but AutoNonVariableTypeMode is not?
Further reading:
- Inference mode RFC https://github.com/.../rfc0011/RFC-0011-InferenceMode.md
- Inference mode docs for C++ frontend users https://github.com/.../cpp/source/notes/inference_mode.rst
- Tracking issue for Python frontend support https://github.com/pytorch/pytorch/issues/56608