-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pytorch GC #592
Comments
@cjdoris @MilesCranmer Any thoughts on this issue? Thanks! |
alright, using PythonCall
torch = pyimport("torch")
torch.cuda.is_available()
n=20000
a = torch.randn((1,n*n),device=torch.device("cuda")) # VRAM increase here
a = torch.randn((1,n*n),device=torch.device("cuda")) # VRAM also increase here
a = torch.randn((1,n*n),device=torch.device("cuda")) # VRAM also increase here
a = nothing # useless
PythonCall.GC.gc() # useless
torch.cuda.empty_cache() # useless
PythonCall.pydel!(a)
PythonCall.GC.gc()
torch.cuda.empty_cache() # Released! |
x66ccff/SymbolicRegressionGPU.jl#22 still problem here |
When using PythonCall and Pytorch together, if tensors are created in Julia code (including any temporary tensors) without keeping a "handle" for Julia to release them through pydel!(), it leads to an inability to release these tensors through gc or torch[].cuda.empty_cache. I wrote a specific example to demonstrate: using PythonCall
torch = pyimport("torch")
torch.cuda.is_available()
n=20000
a = torch.randn((1,n*n),device=torch.device("cuda")) # VRAM increase here
f(x) = begin
1 + 1
3 * 1
x + x # ✅ can be released
end
g = f(a)
PythonCall.pydel!(a)
PythonCall.pydel!(g)
println(torch.cuda.memory_summary())
using PythonCall
torch = pyimport("torch")
torch.cuda.is_available()
n=20000
a = torch.randn((1,n*n),device=torch.device("cuda")) # VRAM increase here
f(x) = begin
x + 1 # ❌ can not release this anymore
x * 1 # ❌ can not release this anymore
x + x # ✅ can be released
end
g = f(a)
PythonCall.pydel!(a)
PythonCall.pydel!(g)
println(torch.cuda.memory_summary())
|
After |
wow! Thanks! That works! 😂 I forgot to try |
@cjdoris However, frequently calling |
No, unless you |
I'm using torch in PythonCall. When I try to create tensors multiple times, even after reassigning the same tensor or setting it to nothing, I don't observe any decrease in GPU memory usage. This persists even after using GC.
Anyone can help?
julia Version 1.11.3
julia> torch.version
Python: '2.6.0+cu126'
The text was updated successfully, but these errors were encountered: