Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Make cudf.pandas and cudf-polars compatible with RAPIDS_NO_INITIALIZE #18210

Open
vyasr opened this issue Mar 10, 2025 · 0 comments · May be fixed by #18224
Open

[BUG] Make cudf.pandas and cudf-polars compatible with RAPIDS_NO_INITIALIZE #18210

vyasr opened this issue Mar 10, 2025 · 0 comments · May be fixed by #18224
Assignees
Labels
bug Something isn't working Python Affects Python cuDF API.

Comments

@vyasr
Copy link
Contributor

vyasr commented Mar 10, 2025

Describe the bug
Currently we support using the environment variables RAPIDS_NO_INITIALIZE and CUDF_NO_INITIALIZE to defer GPU initialization on import so that cudf will import successfully on machines without a GPU. However, this does not currently work with cudf.pandas because cudf.pandas tries to initialize a managed memory resource when it starts, leading to an error failing to find the CUDA driver.

Note that I would not expect this to occur with cudf-polars since cudf-polars only initializes the mr when the first collect call happens, not on import.

Steps/Code to reproduce bug
Install cudf on a machine without a GPU (probably easiest to test in a container without GPUs mounted) and run RAPIDS_NO_INITIALIZE=1 python -m cudf.pandas -c 'import pandas as pd'

Expected behavior
The above command should work. We can probably fix this by nesting the calls to pylibcudf.utils._is_concurrent_managed_access_supported inside a try-except block that can catch these failures and ignore them. We could also do some eager check for a GPU existing, but since the vast majority of practical use cases will have them I don't think LBYL is worthwhile.

@vyasr vyasr added the bug Something isn't working label Mar 10, 2025
@galipremsagar galipremsagar self-assigned this Mar 10, 2025
@galipremsagar galipremsagar added the Python Affects Python cuDF API. label Mar 10, 2025
@galipremsagar galipremsagar linked a pull request Mar 10, 2025 that will close this issue
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Python Affects Python cuDF API.
Projects
Status: Todo
Development

Successfully merging a pull request may close this issue.

2 participants