[FEA] Run pylibcudf test with optional dependencies uninstalled

[mroeschke]

Is your feature request related to a problem? Please describe.
#18020 added a pylibcudf.Scalar.from_numpy API to interact with numpy scalars. Since we want pylibcudf to have minimal dependencies, the API was designed to assume numpy was an option dependency and may possibly not be installed. Currently we don't test scenario this since we run pylibcudf tests with cudf tests which the latter requires numpy.

Describe the solution you'd like
We should ideally have a step which tests pylibcudf without numpy (or any optional dependency xref #17046) installed. Possibly in the same job we can uninstall numpy or create a new virtual env in the same job without pylibcudf optional dependencies.

Describe alternatives you've considered
A whole separate job that just test pylibcudf without optional dependencies (probably unnecessary to have a dedicated job for this just to isolate environments)

Additional context
Add any other context, code examples, or references to existing implementations about the feature request here.