Implicit namespace package leads to autocompleter crash in IPython

[maxalbert]

Versions:

• IPython: 8.31.0
• Jedi: 0.19.2

Given the following directory structure:

$ tree .

.
├── dir1
│   └── foobarpkg
│       └── pkgA
│           └── __init__.py
└── dir2
    └── foobarpkg
        └── pkgB
            └── __init__.py

The following produces an error when trying to use tab completion in step 5:

$ ipython --Completer.debug=True

Python 3.13.1 (main, Dec  3 2024, 17:59:52) [Clang 19.1.5 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import os, sys

In [2]: curdir = os.getcwd()

In [3]: sys.path = [
    os.path.join(curdir, "dir1"),
    os.path.join(curdir, "dir2")
    ]

In [4]: import foobarpkg

In [5]: foobarpkg.<tab>

Error message:

Oops Jedi has crashed, please report a bug with the following:
   ...: """
   ...: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'
   ...: s"""

[maxalbert]

FYI, the bug is triggered inside the method DirectObjectAccess.py__file__ (here).

It is caused by the fact that in the scenario described above, the value of self._obj.__file__ will be None. This in turn causes Path(self._obj.__file__) to throw a TypeError, which is not handled.

[davidhalter]

It makes sense that that would be involved, but since we don't have the traceback it's kind of hard to say what happens and where we should change something. DirectObjectAccess.py__file__ seems fine. But it feels like there is an is not None missing somewhere.

[maxalbert]

@davidhalter Agreed. I (briefly) played with it locally and tried to simply return None in case self._obj.__file__ is None . While this prevented the crash, it didn't seem to lead to the correct behaviour.

My main blocker for going further was that I didn't know how to set up automated tests that reproduce this bug so that I could investigate efficiently. I found a few locations for different kinds of tests around namespace packages but didn't have time to dig deeper. Are you able to point me to an example test that I can copy? Otherwise, which additional information would be useful to debug this further?

[davidhalter]

You probably need to use something like get_example_dir('implicit_namespace_package') and the tests are typically in test/test_api/test_interpreter.py and use something like Interpreter(..., project=project), where project is defined by also using get_example_dir. If you grep around a bit you should be able to find it.

[maxalbert]

Thanks for the pointers @davidhalter. I created draft PR #2045 which adds a regression test for the crash described above. Strangely, the test itself passes. But when I run a script with (seemingly) identical behaviour from an interactive IPython session, it still exhibits the error. See the PR description for details.

I suspect that there are subtle differences in my test setup compared to a "real" IPython session, but I don't have enough knowledge of jedi's internals to figure out what I'm doing wrong. Is there anything obvious that sticks out to you? Many thanks!