You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@pyang30 as you mentioned in #806 I'm seeing bpython access a ton of files it shouldn't need to in order to initialize the import completion set. Like for you, it spends most of its time in node_modules folders.
There's something very broken here, we're crawling way too much.
This isn't broken! Our import completion correctly traverses directories which do not contain an __init__.py, since #792, because empty __init__.py files are no longer required for namespace packages.
Maybe we shouldn't crawl these at all. Apparently unittest doesn't crawl them, a sort of similar problem. At some point in the past, IPython skipped these too.
Maybe we should use different crawling logic for the current directory that bpython is run from than we do for other locations on sys.path: namespace packages in other locations will still be picked up, but running bpython from a filesystem path which contains a massive node_modules directory or similar won't chew up CPU. Users who manually add e.g. their home directory to sys.path would still see this issue.
We could implement a (user-configurable) skip list which is checked before traversing into directories. We could make it default to node_modules, .git, and maybe other well known directories which won't contain Python modules.
from all the junk I have in my home directory so I'm thinking about this again. Same ideas as before: since running bpython from my home directory is very common for me and it's the '' entry in sys.path that causes all the trouble for me, maybe this entry could be treated differently; and maybe traversing non-init.py directories could be an option to turn off, or even off by default.
Activity
thomasballinger commentedon Aug 23, 2020
This isn't broken! Our import completion correctly traverses directories which do not contain an
__init__.py, since #792, because empty__init__.pyfiles are no longer required for namespace packages.Maybe we shouldn't crawl these at all. Apparently unittest doesn't crawl them, a sort of similar problem. At some point in the past, IPython skipped these too.
Maybe we should use different crawling logic for the current directory that bpython is run from than we do for other locations on
sys.path: namespace packages in other locations will still be picked up, but running bpython from a filesystem path which contains a massive node_modules directory or similar won't chew up CPU. Users who manually add e.g. their home directory to sys.path would still see this issue.sebastinas commentedon Aug 23, 2020
We could implement a (user-configurable) skip list which is checked before traversing into directories. We could make it default to
node_modules,.git, and maybe other well known directories which won't contain Python modules.thomasballinger commentedon Nov 18, 2021
I still see a lot of directories visited
(this is a script from #943)
from all the junk I have in my home directory so I'm thinking about this again. Same ideas as before: since running bpython from my home directory is very common for me and it's the
''entry in sys.path that causes all the trouble for me, maybe this entry could be treated differently; and maybe traversing non-init.py directories could be an option to turn off, or even off by default.Another IPython reference bug ipython/ipython#8875, they suggest Jedi for this.
Implement a non-configurable skiplist for import completion (fixes bp…