Repository navigation
pythonnet.load crashes if the Python interpreter is called from PATH in Ubuntu 20.04 #1388
Description
Environment
- Pythonnet version: 3.0.0-dev (latest master on 18/2/2021)
- Python version: 3.8.5
- Operating System: Ubuntu 20.04
- .NET Runtime: .NET 5.0.2
Details
- Describe what you were trying to get done.
I am trying to use the latestpythonnetin a virtual environment. If I call my simple demo script usingpythoncommand in PATH, pythonnet.load() crashes. It is the same with any other PATH-basedpythonXcommand. Using a full path works fine. See the details and the analysis below.
Preparation
# Create virtual env.
python3 -m virtualenv venv
. venv/bin/activate
# Install the latest pythonnet
git clone https://github.com/pythonnet/pythonnet.git
pip install ./pythonnet/
# Get .NET 5 runtime
wget https://download.visualstudio.microsoft.com/download/pr/824175f2-5577-46ec-a675-95d2f652575f/07dafc2398e669afa7d25f2d47398c36/dotnet-runtime-5.0.2-linux-x64.tar.gz
mkdir dotnet_5.0.2
tar xfzv dotnet-runtime-5.0.2-linux-x64.tar.gz -C dotnet_5.0.2/
Runtime config (cat runtimeconfig.json)
{
"runtimeOptions": {
"tfm": "net5.0",
"framework": {
"name": "Microsoft.NETCore.App",
"version": "5.0.2"
}
}
}
Python script (cat bug.py):
import os
import clr_loader
import pythonnet
runtime_config = os.path.join(os.path.dirname(__file__), "runtimeconfig.json")
dotnet_root = os.path.join(os.path.dirname(__file__), "dotnet_5.0.2")
runtime = clr_loader.get_coreclr(runtime_config=runtime_config, dotnet_root=dotnet_root)
pythonnet.set_runtime(runtime)
pythonnet.load()
import System
from System import String
s = String("Hello World from Python.NET for .NET 5!")
System.Console.WriteLine(s)- What commands did you run to trigger this issue? If you can provide a
Minimal, Complete, and Verifiable example
this will help us understand the issue.
> which python
/home/takacs/src/pythonnet/venv/bin/python
>`which python` bug.py
Hello World from Python.NET for .NET 5!
> python bug.py
Traceback (most recent call last):
File "bug.py", line 12, in <module>
pythonnet.load()
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/pythonnet/__init__.py", line 50, in load
_FFI.dlopen(libpython, posix.RTLD_NODELETE | posix.RTLD_LOCAL)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 150, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 832, in _make_ffi_library
backendlib = _load_backend_lib(backend, libname, flags)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 827, in _load_backend_lib
raise OSError(msg)
OSError: cannot load library '/home/takacs/src/pythonnet/python': /home/takacs/src/pythonnet/python: cannot open shared object file: No such file or directory. Additionally, ctypes.util.find_library() did not manage to locate a library called '/home/takacs/src/pythonnet/python'
- If there was a crash, please include the traceback here.
Traceback (most recent call last):
File "bug.py", line 12, in <module>
pythonnet.load()
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/pythonnet/__init__.py", line 50, in load
_FFI.dlopen(libpython, posix.RTLD_NODELETE | posix.RTLD_LOCAL)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 150, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 832, in _make_ffi_library
backendlib = _load_backend_lib(backend, libname, flags)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 827, in _load_backend_lib
raise OSError(msg)
OSError: cannot load library '/home/takacs/src/pythonnet/python': /home/takacs/src/pythonnet/python: cannot open shared object file: No such file or directory. Additionally, ctypes.util.find_library() did not manage to locate a library called
Analysis
This issue is in the pythonnet.find_libpython module. The dlinfo.dli_fname string contains the actual command that was used to start the Python interpreter. In the case of pure python command, os.path.realpath fails the properly translate this back to the real executable.
Adding a few debug lines properly demonstrate this:
def _linked_libpython_unix():
libdl = ctypes.CDLL(ctypes.util.find_library("dl"))
libdl.dladdr.argtypes = [ctypes.c_void_p, ctypes.POINTER(Dl_info)]
libdl.dladdr.restype = ctypes.c_int
dlinfo = Dl_info()
retcode = libdl.dladdr(
ctypes.cast(ctypes.pythonapi.Py_GetVersion, ctypes.c_void_p),
ctypes.pointer(dlinfo))
if retcode == 0: # means error
return None
print(f"DEBUG dlinfo.dli_fname.decode(): {dlinfo.dli_fname.decode()}")
path = os.path.realpath(dlinfo.dli_fname.decode())
print(f"DEBUG path: {path}")
if path == os.path.realpath(sys.executable):
return None
return path> ./venv/bin/python bug.py
DEBUG dlinfo.dli_fname.decode(): ./venv/bin/python
DEBUG path: /usr/bin/python3.8
Hello World from Python.NET for .NET 5!
> ../pythonnet/venv/bin/python bug.py
DEBUG dlinfo.dli_fname.decode(): ../pythonnet/venv/bin/python
DEBUG path: /usr/bin/python3.8
Hello World from Python.NET for .NET 5!
>python bug.py
DEBUG dlinfo.dli_fname.decode(): python
DEBUG path: /home/takacs/src/pythonnet/python
Traceback (most recent call last):
...
The last case demonstrates that os.path.realpath just extends python with the current path which is obviously incorrect.
As a result, pythonnet.load() method's libpython variable will point to a library which cannot be loaded (
pythonnet/pythonnet/__init__.py
Line 50 in 0f5e781
However, if this part is disabled, the application will crash later in the .NET
Python.Runtime for the same reason.
Activity
[-]New pythonnet.load crashes in virtual environment using symlink in Ubuntu 20.04[/-][+]pythonnet.load crashes if python interpreter is called from PATH Ubuntu 20.04[/+]tibortakacs commentedon Feb 18, 2021
A possible solution in Python 3:
pythonnet/pythonnet/find_libpython/__init__.py
Line 79 in 1ab9cb1
The
path = shutil.which(dlinfo.dli_fname.decode())decodes the PATH-based python executable.If this high-level idea is supported, I am happy to move forward and create a production-ready PR.
[-]pythonnet.load crashes if python interpreter is called from PATH Ubuntu 20.04[/-][+]pythonnet.load crashes if the Python interpreter is called from PATH in Ubuntu 20.04[/+]filmor commentedon Feb 22, 2021
Thank you for your thorough analysis. I'm not quite sure about the fix, though.
whichis in the usual case the wrong tool, as a dynamically linkedlibpythonis not going to be in thePATH. I'll see if I can find out how to do the check fordli_fnameagainstsys.executablemore reliably, but there seems to be little documentation on that.filmor commentedon Feb 22, 2021
This should be fixed by #1396, please verify and reopen if that's not the case :)
tibortakacs commentedon Feb 24, 2021
Thank you for the fast response and fix. I am afraid it does not solve the full issue though, in some sense, it made it worst. I do not have deep knowledge of this, please bear with me, but it seems to me that
Py_ENABLE_SHAREDdoes not serve our need here. It seems that although the Python application was built with--enable-shared, the executable does not always use the shared library.The same topic was mentioned in
conda:conda/conda-build#2738 (comment)
"We do build the shared library, but the Python interpreter does not use it. We build it only for projects that want to embed Python as a shared library."
I did some experiment on my computer that demonstrates this:
System Python in Ubuntu 20.04
As it is shown, the executable does not directly depend on any
libpython*.sobutPy_ENABLE_SHAREDis1.However, it works in my custom builds:
Custom built Python3 without shared flag
This demonstrates that without the
--enabled-sharedflag, noCustom build Python3 with
--enable-sharedflagBased on this, and the
condaexperience, it seems that another solution is required here.@filmor What do you think? (I cannot reopen the issue.)
filmor commentedon Feb 27, 2021
I think that I really hate this mess ;)
I'll create a new PR that compares the
basenames instead, that should do it.filmor commentedon Feb 27, 2021
Could you check whether #1401 finally solves this?
tibortakacs commentedon Mar 2, 2021
Unfortunately, it does not. This is the recommended change:
If
pythonwas called using a command in thePATH, the "basename" of thepathwill be just "python", butsys.executablerefers to the real name of the executable, in my example "python3.8".This is a reason why I believe a "which" type of query is needed in advance. This code works in all my cases (virtual environment, self-build Python with and without shared library):
The modification is this part:
This idea is that if the
dlinfopoints to an executable, it either returns the real path of it (that thewhichcommand would return), or just returns the same path. If it is not an executable (e.g. it is a shared library), it just does the same that the current code does. This might resolve your concern you mentioned above.filmor commentedon Mar 2, 2021
Okay, I'm giving up on this for now, can you create a PR with your change? I'm still not happy with it, but
dladdris just (consistently) broken in this regard and there doesn't seem to be a way to get the realargv[0](which is whatdli_fnamecontains at least with glibc and the BSD libcs) in a running Python process.tibortakacs commentedon Mar 2, 2021
Happy to file a PR. This is my first time contributing to this project, let me ask for a few clarification questions:
pythonnetfrom source, and execute a simple operation with it), or is there a better way/infrastructure for this? Could you give some hints about the testing principles inpythonnet? Thank you in advance.filmor commentedon Mar 2, 2021
Please don't introduce an additional dependency. In this particular case it's also absolutely not required, as
shutil.whichworks fine on Unix (where we use it) and we have dropped Python 2 support a while ago.Regarding tests, I'm not sure. If you can create an additional workflow that creates a venv in this form for one Python version and checks that
pythonnet.loadworks, that is fine. Don't add the case to the existing build matrix.tibortakacs commentedon Mar 2, 2021
Sounds great to me. I have not realized that no more Python 2 support, then
shutilis perfect.Regarding the test, can I ask what does "create an additional workflow" mean?
lostmsu commentedon Mar 2, 2021
@tibortakacs https://docs.github.com/en/actions
For an example, see https://github.com/pythonnet/pythonnet/blob/master/.github/workflows/main.yml
filmor commentedon Mar 3, 2021
Looking a bit more into the messy details of ctypes, on Unix it will unconditionally refer to the main executable for lookup of Python symbols:
https://github.com/python/cpython/blob/master/Lib/ctypes/__init__.py#L459-L464
That means for me that we can simplify this code:
find_libpythonin favour of just vendoring the function to get the path on Windows (linked below)find_libpythonas a test-time dependency to fill our environment variable correctlyhttps://github.com/ktbarrett/find_libpython/blob/master/src/find_libpython/__init__.py#L151-L179
@tibortakacs The test case would still be appreciated, but I currently can't think of any reason why we would need to do this lookup while already running anymore.
tibortakacs commentedon Mar 8, 2021
(Sorry for the delay in my response.)
@lostmsu Thank you for pointing out these docs and pythonnet's workflow, going to read/watch them.
@filmor Just to be sure: you are planning to create a PR following your latest proposal, or I should create a PR regarding this?
ktbarrett commentedon Mar 19, 2021
@filmor See ktbarrett/find_libpython#32 and conda/conda-build#2738.
Py_ENABLED_SHAREDisn't a reliable flag to test, andctypes.pythonapican return a handle to the current Python executable. I suggest vendoring in find_libpython@master, or just add it as a dependency. I have a CI system in place now that should prevent any breaks.filmor commentedon Jun 23, 2022
This should be obsolete.