Repository navigation
Crashes on .NET Core 3 and 2 on Linux #967
Description
Can't make this work on Linux Mint 19 (Ubuntu 18.04).
Compiled ReleaseMonoPy3 with XPLAT;;PYTHON3;PYTHON37;UCS4;MONO_LINUX;PYTHON_WITH_PYMALLOC.
It seems to crash on Py.GIL():
using (Py.GIL())
{
dynamic np = Py.Import("numpy");
Console.WriteLine(np.cos(np.pi * 2));
dynamic sin = np.sin;
Console.WriteLine(sin(5));
double c = np.cos(5) + sin(5);
Console.WriteLine(c);
dynamic a = np.array(new List<float> { 1, 2, 3 });
Console.WriteLine(a.dtype);
dynamic b = np.array(new List<float> { 6, 5, 4 }, dtype: np.int32);
Console.WriteLine(b.dtype);
Console.WriteLine(a * b);
Console.ReadKey();
}
Upon further investigation, it seems to crash in InitializePlatformData(), as
IntPtr platformModule = PyImport_ImportModule("platform"); // Becomes NULL pointer
Running this line immediately after setting platformModule = ...
Console.WriteLine(new PythonException().Message);
yields:
ModuleNotFoundError : No module named '_posixsubprocess'
I also tried making sure the python home and path variables are correctly set:
PythonEngine.PythonHome = @"/usr/local/bin/python3.7";
PythonEngine.PythonPath = @"/usr/local/lib/python3.7";
I have no problem importing _posixsubprocess module when running the python3.7 binary from the terminal.
Activity
filmor commentedon Oct 10, 2019
Please verify your installation. If you already get a
PythonException, that means thatlibpythonhas been found and is loaded, so not finding modules is a pure configuration problem that has nothing to do with Python.NET and would affect any other embedding scenario as well. Check whatpython -c "import sys; print(sys.path)spits out and try setting that as yourPythonPath(separated by:on Unix and;on Windows).Jeff17Robbins commentedon Dec 15, 2019
@filmor I respectfully disagree about this issue having nothing to do with Python.NET. Please see my comment in #891.
I think the potential source of confusion is that the
[DllImport]that finds and loadslibpythondoes not set theRTLD_GLOBALflag on itsdlopen()call. And, furthermore, some Linuxes like Debian explicitly "does not link extensions to libpython".[I assume that this is why
runtime.cstakes such trouble to find out theOperatingSystemNameandMachineNameinInitializePlatformData, precisely in order to explictly loadlibpythonwith saidRTLD_GLOBALflag?]So when
platform.pychanged to usingsubprocess.py, which on Linux attempts to import a C extension library_posixsubprocess, there's a crash.This crash is, I think, precisely due to Python.NET running C Python code (importing
platform.py) in a"semi-initialized state" inside itsInitializebefore it has done that explicit Load oflibpythonwithRTLD_GLOBAL.I don't think this crash reflects a configuration problem. I think this crash reflects on why
runtime.csneeds to reloadlibpythonin the first place!For .NET Core 3.x, we could run this code instead of using
platform.py, and get our required strings quicker, without this crash. For example, with code like this (tested on Debian 10 .NET Core 3.1):PS, I believe this analysis explains issue #946 also.
filmor commentedon Dec 15, 2019
@Jeff17Robbins Well, Python.NET didn't decide to default to
RTLD_LOCALon .NET Core inDllImport(which I think is an unfortunate choice).Nevertheless, you are completely right that this whole
InitializePlatformDatadance can be avoided by usingRuntimeInformation(I think there is even a comment in our code that we could do that), but for this we need to "officially" update to .NET >4.7.1. There is a ticket for this (#897) and we'll decide this Tuesday, but as far as I can see everyone is in favour.We wouldn't even need to assign our own values here anymore, we can just use the
RuntimeInformationdata directly.Jeff17Robbins commentedon Dec 16, 2019
@filmor Conceivably, for .NET Core 3.1 anyway, we could override that unfortunate choice using System.Runtime.InteropServices.DllImportResolver. I'm not clear on how to get a call to this to run before all those
[DllImport]s, but if you have any ideas about that, I'd be happy to give it a try. Theresolvercan calldlopenwith theRTLD_GLOBALflag.It's not a solution for .NET Framework 4.7.x.
But it could tidy things up for .NET Core on Linux and remove the need for the second call to
dlopen.