Access Violation when using 32bit version of python

Environment

Pythonnet version: 3.0.0-preview2021-12-25
Python version: 3.9.10
Operating System: Windows 10
.NET Runtime: 4.7.2

Details

Describe what you were trying to get done.

Embed python into C# using 32bit python.

What commands did you run to trigger this issue? If you can provide a
Minimal, Complete, and Verifiable example
this will help us understand the issue.

Example assumes python3.9.10 32bit embed to be extracted to ./python

Python.Runtime.Runtime.PythonDLL = "./python/python39.dll";

Python.Runtime.PythonEngine.PythonHome = ./python/python39.dll;
Python.Runtime.PythonEngine.PythonPath = $"{PythonEngine.PythonPath}{Path.PathSeparator}.";

Python.Runtime.PythonEngine.Initialize();

Last line causes an access violation after preview2021-12-25 (preview2021-12-03 works).
python3.9.10 64bit embed version of python does not cause the same issue.

If there was a crash, please include the traceback here.

   System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Python.Runtime.Runtime.PyType_FromSpecWithBases(NativeTypeSpec& spec, BorrowedReference bases)
   at Python.Runtime.PyType.FromSpec(TypeSpec spec, PyTuple bases)
   at Python.Runtime.TypeManager.CreateMetatypeWithGCHandleOffset()
   at Python.Runtime.TypeManager.CreateMetaType(Type impl, SlotsHolder& slotsHolder)
   at Python.Runtime.MetaType.Initialize()
   at Python.Runtime.Runtime.Initialize(Boolean initSigs)
   at Python.Runtime.PythonEngine.Initialize(IEnumerable`1 args, Boolean setSysArgv, Boolean initSigs)
   at Python.Runtime.PythonEngine.Initialize()

I am also interested in hearing about whether there are plans to continue to support 32-bit Python (e.g., issue #1210).

Hopefully the following can help track down a fix to this issue with 32-bit Python (on Windows). It appears that commit 53836de is the culprit. My criteria for deciding what commit caused this memory error was simply whether import clr succeeded or failed after installing a particular commit.

The commands below demonstrate when the issue appeared (note that dfeb354 is the parent commit to 53836de)

C:\Users\username>python -VV
Python 3.9.11 (tags/v3.9.11:2de452f, Mar 16 2022, 14:21:37) [MSC v.1929 32 bit (Intel)]

C:\Users\username>python -m venv venv-importable

C:\Users\username>python -m venv venv-memory-error

C:\Users\username>venv-importable\Scripts\activate

(venv-importable) C:\Users\username>pip install git+https://github.com/pythonnet/pythonnet@dfeb354de6c396afe3b44d4905c10fbc6445531e
Collecting git+https://github.com/pythonnet/pythonnet@dfeb354de6c396afe3b44d4905c10fbc6445531e
  Cloning https://github.com/pythonnet/pythonnet (to revision dfeb354de6c396afe3b44d4905c10fbc6445531e) to c:\users\username\appdata\local\temp\pip-req-build-ztlwmikh
  Running command git clone --filter=blob:none --quiet https://github.com/pythonnet/pythonnet 'C:\Users\username\AppData\Local\Temp\pip-req-build-ztlwmikh'
  Running command git rev-parse -q --verify 'sha^dfeb354de6c396afe3b44d4905c10fbc6445531e'
  Running command git fetch -q https://github.com/pythonnet/pythonnet dfeb354de6c396afe3b44d4905c10fbc6445531e
  Running command git checkout -q dfeb354de6c396afe3b44d4905c10fbc6445531e
  Resolved https://github.com/pythonnet/pythonnet to commit dfeb354de6c396afe3b44d4905c10fbc6445531e
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting clr-loader>=0.1.7
  Using cached clr_loader-0.1.7-py3-none-any.whl (45 kB)
Collecting cffi>=1.13
  Using cached cffi-1.15.0-cp39-cp39-win32.whl (167 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Building wheels for collected packages: pythonnet
  Building wheel for pythonnet (pyproject.toml) ... done
  Created wheel for pythonnet: filename=pythonnet-3.0.0.dev1-py3-none-any.whl size=252204 sha256=a0c37002a45b159d039d69f51090b594d011640c00f26cadd5ea6a84ef8dc99a
  Stored in directory: c:\users\username\appdata\local\pip\cache\wheels\9a\05\8f\6c115b72abdc7a601fc6a5acd89fdd34c1fc7f5c7371d48fa6
Successfully built pythonnet
Installing collected packages: pycparser, cffi, clr-loader, pythonnet
Successfully installed cffi-1.15.0 clr-loader-0.1.7 pycparser-2.21 pythonnet-3.0.0.dev1

(venv-importable) C:\Users\username>python -c "import clr"

(venv-importable) C:\Users\username>deactivate

C:\Users\username>venv-memory-error\Scripts\activate

(venv-memory-error) C:\Users\username>pip install git+https://github.com/pythonnet/pythonnet@53836defd715d30e7aec95a4c1d724e843988c17
Collecting git+https://github.com/pythonnet/pythonnet@53836defd715d30e7aec95a4c1d724e843988c17
  Cloning https://github.com/pythonnet/pythonnet (to revision 53836defd715d30e7aec95a4c1d724e843988c17) to c:\users\username\appdata\local\temp\pip-req-build-i89wqcl_
  Running command git clone --filter=blob:none --quiet https://github.com/pythonnet/pythonnet 'C:\Users\username\AppData\Local\Temp\pip-req-build-i89wqcl_'
  Running command git rev-parse -q --verify 'sha^53836defd715d30e7aec95a4c1d724e843988c17'
  Running command git fetch -q https://github.com/pythonnet/pythonnet 53836defd715d30e7aec95a4c1d724e843988c17
  Running command git checkout -q 53836defd715d30e7aec95a4c1d724e843988c17
  Resolved https://github.com/pythonnet/pythonnet to commit 53836defd715d30e7aec95a4c1d724e843988c17
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting clr-loader>=0.1.7
  Using cached clr_loader-0.1.7-py3-none-any.whl (45 kB)
Collecting cffi>=1.13
  Using cached cffi-1.15.0-cp39-cp39-win32.whl (167 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Building wheels for collected packages: pythonnet
  Building wheel for pythonnet (pyproject.toml) ... done
  Created wheel for pythonnet: filename=pythonnet-3.0.0.dev1-py3-none-any.whl size=252095 sha256=03ba2b123b79b477a2139bc3bb51bdcbcda51b51b34c87e87b31681ded5d8603
  Stored in directory: c:\users\username\appdata\local\pip\cache\wheels\0b\25\a1\ca634d83fc4e35c137300efeaca0a37ff5d8dcfcdc0ddc9c00
Successfully built pythonnet
Installing collected packages: pycparser, cffi, clr-loader, pythonnet
Successfully installed cffi-1.15.0 clr-loader-0.1.7 pycparser-2.21 pythonnet-3.0.0.dev1

(venv-memory-error) C:\Users\username>python -c "import clr"

Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Python.Runtime.Runtime.PyType_FromSpecWithBases(NativeTypeSpec& spec, BorrowedReference bases)
   at Python.Runtime.PyType.FromSpec(TypeSpec spec, PyTuple bases)
   at Python.Runtime.TypeManager.CreateMetatypeWithGCHandleOffset()
   at Python.Runtime.TypeManager.CreateMetaType(Type impl, SlotsHolder& slotsHolder)
   at Python.Runtime.MetaType.Initialize()
   at Python.Runtime.Runtime.Initialize(Boolean initSigs)
   at Python.Runtime.PythonEngine.Initialize(IEnumerable`1 args, Boolean setSysArgv, Boolean initSigs)
   at Python.Runtime.PythonEngine.InitExt()
   at Python.Runtime.Loader.Initialize(IntPtr data, Int32 size)

I really would like to support it on both Windows and Linux.

On Linux it's a bit tricky because the dotnet toolchain only works on 64bit, so one has to always shuffle between those two user-spaces for debugging and building (I don't have a multilib setup right now, so I'm using a 32bit docker for the debugging). Also, I'm really stuck on this #1210 (comment), I haven't had the time to check with Mono upstream (and .NET Core is not supported on 32bit Linux).

For Windows (which I think has a slightly different issue), I dropped the 32bit CI in #1591 while dropping AppVeyor and didn't bring it back as a Github action, I'll try to fix that now.

So, in summary:

I would like to support x86-32 on both Windows and Linux
I'll add 32bit CI for Windows (for Linux it's a bit trickier as I'll need to run the build on 64bit and then use the artifacts for the tests, I've started work on this a while ago in Separate building from testing #1616)
Regarding Linux, I'm stuck
Regarding Windows, I haven't looked at it, yet, assuming this should've been caught by CI. Thanks for the bisect, I'll have a look at what changed concretely. That PR (Removed ShutdownMode. Now always behaves like original Reload #1638) was supposed to just drop optional behaviour.

Thank you for providing a detailed update. I can appreciate the complexities involved, especially with the lack of dotnet toolchain support on 32-bit Linux.

@lostmsu

@lostmsu If you have a Windows machine to test on, could you have a look at this? I don't think it makes sense to block the release for Linux 32bit, but Windows 32bit support only got lost because I messed up the CI, it's something that we should fix.

Navigation Menu

Repository navigation

Access Violation when using 32bit version of python #1729

Environment

Details

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Access Violation when using 32bit version of python #1729

Description

Environment

Details

Activity

jborbely commented on Mar 24, 2022

filmor commented on Mar 24, 2022

jborbely commented on Mar 24, 2022

filmor commented on Apr 10, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions