What’s new in PyPy2.7 5.3

Reduce the size of generated C sources.

Remove a number of options from the build process that were never tested and never set. Fix a performance bug in the method cache.

JIT: use bitstrings to compress the lists of read or written descrs that we attach to EffectInfo. Fixes a problem we had in remove-objspace-options.

Update cpyext C-API support After this branch, we are almost able to support upstream numpy via cpyext, so we created (yet another) fork of numpy at github.com/pypy/numpy with the needed changes. Among the significant changes to cpyext:

  • allow c-snippet tests to be run with -A so we can verify we are compatible

  • fix many edge cases exposed by fixing tests to run with -A

  • issequence() logic matches cpython

  • make PyStringObject and PyUnicodeObject field names compatible with cpython

  • add prelminary support for PyDateTime_*

  • support PyComplexObject, PyFloatObject, PyDict_Merge, PyDictProxy, PyMemoryView_*, _Py_HashDouble, PyFile_AsFile, PyFile_FromFile,

  • PyAnySet_CheckExact, PyUnicode_Concat

  • improve support for PyGILState_Ensure, PyGILState_Release, and thread primitives, also find a case where CPython will allow thread creation before PyEval_InitThreads is run, dissallow on PyPy

  • create a PyObject-specific list strategy

  • rewrite slot assignment for typeobjects

  • improve tracking of PyObject to rpython object mapping

  • support tp_as_{number, sequence, mapping, buffer} slots

(makes the pypy-c bigger; this was fixed subsequently by the share-cpyext-cpython-api branch)

Reduce generated code for subclasses by using the same function objects in all generated subclasses.

CPyExt tweak: instead of “GIL not held when a CPython C extension module calls PyXxx”, we now silently acquire/release the GIL. Helps with CPython C extension modules that call some PyXxx() functions without holding the GIL (arguably, they are theorically buggy).

Get the cpyext tests to pass with “-A” (i.e. when tested directly with CPython).

Compile c snippets with -Werror in cpyext

Add rgc.FinalizerQueue, documented in pypy/doc/discussion/finalizer-order.rst. It is a more flexible way to make RPython finalizers.

Use the new rgc.FinalizerQueue mechanism to clean up the handling of __del__ methods. Fixes notably issue #2287. (All RPython subclasses of W_Root need to use FinalizerQueue now.)

Implement ufunc.outer on numpypy

Support pypy -v: verbose imports. It does not log as much as cpython, but it should be enough to help when debugging package layout problems.

Fix some warnings when compiling CPython C extension modules

Remove most of the _ovf, _zer and _val operations from RPython. Kills quite some code internally, and allows the JIT to do better optimizations: for example, app-level code like x / 2 or x % 2 can now be turned into x >> 1 or x & 1, even if x is possibly negative.

Generalize cpyext old-style buffers to more than just str/buffer, add support for mmap

Move _numpypy headers into a directory so they are not picked up by upstream numpy, scipy This allows building upstream numpy and scipy in pypy via cpyext

Teach RPython JIT’s off-line traceviewer the most common debug_merge_point formats.

Enable pickling of W_PyCFunctionObject by monkeypatching pickle.Pickler.dispatch at cpyext import time

Add a way to ask “give me a raw pointer to this list’s items”. Only for resizable lists of primitives. Turns the GcArray nonmovable, possibly making a copy of it first.

Finish the work already partially merged in cpyext-for-merge. Adds support for ByteArrayObject using the nonmovable-list, which also enables buffer(bytearray(<some-list>))