================
Cython Changelog
================

3.0.11 (2024-08-05)
===================

Features added
--------------

* The C++11 ``emplace*`` methods were added to ``libcpp.deque``.
  Patch by Somin An.  (Github issue :issue:`6159`)

Bugs fixed
----------

* The exception check value of functions declared in pxd files was not always applied in 3.0.10.
  (Github issue :issue:`6122`)

* A crash on exception deallocations was fixed.
  (Github issue :issue:`6022`)

* A crash was fixed when assigning a zero-length slice to a memoryview.
  Patch by Michael Man.  (Github issue :issue:`6227`)

* ``libcpp.optional.value()`` could crash if it raised a C++ exception.
  Patch by Alexander Condello.  (Github issue :issue:`6190`)

* The return type of ``str()`` was mishandled, leading to crashes with ``language_level=3``.
  (Github issue :issue:`6166`)

* ``bytes.startswith/endswith()`` failed for non-bytes substrings (e.g. ``bytearray``).
  (Github issue :issue:`6168`)

* Fused ctuples crashed Cython.
  (Github issue :issue:`6068`)

* A compiler crash was fixed when using extension types in fused types.
  (Github issue :issue:`6204`)

* The module cleanup code was incorrect for globally defined memory view slices.
  (Github issue :issue:`6276`)

* Some adaptations were made to enable compilation in Python 3.13.
  (Github issues :issue:`5997`, :issue:`6182`, :issue:`6251`)


3.0.10 (2024-03-30)
===================

Bugs fixed
----------

* Cython generated incorrect self-casts when directly calling final methods of subtypes.
  Patch by Lisandro Dalcin.  (Github issue :issue:`2747`)

* Internal C names generated from C function signatures could become too long for MSVC.
  (Github issue :issue:`6052`)

* The ``noexcept`` warnings could be misleading in some cases.
  Patch by Gonzalo Tornaría.  (Github issue :issue:`6087`)

* The ``@cython.ufunc`` implementation could generate incomplete C code.
  (Github issue :issue:`6064`)

* The ``libcpp.complex`` declarations could result in incorrect C++ code.
  Patch by Raffi Enficiaud.  (Github issue :issue:`6037`)

* Several tests were adapted to work with both NumPy 1.x and 2.0.
  Patch by Matti Picus.  (Github issues :issue:`6076`, :issue:`6100`)

* C compiler warnings when the freelist implementation is disabled (e.g. on PyPy) were fixed.
  It can now be disabled explicitly with the C macro guard ``CYTHON_USE_FREELISTS=0``.
  (Github issue :issue:`6099`)

* Some C macro guards for feature flags were missing from the NOGIL Python configuration.

* Some recently added builtins were unconditionally looked up at module import time
  (if used by user code) that weren't available on all Python versions and could thus
  fail the import.

* A performance hint regarding exported pxd declarations was improved.
  (Github issue :issue:`6001`)


3.0.9 (2024-03-05)
==================

Features added
--------------

* Assigning ``const`` values to non-const variables now issues a warning.
  (Github issue :issue:`5639`)

* Using ``noexcept`` on a function returning Python objects now issues a warning.
  (Github issue :issue:`5661`)

* Some C-API usage was updated for the upcoming CPython 3.13.
  Patches by Victor Stinner et al.  (Github issues :issue:`6003`, :issue:`6020`)

* The deprecated ``Py_UNICODE`` type is no longer used, unless required by user code.
  (Github issue :issue:`5982`)

* ``std::string.replace()`` declarations were added to libcpp.string.
  Patch by Kieran Geary.  (Github issue :issue:`6037`)

Bugs fixed
----------

* Cython generates incorrect (but harmless) self-casts when directly calling
  final methods of subtypes.  Lacking a better solution, the errors that recent
  gcc versions produce have been silenced for the time being.
  Original patch by Michał Górny.  (Github issue :issue:`2747`)

* Unused variable warnings about clineno were fixed when C lines in tracebacks are disabled.
  (Github issue :issue:`6035`)

* Subclass deallocation of extern classes could crash if the base class uses GC.
  Original patch by Jason Fried.  (Github issue :issue:`5971`)

* Type checks for Python ``memoryview`` could use an invalid C function.
  Patch by Xenia Lu.  (Github issue :issue:`5988`)

* Calling final fused functions could generate invalid C code.
  (Github issue :issue:`5989`)

* Declaring extern enums multiple times could generate invalid C code.
  (Github issue :issue:`5905`)

* ``pyximport`` used relative paths incorrectly.
  Patch by Stefano Rivera.  (Github issue :issue:`5957`)

* Running Cython with globbing characters (``[]*?``) in the module search path could fail.
  Patch by eewanco.  (Github issue :issue:`5942`)

* Literal strings that include braces could change the C code indentation.

Other changes
-------------

* The "enum class not importable" warning is now only issued once per enum type.
  (Github issue :issue:`5941`)


3.0.8 (2024-01-10)
==================

Bugs fixed
----------

* Using ``const`` together with defined fused types could fail to compile.
  (Github issue :issue:`5230`)

* A "use after free" bug was fixed in parallel sections.
  (Github issue :issue:`5922`)

* Several types were not available as ``cython.*`` types in pure Python code.

* The generated code is now correct C89 again, removing some C++ style ``//`` comments
  and C99-style declaration-after-code code ordering.  This is still relevant for some
  ols C compilers, specifically ones that match old Python 2.7 installations.


3.0.7 (2023-12-19)
==================

Bugs fixed
----------

* In the iterator of generator expressions, ``await`` and ``yield`` were not correctly analysed.
  (Github issue :issue:`5851`)

* ``cpdef`` enums with the same name cimported from different modules could lead to
  invalid C code.
  (Github issue :issue:`5887`)

* Some declarations in ``cpython.unicode`` were fixed and extended.
  (Github issue :issue:`5902`)

* Compiling fused types used in pxd files could crash Cython in Python 3.11+.
  (Github issues :issue:`5894`,  :issue:`5588`)

* Source files with non-ASCII file names could crash Cython.
  (Github issue :issue:`5873`)

* Includes all bug-fixes and features from the 0.29 maintenance branch
  up to the :ref:`0.29.37` release.


3.0.6 (2023-11-26)
==================

Features added
--------------

* Fused def function dispatch is a bit faster.

* Declarations for the ``wchar`` PyUnicode API were added.
  (Github issue :issue:`5836`)

* The Python "nogil" fork is now also detected with the new ``Py_GIL_DISABLED`` macro.
  Patch by Hugo van Kemenade.  (Github issue :issue:`5852`)

Bugs fixed
----------

* Comparing dataclasses could give different results than Python.
  (Github issue :issue:`5857`)

* ``float(std::string)`` generated invalid C code.
  (Github issue :issue:`5818`)

* Using ``cpdef`` functions with ``cimport_from_pyx`` failed.
  (Github issue :issue:`5795`)

* A crash was fixed when string-formatting a Python value fails.
  (Github issue :issue:`5787`)

* On item access, Cython could try the sequence protocol before the mapping protocol
  in some cases if an object supports both.
  (Github issue :issue:`5776`)

* A C compiler warning was resolved.
  (Github issue :issue:`5794`)

* Complex numbers failed to compile in MSVC with C11.
  Patch by Lysandros Nikolaou.  (Github issue :issue:`5809`)

* Some issues with the Limited API and with PyPy were resolved.
  (Github issues :issue:`5695`, :issue:`5696`)

* A C++ issue in Python 3.13 was resolved.
  (Github issue :issue:`5790`)

* Several directives are now also available (as no-ops) in Python code.
  (Github issue :issue:`5803`)

* An error message was corrected.
  Patch by Mads Ynddal.  (Github issue :issue:`5805`)


3.0.5 (2023-10-31)
==================

Features added
--------------

* Preliminary support for CPython 3.13a1 was added to allow early testing.
  (Github issue :issue:`5767`)

Bugs fixed
----------

* A compiler crash was fixed.
  (Github issue :issue:`5771`)

* A typo in the ``always_allow_keywords`` directive for Python code was fixed.
  Patch by lk-1984.  (Github issue :issue:`5772`)

* Some C compiler warnings were resolved.
  Patch by Pierre Jolivet.  (Github issue :issue:`5780`)


3.0.4 (2023-10-17)
==================

Features added
--------------

* A new compiler directive ``show_performance_hints`` was added to disable the
  newly added performance hint output.
  (Github issue :issue:`5748`)

Bugs fixed
----------

* ```cythonize` required ``distutils`` even for operations that did not build binaries.
  (Github issue :issue:`5751`)

* A regression in 3.0.3 was fixed that prevented calling inline functions
  from another inline function in ``.pxd`` files.
  (Github issue :issue:`5748`)

* Some C compiler warnings were resolved.
  Patch by Pierre Jolivet.  (Github issue :issue:`5756`)


3.0.3 (2023-10-05)
==================

Features added
--------------

* More warnings were added to help users migrate and avoid bugs.
  (Github issue :issue:`5650`)

* A warning-like category for performance hints was added that bypasses ``-Werror``.
  (Github issue :issue:`5673`)

* FastGIL now uses standard ``thread_local`` in C++.
  (Github issue :issue:`5640`)

* ``reference_wrapper`` was added to ``libcpp.functional``.
  Patch by Vyas Ramasubramani.  (Github issue :issue:`5671`)

* The ``cythonize`` command now supports the ``--cplus`` option known from the ``cython`` command.
  (Github issue :issue:`5736`)

Bugs fixed
----------

* Performance regressions where the GIL was needlessly acquired were fixed.
  (Github issues :issue:`5670`, :issue:`5700`)

* A reference leak for exceptions in Python 3.12 was resolved.
  Patch by Eric Johnson.  (Github issue :issue:`5724`)

* ``fastcall`` calls with keyword arguments generated incorrect C code.
  (Github issue :issue:`5665`)

* Assigning the type converted result of a conditional (if-else) expression
  to ``int`` or ``bool`` variables could lead to incorrect C code.
  (Github issue :issue:`5731`)

* Early (unlikely) failures in Python function wrappers no longer set a
  traceback in order to simplify the C code flow.  Being mostly memory
  allocation errors, they probably would never have created a traceback anyway.
  (Github issue :issue:`5681`)

* Relative cimports from packages with ``__init__.py`` files could fail.
  (Github issue :issue:`5715`)

* Several issues with the Limited API support were resolved.
  (Github issues :issue:`5641`, :issue:`5648`, :issue:`5689`)

* The code generated for special-casing both Cython functions and PyCFunctions was cleaned up
  to avoid calling C-API functions that were not meant for the other type respectively.
  This could previously trigger assertions in CPython debug builds and now also plays better
  with the Limited API.
  (Github issues :issue:`4804`, :issue:`5739`)

* Fix some C compiler warnings.
  Patches by Ralf Gommers, Oleksandr Pavlyk, Sebastian Koslowski et al.
  (Github issues :issue:`5651`, :issue:`5663`, :issue:`5668`, :issue:`5717`, :issue:`5726`, :issue:`5734`)

* Generating gdb debugging information failed when using generator expressions.
  Patch by Oleksandr Pavlyk.  (Github issue :issue:`5552`)

* Passing a ``setuptools.Extension`` into ``cythonize()`` instead of a
  ``distutils.Extension`` could make it miss the matching extensions.

* ``cython -M`` needlessly required ``distutils``, which made it fail in Python 3.12.
  (Github issue :issue:`5681`)

Other changes
-------------

* The visible deprecation warning for ``DEF`` was removed again since it proved
  difficult for some users to migrate away from it.  The statement is still
  meant to be removed at some point (and thus, like ``IF``, should not be
  used in new code), but the time for sunset is probably not around the corner.
  (Github issue :issue:`4310`)

* The ``np_pythran`` option raise a ``DeprecationWarning`` if it receives other values
  than ``True`` and ``False``.  This will eventually be disallowed (in line with all
  other boolean options).


3.0.2 (2023-08-27)
==================

Bugs fixed
----------

* Using ``None`` as default value for arguments annotated as ``int`` could crash Cython.
  (Github issue :issue:`5643`)

* Default values of fused types that include ``complex`` could generate invalid C code
  with ``-DCYTHON_CCOMPLEX=0``.
  (Github issue :issue:`5644`)

* Using C++ enum class types in extension type method signatures could generate invalid C code.
  (Github issue :issue:`5637`)


3.0.1 (2023-08-25)
==================

Features added
--------------

* The error messages regarding exception declarations were improved in order to give
  better help about possible reasons and fixes.
  (Github issue :issue:`5547`)

Bugs fixed
----------

* Memory view types in Python argument annotations no longer accept ``None``.  They now
  require an explicit ``Optional[]`` or a ``None`` default value in order to allow ``None``
  to be passed.  This was an oversight in the 3.0.0 release and is a BACKWARDS INCOMPATIBLE
  change.  However, since it only applies to code using Python syntax, it probably only
  applies to newly written code that was written for Cython 3.0 and can easily be adapted.
  In most cases, we expect that this change will avoid bugs in user code rather than
  produce problems.
  (Github issue :issue:`5612`)

* ``nogil`` functions using parallel code could freeze when called with the GIL held.
  (Github issues :issue:`5564`, :issue:`5573`)

* Relative cimports could end up searching globally and find the same package installed
  elsewhere, potentially in another version.
  (Github issue :issue:`5511`)

* Attribute lookups on known standard library modules could accidentally search
  in the module namespace instead.
  (Github issue :issue:`5536`)

* Using constructed C++ default arguments could generate invalid C++ code.
  (Github issue :issue:`5553`)

* ``libcpp.memory.make_unique()`` was lacking C++ exception handling.
  (Github issue :issue:`5560`)

* Some non-public and deprecated CAPI usages were replaced by public
  (and thus more future proof) API code.

* Many issues with the Limited API support were resolved.
  Patches by Lisandro Dalcin et al.
  (Github issues :issue:`5549`, :issue:`5550`, :issue:`5556`, :issue:`5605`, :issue:`5617`)

* Some C compiler warnings were resolved.
  Patches by Matti Picus et al.  (Github issues :issue:`5557`, :issue:`5555`)

* Large Python integers are now stored in hex instead of decimal strings to work around
  security limits in Python and generally speed up their Python object creation.

* ``NULL`` could not be used as default for fused type pointer arguments.
  (Github issue :issue:`5554`)

* C functions that return pointer types now return ``NULL`` as default exception value.
  Previously, calling code wasn't aware of this and always tested for raised exceptions.
  (Github issue :issue:`5554`)

* Untyped literal default arguments in fused functions could generate invalid C code.
  (Github issue :issue:`5614`)

* C variables declared as ``const`` could generate invalid C code when used in closures,
  generator expressions, ctuples, etc.
  (Github issues :issue:`5558`,  :issue:`5333`)

* Enums could not refer to previously defined enums in their definition.
  (Github issue :issue:`5602`)

* The Python conversion code for anonymous C enums conflicted with regular int conversion.
  (Github issue :issue:`5623`)

* Using memory views for property methods (and other special methods) could lead to
  refcounting problems.
  (Github issue :issue:`5571`)

* Star-imports could generate code that tried to assign to constant C macros like
  ``PY_SSIZE_T_MAX`` and ``PY_SSIZE_T_MIN``.
  Patch by Philipp Wagner.  (Github issue :issue:`5562`)

* ``CYTHON_USE_TYPE_SPECS`` can now be (explicitly) enabled in PyPy.

* The template parameter "delimeters" in the Tempita ``Template`` class was corrected
  to "delimiters".  The old spelling is still available in the main template API but
  now issues a ``DeprecationWarning``.
  (Github issue :issue:`5608`)

* The ``cython --version`` output is now  less likely to reach both stdout and stderr.
  Patch by Eli Schwartz.  (Github issue :issue:`5504`)

* The sdist was missing the `Shadow.pyi` stub file.


3.0.0 unified release notes
===========================

Cython 3.0.0 has been a very large effort that cleaned up many old warts,
introduced many new features, and introduces a couple of intentional
behaviour changes, even though the goal remained to stay compatible as
much as possible with Cython 0.29.x. For details, see the `migration guide`_.

.. _`migration guide`: https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html

As the development was spread out over several years, a lot of things have
happened in the meantime. Many crucial bugfixes and some features were
backported to 0.29.x and are not strictly speaking "new" in Cython 3.0.0.

Major themes in 3.0.0
=====================

Compatibility with CPython and the Python C API
-----------------------------------------------

Since Cython 3.0.0 started development, CPython 3.8-3.11 were released.
All these are supported in Cython, including experimental support for the
in-development CPython 3.12. On the other end of the spectrum, support for
Python 2.6 was dropped.

Cython interacts very closely with the C-API of Python, which is where most
of the adaptation work happens.


Related changes
^^^^^^^^^^^^^^^

* The long deprecated include files ``python_*``, ``stdio``, ``stdlib`` and
  ``stl`` in ``Cython/Includes/Deprecated/`` were removed.  Use the ``libc.*``
  and ``cpython.*`` pxd modules instead.
  Patch by Jeroen Demeyer.  (Github issue :issue:`2904`)

* The ``Py_hash_t`` type failed to accept arbitrary "index" values.
  (Github issue :issue:`2752`)

* ``@cython.trashcan(True)`` can be used on an extension type to enable the
  CPython :ref:`trashcan`. This allows deallocating deeply recursive objects
  without overflowing the stack. Patch by Jeroen Demeyer.  (Github issue :issue:`2842`)

* ``PyEval_InitThreads()`` is no longer used in Py3.7+ where it is a no-op.

* A low-level inline function ``total_seconds(timedelta)`` was added to
  ``cpython.datetime`` to bypass the Python method call.  Note that this function
  is not guaranteed to give exactly the same results for very large time intervals.
  Patch by Brock Mendel.  (Github issue :issue:`3616`)

* The internal CPython macro ``Py_ISSPACE()`` is no longer used.
  Original patch by Andrew Jones.  (Github issue :issue:`4111`)

* The value ``PyBUF_MAX_NDIM`` was added to the ``cpython.buffer`` module.
  Patch by John Kirkham.  (Github issue :issue:`3811`)

* A new module ``cpython.time`` was added with some low-level alternatives to
  Python's ``time`` module.
  Patch by Brock Mendel.  (Github issue :issue:`3767`)

* More C-API declarations for ``cpython.datetime``  were added.
  Patch by Bluenix2.  (Github issue :issue:`4128`)

* C-API declarations for context variables in Python 3.7 were added.
  Original patch by Zolisa Bleki.  (Github issue :issue:`2281`)

* C-API declarations for ``cpython.fileobject`` were added.
  Patch by Zackery Spytz.  (Github issue :issue:`3906`)

* The signature of ``PyFloat_FromString()`` in ``cpython.float`` was changed
  to match the signature in Py3.  It still has an automatic fallback for Py2.
  (Github issue :issue:`3909`)

* ``PyMem_[Raw]Calloc()`` was added to the ``cpython.mem`` declarations.
  Note that the ``Raw`` versions are no longer #defined by Cython.  The previous
  macros were not considered safe.
  Patch by William Schwartz and David Woods.  (Github issue :issue:`3047`)

* The runtime size check for imported ``PyVarObject`` types was improved
  to reduce false positives and adapt to Python 3.11.
  Patch by David Woods.  (Github issues :issue:`4827`, :issue:`4894`)

* The generated C code failed to compile in CPython 3.11a4 and later.
  (Github issue :issue:`4500`)

* ``pyximport`` no longer uses the deprecated ``imp`` module.
  Patch by Matúš Valo.  (Github issue :issue:`4560`)

* Improvements to ``PyTypeObject`` definitions in pxd wrapping of libpython.
  Patch by John Kirkham. (Github issue :issue:`4699`)

* Some old usages of the deprecated Python ``imp`` module were replaced with ``importlib``.
  Patch by Matúš Valo.  (Github issue :issue:`4640`)

* ``cpdef`` enums no longer use ``OrderedDict`` but ``dict`` in Python 3.6 and later.
  Patch by GalaxySnail.  (Github issue :issue:`5180`)

* Several problems with CPython 3.12 were resolved.
  (Github issue :issue:`5238`)

* The exception handling code was adapted to CPython 3.12.
  (Github issue :issue:`5442`)

* The Python ``int`` handling code was adapted to make use of the new ``PyLong``
  internals in CPython 3.12.
  (Github issue :issue:`5353`)

* A compile error when using ``__debug__`` was resolved.

* The deprecated ``_PyGC_FINALIZED()`` C-API macro is no longer used.
  Patch by Thomas Caswell and Matúš Valo.  (Github issue :issue:`5481`)

* A crash in Python 2.7 was fixed when cleaning up extension type instances
  at program end.


Compatibility with other Python implementations
-----------------------------------------------

Cython tries to support other Python implementations, largely on a best-effort
basis. The most advanced support exists for PyPy, which is tested in our CI
and considered supported.

Related changes
^^^^^^^^^^^^^^^

* An unsupported C-API call in PyPy was fixed.
  Patch by Max Bachmann.  (Github issue :issue:`4055`)

* Support for the now unsupported Pyston V1 was removed in favour of Pyston V2.
  Patch by Marius Wachtler.  (Github issue :issue:`4211`)

* A C compiler warning in PyPy3 regarding ``PyEval_EvalCode()`` was resolved.

* Some compatibility issues with PyPy were resolved.
  Patches by Max Bachmann, Matti Picus.
  (Github issues :issue:`4454`, :issue:`4477`, :issue:`4478`, :issue:`4509`, :issue:`4517`)

* An initial set of adaptations for GraalVM Python was implemented.  Note that
  this does not imply any general support for this target or that your code
  will work at all in this environment.  But testing should be possible now.
  Patch by David Woods.  (Github issue :issue:`4328`)

* A work-around for StacklessPython < 3.8 was disabled in Py3.8 and later.
  (Github issue :issue:`4329`)


Initial support for Limited API
-------------------------------

CPython provides a stable, limited subset of its C-API as the so-called Limited API.
This C-API comes with the guarantee of a stable ABI, meaning that extensions modules
that were compiled for one version of CPython can also be imported in later versions
without recompilation.

There is initial support for this in Cython.  By defining the ``CYTHON_LIMITED_API``
macro, Cython cuts down its C-API usage and tries to adhere to the Limited C-API,
probably at the cost of a bit of performance.
In order to get full benefit from the limited API you will also need to define the
CPython macro ``Py_LIMITED_API`` to a specific CPython compatibility version,
which additionally restricts the C-API during the C compilation,
thus enforcing the forward compatibility of the extension module.

Note that "initial support" in Cython really means that setting the ``Py_LIMITED_API``
macro will almost certainly not yet work for your specific code.
There are limitations in the Limited C-API
that are difficult for Cython to generate C code for, so some advanced Python features
(like async code) may not lead to C code that cannot adhere to the Limited C-API, or
where Cython simply does not know yet how to adhere to it.  Basically, if you get your
code to compile with both macros set, and it passes your test suite, then it should be
possible to import the extension module also in later CPython versions.

The experimental feature flags ``CYTHON_USE_MODULE_STATE`` and
``CYTHON_USE_TYPE_SPECS`` enable some individual aspects of the Limited API
implementation independently.

Related changes
^^^^^^^^^^^^^^^

* Preliminary support for the CPython's ``Py_LIMITED_API`` (stable ABI) is
  available by setting the  ``CYTHON_LIMITED_API`` C macro.  Note that the
  support is currently in an early stage and many features do not yet work.
  You currently still have to define ``Py_LIMITED_API`` externally in order
  to restrict the API usage.  This will change when the feature stabilises.
  Patches by Eddie Elizondo and David Woods.  (Github issues :issue:`3223`,
  :issue:`3311`, :issue:`3501`)

* Limited API support was improved.
  Patches by Matthias Braun.  (Github issues :issue:`3693`, :issue:`3707`)

* New C feature flags: ``CYTHON_USE_MODULE_STATE``, ``CYTHON_USE_TYPE_SPECS``
  Both are currently considered experimental.
  (Github issue :issue:`3611`)

* ``_Py_TPFLAGS_HAVE_VECTORCALL`` was always set on extension types when using the limited API.
  Patch by David Woods.  (Github issue :issue:`4453`)

* Limited API C preprocessor warning is compatible with MSVC. Patch by
  Victor Molina Garcia.  (Github issue :issue:`4826`)

* The embedding code no longer calls deprecated C-API functions but uses the new ``PyConfig``
  API instead on CPython versions that support it (3.8+).
  Patch by Alexander Shadchin.  (Github issue :issue:`4895`)

* Some C code issue were resolved for the Limited API target.
  (Github issues :issue:`5264`, :issue:`5265`, :issue:`5266`)

* Conversion of Python ints to C ``int128`` is now always supported, although slow
  if dedicated C-API support is missing (``_PyLong_AsByteArray()``), specifically in
  the Limited C-API.
  (Github issue :issue:`5419`)

* Custom buffer slot methods are now supported in the Limited C-API of Python 3.9+.
  Patch by Lisandro Dalcin.  (Github issue :issue:`5422`)


Improved fidelity to Python semantics
-------------------------------------

Implemented PEPs
^^^^^^^^^^^^^^^^

* `PEP-3131`_: Supporting Non-ASCII Identifiers (Github issue :issue:`2601`)
* `PEP-479`_: `generator_stop` (enabled by default for `language_level=3`) (Github issue :issue:`2580`)
* `PEP-487`_: Simpler customisation of class creation (Github issue :issue:`2781`)
* `PEP-563`_: Postponed Evaluation of Annotations (Github issue :issue:`3285`)
* `PEP-570`_: Positional-Only Parameters (Github issue :issue:`2915`)
* `PEP-572`_: Assignment Expressions (a.k.a. the walrus operator `:=`) (Github issue :issue:`2636`)
* `PEP-590`_: Vectorcall protocol (Github issue :issue:`2263`)
* `PEP-614`_: Relaxing Grammar Restrictions On Decorators (Github issue :issue:`4570`)

Typing support in the sense of `PEP-484`_ (Github issues :issue:`3949`, :issue:`4243`)
and `PEP-560`_ (Github issues :issue:`2753`, :issue:`3537`, :issue:`3764`) was also improved.

.. _`PEP-3131`: https://www.python.org/dev/peps/pep-3131
.. _`PEP-479`: https://www.python.org/dev/peps/pep-0479
.. _`PEP-484`: https://www.python.org/dev/peps/pep-0484
.. _`PEP-487`: https://www.python.org/dev/peps/pep-0487
.. _`PEP-560`: https://www.python.org/dev/peps/pep-0560
.. _`PEP-563`: https://www.python.org/dev/peps/pep-0563
.. _`PEP-570`: https://www.python.org/dev/peps/pep-0570
.. _`PEP-572`: https://www.python.org/dev/peps/pep-0572
.. _`PEP-590`: https://www.python.org/dev/peps/pep-0590
.. _`PEP-614`: https://www.python.org/dev/peps/pep-0614

The default language level was changed to ``3str``, i.e. Python 3 semantics,
but with ``str`` literals (also in Python 2.7).  This is a backwards incompatible
change from the previous default of Python 2 semantics.  The previous behaviour
is available through the directive ``language_level=2``.
(Github issue :issue:`2565`).  This covers changes such as using the
``print``-function instead of the ``print``-statement, and integer-integer
division giving a floating point answer. Most of these changes were available
in earlier versions of Cython but are now the default.

Cython 3.0.0 also aligns its own language semantics more closely with Python, in particular:

* the power operator has changed to give a result matching what Python does rather than
  keeping the same types as the input (as in C),
* operator overloading of ``cdef classes`` behaves much more like Python classes,
* Cython's behaviour when using type annotations aligns more closely with their
  standard use in Python.

Related changes
^^^^^^^^^^^^^^^

* Cython no longer generates ``__qualname__`` attributes for classes in Python
  2.x since they are problematic there and not correctly maintained for subclasses.
  Patch by Jeroen Demeyer.  (Github issue :issue:`2772`)

* Binding staticmethods of Cython functions were not behaving like Python methods.
  Patch by Jeroen Demeyer.  (Github issue :issue:`3106`, :issue:`3102`)

* Compiling package ``__init__`` files could fail under Windows due to an
  undefined export symbol.  (Github issue :issue:`2968`)

* ``__init__.pyx`` files were not always considered as package indicators.
  (Github issue :issue:`2665`)

* Setting ``language_level=2`` in a file did not work if ``language_level=3``
  was enabled globally before.
  Patch by Jeroen Demeyer.  (Github issue :issue:`2791`)

* ``__doc__`` was not available inside of the class body during class creation.
  (Github issue :issue:`1635`)

* The first function line number of functions with decorators pointed to the
  signature line and not the first decorator line, as in Python.
  Patch by Felix Kohlgrüber.  (Github issue :issue:`2536`)

* Pickling unbound methods of Python classes failed.
  Patch by Pierre Glaser.  (Github issue :issue:`2972`)

* Item access (subscripting) with integer indices/keys always tried the
  Sequence protocol before the Mapping protocol, which diverged from Python
  semantics.  It now passes through the Mapping protocol first when supported.
  (Github issue :issue:`1807`)

* Nested dict literals in function call kwargs could incorrectly raise an
  error about duplicate keyword arguments, which are allowed when passing
  them from dict literals.
  (Github issue :issue:`2963`)

* Diverging from the usual behaviour, ``len(memoryview)``, ``len(char*)``
  and ``len(Py_UNICODE*)`` returned an unsigned ``size_t`` value.  They now
  return a signed ``Py_ssize_t``, like other usages of ``len()``.

* The unicode methods ``.upper()``, ``.lower()`` and ``.title()`` were
  incorrectly optimised for single character input values and only returned
  the first character if multiple characters should have been returned.
  They now use the original Python methods again.

* The ``cython.view.array`` type supports inheritance.
  Patch by David Woods.  (Github issue :issue:`3413`)

* The builtin ``abs()`` function can now be used on C numbers in nogil code.
  Patch by Elliott Sales de Andrade.  (Github issue :issue:`2748`)

* The attributes ``gen.gi_frame`` and ``coro.cr_frame`` of Cython compiled
  generators and coroutines now return an actual frame object for introspection.
  (Github issue :issue:`2306`)

* Inlined properties can be defined for external extension types.
  Patch by Matti Picus. (Github issue :issue:`2640`, redone later in :issue:`3571`)

* Unicode module names and imports are supported.
  Patch by David Woods.  (Github issue :issue:`3119`)

* ``__arg`` argument names in methods were not mangled with the class name.
  Patch by David Woods.  (Github issue :issue:`1382`)

* With ``language_level=3/3str``, Python classes without explicit base class
  are now new-style (type) classes also in Py2.  Previously, they were created
  as old-style (non-type) classes.
  (Github issue :issue:`3530`)

* Conditional blocks in Python code that depend on ``cython.compiled`` are
  eliminated at an earlier stage, which gives more freedom in writing
  replacement Python code.
  Patch by David Woods.  (Github issue :issue:`3507`)

* Python private name mangling now falls back to unmangled names for non-Python
  globals, since double-underscore names are not uncommon in C.  Unmangled Python
  names are also still found as a legacy fallback but produce a warning.
  Patch by David Woods.  (Github issue :issue:`3548`)

* The ``print`` statement (not the ``print()`` function) is allowed in
  ``nogil`` code without an explicit ``with gil`` section.

* ``repr()`` was assumed to return ``str`` instead of ``unicode`` with ``language_level=3``.
  (Github issue :issue:`3736`)

* Type inference now understands that ``a, *b = x`` assigns a list to ``b``.

* No/single argument functions now accept keyword arguments by default in order
  to comply with Python semantics.  The marginally faster calling conventions
  ``METH_NOARGS`` and ``METH_O`` that reject keyword arguments are still available
  with the directive ``@cython.always_allow_keywords(False)``.
  (Github issue :issue:`3090`)

* Special methods for binary operators now follow Python semantics.
  Rather than e.g. a single ``__add__`` method for cdef classes, where
  "self" can be either the first or second argument, one can now define
  both ``__add__`` and ``__radd__`` as for standard Python classes.
  This behavior can be disabled with the ``c_api_binop_methods`` directive
  to return to the previous semantics in Cython code (available from Cython
  0.29.20), or the reversed method (``__radd__``) can be implemented in
  addition to an existing two-sided operator method (``__add__``) to get a
  backwards compatible implementation.
  (Github issue :issue:`2056`)

* Generator expressions in pxd-overridden ``cdef`` functions could
  fail to compile.
  Patch by Matúš Valo.  (Github issue :issue:`3477`)

* Calls to ``.__class__()`` of a known extension type failed.
  Patch by David Woods.  (Github issue :issue:`3954`)

* Structs could not be instantiated with positional arguments in
  pure Python mode.

* Annotations were not exposed on annotated (data-)classes.
  Patch by matsjoyce.  (Github issue :issue:`4151`)

* Docstrings of ``cpdef`` enums are now copied to the enum class.
  Patch by matham.  (Github issue :issue:`3805`)

* ``asyncio.iscoroutinefunction()`` now recognises coroutine functions
  also when compiled by Cython.
  Patch by Pedro Marques da Luz.  (Github issue :issue:`2273`)

* Self-documenting f-strings (``=``) were implemented.
  Patch by davfsa.  (Github issue :issue:`3796`)

* ``cython.array`` supports simple, non-strided views.
  (Github issue :issue:`3775`)

* Attribute annotations in Python classes are now ignored, because they are
  just Python objects in a dict (as opposed to the fields of extension types).
  Patch by David Woods.  (Github issues :issue:`4196`, :issue:`4198`)

* A warning was added when ``__defaults__`` or ``__kwdefaults__`` of Cython compiled
  functions were re-assigned, since this does not current have an effect.
  Patch by David Woods.  (Github issue :issue:`2650`)

* The ``self`` argument of static methods in .pxd files was incorrectly typed.
  Patch by David Woods.  (Github issue :issue:`3174`)

* Default values for memory views arguments were not properly supported.
  Patch by Corentin Cadiou.  (Github issue :issue:`4313`)

* Python object types were not allowed as ``->`` return type annotations.
  Patch by Matúš Valo.  (Github issue :issue:`4433`)

* The excess arguments in a for-in-range loop with more than 3 arguments to `range()`
  were silently ignored.
  Original patch by Max Bachmann. (Github issue :issue:`4550`)

* Unsupported decorators on cdef functions were not rejected in recent releases.
  Patch by David Woods.  (Github issue :issue:`4322`)

* Fused functions were binding unnecessarily, which prevented them from being pickled.
  Patch by David Woods.  (Github issue :issue:`4370`)

* Decorators on inner functions were not evaluated in the right scope.
  Patch by David Woods.  (Github issue :issue:`4367`)

* Cython did not type the ``self`` argument in special binary methods.
  Patch by David Woods.  (Github issue :issue:`4434`)

* Circular imports of compiled modules could fail needlessly even when the import
  could already be resolved from ``sys.modules``.
  Patch by Syam Gadde.  (Github issue :issue:`4390`)

* ``__del__(self)`` on extension types now maps to ``tp_finalize`` in Python 3.
  Original patch by ax487.  (Github issue :issue:`3612`)

* Reusing an extension type attribute name as a method name is now an error.
  Patch by 0dminnimda.  (Github issue :issue:`4661`)

* When using type annotations, ``func(x: list)`` or ``func(x: ExtType)`` (and other
  Python builtin or extension types) no longer allow ``None`` as input argument to ``x``.
  This is consistent with the normal typing semantics in Python, and was a common gotcha
  for users who did not expect ``None`` to be allowed as input.  To allow ``None``, use
  ``typing.Optional`` as in ``func(x: Optional[list])``.  ``None`` is also automatically
  allowed when it is used as default argument, i.e. ``func(x: list = None)``.
  ``int`` and ``float`` are now also recognised in type annotations and restrict the
  value type at runtime.  They were previously ignored.
  Note that, for backwards compatibility reasons, the new behaviour does not apply when using
  Cython's C notation, as in ``func(list x)``.  Here, ``None`` is still allowed, as always.
  Also, the ``annotation_typing`` directive can now be enabled and disabled more finely
  within the module.
  (Github issues :issue:`2696`, :issue:`3883`, :issue:`4606`, :issue:`4669`, :issue:`4886`)

* The parser allowed some invalid spellings of ``...``.
  Patch by 0dminnimda.  (Github issue :issue:`4868`)

* The ``__self__`` attribute of fused functions reports its availability correctly
  with ``hasattr()``.  Patch by David Woods.
  (Github issue :issue:`4808`)

* Several optimised string methods failed to accept ``None`` as arguments to their options.
  Test patch by Kirill Smelkov.  (Github issue :issue:`4737`)

* Cython generators and coroutines now identify as ``CO_ASYNC_GENERATOR``,
  ``CO_COROUTINE`` and ``CO_GENERATOR`` accordingly.
  (Github issue :issue:`4902`)

* Memory views and the internal Cython array type now identify as ``collections.abc.Sequence``.
  Patch by David Woods.  (Github issue :issue:`4817`)

* Context managers can be written in parentheses.
  Patch by David Woods.  (Github issue :issue:`4814`)

* Some parser issues were resolved.
  (Github issue :issue:`4992`)

* Unused ``**kwargs`` arguments did not show up in ``locals()``.
  (Github issue :issue:`4899`)

* Relative imports failed in compiled ``__init__.py`` package modules.
  Patch by Matúš Valo.  (Github issue :issue:`3442`)

* Extension types are now explicitly marked as immutable types to prevent them from
  being considered mutable.
  Patch by Max Bachmann.  (Github issue :issue:`5023`)

* ``int(Py_UCS4)`` returned the code point instead of the parsed digit value.
  (Github issue :issue:`5216`)

* Calling bound classmethods of builtin types could fail trying to call the unbound method.
  (Github issue :issue:`5051`)

* Generator expressions and comprehensions now look up their outer-most iterable
  on creation, as Python does, and not later on start, as they did previously.
  (Github issue :issue:`1159`)

* Bound C methods can now coerce to Python objects.
  (Github issues :issue:`4890`, :issue:`5062`)

* ``cpdef`` enums can now be pickled.
  (Github issue :issue:`5120`)

* The Python Enum of a ``cpdef enum`` now inherits from ``IntFlag`` to better match
  both Python and C semantics of enums.
  (Github issue :issue:`2732`)

* The special ``__*pow__`` methods now support the 2- and 3-argument variants.
  (Github issue :issue:`5160`)

* The ``**`` power operator now behaves more like in Python by returning the correct complex
  result if required by math.  A new ``cpow`` directive was added to turn on the previous
  C-like behaviour.
  (Github issue :issue:`4936`)

* With ``language_level=2``, imports of modules in packages could return the wrong module in Python 3.
  (Github issue :issue:`5308`)

* Function signatures containing a type like `tuple[()]` could not be printed.
  Patch by Lisandro Dalcin.  (Github issue :issue:`5355`)

* ``__qualname__`` and ``__module__`` were not available inside of class bodies.
