PR#33: Add flatpackage conditionals - rpms/python3

rpms / python3

#33 Add flatpackage conditionals

Merged 6 years ago by churchyard. Opened 6 years ago by churchyard.

rpms/ churchyard/python3 flatpackage into master

Add flatpackage conditionals

Miro Hrončok • 6 years ago

8c67baf

python3.spec

file modified

+128 -16

		`@@ -2,19 +2,19 @@`
		`# Top-level metadata`
		`# ==================`

		`- Name: python3`
		`- Summary: Interpreter of the Python programming language`
		`- URL: https://www.python.org/`
		`-`
		`%global pybasever 3.6`

		`# pybasever without the dot:`
		`%global pyshortver 36`

		`+ Name: python3`
		`+ Summary: Interpreter of the Python programming language`
		`+ URL: https://www.python.org/`
		`+`
		`# WARNING When rebasing to a new Python version,`
		`# remember to update the python3-docs package as well`
		`Version: %{pybasever}.5`
		`- Release: 1%{?dist}`
		`+ Release: 2%{?dist}`
		`License: Python`


		`@@ -25,6 +25,14 @@`
		`# Note that the bcond macros are named for the CLI option they create.`
		`# "%%bcond_without" means "ENABLE by default and create a --without option"`

		`+ # Flat package, i.e. python36, python37, python38 for tox etc.`
		`+ # warning: changes some other defaults`
		`+ # in Fedora, never turn this on for the python3 package`
		`+ # and always keep it on for python36 etc.`
		`+ # WARNING: This does not change the package name and summary above`
		`+ %bcond_with flatpackage`
		`+`
		`+`
		`# Expensive optimizations (mainly, profile-guided optimizations)`
		`%ifarch %{ix86} x86_64`
		`%bcond_without optimizations`
		`@@ -38,11 +46,19 @@`
		`%bcond_without tests`

		`# Ability to reuse RPM-installed pip using rewheel`
		`+ %if %{with flatpackage}`
		`+ %bcond_with rewheel`
		`+ %else`
		`%bcond_without rewheel`
		`+ %endif`

		`# Extra build for debugging the interpreter or C-API extensions`
		`# (the -debug subpackages)`
		`+ %if %{with flatpackage}`
		`+ %bcond_with debug_build`
		`+ %else`
		`%bcond_without debug_build`
		`+ %endif`

		`# Support for the GDB debugger`
		`%bcond_without gdb_hooks`
		`@@ -372,6 +388,8 @@`
		`# Descriptions, and metadata for subpackages`
		`# ==========================================`

		`+ %if %{without flatpackage}`
		`+`
		`# Packages with Python modules in standard locations automatically`
		`# depend on python(abi). Provide that here.`
		`Provides: python(abi) = %{pybasever}`
		`@@ -385,7 +403,7 @@`
		`Requires: %{name}-libs%{?_isa} = %{version}-%{release}`

		`# In order to support multiple Python interpreters for development purposes,`
		`- # packages with with the naming scheme pythonXY (e.g. python35) exist for`
		`+ # packages with the naming scheme flatpackage (e.g. python35) exist for`
		`# non-default versions of Python 3.`
		`# For consistency, and to keep the upgrade path clean, we Provide/Obsolete`
		`# these names here.`
		`@@ -605,6 +623,30 @@`
		`so extensions for both versions can co-exist in the same directory.`
		`%endif # with debug_build`

		`+ %else # with flatpackage`
		`+`
		`+ Requires: redhat-rpm-config`
		`+`
		`+ # We'll not provide this, on purpose`
		`+ # No package in Fedora shall ever depend on flatpackage via this`
		`+ %global __requires_exclude ^python\$abi\$ = 3\\..$`
		`+ %global __provides_exclude ^python\$abi\$ = 3\\..$`
		`+`
		`+ # We keep those inside on purpose`
		`+ Provides: bundled(python3-pip) = 9.0.3`
		`+ Provides: bundled(python3-setuptools) = 39.0.1`
		`+`
		`+ # The description for the flat package`
		`+ %description`
		`+ Python %{pybasever} package for developers.`
		`+`
		`+ This package exists to allow developers to test their code against an older`
		`+ version of Python. This is not a full Python stack and if you wish to run`
		`+ your applications with Python %{pybasever}, see other distributions`
cstratak commented 6 years ago I would change the wording here because it sort of implies that other distributions support the whatever Python %{pybasever} is at the time which might not always hold true. How about something like (just a suggestion though): This package exists to allow developers to test their code against an older version of Python. However this is not a full Python stack and running your applications with Python %{pybasever} is not a supported use case.
churchyard commented 6 years ago The description is carefully designed not to trigger anyone to say we are packaging "unsupported" bits into Fedora.
cstratak commented 6 years ago I'm fine in regards to not duel much on the 'unsupported' issue, however I wouldn't like people expecting 'support' from mentioned distros and quoting descriptions here.
churchyard commented 6 years ago This exists https://www.softwarecollections.org/en/scls/rhscl/rh-python36/ so I don't know what's wrong. (The description of python33 mentions software collections as well, which might be more problematic.)
cstratak commented 6 years ago Let's fix it at another time.
		`+ that support it, such as CentOS or RHEL with Software Collections`
		`+ or older Fedora releases.`
		`+`
		`+ %endif # with flatpackage`

		`# ======================================================`
		`# The prep phase of the build:`
		`@@ -870,6 +912,7 @@`
		`install -d -m 0755 %{buildroot}%{_prefix}/lib/python%{pybasever}/site-packages/__pycache__`
		`%endif`

		`+ %if %{without flatpackage}`
		`# add idle3 to menu`
		`install -D -m 0644 Lib/idlelib/Icons/idle_16.png %{buildroot}%{_datadir}/icons/hicolor/16x16/apps/idle3.png`
		`install -D -m 0644 Lib/idlelib/Icons/idle_32.png %{buildroot}%{_datadir}/icons/hicolor/32x32/apps/idle3.png`
		`@@ -880,6 +923,7 @@`
		`mkdir -p %{buildroot}%{_datadir}/appdata`
		`cp -a %{SOURCE11} %{buildroot}%{_datadir}/appdata`
		`appstream-util validate-relax --nonet %{buildroot}%{_datadir}/appdata/idle3.appdata.xml`
		`+ %endif`

		`# Make sure distutils looks at the right pyconfig.h file`
		`# See https://bugzilla.redhat.com/show_bug.cgi?id=201434`
		`@@ -942,15 +986,17 @@`
		`# Fixup permissions for shared libraries from non-standard 555 to standard 755:`
		`find %{buildroot} -perm 555 -exec chmod 755 {} \;`

		`+ %if %{without flatpackage}`
		`# Install macros for rpm:`
		`mkdir -p %{buildroot}/%{_rpmconfigdir}/macros.d/`
		`install -m 644 %{SOURCE3} %{buildroot}/%{_rpmconfigdir}/macros.d/`
		`install -m 644 %{SOURCE9} %{buildroot}/%{_rpmconfigdir}/macros.d/`
		`+ %endif`

		`# Create "/usr/bin/python3-debug", a symlink to the python3 debug binary, to`
		`# avoid the user having to know the precise version and ABI flags.`
		`# See e.g. https://bugzilla.redhat.com/show_bug.cgi?id=676748`
		`- %if %{with debug_build}`
		`+ %if %{with debug_build} && %{without flatpackage}`
		`ln -s \`
		`%{_bindir}/python%{LDVERSION_debug} \`
		`%{buildroot}%{_bindir}/python3-debug`
		`@@ -958,9 +1004,24 @@`

		`# System Python: Link the executable to libexec`
		`# This is for backwards compatibility only and should be removed in Fedora 29`
		`+ %if %{without flatpackage}`
		`mkdir -p %{buildroot}%{_libexecdir}`
		`ln -s %{_bindir}/python%{pybasever} %{buildroot}%{_libexecdir}/system-python`
		`+ %endif`

		`+ %if %{with flatpackage}`
		`+ # Remove stuff that would conflict with python3 package`
		`+ rm %{buildroot}%{_bindir}/python3`
		`+ rm %{buildroot}%{_bindir}/pydoc3`
		`+ rm %{buildroot}%{_bindir}/pathfix.py`
		`+ rm %{buildroot}%{_bindir}/idle3`
		`+ rm %{buildroot}%{_bindir}/python3-*`
		`+ rm %{buildroot}%{_bindir}/pyvenv`
		`+ rm %{buildroot}%{_bindir}/2to3*`
		`+ rm %{buildroot}%{_libdir}/libpython3.so`
		`+ rm %{buildroot}%{_mandir}/man1/python3.1*`
		`+ rm %{buildroot}%{_libdir}/pkgconfig/python3.pc`
		`+ %endif`

		`# ======================================================`
		`# Checks for packaging issues`
		`@@ -1052,26 +1113,37 @@`
		`%defattr(-, root, root)`
		`%license LICENSE`
		`%doc README.rst`
		`+`
		`+ %if %{without flatpackage}`
		`%{_bindir}/pydoc*`
		`%{_bindir}/python3`
		`- %{_bindir}/python%{pybasever}`
		`- %{_bindir}/python%{pybasever}m`
		`%{_bindir}/pyvenv`
		`- %{_bindir}/pyvenv-%{pybasever}`
		`%{_mandir}//`
		`+ %{_bindir}/pyvenv`
		`# Remove in Fedora 29:`
		`%{_libexecdir}/system-python`
		`+ %else`
		`+ %{_bindir}/pydoc%{pybasever}`
		`+ %{_mandir}//python%{pybasever}`
		`+ %endif`
		`+ %{_bindir}/pyvenv-%{pybasever}`
		`+ %{_bindir}/python%{pybasever}`
		`+ %{_bindir}/python%{pybasever}m`

		`+ %if %{without flatpackage}`
		`%files libs`
		`%defattr(-,root,root,-)`
		`%license LICENSE`
		`%doc README.rst`
		`+ %endif`

		`%dir %{pylibdir}`
		`%dir %{dynload_dir}`

		`%{pylibdir}/lib2to3`
		`+ %if %{without flatpackage}`
		`%exclude %{pylibdir}/lib2to3/tests`
		`+ %endif`

		`%dir %{pylibdir}/unittest/`
		`%dir %{pylibdir}/unittest/__pycache__/`
		`@@ -1096,7 +1168,13 @@`
		`%dir %{pylibdir}/ensurepip/__pycache__/`
		`%{pylibdir}/ensurepip/*.py`
		`%{pylibdir}/ensurepip/__pycache__/*%{bytecode_suffixes}`
		`+`
		`+ %if %{without flatpackage}`
		`%exclude %{pylibdir}/ensurepip/_bundled`
		`+ %else`
		`+ %dir %{pylibdir}/ensurepip/_bundled`
		`+ %{pylibdir}/ensurepip/_bundled/*.whl`
		`+ %endif`

		`%if %{with rewheel}`
		`%dir %{pylibdir}/ensurepip/rewheel/`
		`@@ -1255,8 +1333,10 @@`
		`%{pylibdir}/sqlite3/*.py`
		`%{pylibdir}/sqlite3/__pycache__/*%{bytecode_suffixes}`

		`+ %if %{without flatpackage}`
		`%exclude %{pylibdir}/turtle.py`
		`%exclude %{pylibdir}/__pycache__/turtle*%{bytecode_suffixes}`
		`+ %endif`

		`%{pylibdir}/urllib`
		`%{pylibdir}/xml`
		`@@ -1276,43 +1356,68 @@`
		`%{_includedir}/python%{LDVERSION_optimized}/%{_pyconfig_h}`

		`%{_libdir}/%{py_INSTSONAME_optimized}`
		`+ %if %{without flatpackage}`
		`%{_libdir}/libpython3.so`
		`+ %endif`

		`+ %if %{without flatpackage}`
		`%files devel`
		`%defattr(-,root,root)`
		`%{_bindir}/2to3`
		`# TODO: Remove 2to3-3.7 once rebased to 3.7`
		`%{_bindir}/2to3-%{pybasever}`
		`+ %endif`
		`+`
		`%{pylibdir}/config-%{LDVERSION_optimized}-%{_arch}-linux%{_gnu}/*`
		`+ %if %{without flatpackage}`
		`%exclude %{pylibdir}/config-%{LDVERSION_optimized}-%{_arch}-linux%{_gnu}/Makefile`
		`+ %exclude %{_includedir}/python%{LDVERSION_optimized}/%{_pyconfig_h}`
		`+ %endif`
		`%{pylibdir}/distutils/command/wininst-*.exe`
		`%{_includedir}/python%{LDVERSION_optimized}/*.h`
		`- %exclude %{_includedir}/python%{LDVERSION_optimized}/%{_pyconfig_h}`
		`%doc Misc/README.valgrind Misc/valgrind-python.supp Misc/gdbinit`
		`+`
		`+ %if %{without flatpackage}`
		`%{_bindir}/python3-config`
		`+ %{_libdir}/pkgconfig/python3.pc`
		`+ %{_rpmconfigdir}/macros.d/macros.pybytecompile%{pybasever}`
		`+ %{_rpmconfigdir}/macros.d/macros.systempython`
		`+ %{_bindir}/pathfix.py`
		`+ %endif`
		`+`
		`%{_bindir}/python%{pybasever}-config`
		`%{_bindir}/python%{LDVERSION_optimized}-config`
		`%{_bindir}/python%{LDVERSION_optimized}-*-config`
		`- %{_bindir}/pathfix.py`
		`%{_libdir}/libpython%{LDVERSION_optimized}.so`
		`%{_libdir}/pkgconfig/python-%{LDVERSION_optimized}.pc`
		`%{_libdir}/pkgconfig/python-%{pybasever}.pc`
		`- %{_libdir}/pkgconfig/python3.pc`
		`- %{_rpmconfigdir}/macros.d/macros.pybytecompile%{pybasever}`
		`- %{_rpmconfigdir}/macros.d/macros.systempython`

		`+ %if %{without flatpackage}`
		`%files idle`
		`%defattr(-,root,root,755)`
		`+`
		`%{_bindir}/idle*`
		`+ %else`
		`+ %{_bindir}/idle%{pybasever}`
		`+ %endif`
		`+`
		`%{pylibdir}/idlelib`
		`+`
		`+ %if %{without flatpackage}`
		`%{_datadir}/appdata/idle3.appdata.xml`
		`%{_datadir}/applications/idle3.desktop`
		`%{_datadir}/icons/hicolor//apps/idle3.`
		`+ %endif`

		`+ %if %{without flatpackage}`
		`%files tkinter`
		`%defattr(-,root,root,755)`
		`+ %endif`
		`+`
		`%{pylibdir}/tkinter`
		`+ %if %{without flatpackage}`
		`%exclude %{pylibdir}/tkinter/test`
		`+ %endif`
		`%{dynload_dir}/_tkinter.%{SOABI_optimized}.so`
		`%{pylibdir}/turtle.py`
		`%{pylibdir}/__pycache__/turtle*%{bytecode_suffixes}`
		`@@ -1322,8 +1427,10 @@`
		`%dir %{pylibdir}/turtledemo/__pycache__/`
		`%{pylibdir}/turtledemo/__pycache__/*%{bytecode_suffixes}`

		`+ %if %{without flatpackage}`
		`%files test`
		`%defattr(-, root, root)`
		`+ %endif`
		`%{pylibdir}/ctypes/test`
		`%{pylibdir}/distutils/tests`
		`%{pylibdir}/sqlite3/test`
		`@@ -1344,12 +1451,14 @@`
		`# all of the other subpackages`

		`%if %{with debug_build}`
		`+ %if %{without flatpackage}`
		`%files debug`
		`%defattr(-,root,root,-)`
		`+ %{_bindir}/python3-debug`
		`+ %endif`

		`# Analog of the core subpackage's files:`
		`%{_bindir}/python%{LDVERSION_debug}`
		`- %{_bindir}/python3-debug`

		`# Analog of the -libs subpackage's files:`
		`# ...with debug builds of the built-in "extension" modules:`
		`@@ -1470,6 +1579,9 @@`
		`# ======================================================`

		`%changelog`
		`+ * Wed Apr 18 2018 Miro Hrončok <mhroncok@redhat.com> - 3.6.5-2`
		`+ - Add flatpackage conditionals`
		`+`
		`* Thu Mar 29 2018 Charalampos Stratakis <cstratak@redhat.com> - 3.6.5-1`
		`- Update to 3.6.5`

churchyard commented 6 years ago

This will allow us to create python36 easily once needed [1].
It shoudl also make it easier to merge stuff between python36 and 37.

This more or less backports:

https://src.fedoraproject.org/rpms/python37/c/63459d7d2a4a8df39f900d753cf742c979b480a9
https://src.fedoraproject.org/rpms/python37/c/7f3ae64d2f98f00b8778b9693ad37fa3c295f4dc
https://src.fedoraproject.org/rpms/python37/c/6a6d4a03245c076ec8686b8f96ac4ffd3f6ad4e4

[1] https://fedoraproject.org/wiki/Changes/Python3.7

churchyard commented 6 years ago

I've tried to compare the results (default, i.e. without flatpackage) with original and the packages seems identical. Comparing results of rpmspec -P (prior to bumping the release) also only gives newlines and order changes.

The flatpackage also doesn't contain anything that should conflict with 3.7 nonflat:

$ rpm -qpl python3-3.6.5-2.fc29.x86_64.rpm | grep -v '3\.6' | grep -v '/usr/lib/\.build-id'
/usr/share/doc/python3
/usr/share/doc/python3/README.rst
/usr/share/doc/python3/README.valgrind
/usr/share/doc/python3/gdbinit
/usr/share/doc/python3/valgrind-python.supp
/usr/share/licenses/python3
/usr/share/licenses/python3/LICENSE

Note that the files above would conflict normally, but the python3 there is from the package name and that will eventually be python36.

churchyard commented 6 years ago

Also, when removing all python3 packages from mock and installing the flat one, python3.6 works. Also venv works and building a native extension in venv with cython numpy and setuptools work.

cstratak commented on line 14 of python3.spec 6 years ago

That was moved below the global macros in order to have them evaluated before the name was evaluated on https://src.fedoraproject.org/rpms/python37/c/63459d7d2a4a8df39f900d753cf742c979b480a9 but is it required here?

cstratak commented on line 96 of python3.spec 6 years ago

typo s/descripton/description

cstratak commented on line 96 of python3.spec 6 years ago

Also please replace 'flat flatpackage package' with either 'flat package' or 'flatpackage'

cstratak commented on line 75 of python3.spec 6 years ago

Please remove one 'with'

cstratak commented on line 98 of python3.spec 6 years ago

nitpick s/an/a

churchyard commented on line 14 of python3.spec 6 years ago

I moved it so once we change the name, we don't need to move it.

churchyard commented on line 98 of python3.spec 6 years ago

I will change the description entirely, because it needs to say older.

cstratak commented on line 263 of python3.spec 6 years ago

side note: If this is intended only for rawhide, maybe we should remove the system-python bits as well. Could be done at a separate PR of course.

churchyard commented on line 263 of python3.spec 6 years ago

I thought the same, yet in rawhide, we will update to 3.7 so this goes away naturally.

1 new commit added

fixup! Add flatpackage conditionals

6 years ago

churchyard commented 6 years ago

Added fixup commit.

BTW I track the changes that are applicable to python37 as well and will send a separate change there as well.

churchyard commented 6 years ago

Also, I want to merge this to F28 as well, so it stays somewhere once python3 in rawhide is updated to 3.7.

cstratak commented on line 102 of python3.spec 6 years ago

I would change the wording here because it sort of implies that other distributions support the whatever Python %{pybasever} is at the time which might not always hold true.

How about something like (just a suggestion though):
This package exists to allow developers to test their code against an older version of Python. However this is not a full Python stack and running your applications with Python %{pybasever} is not a supported use case.

churchyard commented on line 102 of python3.spec 6 years ago

The description is carefully designed not to trigger anyone to say we are packaging "unsupported" bits into Fedora.

cstratak commented on line 102 of python3.spec 6 years ago

I'm fine in regards to not duel much on the 'unsupported' issue, however I wouldn't like people expecting 'support' from mentioned distros and quoting descriptions here.

churchyard commented on line 102 of python3.spec 6 years ago

This exists https://www.softwarecollections.org/en/scls/rhscl/rh-python36/ so I don't know what's wrong. (The description of python33 mentions software collections as well, which might be more problematic.)

cstratak commented on line 102 of python3.spec 6 years ago

Let's fix it at another time.

cstratak commented 6 years ago

LGTM on my side! Feel free to merge.

rebased onto 8c67baf

6 years ago

churchyard commented 6 years ago

BTW tests hang after test_multiprocessing_fork but they did before as well.

Pull-Request has been merged by churchyard

6 years ago