From 37da9eb38a67535974e51753a62a3f81f1ec1755 Mon Sep 17 00:00:00 2001 From: Karsten Hopp Date: May 25 2018 21:32:41 +0000 Subject: first Fedora version --- diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..11ba31e --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +/fedmod-0.1.tar.gz diff --git a/README.md b/README.md index 5d7b6ad..ed56a90 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,175 @@ -# fedmod +# fedmod modularity tools -The fedmod package \ No newline at end of file +fedmod provides tools for working with Fedora's modulemd metadata format +that aren't related to actually building them (for build commands, see +fedpkg and mbs-build). + +Currently, this consists of: + +* `fedmod `: simple repoquery-like commands providing operations + like listing modules, resolving dependencies for packages, finding out + where a certain package is, etc. See the + [user docs](https://pagure.io/modularity/fedmod/blob/master/f/src/README.md) + for a full list of available query subcommands. +* `fedmod rpm2module`: generates a draft modulemd file based on + the given RPM name (multiple RPM names can be given, but the resulting + draft module will lack any descriptive metadata in that case) +* `fedmod fetch-metadata`: download the F28 package and module metadata needed + to generate draft module definitions (the metadata sets to use are not yet + configurable) + +## Project status + +`fedmod` is not yet available from the main Fedora repos, but can be installed +from [COPR](https://copr.fedorainfracloud.org/coprs/g/modularity/fedmod/) +as follows: + + $ sudo dnf copr enable @modularity/fedmod + $ sudo dnf install fedmod + $ fedmod fetch-metadata + $ fedmod rpm2module graphite-web + +This will generate a draft `modulemd` file for Fedora's `graphite-web` +package on stdout. + +See the local development instructions below for info on running directly +from a local development clone with `pipenv`. + +## User documentation + +Please see the [User docs of fedmod modularity tools](https://pagure.io/modularity/fedmod/blob/master/f/src/README.md). + + +## Local development + +### Runtime dependencies + +The preferred dependency management tool for development is `pipenv`: + + $ pipenv --three --site-packages + $ PIP_IGNORE_INSTALLED=1 pipenv install --dev + +The `PIP_IGNORE_INSTALLED=1` setting means that everything available to `pip` +will be installed into the virtual environment based on `Pipfile.lock`, and only +components that aren't installable with `pip` will be used from the system +Python installation. + +Some dependencies aren't currently available from PyPI, and hence need to be +installed system-wide: + + $ sudo dnf install libmodulemd python3-gobject-base python3-solv + +### Additional development dependencies + +`pipenv` itself isn't packaged for Fedora yet, so the recommended bootstrapping +approach is to use the "pip script installer", `pipsi`: + + $ sudo dnf install pipsi + $ pipsi install pew + $ pipsi install pipenv + +This will create a pair of isolated virtual environments in your home directory +specifically for `pipenv` and the tool it uses for virtual environment +management, `pew`. These can later be updated to newer versions using `pipsi`: + + $ pipsi upgrade pew + $ pipsi upgrade pipenv + +(Note: the `pipsi list` command will list all packages installed via `pipsi`, +and the commands they provide) + +### Running the development version + +After setting up the `pipenv` environment, the development version can be +run as follows + + $ pipenv run fedmod fetch-metadata + $ pipenv run fedmod rpm2module graphite-web + +Alternatively, start an interactive shell as described below for running the +tests, and `fedmod` will refer to the development version. + +### Running the tests + +After going through the environment setup steps above, start a shell that's +correctly configured to run the tests with `fedmod` and all of its +dependencies available: + + $ pipenv shell + +The metadata needed by the module generator tests can then be installed with +`fedmod` itself: + + $ fedmod fetch-metadata + +The tests can then be run in the launched subshell with: + + $ pytest tests + +To test the package build process, tox is also supported: + + $ tox -e py36 + + +### Reviewing project dependencies + +To see the Python level dependencies graph: + + $ pew toggleglobalsitepackages + $ pipenv graph + $ pew toggleglobalsitepackages + +(If you don't turn off global site-packages access first, you'll get the +dependency graph of all the installed system Python components as well) + + +### Testing Fedora system package compatibility + +While the default development environment is managed with `pipenv` for a more +consistent cross-platform development experience, `fedmod` is intended to +support installation as a system package in Fedora 26 and later. + +A specific tox environment is provided to enable this testing: + + $ tox -e system + +The only component this installs into the environment is `fedmod` itself: all +other dependencies must be available as Python 3 system packages. + +Unlike the regular test environment, this environment also implicitly runs +`fedmod fetch-metadata` in order to ensure that the metadata fetching operation +also works correctly given only system packages as dependencies. + +## Publishing new releases + +### COPR releases + +The main current release mechanism is through COPR at +https://copr.fedorainfracloud.org/coprs/g/modularity/fedmod/. + +This is configured to automatically build a new release every time a new tag +is pushed to the `fedmod` git repository. + +`fedmod`'s RPMs are built with `tito`, but version tagging is handled with a +helper script. To publish a new release, run: + + $ ./tag-release.sh + $ git push && git push --tags + + +### PyPI releases + +`fedmod` is also published to PyPI here: https://pypi.org/project/fedmod/ + +After releasing to COPR to ensure everything is properly tagged, a new PyPI +release can be made by doing: + + $ cd src + $ rm dist/* + $ python setup.py sdist bdist_wheel + $ twine upload dist/* + +The `dnf` and `solv` dependencies unfortunately mean the PyPI release isn't +particularly useful at this point (`pipsi` doesn't allow system level +dependencies, and even if it did, platforms that provide these libraries are +also likely to provide access to COPR). diff --git a/fedmod.spec b/fedmod.spec new file mode 100644 index 0000000..71eee96 --- /dev/null +++ b/fedmod.spec @@ -0,0 +1,229 @@ +# Based on initial spec generated by pyp2rpm-3.2.3 + +Name: fedmod +Version: 0.1 +Release: 2%{?dist} +Summary: Utilities for generating & maintaining modulemd files + +License: GPLv2+ and GPLv3+ and MIT +URL: https://pagure.io/modularity/fedmod +Source0: %{name}-%{version}.tar.gz +BuildArch: noarch + +BuildRequires: python3-devel +BuildRequires: python3-setuptools + +Requires: libmodulemd >= 1.2.0 +Requires: python3-attrs +Requires: python3-click +Requires: python3-gobject-base +Requires: python3-lxml +Requires: python3-modulemd +Requires: python3-PyYAML +Requires: python3-requests +Requires: python3-requests-toolbelt +Requires: python3-setuptools +Requires: python3-smartcols +Requires: python3-solv + +%description +fedmod provides tools for converting existing RPMs (most notably metapackages) +into module definitions in Fedora's modulemd format. + +%prep +%autosetup -n %{name}-%{version} +# Remove bundled egg-info +rm -rf %{name}.egg-info + +%build +cd src +%py3_build + +%install +cd src +%py3_install + + +%files +%doc +%license LICENSE +%{_bindir}/fedmod +%{python3_sitelib}/_fedmod +%{python3_sitelib}/%{name}-%{version}-py?.?.egg-info + +%changelog +* Fri May 25 2018 Karsten Hopp 0.1-2 +- update License tag for package review add %%license (karsten@redhat.com) +- add changelog (karsten@redhat.com) +- Update version metadata for release 0.1 (karsten@redhat.com) +- _depchase: Remove some stray assignments (otaylor@fishsoup.net) +- add mapping package/module for dependencies (karsten@redhat.com) +- Enable the Fedora updates repository (otaylor@fishsoup.net) +- depchase: Handle duplicate packages in the pool (otaylor@fishsoup.net) +- Fix problem resolving relative paths to updates repo (otaylor@fishsoup.net) +- Use RPM metadata as the basis for summary and description + (otaylor@fishsoup.net) +- Use a cache for dependency details (otaylor@fishsoup.net) +- update maintainer (karsten@redhat.com) +- disable updates for now, move that to a new issue (karsten@redhat.com) +- break out unrelated commit (karsten@redhat.com) +- add module updates (karsten@redhat.com) +- libmodulemd fixes (karsten@redhat.com) +- Use libmodulemd instead of modulemd (otaylor@fishsoup.net) + +* Fri May 25 2018 Karsten Hopp +- update License tag for package review add %%license (karsten@redhat.com) +- add changelog (karsten@redhat.com) +- Update version metadata for release 0.1 (karsten@redhat.com) +- _depchase: Remove some stray assignments (otaylor@fishsoup.net) +- add mapping package/module for dependencies (karsten@redhat.com) +- Enable the Fedora updates repository (otaylor@fishsoup.net) +- depchase: Handle duplicate packages in the pool (otaylor@fishsoup.net) +- Fix problem resolving relative paths to updates repo (otaylor@fishsoup.net) +- Use RPM metadata as the basis for summary and description + (otaylor@fishsoup.net) +- Use a cache for dependency details (otaylor@fishsoup.net) +- update maintainer (karsten@redhat.com) +- disable updates for now, move that to a new issue (karsten@redhat.com) +- break out unrelated commit (karsten@redhat.com) +- add module updates (karsten@redhat.com) +- libmodulemd fixes (karsten@redhat.com) +- Use libmodulemd instead of modulemd (otaylor@fishsoup.net) + +* Fri May 25 2018 Karsten Hopp +- Update version metadata for release 0.1 (karsten@redhat.com) +- _depchase: Remove some stray assignments (otaylor@fishsoup.net) +- add mapping package/module for dependencies (karsten@redhat.com) +- Enable the Fedora updates repository (otaylor@fishsoup.net) +- depchase: Handle duplicate packages in the pool (otaylor@fishsoup.net) +- Fix problem resolving relative paths to updates repo (otaylor@fishsoup.net) +- Use RPM metadata as the basis for summary and description + (otaylor@fishsoup.net) +- Use a cache for dependency details (otaylor@fishsoup.net) +- update maintainer (karsten@redhat.com) +- disable updates for now, move that to a new issue (karsten@redhat.com) +- break out unrelated commit (karsten@redhat.com) +- add module updates (karsten@redhat.com) +- libmodulemd fixes (karsten@redhat.com) +- Use libmodulemd instead of modulemd (otaylor@fishsoup.net) + +* Thu Apr 05 2018 Karsten Hopp 0.0.10-1 +- Update version metadata for release 0.0.10 (karsten@redhat.com) +- Strip down module generation (otaylor@fishsoup.net) +- Comment out references to updates repo (otaylor@fishsoup.net) +- Move from F27 to F28 (otaylor@fishsoup.net) +- Remove special handling of fedora-release (otaylor@fishsoup.net) +- Remove references to bootstrap module (otaylor@fishsoup.net) +- Move gzip import into the right file (otaylor@fishsoup.net) +- When tracking package dependencies, look at pre-requires, not just normal + reqs (otaylor@fishsoup.net) +- add PyYAML dependency to spec file (nils@redhat.com) +- allow marking blocks as "explicitly empty" (nils@redhat.com) +- don't trip over 'filter: ~' (nils@redhat.com) +- improve wording in some detailed descriptions (nils@redhat.com) +- fix grammar (nils@redhat.com) +- check that filter/rpms exists before accessing it (nils@redhat.com) +- check for optional dependencies/(build)requires (nils@redhat.com) +- fix some method/prerequisite names (nils@redhat.com) +- fix PyYAML dependency (nils@redhat.com) +- we don't really support UTF-16 at the moment (nils@redhat.com) +- return a non-zero exit code on warnings, errors (nils@redhat.com) +- add checks for optional blocks (nils@redhat.com) +- load and process the file in linter methods (nils@redhat.com) +- allow empty guidelines link (nils@redhat.com) +- add @prerequisite_for decorator (nils@redhat.com) +- add @option decorator (nils@redhat.com) +- add license blurb (nils@redhat.com) +- more comments (nils@redhat.com) +- check types of content components' metadata (nils@redhat.com) +- check components (nils@redhat.com) +- check filters (nils@redhat.com) +- check the API (nils@redhat.com) +- add check_is_list() and use it (nils@redhat.com) +- check the profiles (nils@redhat.com) +- check the references (nils@redhat.com) +- check that all license elements are scalars (nils@redhat.com) +- check that all license blocks are lists in one go (nils@redhat.com) +- warn about unknown license keys/blocks (nils@redhat.com) +- let check_is_(dict|scalar) accept multiple values (nils@redhat.com) +- make assertion errors more informative (nils@redhat.com) +- assert that every linter method has a docstring (nils@redhat.com) +- check that a potential 'xmd' block is a dict (nils@redhat.com) +- don't use modulemd for linting (nils@redhat.com) +- add check_is_dict() and use it (nils@redhat.com) +- add check_is_scalar() method and use it (nils@redhat.com) +- add checks for the dependencies block (nils@redhat.com) +- flag fields which should be set during build (nils@redhat.com) +- mention recognized license keys (nils@redhat.com) +- format guideline blurbs and problem details better (nils@redhat.com) +- use click.Choice for validating --min-level (nils@redhat.com) +- add --min-level option (nils@redhat.com) +- use check*() methods rather than assert (nils@redhat.com) +- add prerequisite decorator (nils@redhat.com) +- check that description exists (nils@redhat.com) +- add license block checks (nils@redhat.com) +- load raw YAML dict for low-level checks (nils@redhat.com) +- reformat output a little (nils@redhat.com) +- add detail decorator (nils@redhat.com) +- execute linter methods in order of their definition (nils@redhat.com) +- check if description ends in a period (nils@redhat.com) +- add 'lint' command (nils@redhat.com) +- don't import from __future__ (nils@redhat.com) + +* Thu Apr 05 2018 Karsten Hopp - 0.0.10-1 +- update for F28 (Owen Taylor) +- look at pre-reqs for package dependencies, too (Owen Taylor) + +* Wed Dec 06 2017 Nick Coghlan 0.0.9-1 +- Update version metadata for release 0.0.9 (ncoghlan@gmail.com) +- Issue #28: Remove dependency on DNF (ncoghlan@gmail.com) +- Add docs for 'resolve-deps --json' (ncoghlan@gmail.com) +- Add a basic 'resolve-deps --json' test case (ncoghlan@gmail.com) +- Keep function signature compatible (ncoghlan@gmail.com) +- Report lists to handle ambiguous deps (ncoghlan@gmail.com) +- resolve-deps: Add a --json flag to get full output in JSON form + (otaylor@fishsoup.net) +- Separate metadata fetching to its own file (ncoghlan@gmail.com) +- README: Document srpm-of-rpm command (crobinso@redhat.com) +- cli: add rpms-from-srpm command (crobinso@redhat.com) +- depchase: Handle epoch packages in get_rpms_for_srpms (crobinso@redhat.com) +- depchase: Drop unnecessary archful lookup for get_rpms_for_srpms + (crobinso@redhat.com) +- depchase: Fix get_rpms_for_srpms lookup mapping (crobinso@redhat.com) +- cli: print sorted output (crobinso@redhat.com) +- Issue #57: Cache metadata lookup tables (ncoghlan@gmail.com) +- Issue #34: Improve handling of duplicate components (ncoghlan@gmail.com) +- Issue #49: Migrate CLI processing to click (ncoghlan@gmail.com) + +* Wed Nov 22 2017 Adam Samalik 0.0.8-1 +- Update version metadata for release 0.0.8 (asamalik@redhat.com) +- add smartcols and solv dependencies to spec (asamalik@redhat.com) +- Issue #52: Handle missing metadata (ncoghlan@gmail.com) +- Add basic test file descriptions (ncoghlan@gmail.com) +- Clarify repoquery subcommand docs (ncoghlan@gmail.com) +- Tidy up whitespace in test file (ncoghlan@gmail.com) + +* Fri Nov 10 2017 Adam Samalik 0.0.7-1 +- Update version metadata for release 0.0.7 (asamalik@redhat.com) +- add srpm-of-rpm functionality (asamalik@redhat.com) + +* Tue Nov 07 2017 Adam Samalik 0.0.6-1 +- Update version metadata for release 0.0.6 (asamalik@redhat.com) +- workaround for https://pagure.io/pagure/issue/2751 (asamalik@redhat.com) +- add user docs (asamalik@redhat.com) +- add tests for module repoquery (asamalik@redhat.com) +- use forward lookup table for modules (asamalik@redhat.com) +- implement repoquery-like commands (asamalik@redhat.com) +- Remove dependency-reports-scripts cross-reference (ncoghlan@gmail.com) +- Fix COPR package name (ncoghlan@redhat.com) +- Add release publication docs (ncoghlan@gmail.com) + +* Thu Nov 02 2017 Nick Coghlan 0.0.5-1 +- Update version metadata for release 0.0.5 (ncoghlan@gmail.com) +- Put requirements before description (ncoghlan@gmail.com) + +* Thu Nov 02 2017 Nick Coghlan 0.0.4-1 +- new package built with tito + +* Wed Nov 01 2017 mockbuilder - 0.0.2-1 +- Initial package. diff --git a/sources b/sources new file mode 100644 index 0000000..9d461d7 --- /dev/null +++ b/sources @@ -0,0 +1 @@ +SHA512 (fedmod-0.1.tar.gz) = 0240510162ada046cb0865e9c4137c65cf5a0043db364c8e5d0a6777f536abb3eea6ff5247b0e53e7cf3b6262f4efc815c109acfb397d6070af63656788c7e78