#2 Add CI tests using the Standard Test Interface
Opened 5 years ago by xiaonan. Modified 5 years ago
https://src.fedoraproject.org/forks/xiaonan/rpms/udisks2.git master  into  master

No commits found

no initial comment

Adds tests according to the CI wiki [0] specifically the standard test interface in the spec [1].

The playbook includes Tier1 level test cases that have been tested in the following contexts and is passing reliably with latest Fedora (classic mode), test logs are stored in the artifacts directory.

Test Logs: (If you you would like a pointer to the complete log, I can include that as well)


# ansible-playbook tests.yml 
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ************************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************
ok: [localhost]

TASK [str-common-init : Check if system is Atomic Host] *********************************************************************************************************************
ok: [localhost]

TASK [str-common-init : Set fact 'is_atomic'] *******************************************************************************************************************************
ok: [localhost]

TASK [str-common-init : Set facts about system] *****************************************************************************************************************************
ok: [localhost]

TASK [str-common-init : Hardcode Python 3 interpreter for ansible modules] **************************************************************************************************
skipping: [localhost]

TASK [str-common-init : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [str-common-init : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [str-common-init : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [str-common-init : Add test-runner to ansible inventory] ***************************************************************************************************************
changed: [localhost]

TASK [str-common-init : Gather facts] ***************************************************************************************************************************************
ok: [localhost - test-runner]

TASK [str-common-init : Set facts about test-runner] ************************************************************************************************************************
ok: [localhost - test-runner]

TASK [str-common-init : Hardcode Python 3 interpreter for ansible modules for test-runner] **********************************************************************************
skipping: [localhost]

TASK [str-common-init : Fetch tests from remote repositories] ***************************************************************************************************************
skipping: [localhost]

TASK [str-common-init : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [str-common-init : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [str-common-init : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [str-common-init : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [include_role : str-common-pkgs] ***************************************************************************************************************************************

TASK [str-common-pkgs : Build packages lists to be present on test environment] *********************************************************************************************
ok: [localhost]

TASK [str-common-pkgs : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [str-common-pkgs : include_tasks] **************************************************************************************************************************************
included: /usr/share/ansible/roles/str-common-pkgs/tasks/pkgs-dnf.yml for localhost

TASK [str-common-pkgs : Remove yum-utils for dnf system] ********************************************************************************************************************
ok: [localhost]

TASK [str-common-pkgs : Install additional dnf packages] ********************************************************************************************************************
ok: [localhost] = (item=dnf-plugins-core)
ok: [localhost] = (item=dnf-utils)

TASK [str-common-pkgs : Enable copr repos] **********************************************************************************************************************************
changed: [localhost] = (item=mvadkert/beakerlib-libraries)
 [WARNING]: Consider using the dnf module rather than running dnf.  If you need to use command because dnf is insufficient you can add warn=False to this command task or
set command_warnings=False in ansible.cfg to get rid of this message.


TASK [str-common-pkgs : Install test-specific package requirements] *********************************************************************************************************
ok: [localhost] = (item=udisks2)
ok: [localhost] = (item=beakerlib)
ok: [localhost] = (item=rsync)
ok: [localhost] = (item=make)
ok: [localhost] = (item=beakerlib-libraries)
ok: [localhost] = (item=libselinux-python3)
ok: [localhost] = (item=python2-lxml)

TASK [str-common-pkgs : Install debuginfo packages] *************************************************************************************************************************

TASK [str-common-init : Make artifacts directory] ***************************************************************************************************************************
changed: [localhost]

TASK [str-common-init : Copy tests to test environment] *********************************************************************************************************************
changed: [localhost]

TASK [str-common-init : Start services defined in `required_services`] ******************************************************************************************************

TASK [standard-test-beakerlib : Get list of RPM packages listed at beakerlib tests' requirements] ***************************************************************************
changed: [localhost - test-runner] = (item=check_pkg)

TASK [standard-test-beakerlib : Set Ansible facts for required RPM packages for beakerlib tests] ****************************************************************************
ok: [localhost]

TASK [standard-test-beakerlib : debug] **************************************************************************************************************************************
skipping: [localhost]

TASK [include_role : str-common-pkgs] ***************************************************************************************************************************************

TASK [str-common-pkgs : Build packages lists to be present on test environment] *********************************************************************************************
ok: [localhost]

TASK [str-common-pkgs : debug] **********************************************************************************************************************************************
skipping: [localhost]

TASK [str-common-pkgs : include_tasks] **************************************************************************************************************************************
included: /usr/share/ansible/roles/str-common-pkgs/tasks/pkgs-dnf.yml for localhost

TASK [str-common-pkgs : Remove yum-utils for dnf system] ********************************************************************************************************************
ok: [localhost]

TASK [str-common-pkgs : Install additional dnf packages] ********************************************************************************************************************
ok: [localhost] = (item=dnf-plugins-core)
ok: [localhost] = (item=dnf-utils)

TASK [str-common-pkgs : Enable copr repos] **********************************************************************************************************************************
changed: [localhost] = (item=mvadkert/beakerlib-libraries)

TASK [str-common-pkgs : Install test-specific package requirements] *********************************************************************************************************

TASK [str-common-pkgs : Install debuginfo packages] *************************************************************************************************************************

TASK [standard-test-beakerlib : Turn on usroverlay for Atomic Host] *********************************************************************************************************
skipping: [localhost]

TASK [standard-test-beakerlib : Create legacy beakerlib directories] ********************************************************************************************************
ok: [localhost] = (item=/usr/lib/beakerlib)
ok: [localhost] = (item=/usr/share/rhts-library)

TASK [standard-test-beakerlib : Create legacy beakerlib links] **************************************************************************************************************
ok: [localhost] = (item=/usr/lib/beakerlib/beakerlib.sh)
ok: [localhost] = (item=/usr/share/rhts-library/rhtslib.sh)

TASK [standard-test-beakerlib : Put beakerlib binaries to test environment Atomic Host] *************************************************************************************
skipping: [localhost] = (item=/usr/share/ansible/roles/standard-test-beakerlib/files/rpm.py) 

TASK [standard-test-beakerlib : Copy rhts files to test environment /usr/share/rhts/lib] ************************************************************************************
ok: [localhost] = (item=/usr/share/ansible/roles/standard-test-beakerlib/files/rhts-make.include)

TASK [standard-test-beakerlib : Copy rhts files to test environment /bin] ***************************************************************************************************
ok: [localhost] = (item=/usr/share/ansible/roles/standard-test-beakerlib/files/rhts-environment.sh)
ok: [localhost] = (item=/usr/share/ansible/roles/standard-test-beakerlib/files/rhts-run-simple-test)

TASK [standard-test-beakerlib : Run beakerlib tests] ************************************************************************************************************************
changed: [localhost] = (item=check_pkg)

TASK [standard-test-beakerlib : Make the master tests summary log artifact] *************************************************************************************************
changed: [localhost] = (item=check_pkg)

TASK [standard-test-beakerlib : Check the results] **************************************************************************************************************************
changed: [localhost]

TASK [standard-test-beakerlib : Set role result] ****************************************************************************************************************************
ok: [localhost]

TASK [include_role : str-common-final] **************************************************************************************************************************************

TASK [str-common-final : Pull out the logs from test environment to test runner] ********************************************************************************************
changed: [localhost]

TASK [str-common-final : Report role result] ********************************************************************************************************************************
ok: [localhost] = {
    "msg": [
        "Tests failed: False", 
        "Tests msg: ", 
        ""
    ]
}

PLAY RECAP ******************************************************************************************************************************************************************
localhost                  : ok=32   changed=10   unreachable=0    failed=0 

The RH QE maintainer contact in case you have questions: xhe@redhat.com

[0] https://fedoraproject.org/wiki/CI
[1] https://fedoraproject.org/wiki/Changes/InvokingTests

More testing
[citest]

Hopefully the last one :)
[citest]

Almost there :)
[citest]