rpms / vim

#1 Add CI tests using the standard test interface

Merged 6 years ago by zdohnal. Opened 6 years ago by astepano.

git://fedorapeople.org/~astepano/vim master into master

Add CI tests using the standard test interface

Andrei Stepanov • 6 years ago

81473c0

tests/netrw-plugin-delete-command-broken/Makefile

file added

+64

		`@@ -0,0 +1,64 @@`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+ #`
		`+ # Makefile of netrw-plugin-delete-command-broken`
		`+ # Description: Test for netrw plugin: delete command broken`
		`+ # Author: Petr Splichal <psplicha@redhat.com>`
		`+ #`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+ #`
		`+ # Copyright (c) 2009 Red Hat, Inc. All rights reserved.`
		`+ #`
		`+ # This copyrighted material is made available to anyone wishing`
		`+ # to use, modify, copy, or redistribute it subject to the terms`
		`+ # and conditions of the GNU General Public License version 2.`
		`+ #`
		`+ # This program is distributed in the hope that it will be`
		`+ # useful, but WITHOUT ANY WARRANTY; without even the implied`
		`+ # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR`
		`+ # PURPOSE. See the GNU General Public License for more details.`
		`+ #`
		`+ # You should have received a copy of the GNU General Public`
		`+ # License along with this program; if not, write to the Free`
		`+ # Software Foundation, Inc., 51 Franklin Street, Fifth Floor,`
		`+ # Boston, MA 02110-1301, USA.`
		`+ #`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+`
		`+ export TEST=/CoreOS/vim/Regression/netrw-plugin-delete-command-broken`
		`+ export TESTVERSION=1.0`
		`+`
		`+ BUILT_FILES=`
		`+`
		`+ FILES=$(METADATA) runtest.sh Makefile PURPOSE`
		`+`
		`+ .PHONY: all install download clean`
		`+`
		`+ run: $(FILES) build`
		`+ ./runtest.sh`
		`+`
		`+ build: $(BUILT_FILES)`
		`+ chmod a+x runtest.sh`
		`+`
		`+ clean:`
		`+ rm -f *~ $(BUILT_FILES)`
		`+`
		`+`
		`+ include /usr/share/rhts/lib/rhts-make.include`
		`+`
		`+ $(METADATA): Makefile`
		`+ @echo "Owner: Petr Splichal <psplicha@redhat.com>" > $(METADATA)`
		`+ @echo "Name: $(TEST)" >> $(METADATA)`
		`+ @echo "TestVersion: $(TESTVERSION)" >> $(METADATA)`
		`+ @echo "Path: $(TEST_DIR)" >> $(METADATA)`
		`+ @echo "Description: Test for netrw plugin: delete command broken" >> $(METADATA)`
		`+ @echo "Type: Regression" >> $(METADATA)`
		`+ @echo "TestTime: 5m" >> $(METADATA)`
		`+ @echo "RunFor: vim" >> $(METADATA)`
		`+ @echo "Requires: vim" >> $(METADATA)`
		`+ @echo "Priority: Normal" >> $(METADATA)`
		`+ @echo "License: GPLv2" >> $(METADATA)`
		`+ @echo "Confidential: no" >> $(METADATA)`
		`+ @echo "Destructive: no" >> $(METADATA)`
		`+ @echo "Requires: vim-enhanced" >> $(METADATA)`
		`+`
		`+ rhts-lint $(METADATA)`

tests/netrw-plugin-delete-command-broken/PURPOSE

file added

		`@@ -0,0 +1,9 @@`
		`+ PURPOSE of netrw-plugin-delete-command-broken`
		`+ Description: Test for netrw plugin: delete command broken`
		`+ Author: Petr Splichal <psplicha@redhat.com>`
		`+ Summary: netrw plugin: delete command broken`
		`+`
		`+ Description:`
		`+`
		`+ Regression to to verify the delete command of netrw plugin`
		`+ successfully deletes files or directories.`

tests/netrw-plugin-delete-command-broken/runtest.sh

file added

+64

		`@@ -0,0 +1,64 @@`
		`+ #!/bin/bash`
		`+ # vim: dict=/usr/share/rhts-library/dictionary.vim cpt=.,w,b,u,t,i,k`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+ #`
		`+ # runtest.sh of /CoreOS/vim/Regression/netrw-plugin-delete-command-broken`
		`+ # Description: Test for netrw plugin: delete command broken`
		`+ # Author: Petr Splichal <psplicha@redhat.com>`
		`+ #`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+ #`
		`+ # Copyright (c) 2009 Red Hat, Inc. All rights reserved.`
		`+ #`
		`+ # This copyrighted material is made available to anyone wishing`
		`+ # to use, modify, copy, or redistribute it subject to the terms`
		`+ # and conditions of the GNU General Public License version 2.`
		`+ #`
		`+ # This program is distributed in the hope that it will be`
		`+ # useful, but WITHOUT ANY WARRANTY; without even the implied`
		`+ # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR`
		`+ # PURPOSE. See the GNU General Public License for more details.`
		`+ #`
		`+ # You should have received a copy of the GNU General Public`
		`+ # License along with this program; if not, write to the Free`
		`+ # Software Foundation, Inc., 51 Franklin Street, Fifth Floor,`
		`+ # Boston, MA 02110-1301, USA.`
		`+ #`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+`
		`+ # Include rhts environment`
		`+ . /usr/bin/rhts-environment.sh`
		`+ . /usr/share/rhts-library/rhtslib.sh`
		`+`
		`+ PACKAGE="vim-enhanced"`
		`+`
		`+ rlJournalStart`
		`+ rlPhaseStartSetup`
		`+ rlAssertRpm $PACKAGE`
		+ rlRun "TmpDir=\`mktemp -d\`" 0 "Creating tmp directory"
		`+ rlRun "pushd $TmpDir"`
		`+ # create test file & dir`
		`+ rlRun "touch testfile" 0 "Creating test file"`
		`+ rlRun "mkdir testdir" 0 "Creating test directory"`
		`+ rlAssertExists "testfile"`
		`+ rlAssertExists "testdir"`
		`+ # create vim scripts`
		`+ rlRun "echo '/testfile Dy :q' > scriptfile" \`
		`+ 0 "Creating file delete script"`
		`+ rlRun "echo '/testdir Dy :q' > scriptdir" \`
		`+ 0 "Creating directory delete script"`
		`+ rlPhaseEnd`
		`+`
		`+ rlPhaseStartTest`
		`+ rlRun "vim . -s scriptfile" 0 "Deleting file using netrw"`
		`+ rlAssertNotExists "testfile"`
		`+ rlRun "vim . -s scriptdir" 0 "Deleting directory using netrw"`
		`+ rlAssertNotExists "testdir"`
		`+ rlPhaseEnd`
		`+`
		`+ rlPhaseStartCleanup`
		`+ rlRun "popd"`
		`+ rlRun "rm -r $TmpDir" 0 "Removing tmp directory"`
		`+ rlPhaseEnd`
		`+ rlJournalPrintText`
		`+ rlJournalEnd`

tests/tests.yml

file added

+23

		`@@ -0,0 +1,23 @@`
		`+ ---`
		`+ # This first play always runs on the local staging system`
		`+ - hosts: localhost`
		`+ roles:`
		`+ - role: standard-test-beakerlib`
		`+ tags:`
		`+ - classic`
		`+ - container`
		`+ tests:`
		`+ - netrw-plugin-delete-command-broken`
		`+ - vim-in-ex-mode-incorrectly-gives-an-eol-error`
		`+ required_packages:`
		`+ - vim-minimal`
		`+ - vim-enhanced`
		`+`
		`+ # Tests for Atomic Host`
		`+ - hosts: localhost`
		`+ roles:`
		`+ - role: standard-test-beakerlib`
		`+ tags:`
		`+ - atomic`
		`+ tests:`
		`+ - vim-in-ex-mode-incorrectly-gives-an-eol-error`

tests/vim-in-ex-mode-incorrectly-gives-an-eol-error/Makefile

file added

+63

		`@@ -0,0 +1,63 @@`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+ #`
		`+ # Makefile of /CoreOS/vim/Regression/vim-in-ex-mode-incorrectly-gives-an-eol-error`
		`+ # Description: Test for vim in ex mode incorrectly gives an eol error`
		`+ # Author: David Kutalek <dkutalek@redhat.com>`
		`+ #`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+ #`
		`+ # Copyright (c) 2010 Red Hat, Inc. All rights reserved.`
		`+ #`
		`+ # This copyrighted material is made available to anyone wishing`
		`+ # to use, modify, copy, or redistribute it subject to the terms`
		`+ # and conditions of the GNU General Public License version 2.`
		`+ #`
		`+ # This program is distributed in the hope that it will be`
		`+ # useful, but WITHOUT ANY WARRANTY; without even the implied`
		`+ # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR`
		`+ # PURPOSE. See the GNU General Public License for more details.`
		`+ #`
		`+ # You should have received a copy of the GNU General Public`
		`+ # License along with this program; if not, write to the Free`
		`+ # Software Foundation, Inc., 51 Franklin Street, Fifth Floor,`
		`+ # Boston, MA 02110-1301, USA.`
		`+ #`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+`
		`+ export TEST=/CoreOS/vim/Regression/vim-in-ex-mode-incorrectly-gives-an-eol-error`
		`+ export TESTVERSION=1.0`
		`+`
		`+ BUILT_FILES=`
		`+`
		`+ FILES=$(METADATA) runtest.sh Makefile PURPOSE exdoublequote expounddoublequote`
		`+`
		`+ .PHONY: all install download clean`
		`+`
		`+ run: $(FILES) build`
		`+ ./runtest.sh`
		`+`
		`+ build: $(BUILT_FILES)`
		`+ chmod a+x runtest.sh`
		`+`
		`+ clean:`
		`+ rm -f *~ $(BUILT_FILES)`
		`+`
		`+`
		`+ include /usr/share/rhts/lib/rhts-make.include`
		`+`
		`+ $(METADATA): Makefile`
		`+ @echo "Owner: David Kutalek <dkutalek@redhat.com>" > $(METADATA)`
		`+ @echo "Name: $(TEST)" >> $(METADATA)`
		`+ @echo "TestVersion: $(TESTVERSION)" >> $(METADATA)`
		`+ @echo "Path: $(TEST_DIR)" >> $(METADATA)`
		`+ @echo "Description: Test for vim in ex mode incorrectly gives an eol error" >> $(METADATA)`
		`+ @echo "Type: Regression" >> $(METADATA)`
		`+ @echo "TestTime: 5m" >> $(METADATA)`
		`+ @echo "RunFor: vim" >> $(METADATA)`
		`+ @echo "Requires: vim vim-minimal" >> $(METADATA)`
		`+ @echo "Priority: Normal" >> $(METADATA)`
		`+ @echo "License: GPLv2" >> $(METADATA)`
		`+ @echo "Confidential: no" >> $(METADATA)`
		`+ @echo "Destructive: no" >> $(METADATA)`
		`+`
		`+ rhts-lint $(METADATA)`

tests/vim-in-ex-mode-incorrectly-gives-an-eol-error/PURPOSE

file added

		`@@ -0,0 +1,3 @@`
		`+ PURPOSE of /CoreOS/vim/Regression/vim-in-ex-mode-incorrectly-gives-an-eol-error`
		`+ Description: Test for vim in ex mode incorrectly gives an eol error`
		`+ Author: David Kutalek <dkutalek@redhat.com>`

tests/vim-in-ex-mode-incorrectly-gives-an-eol-error/exdoublequote

file added

		`@@ -0,0 +1,2 @@`
		`+ " comment`
		`+ q`

tests/vim-in-ex-mode-incorrectly-gives-an-eol-error/expounddoublequote

file added

		`@@ -0,0 +1,2 @@`
		`+ #" comment`
		`+ q`

tests/vim-in-ex-mode-incorrectly-gives-an-eol-error/runtest.sh

file added

+46

		`@@ -0,0 +1,46 @@`
		`+ #!/bin/bash`
		`+ # vim: dict=/usr/share/beakerlib/dictionary.vim cpt=.,w,b,u,t,i,k`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+ #`
		`+ # runtest.sh of /CoreOS/vim/Regression/vim-in-ex-mode-incorrectly-gives-an-eol-error`
		`+ # Description: Test for vim in ex mode incorrectly gives an eol error`
		`+ # Author: David Kutalek <dkutalek@redhat.com>`
		`+ #`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+ #`
		`+ # Copyright (c) 2010 Red Hat, Inc. All rights reserved.`
		`+ #`
		`+ # This copyrighted material is made available to anyone wishing`
		`+ # to use, modify, copy, or redistribute it subject to the terms`
		`+ # and conditions of the GNU General Public License version 2.`
		`+ #`
		`+ # This program is distributed in the hope that it will be`
		`+ # useful, but WITHOUT ANY WARRANTY; without even the implied`
		`+ # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR`
		`+ # PURPOSE. See the GNU General Public License for more details.`
		`+ #`
		`+ # You should have received a copy of the GNU General Public`
		`+ # License along with this program; if not, write to the Free`
		`+ # Software Foundation, Inc., 51 Franklin Street, Fifth Floor,`
		`+ # Boston, MA 02110-1301, USA.`
		`+ #`
		`+ # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
		`+`
		`+ # Include rhts environment`
		`+ . /usr/bin/rhts-environment.sh`
		`+ . /usr/lib/beakerlib/beakerlib.sh`
		`+`
		`+ PACKAGE="vim"`
		`+`
		`+ rlJournalStart`
		`+ rlPhaseStartSetup`
		`+ rlAssertRpm vim-minimal`
		`+ rlPhaseEnd`
		`+`
		`+ rlPhaseStartTest`
		`+ rlRun "ex -u NONE ~/.bash_profile < ./exdoublequote"`
		`+ rlRun "ex -u NONE ~/.bash_profile < ./expounddoublequote"`
		`+ rlPhaseEnd`
		`+`
		`+ rlJournalPrintText`
		`+ rlJournalEnd`

no initial comment

astepano commented 6 years ago

Justification

Adds tests according to the CI wiki specifically the standard test interface
in the spec.

Adds tests according to the CI wiki specifically the standard test interface
in the spec. The playbook includes Tier1 level test cases that have been
tested in the following contexts and is passing reliably: Classic,
Atomic, Container. Test logs are stored in the artifacts directory.

The following steps are used to execute the tests using the standard test
interface:

$ ansible-playbook --list-tags tests.yml

playbook: tests.yml

play #1 (localhost): localhost    TAGS: []
     TASK TAGS: [atomic, classic, container]

The following steps are used to execute the tests using the standard test
interface.

Test environment

Make sure you have installed packages from the spec.

$ rpm -q ansible python2-dnf libselinux-python standard-test-roles
ansible-2.3.2.0-1.fc26.noarch
python2-dnf-2.6.3-11.fc26.noarch
libselinux-python-2.6-7.fc26.x86_64
standard-test-roles-2.4-1.fc26.noarch

Run tests as root:

For classic

# export TEST_SUBJECTS=''
# export ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory)
# ansible-playbook --tags=classic tests.yml

For atomic

# curl -Lo ~/atomic.qcow2 https://ftp-stud.hs-esslingen.de/pub/Mirrors/alt.fedoraproject.org/atomic/stable/Fedora-Atomic-26-20170707.1/CloudImages/x86_64/images/Fedora-Atomic-26-20170707.1.x86_64.qcow2
# export TEST_SUBJECTS=~/atomic.qcow2
# export ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory)
# ansible-playbook --tags=atomic tests.yml

For container

# export ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory)
# export TEST_SUBJECTS=docker:docker.io/library/fedora:26
# ansible-playbook --tags=container tests.yml

Snippet of test run

TASK [standard-test-beakerlib : Execute beakerlib tests] ****************************************************************
changed: [/home/fedora/atomic.qcow2] => (item=vim-in-ex-mode-incorrectly-gives-an-eol-error)

TASK [standard-test-beakerlib : Make the master test summary log artifact] **********************************************
changed: [/home/fedora/atomic.qcow2] => (item=vim-in-ex-mode-incorrectly-gives-an-eol-error)

TASK [standard-test-beakerlib : Pull out the logs] **********************************************************************
changed: [/home/fedora/atomic.qcow2]

TASK [standard-test-beakerlib : Check the results] **********************************************************************
changed: [/home/fedora/atomic.qcow2]

PLAY RECAP **************************************************************************************************************
/home/fedora/atomic.qcow2  : ok=15   changed=9    unreachable=0    failed=0

Notes

Tests will be enabled in CI, yet gating is currently disabled, so nothing will
change. Tests will run on each dist-git commit, they are not triggered on koji
builds and if you are using FMN, it should notify you of failures normally.

The RH QE maintainer contact in case you have questions: akhaitov @redhat.com
The idea is that these tests become yours just as you are maintaining the
package, there will of course be people around if you have questions or
troubles.