#1 Add tests ported from OS CI project
Opened 6 years ago by mgahagan. Modified 6 years ago
rpms/ mgahagan/json-glib master  into  master

No commits found

Adds tests according to the CI wiki [0] specifically the standard test interface in the spec [1].

The playbook includes Tier1 level test cases that have been tested in the following contexts and is passing reliably: Docker, and Classic. No tests are suitable for Atomic Host at this time due to package requirements
Test logs are stored in the artifacts directory.

The following steps are used to execute the tests using the standard test interface:

Docker
sudo ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory) TEST_SUBJECTS=docker:docker.io/library/fedora:26 TEST_ARTIFACTS=$PWD/artifacts ansible-playbook --tags container tests.yml

Classic
sudo ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory) TEST_SUBJECTS="" TEST_ARTIFACTS=$PWD/artifacts ansible-playbook --tags classic tests.yml

[0] https://fedoraproject.org/wiki/CI
[1] https://fedoraproject.org/wiki/Changes/InvokingTests

Sample test output (container)

<snipping package installation logs>
TASK [standard-test-basic : Execute tests] *********
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-array.test': {u'run': u"./runtest.sh 'json-glib-1.0/array.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-boxed.test': {u'run': u"./runtest.sh 'json-glib-1.0/boxed.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-builder.test': {u'run': u"./runtest.sh 'json-glib-1.0/builder.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-generator.test': {u'run': u"./runtest.sh 'json-glib-1.0/generator.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-gvariant.test': {u'run': u"./runtest.sh 'json-glib-1.0/gvariant.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-invalid.test': {u'run': u"./runtest.sh 'json-glib-1.0/invalid.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-node.test': {u'run': u"./runtest.sh 'json-glib-1.0/node.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-object.test': {u'run': u"./runtest.sh 'json-glib-1.0/object.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-parser.test': {u'run': u"./runtest.sh 'json-glib-1.0/parser.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-path.test': {u'run': u"./runtest.sh 'json-glib-1.0/path.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-reader.test': {u'run': u"./runtest.sh 'json-glib-1.0/reader.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-serialize-complex.test': {u'run': u"./runtest.sh 'json-glib-1.0/serialize-complex.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-serialize-full.test': {u'run': u"./runtest.sh 'json-glib-1.0/serialize-full.test'", u'dir': u'upstream-testset'}})
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b] => (item={u'json-glib-serialize-simple.test': {u'run': u"./runtest.sh 'json-glib-1.0/serialize-simple.test'", u'dir': u'upstream-testset'}})

TASK [standard-test-basic : Pull out the logs] *********
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b]

TASK [standard-test-basic : Check the results] *********
changed: [be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b]

PLAY RECAP ***************
be83559b3a47217eee3d53aab91af1650c6babe7f08da76bcb79a0d3079a6a7b : ok=12 changed=8 unreachable=0 failed=0

sample test output log (each test logs details to its own file):

  • set -x
    ++ uname -m
  • ARCH=x86_64
  • LOG=/tmp/json-glib-array.test
  • T=json-glib-1.0/array.test
  • user=test
  • grep -e '^test:' /etc/passwd
  • '[' 0 -ne 0 ']'
  • rpm -q gnome-desktop-testing
  • echo 'gnome-desktop-testing already installed'
    gnome-desktop-testing already installed
  • sudo -u test ginsttest-runner json-glib-1.0/array.test
    Running test: json-glib-1.0/array.test
    /array/empty-array: OK
    /array/add-element: OK
    /array/remove-element: OK
    /array/foreach-element: OK
    PASS: json-glib-1.0/array.test
    SUMMARY: total=1; passed=1; skipped=0; failed=0; user=0.0s; system=0.0s; maxrss=4680
  • rc=0
  • RESULT=FAIL
  • '[' 0 -eq 0 ']'
  • RESULT=PASS
  • echo 'Result is: PASS'
    Result is: PASS
  • exit 0
Metadata