[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v2 1/2] pickles: Add new pickle `pktest.pk`
From: |
Mohammad-Reza Nabipoor |
Subject: |
[PATCH v2 1/2] pickles: Add new pickle `pktest.pk` |
Date: |
Wed, 16 Dec 2020 07:30:20 +0330 |
This commit adds `pktest.pk` that provides testing facilities. It
is a TAP (Test Anything Protocol) producer. That means tests written
using `pktest` will generate TAP output. The output can be used by
any TAP consumer (test harnesses that read TAP).
2020-12-16 Mohammad-Reza Nabipoor <m.nabipoor@yahoo.com>
* pickles/pktest.pk: New file.
* pickles/Makefile.am (dist_pickles_DATA): Add `pktest.pk`.
* testsuite/poke.pktest/pktest.exp: New file.
* testsuite/poke.pktest/pktest-1.pk: Likewise.
* testsuite/poke.pktest/pktest-2.pk: Likewise.
* testsuite/poke.pktest/pktest-3.pk: Likewise.
* testsuite/poke.pktest/pktest-4.pk: Likewise.
* testsuite/poke.pktest/pktest-5.pk: Likewise.
* testsuite/poke.pktest/pktest-6.pk: Likewise.
* testsuite/poke.pktest/pktest-7.pk: Likewise.
* testsuite/poke.pktest/pktest-8.pk: Likewise.
* testsuite/poke.pktest/pktest-9.pk: Likewise.
* testsuite/poke.pktest/pktest-10.pk: Likewise.
* testsuite/poke.pktest/pktest-11.pk: Likewise.
* testsuite/poke.pktest/pktest-12.pk: Likewise.
* testsuite/Makefile.am (EXTRA_DIST): Add new tests.
---
Hi, Jose!
Now `pktest` speaks TAP :)
It provides two functions:
- pktest_run_noexit
- pktest_run
I changed the semantics of `pktest_run` to exit after the termination.
The `PkTest` struct has two more `string` fields to support SKIP and TODO
outputs of TAP.
And still no documentation!
Regards,
Mohammad-Reza
ChangeLog | 19 +++++++
pickles/Makefile.am | 2 +-
pickles/pktest.pk | 84 ++++++++++++++++++++++++++++++
testsuite/Makefile.am | 13 +++++
testsuite/poke.pktest/pktest-1.pk | 6 +++
testsuite/poke.pktest/pktest-10.pk | 29 +++++++++++
testsuite/poke.pktest/pktest-11.pk | 31 +++++++++++
testsuite/poke.pktest/pktest-2.pk | 6 +++
testsuite/poke.pktest/pktest-3.pk | 6 +++
testsuite/poke.pktest/pktest-4.pk | 17 ++++++
testsuite/poke.pktest/pktest-5.pk | 17 ++++++
testsuite/poke.pktest/pktest-6.pk | 26 +++++++++
testsuite/poke.pktest/pktest-7.pk | 26 +++++++++
testsuite/poke.pktest/pktest-8.pk | 28 ++++++++++
testsuite/poke.pktest/pktest-9.pk | 29 +++++++++++
testsuite/poke.pktest/pktest.exp | 22 ++++++++
16 files changed, 360 insertions(+), 1 deletion(-)
create mode 100644 pickles/pktest.pk
create mode 100644 testsuite/poke.pktest/pktest-1.pk
create mode 100644 testsuite/poke.pktest/pktest-10.pk
create mode 100644 testsuite/poke.pktest/pktest-11.pk
create mode 100644 testsuite/poke.pktest/pktest-2.pk
create mode 100644 testsuite/poke.pktest/pktest-3.pk
create mode 100644 testsuite/poke.pktest/pktest-4.pk
create mode 100644 testsuite/poke.pktest/pktest-5.pk
create mode 100644 testsuite/poke.pktest/pktest-6.pk
create mode 100644 testsuite/poke.pktest/pktest-7.pk
create mode 100644 testsuite/poke.pktest/pktest-8.pk
create mode 100644 testsuite/poke.pktest/pktest-9.pk
create mode 100644 testsuite/poke.pktest/pktest.exp
diff --git a/ChangeLog b/ChangeLog
index 39290b18..9f55f5f6 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,22 @@
+2020-12-16 Mohammad-Reza Nabipoor <m.nabipoor@yahoo.com>
+
+ * pickles/pktest.pk: New file.
+ * pickles/Makefile.am (dist_pickles_DATA): Add `pktest.pk`.
+ * testsuite/poke.pktest/pktest.exp: New file.
+ * testsuite/poke.pktest/pktest-1.pk: Likewise.
+ * testsuite/poke.pktest/pktest-2.pk: Likewise.
+ * testsuite/poke.pktest/pktest-3.pk: Likewise.
+ * testsuite/poke.pktest/pktest-4.pk: Likewise.
+ * testsuite/poke.pktest/pktest-5.pk: Likewise.
+ * testsuite/poke.pktest/pktest-6.pk: Likewise.
+ * testsuite/poke.pktest/pktest-7.pk: Likewise.
+ * testsuite/poke.pktest/pktest-8.pk: Likewise.
+ * testsuite/poke.pktest/pktest-9.pk: Likewise.
+ * testsuite/poke.pktest/pktest-10.pk: Likewise.
+ * testsuite/poke.pktest/pktest-11.pk: Likewise.
+ * testsuite/poke.pktest/pktest-12.pk: Likewise.
+ * testsuite/Makefile.am (EXTRA_DIST): Add new tests.
+
2020-12-15 Jose E. Marchesi <jemarch@gnu.org>
* libpoke/pkl-typify.c (TYPIFY_BIN): DIV and CEILDIV accept
diff --git a/pickles/Makefile.am b/pickles/Makefile.am
index aece5326..d5452e52 100644
--- a/pickles/Makefile.am
+++ b/pickles/Makefile.am
@@ -2,4 +2,4 @@ picklesdir = $(pkgdatadir)/pickles
dist_pickles_DATA = elf.pk ctf.pk leb128.pk bpf.pk btf.pk bmp.pk \
color.pk rgb24.pk id3v1.pk \
dwarf.pk dwarf-common.pk dwarf-frame.pk dwarf-pubnames.pk \
- dwarf-types.pk time.pk argp.pk
+ dwarf-types.pk time.pk argp.pk pktest.pk
diff --git a/pickles/pktest.pk b/pickles/pktest.pk
new file mode 100644
index 00000000..f0f03e8a
--- /dev/null
+++ b/pickles/pktest.pk
@@ -0,0 +1,84 @@
+/* pktest.pk - Facilities to write tests for pickles. */
+
+/* Copyright (C) 2020 The poke authors */
+
+/* This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/* This is a TAP (Test Anything Protocol) producer.
+ * TAP is a simple text-based interface between testing modules in a test
+ * harness.
+ *
+ * For more info, see https://testanything.org/.
+ */
+
+type PkTestFn = (string) void;
+type PkTest = struct
+ {
+ string name;
+ string skip; /* Skip reason. If non-empty, test will be skipped */
+ string todo; /* TODO reason. If non-empty, test will be marked as TODO */
+ PkTestFn func;
+ };
+
+fun pktest_run_noexit = (PkTest[] tests, string skip = "") int:
+ {
+ var ok = 1;
+ var i = 0UL;
+
+ if (skip != "")
+ {
+ printf "1..0 # Skipped: %s\n", skip;
+ return ok;
+ }
+ else
+ printf "1..%u64d\n", tests'length;
+
+ for (t in tests)
+ {
+ ++i;
+
+ if (t.skip != "")
+ {
+ printf "ok %u64d %s # SKIP %s\n", i, t.name, t.skip;
+ continue;
+ }
+
+ fun todo = void:
+ {
+ if (t.todo == "")
+ print "\n";
+ else
+ printf " # TODO %s\n", t.todo;
+ }
+
+ try t.func(t.name);
+ catch (Exception ex)
+ {
+ ok = 0;
+ printf "not ok %u64d %s: %s", i, t.name, ex.msg;
+ todo ();
+ continue;
+ }
+ printf "ok %u64d %s", i, t.name;
+ todo ();
+ }
+
+ return ok;
+ }
+
+fun pktest_run = (PkTest[] tests, string skip = "") void:
+ {
+ exit (pktest_run_noexit (tests, skip) ? 0 : 1);
+ }
diff --git a/testsuite/Makefile.am b/testsuite/Makefile.am
index e79361c0..950e2ecf 100644
--- a/testsuite/Makefile.am
+++ b/testsuite/Makefile.am
@@ -1801,6 +1801,19 @@ EXTRA_DIST = \
poke.pkl/xor-offsets-2.pk \
poke.pkl/xora-int-1.pk \
poke.pkl/xora-offset-1.pk \
+ poke.pktest/pktest.exp \
+ poke.pktest/pktest-1.pk \
+ poke.pktest/pktest-2.pk \
+ poke.pktest/pktest-3.pk \
+ poke.pktest/pktest-4.pk \
+ poke.pktest/pktest-5.pk \
+ poke.pktest/pktest-6.pk \
+ poke.pktest/pktest-7.pk \
+ poke.pktest/pktest-8.pk \
+ poke.pktest/pktest-9.pk \
+ poke.pktest/pktest-10.pk \
+ poke.pktest/pktest-11.pk \
+ poke.pktest/pktest-12.pk \
poke.repl/repl.exp \
poke.rgb24/rgb24.exp \
poke.rgb24/rgb24-1.pk \
diff --git a/testsuite/poke.pktest/pktest-1.pk
b/testsuite/poke.pktest/pktest-1.pk
new file mode 100644
index 00000000..607c5405
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-1.pk
@@ -0,0 +1,6 @@
+/* { dg-do run } */
+
+load pktest;
+
+/* { dg-command {pktest_run_noexit (PkTest[] ())} } */
+/* { dg-output "1..0\\n" } */
diff --git a/testsuite/poke.pktest/pktest-10.pk
b/testsuite/poke.pktest/pktest-10.pk
new file mode 100644
index 00000000..b0b18095
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-10.pk
@@ -0,0 +1,29 @@
+/* { dg-do run } */
+
+load pktest;
+
+var tests = [
+ PkTest {
+ name = "a failing test",
+ skip = "reason to skip this test case",
+ func = lambda (string name) void:
+ {
+ /* Because of the SKIP, the result will be "ok" */
+ assert (1 != 1);
+ },
+ },
+ PkTest {
+ name = "another descriptive name or phrase",
+ todo = "WIP",
+ func = lambda (string name) void:
+ {
+ assert (255 == 0xff);
+ assert ([1,2] + [3,4] == [1,2,3,4]);
+ },
+ },
+];
+
+/* { dg-command {pktest_run_noexit (tests)} } */
+/* { dg-output "1..2\\nok 1 a failing test # SKIP" } */
+/* { dg-output " reason to skip this test case\\n" } */
+/* { dg-output "ok 2 another descriptive name or phrase # TODO WIP" } */
diff --git a/testsuite/poke.pktest/pktest-11.pk
b/testsuite/poke.pktest/pktest-11.pk
new file mode 100644
index 00000000..1d2351cc
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-11.pk
@@ -0,0 +1,31 @@
+/* { dg-do run } */
+
+load pktest;
+
+var tests = [
+ PkTest {
+ name = "a failing test",
+ skip = "reason to skip this test case",
+ func = lambda (string name) void:
+ {
+ /* Because of the SKIP, the result will be "ok" */
+ assert (1 != 1);
+ },
+ },
+ PkTest {
+ name = "another failing test",
+ todo = "WIP",
+ func = lambda (string name) void:
+ {
+ /* This will report "not ok".
+ Test harness should this as expected failure. */
+ assert (255 != 0xff);
+ },
+ },
+];
+
+/* { dg-command {pktest_run_noexit (tests)} } */
+/* { dg-output "1..2\\nok 1 a failing test # SKIP" } */
+/* { dg-output " reason to skip this test case\\n" } */
+/* { dg-output "not ok 2 another failing test: assertion failed at" } */
+/* { dg-output ".*:22:9 # TODO WIP\\n" } */
diff --git a/testsuite/poke.pktest/pktest-2.pk
b/testsuite/poke.pktest/pktest-2.pk
new file mode 100644
index 00000000..730c5e9f
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-2.pk
@@ -0,0 +1,6 @@
+/* { dg-do run } */
+
+load pktest;
+
+/* { dg-command {pktest_run_noexit (PkTest[2] ())} } */
+/* { dg-output "1..2\\nok 1 \\nok 2 \\n" } */
diff --git a/testsuite/poke.pktest/pktest-3.pk
b/testsuite/poke.pktest/pktest-3.pk
new file mode 100644
index 00000000..d99d1526
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-3.pk
@@ -0,0 +1,6 @@
+/* { dg-do run } */
+
+load pktest;
+
+/* { dg-command {pktest_run_noexit (PkTest[2] (), "Reason to skip")} } */
+/* { dg-output "1..0 # Skipped: Reason to skip\\n" } */
diff --git a/testsuite/poke.pktest/pktest-4.pk
b/testsuite/poke.pktest/pktest-4.pk
new file mode 100644
index 00000000..789f65eb
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-4.pk
@@ -0,0 +1,17 @@
+/* { dg-do run } */
+
+load pktest;
+
+var tests = [
+ PkTest {
+ name = "a descriptive name or phrase",
+ func = lambda (string name) void:
+ {
+ assert (1 == 1);
+ assert ("elf" != "ELF");
+ },
+ },
+];
+
+/* { dg-command {pktest_run_noexit (tests)} } */
+/* { dg-output "1..1\\nok 1 a descriptive name or phrase\\n" } */
diff --git a/testsuite/poke.pktest/pktest-5.pk
b/testsuite/poke.pktest/pktest-5.pk
new file mode 100644
index 00000000..69395317
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-5.pk
@@ -0,0 +1,17 @@
+/* { dg-do run } */
+
+load pktest;
+
+var tests = [
+ PkTest {
+ name = "a failing test",
+ func = lambda (string name) void:
+ {
+ assert (1 != 1); /* `assert` starts at line 10, column 9 */
+ },
+ },
+];
+
+/* { dg-command {pktest_run_noexit (tests)} } */
+/* { dg-output "1..1\\nnot ok 1 a failing test: assertion failed at" } */
+/* { dg-output " .*:10:9\\n" } */
diff --git a/testsuite/poke.pktest/pktest-6.pk
b/testsuite/poke.pktest/pktest-6.pk
new file mode 100644
index 00000000..47ac5414
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-6.pk
@@ -0,0 +1,26 @@
+/* { dg-do run } */
+
+load pktest;
+
+var tests = [
+ PkTest {
+ name = "a descriptive name or phrase",
+ func = lambda (string name) void:
+ {
+ assert (1 == 1);
+ assert ("elf" != "ELF");
+ },
+ },
+ PkTest {
+ name = "another descriptive name or phrase",
+ func = lambda (string name) void:
+ {
+ assert (255 == 0xff);
+ assert ([1,2] + [3,4] == [1,2,3,4]);
+ },
+ },
+];
+
+/* { dg-command {pktest_run_noexit (tests)} } */
+/* { dg-output "1..2\\nok 1 a descriptive name or phrase\\n" } */
+/* { dg-output "ok 2 another descriptive name or phrase\\n" } */
diff --git a/testsuite/poke.pktest/pktest-7.pk
b/testsuite/poke.pktest/pktest-7.pk
new file mode 100644
index 00000000..bddf9cf6
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-7.pk
@@ -0,0 +1,26 @@
+/* { dg-do run } */
+
+load pktest;
+
+var tests = [
+ PkTest {
+ name = "a failing test",
+ func = lambda (string name) void:
+ {
+ assert (1 != 1); /* `assert` starts at line 10, column 9 */
+ },
+ },
+ PkTest {
+ name = "another failing test",
+ func = lambda (string name) void:
+ {
+ assert (255 != 0xff); /* `assert` starts at line 17, column 9 */
+ },
+ },
+];
+
+/* { dg-command {pktest_run_noexit (tests)} } */
+/* { dg-output "1..2\\nnot ok 1 a failing test:" } */
+/* { dg-output " assertion failed at .*:10:9\\n" } */
+/* { dg-output "not ok 2 another failing test:" } */
+/* { dg-output " assertion failed at .*:17:9\\n" } */
diff --git a/testsuite/poke.pktest/pktest-8.pk
b/testsuite/poke.pktest/pktest-8.pk
new file mode 100644
index 00000000..000bba6e
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-8.pk
@@ -0,0 +1,28 @@
+/* { dg-do run } */
+
+load pktest;
+
+var tests = [
+ PkTest {
+ name = "a descriptive name or phrase",
+ skip = "my reason to skip this test case",
+ func = lambda (string name) void:
+ {
+ assert (1 == 1);
+ assert ("elf" != "ELF");
+ },
+ },
+ PkTest {
+ name = "another descriptive name or phrase",
+ func = lambda (string name) void:
+ {
+ assert (255 == 0xff);
+ assert ([1,2] + [3,4] == [1,2,3,4]);
+ },
+ },
+];
+
+/* { dg-command {pktest_run_noexit (tests)} } */
+/* { dg-output "1..2\\nok 1 a descriptive name or phrase # SKIP" } */
+/* { dg-output " my reason to skip this test case\\n" } */
+/* { dg-output "ok 2 another descriptive name or phrase\\n" } */
diff --git a/testsuite/poke.pktest/pktest-9.pk
b/testsuite/poke.pktest/pktest-9.pk
new file mode 100644
index 00000000..97880d21
--- /dev/null
+++ b/testsuite/poke.pktest/pktest-9.pk
@@ -0,0 +1,29 @@
+/* { dg-do run } */
+
+load pktest;
+
+var tests = [
+ PkTest {
+ name = "a descriptive name or phrase",
+ skip = "my reason to skip this test case",
+ func = lambda (string name) void:
+ {
+ assert (1 == 1);
+ assert ("elf" != "ELF");
+ },
+ },
+ PkTest {
+ name = "another descriptive name or phrase",
+ todo = "WIP",
+ func = lambda (string name) void:
+ {
+ assert (255 == 0xff);
+ assert ([1,2] + [3,4] == [1,2,3,4]);
+ },
+ },
+];
+
+/* { dg-command {pktest_run_noexit (tests)} } */
+/* { dg-output "1..2\\nok 1 a descriptive name or phrase # SKIP" } */
+/* { dg-output " my reason to skip this test case\\n" } */
+/* { dg-output "ok 2 another descriptive name or phrase # TODO WIP" } */
diff --git a/testsuite/poke.pktest/pktest.exp b/testsuite/poke.pktest/pktest.exp
new file mode 100644
index 00000000..f579b167
--- /dev/null
+++ b/testsuite/poke.pktest/pktest.exp
@@ -0,0 +1,22 @@
+# pktest.exp - Tests for the pktest pickle
+#
+# Copyright (C) 2020 The poke authors
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA 02110-1301,
USA.
+
+load_lib ${tool}-dg.exp
+dg-init
+dg-runtest [lsort [glob -nocomplain $srcdir/poke.pktest/*.pk]] {} {}
+dg-finish
--
2.29.2
- [PATCH v2 1/2] pickles: Add new pickle `pktest.pk`,
Mohammad-Reza Nabipoor <=