Commit Graph

58 Commits

Author SHA1 Message Date
Daan De Meyer
1d77ac19cf test: Only move journal file if we didn't just unlink it 2025-01-14 10:21:45 +01:00
Daan De Meyer
79ac78e368 test: Fix bug in integration test wrapper 2025-01-13 11:06:37 +01:00
Daan De Meyer
1f17ec0ed4 test: Move StateDirectory= directive into dropin
The integration-test-setup calls require StateDirectory= but some
tests override the test unit used which then won't have StateDirectory=
so let's move StateDirectory= into the dropin as well to avoid this
issue.
2025-01-10 16:14:01 +01:00
Daan De Meyer
84b30442d2 test: Don't register machines with machined unless we're in interactive mode 2025-01-10 16:14:01 +01:00
Daan De Meyer
e49fdecd16 test: Add option to save in progress test journals to /tmp
The journal isn't the best at being fast, especially when writing
to disk and not to memory, which can cause integration tests to
grind to a halt on beefy systems due to all the systemd-journal-remote
instances not being able to write journal entries to disk fast enough.

Let's introduce an option to allow writing in progress test journals
to use /tmp which can be used on beefy systems with lots of memory to
speed things up.
2025-01-10 16:13:58 +01:00
Daan De Meyer
96403d5121 tree-wide: Fix python formatting
The new release of ruff formats a few more things which causes linter
failures in CI so let's fix those formatting nits.
2025-01-10 14:58:56 +09:00
Daan De Meyer
794d456cf4 test: Only plug in integration-test-setup.sh in interactive mode
If we're not running interactively, there's no point in the features
from integration-test-setup.sh which are intended for interactive
development and debugging so lets skip adding it in that case.
2025-01-09 11:28:15 +01:00
Daan De Meyer
28baef47a3 mkosi: Migrate more deprecated options 2025-01-06 12:07:53 +01:00
Yu Watanabe
1bdb9e808f test: extract sanitizer reports from journal 2024-12-10 11:01:48 +09:00
Daan De Meyer
8f51cf6981 test: Set kernel loglevel to INFO when running tests unattended
This makes sure all kernel log messages are logged to the console.
This should be helpful during shutdown to detect possible issues with
journald when the logs can't be written to the journal itself anymore
but are written to kmsg.
2024-12-08 12:55:43 +01:00
Daan De Meyer
e022e73e3f test: Implement TEST_PREFER_QEMU and use it in one of the mkosi jobs
We want to make sure the integration tests that don't require qemu
can run successfully both in an nspawn container and in a qemu VM.
So let's add one more knob TEST_PREFER_QEMU=1 to run jobs that normally
require nspawn in qemu instead.

Running these tests in qemu is also possible by not running as root but
that's very implicit so we add an explicit knob instead to make it explicit
that we want to run these in qemu instead of nspawn.
2024-12-05 16:43:11 +01:00
Daan De Meyer
c45174f05d ci: Implement coverage on top of mkosi 2024-12-05 00:21:57 +01:00
Daan De Meyer
4a346b779a test: Dump coredumps from journal in the integration test wrapper
Fixes #35277
2024-11-25 19:12:11 +09:00
Daan De Meyer
ceca7c5005 test: Fix typing errors in integration-test-wrapper.py 2024-11-24 16:47:20 +01:00
Daan De Meyer
4f969b20b0 test: Format integration-test-wrapper.py 2024-11-24 16:47:20 +01:00
Daan De Meyer
6d2fd490cf integration-test-wrapper: Remove unneeded format strings 2024-11-24 16:47:20 +01:00
Luca Boccassi
0d7f5a9ae6 mkosi: mark test as skipped when QEMU crashes
On Ubuntu/Debian infrastructure QEMU crashes a lot, so mark the test
as skipped in that case as there's nothing we can do about it and
we shouldn't mark runs as failed
2024-10-07 23:58:38 +01:00
Luca Boccassi
8f4bbd096b test: fix TEST_SKIP for test cases with subtests
TEST-64-UDEV-STORAGE is invoked with the subtest appended, so TEST_SKIP=TEST-64-UDEV-STORAGE
does not work. Fix it by using TEST_SKIP as a partial match.

Follow-up for ddc91af4ea
2024-09-11 13:55:53 +02:00
Daan De Meyer
7560a5393a test: Set show_status=error
The TEST-64-UDEV-STORAGE tests fail before we even start the test.
Let's set show_status=error to get more information when those failures
happen.
2024-08-28 19:20:56 +02:00
Daan De Meyer
c8e7cfeddc tests: Don't override QemuKvm= value if TEST_NO_KVM=0
Let's disable KVM if TEST_NO_KVM=1 is set but let's not specify anything
if it's not set so the QemuKvm= setting from mkosi.conf is used.
2024-08-21 01:52:09 +09:00
Luca Boccassi
ddc91af4ea test: add TEST_SKIP to mkosi integration test wrapper
Takes a space-separate list of test names, allows to skip one or more tests
2024-08-15 10:39:51 +02:00
Daan De Meyer
dd1c01b20f test: Implement TEST_SHELL for mkosi based test runner
TEST_SHELL can be used to get a shell in the integration test
environment without actually immediately starting the test.
2024-08-05 15:00:24 +02:00
Daan De Meyer
578ee05155 test: Don't mount build sources into image when running non-interactively 2024-07-29 13:40:42 +02:00
Daan De Meyer
20345a86b7 mkosi: Adapt configuration to take into account configuration rework
In https://github.com/systemd/mkosi/pull/2847, the '@' specifier is
removed, CLI arguments take priority over configuration files again
and the "main" image is defined at the top level instead of in
mkosi.images/. Additionally, not every setting from the top level
configuration is inherited by the images in mkosi.images/ anymore,
only settings which make sense to be inherited are inherited.

This commit gets rid of all the usages of '@', moves the "main" image
configuration from mkosi.images/system to the top level and gets rid
of various hacks we had in place to deal with quirks of the old
configuration parsing logic.

We also remove usages of Images= and --append as these options are
removed by the mentioned PR.
2024-07-09 08:07:09 +02:00
Daan De Meyer
32f3617fd7 test: pull in multi-user.target explicitly
Let's make sure we pull in multi-user.target so that we get a console
when a test fails and we're running meson test with --interactive.
2024-06-28 13:34:37 +02:00
Luca Boccassi
59d4970a55 mkosi: add support for TEST_SAVE_JOURNAL to integration test wrapper
Same variable and values as supported by the shell suite
2024-06-25 12:16:59 +02:00
Luca Boccassi
464d182b3e test: support TEST_NO_QEMU in mkosi integration wrapper
Same as the old integration test suite, allow skipping tests that
require qemu.
ppc64el's vsock support doesn't appear to work, so we'll skip it,
as it is already done in the legacy framework.
2024-06-17 15:52:50 +01:00
Luca Boccassi
7d2701e7d1 test: support TEST_NO_KVM
The shell integration suite allows to manually deselect KVM, so
suppor the same env var for the same purpose in python.
2024-06-17 15:52:50 +01:00
Luca Boccassi
626518ecd5 test: drop obsolete comment
We want to keep various logic here instead of mkosi, so drop the
temporary comment
2024-06-17 15:52:50 +01:00
Luca Boccassi
88c2ef1f75 Revert "test: Run end.sh when running integration tests with mkosi"
It seems this introduced a regression in the CentOS CI;

14:25:58 FAILED TASKS:14:25:58 -------------
14:25:58 TEST-03-JOBS
14:25:58 TEST-52-HONORFIRSTSHUTDOWN
14:25:58 TEST-63-PATH

Revert for now.

This reverts commit da3c6fc553.
2024-06-03 16:29:23 +01:00
Daan De Meyer
da3c6fc553 test: Run end.sh when running integration tests with mkosi 2024-06-02 19:15:21 +02:00
Daan De Meyer
1e5a1bbe02 test: Run tests that don't need a vm in systemd-nspawn
If we're not running the test as root, stick to using a virtual
machine, as mkosi can't do rootless nspawn yet.
2024-05-29 14:10:50 +02:00
Daan De Meyer
e2f9f3409a meson: Look up mkosi once 2024-05-27 12:02:31 +02:00
Daan De Meyer
b13b7978e7 test: Enable TEST-08-INITRD on mkosi 2024-05-15 10:09:53 +02:00
Daan De Meyer
eb4c962aa4 test: Ignore configure scripts in minimal images
These are always aimed at the system image so ignore any specified
configure scripts in the minimal images.
2024-05-14 12:43:28 +02:00
Daan De Meyer
348f5017a4 test: Generate basic testsuite services with meson
Most of these are identical, let's just generate from a meson
template.
2024-05-14 12:43:28 +02:00
Daan De Meyer
f483e08313 test: Simplify argument/variable names 2024-05-14 12:43:28 +02:00
Daan De Meyer
7a321b5a21 test: Rename testsuite-XX units to match test name
Having these named differently than the test itself mostly creates
unecessary confusion and makes writing logic against the tests harder
so let's rename the testsuite-xx units and scripts to just use the
test name itself.
2024-05-14 12:43:28 +02:00
Daan De Meyer
f1f87f3b2e test: Add infra to mark integration tests as slow
Some integration tests take much more time than others, let's add
a test param that can be used to configure this and integrate it
with the slow-tests meson option.
2024-05-07 19:06:42 +02:00
Daan De Meyer
a77f65d0c9 ci: Print a helpful link to download and view a failed test's journal 2024-05-07 19:06:39 +02:00
Daan De Meyer
ca2e19f2b7 test: Don't keep journals for skipped tests
Let's make sure we don't save journals for tests that were skipped.
2024-05-07 17:34:42 +02:00
Daan De Meyer
1f2c9bda49 test: Default to linux qemu firmware
Direct kernel boot results in much faster boot times so let's use
it by default.

We disable it for tests that need to reboot because +-50% of the
time, doing a reboot when using direct kernel boot causes qemu to
hang on reboot. Until we figure that out, let's use UEFI for the
tests that need to reboot.
2024-05-06 11:57:09 +02:00
Daan De Meyer
d91bb1cbf0 test: Always shutdown on test success in mkosi
When we want to get an interactive shell in a test that fails because
of a race condition, we might need to run the test a few times with
--repeat before it fails. However, currently, when -i is used, the VM
needs to be shut down manually each time before the next run can start.

Let's always shut down the VM if the test succeeds so that --repeat can
be used with -i to run the test until it fails and then get an interactive
shell in the VM.
2024-05-06 10:56:54 +02:00
Daan De Meyer
0596237e0e test: Use MESON_TEST_ITERATION if available
This allows running the same test multiple times concurrently with
meson's --repeat option. Proposed upstream but not yet merged:
https://github.com/mesonbuild/meson/pull/13177.
2024-05-06 10:56:54 +02:00
Daan De Meyer
3cb61e0d1c test: Pass through test matching environment variables to the mkosi VM 2024-05-06 10:56:54 +02:00
Daan De Meyer
bdade5f597 test: Formatting fixes 2024-05-06 10:56:43 +02:00
Daan De Meyer
69e02d94f5 meson: Remove --debug from mkosi arguments
The exit status issue for which we introduced this was fixed so
let's remove --debug again to make the meson output less verbose.
2024-05-05 14:49:21 +02:00
Daan De Meyer
9a69900a4b test: Make journal storage configurable per test and make persistent for TEST-09-JOURNAL
Also fix a bug in the logic since it's 'volatile' and not 'runtime'.
2024-04-30 22:10:05 +02:00
Daan De Meyer
27f166c523 mkosi: Use systemd.crash_action=poweroff in integration tests
Some integration tests expect to be able to reboot so switch to using
the new systemd.crash_action=poweroff instead.
2024-04-30 17:01:41 +02:00
Daan De Meyer
e911a335ed mkosi: Mask systemd-networkd-wait-online when --runtime-network=none
Otherwise it will wait forever waiting for an interface to appear.
See https://github.com/systemd/systemd/issues/29388.
2024-04-30 17:01:41 +02:00