src/tests/conftest.py, branch releng/14.3

Testing: add framework for the kernel unit tests.

2023-04-14T15:47:55Z

This changes intends to reduce the bar to the kernel unit-testing by introducing a new kernel-testing framework ("ktest") based on Netlink, loadable test modules and python test suite integration. This framework provides the following features: * Integration to the FreeBSD test suite * Automatic test discovery * Automatic test module loading * Minimal boiler-plate code in both kernel and userland * Passing any metadata to the test * Convenient environment pre-setup using python testing framework * Streaming messages from the kernel to the userland * Running tests in the dedicated taskqueues * Skipping or parametrizing tests Differential Revision: https://reviews.freebsd.org/D39385 MFC after: 2 weeks

tests: refactor atf_python a bit

2023-04-01T19:44:37Z

* Move more logic from conftest.py to the actual atf_pytest handler * Move nodeid_to_method_name() to the utils.py so it can be shared MFC after: 2 weeks

testing: handling non-root users with VNETs in pytest-based tests.

2023-02-09T14:49:27Z

Currently isolation and resource requirements are handled directly by the kyua runner, based on the requirements specified by the test. It works well for simple tests, but may cause discrepancy with tests doing complex pre-setups. For example, all tests that perform VNET setups require root access to properly function. This change adds additional handling of the "require_user" property within the python testing framework. Specifically, it requests root access if the test class signals its root requirements and drops privileges to the desired user after performing the pre-setup. Differential Revision: https://reviews.freebsd.org/D37923 MFC after: 2 weeks

pytest: silence deprecation warning in the pytest atf wrapper

2022-12-28T15:53:48Z

Convert `pytest.mark` decorator to the `pytest.hookimpl()` one, as suggested by the https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers Differential Revision: https://reviews.freebsd.org/D37884 MFC after: 2 weeks

testing: pass ATF vars to pytest via env instead of arguments.

2022-06-28T12:20:16Z

This change is a continuation of 9c42645a1e4d workaround. Apparently pytest argument parser is not happy when parsing values with spaces or just more than one --atf-var argument. Switch wrapper to send these kv pairs as env variables. Specifically, use _ATF_VAR_key=value format to distinguish from the other vars. Add the `atf_vars` fixture returning all passed kv pairs as a dict. Reviewed by: lwhsu Differential Revision: https://reviews.freebsd.org/D35625 MFC after: 2 weeks

testing: Add basic atf support to pytest.

2022-06-25T19:25:15Z

Implementation consists of the pytest plugin implementing ATF format and a simple C++ wrapper, which reorders the provided arguments from ATF format to the format understandable by pytest. Each test has this wrapper specified after the shebang. When kyua executes the test, wrapper calls pytest, which loads atf plugin, does the work and returns the result. Additionally, a separate python "package", `/usr/tests/atf_python` has been added to collect code that may be useful across different tests. Current limitations: * Opaque metadata passing via X-Name properties. Require some fixtures to write * `-s srcdir` parameter passed by the runner is ignored. * No `atf-c-api(3)` or similar - relying on pytest framework & existing python libraries * No support for `atf_tc__config_var()` & `atf_tc_set_md_var()`. Can be probably implemented with env variables & autoload fixtures Differential Revision: https://reviews.freebsd.org/D31084 Reviewed by: kp, ngie