Regression test suite [Admidio-Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Regression test suite ======

The Admidio regression test suite protects the behaviour of Admidio Core against regressions during development. It is useful both for developers contributing to Admidio itself and for third-party developers who build modules, plugins or integrations against Admidio.

The suite does not only test isolated PHP classes. Depending on the layer it exercises the real Admidio database abstraction, Entities, Services, the production installer, the command-line interface, filesystem handling and mail delivery.

Once the environment is set up, running the tests after a change is a single command ''composer test:all'' in the Admidio installation directory:

<code bash>
PS C:\Users\OpenTools\Development\admidio> composer test:all
PHPUnit 9.6.36 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.3
Configuration: C:\Users\OpenTools\Development\admidio\phpunit.xml

......
  Setting up Admidio test database...
  Installing Admidio production setup...
  ✓ Dropped 49 existing tables
  ✓ Database initialized
  ✓ Schema created
  ✓ Default data installed
  ✓ Administrator user created

.........................................................  63 / 412 ( 15%)
............................................................... 126 / 412 ( 30%)
............................................................... 189 / 412 ( 45%)
............................................................... 252 / 412 ( 61%)
............................................................... 315 / 412 ( 76%)
............................................................... 378 / 412 ( 91%)
..................................                              412 / 412 (100%)

Time: 07:05.459, Memory: 50.00 MB

OK (412 tests, 6450 assertions)
</code>

The test count in this example output is only illustrative, it grows with the suite.

**The suite is intentionally destructive. Never point it at a production database or at a development database whose content you want to keep.** Several guards make this hard to do by accident, see [[#safety_mechanisms|Safety mechanisms]]. Do not disable them.

The suite is not a browser UI test suite. Many tests deliberately stop at the Entity, Service or CLI acceptance boundary. Historic version-to-version database upgrade coverage is a separate lifecycle concern: a green regression run only covers upgrade paths if the current branch actually contains upgrade tests.

For the general setup of an Admidio development installation also see [[en:entwickler:testumgebung_einrichten|Set up a test environment]].

===== Quick start =====

==== First run ====

From the root of your Admidio checkout:

<code bash>
# 1. install the development dependencies (PHPUnit, symfony/process)
composer install

# 2. start the test services (MariaDB, PostgreSQL, Mailpit)
docker compose -f docker-compose.test.yml up -d

# 3. create the test configuration
cp .env.test.example .env.test

# 4. prepare and verify the environment
php tests/bin/setup-test-env.php

# 5. run everything
composer test:all
</code>

Step 2 takes about 30 seconds until the databases accept connections. The setup script in step 4 waits for them, so you do not have to.

On Windows use ''copy .env.test.example .env.test'' or ''Copy-Item'' instead of ''cp''.

The defaults in ''.env.test.example'' match the services in ''docker-compose.test.yml'', so on the recommended setup no value has to be changed.

==== Everyday commands ====

^ Command ^ What it runs ^ Database needed ^
| ''composer test:unit'' | The ''Unit Tests'' suite, ''tests/Unit'' | no |
| ''composer test:integration'' | The ''Integration Tests'' suite, ''tests/Integration'' | yes |
| ''composer test:cli'' | The ''CLI Tests'' suite, ''tests/Cli'' | yes |
| ''composer test'' | Unit and Integration tests | yes |
| ''composer test:all'' | Everything ''phpunit.xml'' defines, all three suites | yes |
| ''composer test:coverage'' | Full run with an HTML report in ''tests/reports/coverage'' | yes |
| ''composer test:setup'' | ''php tests/bin/setup-test-env.php'' | yes |

While developing, run the test you are working on directly:

<code bash>
# one file
vendor/bin/phpunit tests/Integration/Inventory/InventoryTest.php

# one method, in the whole suite or in one file
vendor/bin/phpunit --filter testInventoryFieldAndItemLifecycleUsesProductionServices
vendor/bin/phpunit --filter testUploadDownloadAndDelete tests/Integration/Filesystem/DocumentsFilesystemServiceTest.php

# one suite with readable descriptions
vendor/bin/phpunit --testsuite="Integration Tests" --testdox
</code>

''composer test:unit'' is the fast feedback loop: it needs neither database nor Docker, because the Admidio bootstrap is only loaded once a database-backed test asks for it.

''composer test:coverage'' additionally needs Xdebug or PCOV, otherwise PHPUnit reports that no code coverage driver is available.

After a focused test passes, run ''composer test:all'' before you consider the change complete.

==== What a run changes on your machine ====

  * **The test database is rebuilt.** The first database-backed test in a PHPUnit process drops every table carrying the Admidio table prefix and reinstalls the schema with the current production installer. Tables without that prefix are left alone, but the database must still be a disposable one.
  * **Files are written below ''tests/adm_my_files''.** For the duration of a run Admidio's ''FOLDER_DATA'' points there, so nothing lands in the ''adm_my_files'' of your checkout.
  * **Mail does not leave your machine.** The mail test talks to the local Mailpit SMTP listener.
  * **Nothing else is touched.** Each database test runs inside a transaction that is rolled back afterwards. The exception are CLI subprocess tests, which commit and therefore clean up after themselves, see [[#cli_tests|CLI tests]].

===== Setting up the test environment =====

==== Requirements ====

  * a checkout of the Admidio source tree;
  * PHP as required by ''composer.json'' (currently ''^8.2'') with the extensions ''curl'', ''gd'', ''iconv'', ''json'', ''pdo'', ''simplexml'' and ''zip'';
  * Composer, to install PHPUnit and ''symfony/process'';
  * a dedicated MariaDB, MySQL or PostgreSQL test database and the matching PDO driver (''pdo_mysql'' or ''pdo_pgsql'');
  * Mailpit or another local SMTP sink for the mail test;
  * Docker, if you want the supplied test services instead of your own servers.

''ext-gd'' and ''ext-zip'' are not optional for the suite, the photo and import/export regressions use them.

Install the dependencies from the Admidio root directory:

<code bash>
composer install
</code>

==== Recommended: the supplied Docker environment ====

The repository contains ''docker-compose.test.yml'' in the root directory. It provides everything the suite needs:

^ Service ^ Image ^ Published ports ^ Database ^ User ^ Password ^
| ''mariadb'' | ''mariadb:10.11'' | 3306 | ''admidio_test'' | ''admidio'' | ''admidio_test'' |
| ''postgres'' | ''postgres:15'' | 5432 | ''admidio_test'' | ''admidio'' | ''admidio_test'' |
| ''mailpit'' | ''axllent/mailpit'' | 1025 (SMTP), 8025 (HTTP API and web UI) | — | — | — |

<code bash>
# start
docker compose -f docker-compose.test.yml up -d

# status and logs
docker compose -f docker-compose.test.yml ps
docker compose -f docker-compose.test.yml logs -f mariadb

# stop, keeping the data volumes
docker compose -f docker-compose.test.yml down

# stop and throw the databases away
docker compose -f docker-compose.test.yml down -v
</code>

The ''-f docker-compose.test.yml'' is not optional. There is no ''docker-compose.yml'' in the repository root, so a plain ''docker compose up -d'' does not start the test services.

The Mailpit web UI is at [[http://localhost:8025|http://localhost:8025]] and is the quickest way to look at what a mail test actually delivered.

The Compose file has no MySQL service, because MariaDB already occupies port 3306. MySQL 8 is a scheduled CI target. To test it locally, point the ''TEST_DB_MYSQL_*'' variables at your own instance and make sure it does not collide with the MariaDB container.

==== Alternative: your own database server ====

If you already run a database server, you only need a disposable database and a user that may create and drop tables in it.

=== MariaDB and MySQL ===

<code sql>
CREATE DATABASE admidio_test CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'admidio'@'%' IDENTIFIED BY 'admidio_test';
GRANT ALL PRIVILEGES ON admidio_test.* TO 'admidio'@'%';
FLUSH PRIVILEGES;
</code>

The user needs ''ALL PRIVILEGES'' on that one database because the suite runs the production installer, which creates the whole schema and drops it again at the start of the next run. Grant nothing outside this database.

=== PostgreSQL ===

<code sql>
CREATE ROLE admidio LOGIN PASSWORD 'admidio_test';
CREATE DATABASE admidio_test OWNER admidio ENCODING 'UTF8';
\c admidio_test
GRANT ALL ON SCHEMA public TO admidio;
</code>

The last statement matters from PostgreSQL 15 on, where the ''public'' schema no longer lets everybody create objects. The suite reads and drops the Admidio tables in the ''public'' schema of the configured database.

=== A local SMTP sink ===

Without Docker, run Mailpit directly. It is a single binary, or:

<code bash>
docker run -d --name admidio-test-mailpit -p 1025:1025 -p 8025:8025 axllent/mailpit
</code>

The mail test needs both the SMTP port and the HTTP API, because it verifies the delivered message through the API rather than trusting that the connection succeeded.

Whatever you use, adjust ''.env.test'' to match, and keep the database name a throwaway one whose name satisfies the safety check described below.

==== The test configuration in .env.test ====

Copy the supplied example and edit it if your services differ from the defaults:

<code bash>
cp .env.test.example .env.test
</code>

''.env.test'' is not in version control. The checked-in ''.env.test.example'' is the authoritative reference for the variables the current branch supports.

^ Variable ^ Meaning ^ Default ^
| ''TEST_DATABASE_ENGINE'' | Engine the run uses: ''mariadb'', ''mysql'' or ''postgres'' | required |
| ''TEST_FILES_PATH'' | Data directory of the test run, becomes Admidio's ''FOLDER_DATA'' | required, normally ''./tests/adm_my_files'' |
| ''TEST_DB_<ENGINE>_HOST'' | Database host | ''127.0.0.1'' |
| ''TEST_DB_<ENGINE>_PORT'' | Database port | ''3306'', for PostgreSQL ''5432'' |
| ''TEST_DB_<ENGINE>_USER'' | Database user | ''admidio'' |
| ''TEST_DB_<ENGINE>_PASS'' | Password of that user | empty |
| ''TEST_DB_<ENGINE>_NAME'' | Name of the test database | ''admidio_test'' |
| ''TEST_MAIL_HOST'', ''TEST_MAIL_PORT'' | SMTP listener of the mail sink | ''127.0.0.1'', ''1025'' |
| ''TEST_MAILPIT_API_HOST'', ''TEST_MAILPIT_API_PORT'' | HTTP API of Mailpit | ''127.0.0.1'', ''8025'' |

''<ENGINE>'' is the uppercase engine name, so the MariaDB host is ''TEST_DB_MARIADB_HOST''. One block exists per engine, which is why switching engines is a matter of one variable.

Only ''TEST_DATABASE_ENGINE'' and ''TEST_FILES_PATH'' are strictly required, everything else falls back to the defaults above.

Two details that regularly cost time:

  * **Use ''127.0.0.1'', not ''localhost''.** A MySQL client reads ''localhost'' as "connect through the unix socket", which does not reach a database that Docker published on a port.
  * **A variable that is already set in the process environment wins over the file.** That is how CI configures a run without any ''.env.test'', and how you override a single value for one shell:

<code bash>
TEST_DATABASE_ENGINE=postgres composer test:integration
</code>

<code powershell>
$env:TEST_DATABASE_ENGINE = 'postgres'; composer test:integration
</code>

==== The setup script ====

<code bash>
php tests/bin/setup-test-env.php
</code>

Run it once after creating ''.env.test'', and again whenever you change the engine or recreate the containers. It:

  - loads ''.env.test'' and the process environment and fails if the run is not configured;
  - creates the subdirectories of the test data directory (''documents'', ''documents_test'', ''photos'', ''temp'', ''logs'', ''import'', ''export'');
  - connects to the database, retrying for up to a minute, because a container answers on its port before the server accepts connections;
  - reports whether the mail sink is reachable, which is optional and does not stop the setup;
  - writes the ''.test-environment-marker'' file.

The script accepts ''--db=mariadb|postgres|mysql'' to check a different engine than the configured one:

<code bash>
php tests/bin/setup-test-env.php --db=postgres
</code>

**This option only affects the script.** The test run itself always reads ''TEST_DATABASE_ENGINE''. The engine is not a command-line option of PHPUnit.

==== Switching the database engine ====

To run the suite against another engine, change the engine in ''.env.test'' (or in the environment), then run the setup script and the tests again:

<code bash>
# in .env.test:  TEST_DATABASE_ENGINE=postgres
php tests/bin/setup-test-env.php
composer test:all
</code>

Each engine has its own connection block, so both configurations can stay in the file side by side. The schema is rebuilt at the start of every run, so no manual cleanup is needed when you switch back and forth.

Before submitting a change that touches SQL, Entities or the installer, run the suite at least against MariaDB and PostgreSQL. Those are the two engines every pull request is tested on.

==== Safety mechanisms ====

The suite refuses to run in an environment that does not look like a test environment. The guards are:

^ Guard ^ Enforced in ^ Rule ^
| Database name | ''tests/bootstrap.php'' | The name must contain ''test'' as a separate token, delimited by start, end, ''_'' or ''-''. ''admidio_test'', ''test'' and ''test-db'' pass, ''admidiotest'' and ''latest'' do not. |
| Data directory | ''tests/env.php'' | ''TEST_FILES_PATH'' must contain ''test'' and must resolve to a directory //inside// the checkout, because Admidio addresses everything it writes as ''ADMIDIO_PATH . FOLDER_DATA''. |
| Table prefix | ''TestDatabaseInitializer'' | Only tables starting with the configured Admidio table prefix are dropped, so unrelated tables in the same database survive. |
| Filesystem root | ''FilesystemTestCase'' | ''FOLDER_DATA'' has to resolve to ''tests/adm_my_files'' and the committed marker ''tests/adm_my_files/.admidio-regression-test'' has to exist, otherwise destructive operations are refused. Every path that is created or cleaned up is checked against that root again. |

There are two marker files in ''tests/adm_my_files'' and they are not interchangeable:

  * ''.admidio-regression-test'' is committed to git and is the guard the filesystem tests check. Do not delete it.
  * ''.test-environment-marker'' is created automatically by the setup script and the bootstrap. It is informational.

Never point ''TEST_FILES_PATH'' at the ''adm_my_files'' of a real installation, never use a shared database, and do not work around a guard that fires. It is telling you that the environment is not the one you think it is.

==== Continuous integration ====

''.github/workflows/regression-tests.yml'' runs the suite on every pull request against ''master'', on pushes to that branch and on a schedule. CI configures everything through workflow environment variables and therefore needs no ''.env.test''.

^ Job ^ When ^ What it does ^
| ''fast-checks'' | always | ''php -l'' over ''src'', ''system'', ''modules'', ''install'', ''tests'', then ''composer validate --strict'' and ''composer test:unit'' |
| MariaDB 10.11 | always, after fast-checks | setup script, ''composer test:integration'', ''composer test:cli'' |
| PostgreSQL 15 | always, after fast-checks | setup script, ''composer test:integration'', ''composer test:cli'' |
| MySQL 8.0 | Mondays 02:00 UTC, or on demand | same as the other database jobs |

The workflow can also be started manually through //workflow_dispatch// with a choice of engine, which is the quickest way to get a MySQL result for one branch.

CI runs no mail job. The Mailpit test is a local integration test.

===== How the suite is built =====

==== Test layers ====

The suite is divided into layers. A test belongs in the lowest layer that can reliably test the behaviour in question.

^ Layer ^ Location ^ Purpose ^
| Unit tests | ''tests/Unit/'' | Pure production logic without database, filesystem or network access. |
| Integration tests | ''tests/Integration/'' | Real Admidio Entities, Services, permissions and database behaviour. |
| Filesystem integration tests | ''tests/Integration/Filesystem/'' | Production document, photo and import/export code against the protected test data directory. |
| Mail integration tests | ''tests/Integration/Mail/'' | The real Admidio mail path against a local SMTP sink. |
| CLI tests | ''tests/Cli/'' | CLI contracts and complete workflows through the real Admidio command-line entry point. |
| Installation coverage | ''TestDatabaseInitializer'' and ''tests/Cli/'' | The current production installer has to produce a usable Admidio database. |

The ''tests/Integration'' directory is organised by subject area — ''Announcements'', ''Cascading'', ''Categories'', ''Changelog'', ''Core'', ''Database'', ''Documents'', ''Events'', ''Exchange'', ''Filesystem'', ''Inventory'', ''Lists'', ''Mail'', ''Menu'', ''Messages'', ''Organizations'', ''Permissions'', ''Photos'', ''ProfileFields'', ''Roles'', ''Security'', ''Services'', ''Users'', ''Weblinks'', ''Workflows''. Add a new directory when a new area gets its first test.

==== Directory layout ====

<code>
tests/
├── bootstrap.php               PHPUnit bootstrap: autoloader, session, safety checks
├── env.php                     reads .env.test and the process environment
├── bootstrap-admidio.php       full Admidio bootstrap, loaded lazily by DatabaseTestCase
├── bin/
│   └── setup-test-env.php      one-time preparation and verification of the environment
├── Support/                    base classes, traits and fixtures - contains no tests
│   ├── AdmidioTestCase.php         root base class, generic assertions
│   ├── DatabaseTestCase.php        database connection, schema, transaction isolation
│   ├── AdministratorTestCase.php   runs a test as the real installed administrator
│   ├── FilesystemTestCase.php      guarded filesystem access and cleanup
│   ├── CliSubprocess.php           trait, starts ./admidio as a real process
│   ├── PermissionContext.php       trait, switches organization and current user
│   ├── AdmidioTestFixture.php      creates prerequisite orgs, users, roles, categories
│   └── TestDatabaseInitializer.php runs the production installer against the test database
├── Unit/                       no database, no filesystem, no network
├── Integration/                one directory per subject area
├── Cli/                        CLI contract and workflow tests
└── adm_my_files/               FOLDER_DATA of a test run, git-ignored except the marker
</code>

The three bootstraps are separate on purpose. ''tests/bootstrap.php'' is what ''phpunit.xml'' loads and stays free of Admidio and of the database, so ''composer test:unit'' remains a true unit-test boundary. ''DatabaseTestCase'' pulls in ''tests/bootstrap-admidio.php'' the first time a test really needs Admidio.

==== Base classes and helpers ====

The base classes build on each other:

<code>
PHPUnit\Framework\TestCase
    └── AdmidioTestCase          generic assertions, no external state
            └── DatabaseTestCase         database, schema, transaction per test
                    └── AdministratorTestCase    the real administrator as current user
                            └── FilesystemTestCase       guarded test data directory
</code>

^ Base class ^ Gives you ^ Choose it when ^
| ''AdmidioTestCase'' | ''assertValidUuid()'', ''assertValidTimestamp()'', ''assertArrayHasKeys()'' | the code under test needs no database, filesystem or network |
| ''DatabaseTestCase'' | the Admidio bootstrap, an installed schema, ''getDatabase()'', an automatic transaction rollback after each test | the test touches Entities, Services or SQL |
| ''AdministratorTestCase'' | additionally ''$gCurrentUser'' set to the administrator the production installer created, and ''$gValidLogin'' | the production path under test requires administrator rights |
| ''FilesystemTestCase'' | additionally ''getTestDataRoot()'', ''createIsolatedDirectory()'', ''createFixtureFile()'', ''registerCleanupPath()'' and automatic, verified cleanup | production code writes or deletes files |

''AdministratorTestCase'' does not mock anything and injects no rights. It looks up the real administrator membership created by the installer and loads that user through the normal ''User'' Entity, then restores the previous globals in ''tearDown()''.

Two traits add capabilities to any of those classes:

^ Trait ^ Provides ^
| ''CliSubprocess'' | ''runCli()'' starts ''./admidio'' as a real process against the test database, ''cliJson()'' decodes its JSON output, ''cliConfigurationFile()'' writes the configuration through the production ''Installation'' service |
| ''PermissionContext'' | ''withOrganization()'', ''withCurrentUser()'', ''loadUserInOrganization()'' and ''settingsOf()'' — Admidio resolves rights against globals, so a test has to set and restore them around the code it exercises |

''AdmidioTestFixture'' is the shared fixture helper. Instantiate it with the test database and use it for prerequisites:

<code php>
protected function getFixture(): AdmidioTestFixture
{
    return new AdmidioTestFixture($this->getDatabase());
}
</code>

It offers, among others, ''createAndSaveOrganization()'', ''createAndSaveUser()'', ''createAndSaveRole()'', ''createAndSaveRoleWithRights()'', ''createAndSaveCategory()'', ''assignUserToRole()'', ''assignUserToRolePeriod()'' and ''seedDefaultPreferences()''. An organization created by the fixture starts without preferences, so seed them if the behaviour under test reads a setting.

==== Schema creation and test isolation ====

Database-backed runs do not load an SQL dump. The first ''DatabaseTestCase'' in a PHPUnit process calls ''TestDatabaseInitializer'', which drops the prefixed tables and then runs the real production installer. An old dump would let an installer or schema regression pass unnoticed; this way a broken ''install/db_scripts'' change fails the run immediately.

The installation the tests work on therefore always contains:

  * one organization, //Test Organization// with the short name ''TEST'';
  * the default categories, roles and profile fields of a fresh installation;
  * one administrator with the login ''admin''.

Every individual test then runs inside a transaction that ''DatabaseTestCase'' rolls back in ''tearDown()''. That keeps tests isolated without recreating the schema for each of them, and it means:

  * a test must not depend on data another test created;
  * a test must not depend on execution order — ''phpunit.xml'' uses ''executionOrder="depends,defects"'', so tests do not even run in source order;
  * anything a test writes through the normal database connection disappears afterwards, and nothing has to be deleted by hand.

The two exceptions are CLI subprocesses, which have their own connection and commit, and files, which the filesystem base class removes explicitly.

==== Core principle: test Admidio, not the test suite ====

The most important rule when adding a regression test is:

**The action being tested must be performed by production Admidio code.**

A test must not reproduce the expected Admidio behaviour inside a fixture, helper or mock and then verify the behaviour it implemented itself. A test for a Service operation should:

  - create only the prerequisites the test needs;
  - call the real Admidio Service;
  - let that Service use the normal Entities and database abstraction;
  - verify the resulting state independently, for example through a new Entity instance or a direct prepared query.

This matters most for operations that do more than one thing: reciprocal user relations, related records, organization boundaries, changelog entries, sequence values, permissions, messages and emails, thumbnails and archives.

Fixtures create prerequisites. They are not substitutes for the production workflow under test. If production code is expected to write two reciprocal relationship records, the test must not write those two records itself and then assert that both exist.

===== Adding a test =====

==== Where does the test belong? ====

^ The behaviour you want to protect ^ Layer ^ Base class ^
| A pure function or value object, no state | ''tests/Unit/'' | ''AdmidioTestCase'' |
| An Entity, a Service, a query, a cascade | ''tests/Integration/<Area>/'' | ''DatabaseTestCase'' |
| The same, but the production path checks administrator rights | ''tests/Integration/<Area>/'' | ''AdministratorTestCase'' |
| Visibility, role rights, organization isolation | ''tests/Integration/Permissions/'' or the module's area | ''DatabaseTestCase'' plus the ''PermissionContext'' trait |
| Anything that writes, reads or deletes files | ''tests/Integration/Filesystem/'' | ''FilesystemTestCase'' |
| Sending mail | ''tests/Integration/Mail/'' | ''AdministratorTestCase'' |
| A command of the Admidio CLI | ''tests/Cli/'' | ''DatabaseTestCase'' plus the ''CliSubprocess'' trait |

When in doubt, pick the lowest layer that can actually fail when the feature breaks.

==== Recipe: a regression test for a bug fix ====

A regression test for a bug should fail before the fix and pass after it. That is the only way to know it protects anything.

  - **Reproduce the defect** and find the production class that misbehaves: Entity, Service, CLI command or query.
  - **Write the smallest test** that calls that production code and asserts the correct behaviour. Put it in the area directory of the module, name it after the behaviour, not after the ticket.
  - **Run it and watch it fail** — and check //why// it fails. A test that errors out because a fixture is missing has not reproduced the bug.
  - **Implement the production fix.**
  - **Run the focused test again**, it must now pass: ''vendor/bin/phpunit --filter yourTestName''.
  - **Run the neighbouring tests**, for instance the whole area directory or ''composer test:integration''.
  - **Run ''composer test:all''** before you open the pull request.

If the fix is in code that only PostgreSQL or only MySQL reaches, also run the suite with that engine, see [[#switching_the_database_engine|Switching the database engine]].

Avoid writing the assertion only after changing the production code, if that makes it impossible to prove that the test actually detects the regression.

==== Recipe: a test for a new feature ====

Structure the test as arrange, act, assert independently. The skeleton below follows ''tests/Integration/Services/InventoryServicePathTest.php'':

<code php>
<?php

namespace Admidio\Tests\Integration\Inventory;

use Admidio\Inventory\Service\ItemFieldService;
use Admidio\Tests\Support\AdministratorTestCase;

class ItemFieldServiceTest extends AdministratorTestCase
{
    /**
     * @testdox ItemFieldService stores a new inventory field for the current organization
     */
    public function testSaveDataStoresTheFieldForTheCurrentOrganization(): void
    {
        global $gCurrentOrgId;

        // Arrange: only the prerequisites, created through production code
        $db = $this->getDatabase();
        $fieldName = 'Regression asset tag ' . bin2hex(random_bytes(5));

        // Act: the real production Service does the work
        $this->assertTrue((new ItemFieldService($db))->saveData(array(
            'inf_name' => $fieldName,
            'inf_type' => 'TEXT',
            'inf_required_input' => 0,
            'inf_disabled' => 0
        )));

        // Assert: read the state back independently of the Service
        $row = $db->queryPrepared(
            'SELECT inf_uuid, inf_name_intern
               FROM ' . TBL_INVENTORY_FIELDS . '
              WHERE inf_org_id = ?
                AND inf_name = ?',
            array($gCurrentOrgId, $fieldName)
        )->fetch();

        $this->assertIsArray($row);
        $this->assertNotSame('', (string)$row['inf_uuid']);
    }
}
</code>

Points worth copying:

  * the namespace mirrors the directory, and the class name ends in ''Test'';
  * a random suffix in names keeps the test independent of leftovers and of parallel data;
  * the Service is called exactly the way a module calls it;
  * the assertion queries the database itself instead of trusting the object the Service returned;
  * ''@testdox'' states the behaviour, so ''--testdox'' output reads like a specification.

==== Using fixtures ====

For prerequisites that Admidio does not already install — extra organizations, users, roles, memberships, categories — use ''AdmidioTestFixture'' instead of writing rows by hand:

<code php>
$fixture = $this->getFixture();

$org = $fixture->createAndSaveOrganization('Second Organization', 'SECOND');
$user = $fixture->createAndSaveUser('regression-user', 'user@example.local', $org['org_id']);
$role = $fixture->createAndSaveRoleWithRights('Editors', $org['org_id'], array('rol_edit_user' => 1));
$fixture->assignUserToRole($user['usr_id'], $role['rol_id']);
$fixture->seedDefaultPreferences($org['org_id']);
</code>

Each method returns an array with the generated ids and UUIDs. Fixtures create their objects through the same Entities production code uses.

The short name of an organization is at most ten characters and unique, and ''TEST'' is already taken by the installed organization.

A fixture may prepare state, but it must never implement the behaviour under test.

==== Asserting persistence ====

Verify the result through a path that is independent of the code you just called:

^ Verification ^ Use it for ^
| A new production Entity, read by id or UUID | the normal case for Entity and Service tests |
| ''$this->getDatabase()->queryPrepared(...)'' | proving exactly which columns were written |
| A second Service read operation | read models, lists, exports |
| The file on disk | filesystem tests, next to the database check |
| A second CLI process | proving a CLI command really committed |
| The Mailpit HTTP API | proving a message was really delivered |

<code php>
$row = $this->getDatabase()->queryPrepared(
    'SELECT ... FROM ' . TBL_EXAMPLE . ' WHERE ... = ?',
    array($value)
)->fetch();
</code>

Direct SQL is the right tool for **asserting** what production code wrote. It is the wrong tool for reproducing the business operation the test claims to exercise.

Keep the SQL portable across the engines Admidio supports. Use the ''TBL_...'' constants and prepared statements, and avoid engine-specific syntax unless the test exists precisely to verify the database abstraction.

==== Naming and scope ====

  * A test file ends in ''Test.php'', otherwise PHPUnit does not discover it.
  * The namespace mirrors the directory, for example ''Admidio\Tests\Integration\Roles''.
  * Test method names and ''@testdox'' descriptions state the behaviour that is verified. Prefer //PreferencesService sends a real email through Mailpit// over //Email works//.
  * One regression test has one clear reason to fail. It may perform several steps when those steps form one production workflow, such as create, update and delete of the same object.

===== Special kinds of tests =====

==== Permissions and organization boundaries ====

Permission tests are the easiest to get wrong. A test that writes

<code sql>
WHERE object_org_id = ?
</code>

itself proves only that the test author knows how to write a safe query. It says nothing about the production code.

Whenever the regression concerns visibility, role rights or organization isolation, run the production Entity, Service, rights object or query that is responsible for enforcing the boundary, and then verify that inaccessible data really is absent.

The ''PermissionContext'' trait exists for this. Admidio resolves rights against globals: ''User::__construct()'' copies ''$gCurrentOrgId'' into the object, and ''Component::isVisible()'' reads ''$gCurrentUser'', ''$gValidLogin'' and ''$gSettingsManager''. The trait sets those globals around a callback and restores them afterwards, so no test leaks its context into the next one:

<code php>
$visible = $this->withCurrentUser($user, $orgId, true, function () {
    // production code that resolves rights for that user
});
</code>

Use ''loadUserInOrganization()'' when the rights have to be resolved against a specific organization — the organization is fixed when the ''User'' object is constructed and cannot be changed afterwards.

==== Filesystem tests ====

Tests that exercise documents, photos, imports, exports or any other file operation extend ''FilesystemTestCase''. The regression filesystem root is ''tests/adm_my_files''.

The base class verifies that Admidio's data directory really resolves to that path and that the committed marker exists. If either check fails, it refuses to run.

When adding a filesystem test:

  * create files only below the test data root, using ''createIsolatedDirectory()'' and ''createFixtureFile()'';
  * register everything production code creates with ''registerCleanupPath()'';
  * call the actual Admidio Service or Entity that performs the file operation;
  * verify both the filesystem and the database state;
  * verify the cleanup path of the feature explicitly, where deleting is part of the behaviour.

Cleanup runs in ''tearDown()'', also when an assertion failed, and it fails the test if a registered path survives. Every path is re-checked against the test data root before anything is deleted.

==== CLI tests ====

CLI coverage has two halves.

=== Contract tests ===

Contract tests inspect command registration: name, description, usage, arguments, options and callback availability. When you add a command, make it satisfy the generic contract tests rather than adding an exception for incomplete metadata.

=== Workflow tests ===

Workflow tests start the real Admidio executable through the ''CliSubprocess'' trait. That process has its own bootstrap, its own database connection, its own exit status and its own output — which makes it a genuine acceptance boundary, and which has consequences:

  * **A subprocess cannot see the PHPUnit transaction.** Prerequisites must either exist in the committed baseline created by the installer, or be created through subprocess commands as part of the scenario.
  * **A subprocess commits.** Its changes survive the rollback, so a mutating test must clean up after itself, normally in a ''finally'' block.
  * **File-writing commands are deliberately not covered here**, because the real CLI bootstrap uses the ''adm_my_files'' of the checkout rather than the test data directory.

The established pattern is create, verify from a second process, delete, verify the deletion:

<code php>
$login = 'cli-e2e-' . bin2hex(random_bytes(5));
$created = false;

try {
    $create = $this->runCli(array('--as=admin', 'user:add', '--login=' . $login, ...));
    $this->assertSame(0, $create->getExitCode(), $create->getErrorOutput());
    $created = true;

    // A second process has a different connection. Seeing the record here proves
    // the production CLI command committed an actual Admidio database write.
    $show = $this->runCli(array('--as=admin', 'user:show', $login, '--format=json'));
    $this->assertSame(0, $show->getExitCode(), $show->getErrorOutput());
    $this->assertSame($login, $this->cliJson($show)['login']);
} finally {
    if ($created) {
        $delete = $this->runCli(array('--as=admin', 'user:delete', $login, '--yes'));
        $this->assertSame(0, $delete->getExitCode(), $delete->getErrorOutput());
    }
}
</code>

Assert the exit code and the error output, and parse machine-readable output such as ''--format=json'' instead of human-oriented console formatting. Use ''CliSubprocess'', do not write another subprocess launcher.

==== Mail tests ====

Mail regressions exercise the real Admidio email stack instead of mocking the sender. The existing test follows this path:

<code>
PreferencesService
    -> Admidio Email
    -> PHPMailer
    -> SMTP
    -> Mailpit
    -> Mailpit HTTP API
</code>

Guidelines:

  * use a unique recipient address per run, so the assertion cannot match a message from an earlier run;
  * verify the delivered message through the Mailpit API, not merely that a TCP port answered;
  * restore the mail settings and any user data you changed, even though the transaction is rolled back, if the code under test caches them.

The mail test deliberately ignores Docker's health status. Some environments report the Mailpit container as ''unhealthy'' while Mailpit works perfectly. What matters is whether Admidio can hand the message to the SMTP listener and whether it then appears in the API. A Docker health label alone is not a reason to disable the test.

==== Installation coverage ====

Every database-backed run installs Admidio from scratch with the production installer, so the installer is covered implicitly: if ''install/db_scripts'' or the installation service breaks, the whole suite fails during setup rather than in a single test.

''tests/Cli'' adds explicit coverage of the installation result and of maintenance mode on top of that.

===== Checklists =====

==== Before you open a pull request ====

  * Does ''composer test:all'' pass on your primary engine?
  * If the change touches SQL, Entities or the installer: does it also pass on the other engine?
  * Does every new behaviour have a test that would fail without your production change?
  * Do new tests clean up subprocess and filesystem changes?
  * Are new language strings, database changes and update steps covered by the change itself?

A green run does not replace code review. Reviewers should still check whether the new tests exercise the correct production path, and whether error, permission and cross-organization cases are covered.

==== Reviewing a new regression test ====

  * Does the test invoke the actual production Entity, Service, CLI command or query it claims to test?
  * Does it verify the resulting state independently?
  * Does it verify a real database write where persistence is part of the feature?
  * Does it avoid duplicating the production business logic inside the test?
  * Is the fixture limited to prerequisites?
  * Is the test isolated from other tests and independent of execution order?
  * Does cleanup also run when an assertion fails?
  * Is filesystem access restricted to ''tests/adm_my_files''?
  * Does a CLI write become visible to another independent process?
  * Does a mail test verify a message that actually reached Mailpit?
  * Are permissions and organization boundaries tested through production code?
  * Is the SQL portable across the supported engines?
  * **Would the test fail if the production behaviour it protects were removed?**

If the answer to the last question is no, the test is probably testing its own setup rather than Admidio.

==== Anti-patterns ====

Do not add a test that:

  * stores expected data in an array and reads it back from the same array;
  * implements a fake Entity or fake Service instead of invoking Admidio;
  * repeats by hand the database changes the production Service is supposed to make;
  * passes when no relevant database record exists;
  * catches an unexpected exception and then succeeds unconditionally;
  * uses assertions such as ''rowCount() >= 0'' that cannot fail meaningfully;
  * depends on another test having run first;
  * writes files outside the protected regression directory;
  * points at a non-test database;
  * assumes a subprocess can see an uncommitted PHPUnit transaction.

A test that cannot fail when the corresponding feature is broken provides false confidence and should be corrected.

===== Third-party modules and plugins =====

Third-party modules and plugins benefit from the same principles even when their tests live outside the Admidio Core repository. Use a checkout of the Admidio version you develop against, and a dedicated test database.

  * invoke real Admidio APIs rather than duplicating them;
  * use Entities and Services following the same patterns as Core;
  * never point tests at a production Admidio database;
  * keep filesystem fixtures out of a real ''adm_my_files'';
  * use Mailpit or another local SMTP sink for mail behaviour;
  * verify organization and permission boundaries through production code;
  * test against every database engine your extension claims to support.

If a third-party change exposes a regression or a missing contract in Admidio Core itself, consider contributing the corresponding regression test to the Core suite.

===== Troubleshooting =====

==== Safety check failed: database name does not contain "test" ====

The name must contain ''test'' as a separate token, so ''admidio_test'' works but ''admidiotest'' does not. Check ''TEST_DATABASE_ENGINE'' and the ''TEST_DB_<ENGINE>_NAME'' that belongs to it — with three connection blocks in the file it is easy to edit the wrong one.

Note that the setup script checks the name more loosely than the test run does. If ''php tests/bin/setup-test-env.php'' is happy but PHPUnit refuses the database, this is why.

==== Database connection fails ====

Check that the service is running (''docker compose -f docker-compose.test.yml ps''), that host and port are right, that the PDO driver for the engine is installed, that the database exists, and that the user may create and drop tables in it.

On MySQL and MariaDB, use ''127.0.0.1'' rather than ''localhost'' so the client does not try the unix socket.

==== "Database not initialized" or a connection error at the first integration test ====

The bootstrap connects lazily, so a wrong configuration only surfaces when the first database-backed test runs. The message repeats the underlying connection error. Run the setup script, it reports the same problem with more context.

==== Unit tests pass but integration tests fail immediately ====

Unit tests deliberately never touch the database. A failure that appears only in the integration suite is almost always the environment: check ''.env.test'' and the database service first.

==== The engine did not change ====

''--db='' is an option of the setup script only. The test run reads ''TEST_DATABASE_ENGINE'' from the environment or ''.env.test''. Also remember that a variable already set in your shell wins over the file.

==== Port 3306 is already in use ====

The MariaDB container publishes 3306. A local MySQL or MariaDB service on the same port will conflict. Stop the local service, or change the published port in ''docker-compose.test.yml'' and in ''.env.test''.

==== PostgreSQL: permission denied for schema public ====

From PostgreSQL 15 on, the ''public'' schema does not let every role create objects. Grant it to the test user:

<code sql>
GRANT ALL ON SCHEMA public TO admidio;
</code>

==== A filesystem test refuses to run ====

Verify that ''.env.test'' contains

<code>
TEST_FILES_PATH=./tests/adm_my_files
</code>

and that the committed marker file ''tests/adm_my_files/.admidio-regression-test'' still exists. Do not build a workaround that disables this protection.

==== Mailpit is shown as unhealthy ====

Ignore the Docker health label and check the endpoints the test actually uses:

<code>
SMTP:     127.0.0.1:1025
HTTP API: 127.0.0.1:8025
</code>

If those work, the mail test can pass even while Docker reports the container as unhealthy.

==== A CLI test cannot see its fixture ====

A real CLI subprocess has its own database connection and cannot see uncommitted rows from the PHPUnit transaction. Create the prerequisite through the CLI subprocess itself, or use data from the committed baseline the installer created.

==== A test leaves data behind ====

Changes made through ''DatabaseTestCase'' disappear with the transaction rollback. Subprocess changes are committed independently and must be removed by the test itself, normally in a ''finally'' block. Filesystem changes must be registered with ''registerCleanupPath()''.

==== Code coverage is empty ====

''composer test:coverage'' needs Xdebug or PCOV. Without a coverage driver PHPUnit runs the tests but writes no report.

==== Windows notes ====

''cp'' does not exist in ''cmd.exe'' or PowerShell, use ''copy'' or ''Copy-Item''. Use ''vendor\bin\phpunit'' or ''vendor/bin/phpunit'' depending on the shell. Everything else, including the Docker environment and the setup script, works the same way.

===== See also =====

  * [[en:entwickler:testumgebung_einrichten|Set up a test environment]]
  * ''docker-compose.test.yml'', ''.env.test.example'', ''phpunit.xml'' and ''tests/bin/setup-test-env.php'' in the repository
  * ''.github/workflows/regression-tests.yml'' for the exact commands CI runs

The goal of the regression suite is not to maximise the number of tests. The goal is to make real Admidio regressions visible as reliable, understandable test failures.