PHPUnit Code Sprint in Autumn

Last week I traveled to Munich to attend PHPCode Sprint. Once again Sebastian Bergmann invited to work together on PHPUnit, close issues, fix bugs and think weird behaviour by tests, PHPUnit itself and PHP through. A big shout out and thank you goes to celebrate company who hosted once gain the hackathon and took care of a dry and warm place and some food during the day.

Optimizing PHPT tests

One of the first topics was to look into PHPT Tests. The current implementation runs each part of the Test in a separate process to make sure no side effects happen. Markus Staab already took a lot of effort to implement a library based on the tokenizer by PHP. But the pull request lacked tests, because back then there was no way to check whether a subprocess was started. The reason to implement this is to check wether side effects may happen and if we (as in Markus <3) are sure there are no side effects, we can run the code without a sub process which saves some overhead and therefore resources and in the end some CO2. Especially based on the amount of installations of PHPUnit code optimisation is quite a a topic.

Adding two new events to PHPUnit

So to know wether a subprocess was started it was quite clear to add events. But answering the questions: Which events? Was already hard and although answered it was reiterated a couple of times. And then the even harder question: Where to dispatch them?

The understanding of an event for the PHPUnit team is not the same as I would describe it in my experience. Events dispatched in Magento or Shopware carry data so you can change the process flow. But this is not what PHPUnit wants. What Sebastian wants (and mostly Arne Blankerts and Andreas Möller implemented – thanks!) is:

• An event fires AFTER something happened – it is always a description of the past
• There is no possibility to change the process. Your subscriber only gets an info, but PHPUnit doesn’t change anything based on EventSubscriber

Especially the first requirement makes it often hard to find the right spot.

Two events were on the wishlist: ChildProcessStarted and ChildProcessEnded and at least for the first the line to add it was found quickly. The second event should have been right behind the proc_close, but as you can see it isn’t. The problem is, that for tests which runInSeparateProcess, the order of events got mixed up.

Welcome to the rabbit hole of JobRunner and all the different use cases in PHPUnit.

What it should look like:

1. Child Process Started
2. Test Preparation Started (PHPUnit\TestFixture\Event\FatalTest::testOne)
3. Test Prepared (PHPUnit\TestFixture\Event\FatalTest::testOne)
4. Test Errored (PHPUnit\TestFixture\Event\FatalTest::testOne)
5. Call to undefined function PHPUnit\TestFixture\Event\doesNotExist()
6. Test Finished (PHPUnit\TestFixture\Event\FatalTest::testOne)
7. Child Process Finished

But it looked like this:

1. Child Process Started
2. Child Process Finished
3. Test Preparation Started (PHPUnit\TestFixture\Event\FatalTest::testOne)
4. Test Prepared (PHPUnit\TestFixture\Event\FatalTest::testOne)
5. Test Errored (PHPUnit\TestFixture\Event\FatalTest::testOne)
6. Call to undefined function PHPUnit\TestFixture\Event\doesNotExist()
7. Test Finished (PHPUnit\TestFixture\Event\FatalTest::testOne)

You want to know why?

Because PHPUnit uses a template to create a php file which then is used to run your test code in its own process and write metadata to a temporary file. This file is then in return read by the original process and contains among other things the events.

The test is run in runTestJob but the events of the child process is only attached to the event collection in processChildResult. Our biggest concern while discussing that is that we we would love to keep the two events on the same level of abstraction, so it is easy to see where the start and finish event occurs. But this doesn’t work. As you can see in the code our solution to this was to add the finish event to the templates of the processRunner. The events are in the right order but not on the same abstraction level. But for the PHPUnit user the order matters more, I think ?

Back to topic: Testing PHPT

So after all this was implemented, we (as in Markus) were able to write tests for the optimization.