assertDirectoryExists and assertDirectoryDoesNotExist failed #5996

Did you know php has a statcache which caches the return values of all kinds of return values from the operating system IO?

That means, as long as you stick to the php universe like stat(), lstat(), file_exists(), is_writable(), is_readable(), is_executable(), is_file(), is_dir(), is_link(), filectime(), fileatime(), filemtime(), fileinode(), filegroup(), fileowner(), filesize(), filetype(), and fileperms() everything is fine, but don’t mix it with exec("rm -rf $yourEscapeDirectory");

More on this on the ticket written down by Sebastian.

Run PHPT tests with -n option #5987

Markus Staab had the idea to run PHPT tests with the -n flag in php, this flag omits the extension loading in PHP and makes therefore the starting of the php process faster.

To do this, we need to know which extensions are needed to run the test and the nice discussion we had didn’t find solution for the most pressing questions:

• How to know which extensions are needed?
• And were are they on the disk

Together with the small improvement of 30ms (which might in return be eaten by the code to answer the two questions above) we decided to not follow this idea.

assertEquals() hides contents of long strings #5846

Another interesting issue was this one. Although we were not sure about the issue itself, a nice discussion started how to improve on the output of string comparison.

The current output of string comparison in PHPUnit is stripped down and the middle part is replaced by …

Two things came up:

1. Show more of the compared string, so we added a optional argument for the length which in the future might be added to the phpunit configuration and is then passed down to the exporter.
2. Remove the common prefix of the two strings
   The interesting part of the two string is the part which differs. So in case they have a long common prefix you only see parts which don’t differ. But implementing this is not as easy as one might think, so we opened a ticket.

Bring back --repeat CLI option #5718

One of the first tickets we discussed was this one. But I hide it at the end of the post, I’m not sure wether we need more opinions ?

In the past you could tell PHPUnit to --repeat which essentially runs your test suite n times. Because it was blocking the release of PHPUnit 10 it was removed. But obviously there are people who used it. We tried to come up with a proper use case but it was hard for us.

Here is my opinion on them.

1. If you want to benchmark your code, PHPUnit is not the right tool for it, better use something like phpbench
2. If you want to find flaky tests run your test suite multiple times. By either running your pipeline multiple times or for i in {1..10}; do command; done
   By the way, this spawned another interesting discussion because someone found an internal php cli command: --repeat
   All these discussions, at PHPUnit but at the php-src as well boil down to the question: Do we want this code? And if yes, do we want to maintain it for today and forever? Added features need maintenance which means effort and time. And there are good alternatives.
3. A sub-use-case of 2. is: Run it twice to make sure, the cache which is filled the first time doesn’t break on a second run – but this should imho be done by running your code twice in your test.

Although I totally understand that changing your toys as a developer is painful and annoying, I don’t think we should reimplement --repeat, but use one of the above solutions.

On the other hand I see a feature which when implemented doesn’t add much benefit but needs maintenance.

Other articles from this category