The beauty of testing is found not in the effort but in the efficiency. Knowing what should be tested is beautiful, and knowing what is being tested is beautiful. | ||
| --Murali Nandigama | ||
In this chapter you will learn all about PHPUnit's code coverage functionality that provides an insight into what parts of the production code are executed when the tests are run. It helps answering questions such as:
How do you find code that is not yet tested -- or, in other words, not yet covered by a test?
How do you measure testing completeness?
An example of what code coverage statistics can mean is that if there is a method with 100 lines of code, and only 75 of these lines are actually executed when tests are being run, then the method is considered to have a code coverage of 75 percent.
PHPUnit's code coverage functionality makes use of the PHP_CodeCoverage component, which in turn leverages the statement coverage functionality provided by the Xdebug extension for PHP.
Note
Xdebug is not distributed as part of PHPUnit. If you receive a notice while running tests that the Xdebug extension is not loaded, it means that Xdebug is either not installed or not configured properly. Before you can use the code coverage analysis features in PHPUnit, you should read the Xdebug installation guide.
Let us generate a code coverage report for a BankAccount
class:
phpunit --coverage-html ./report BankAccountTest
PHPUnit 4.2.0 by Sebastian Bergmann.
...
Time: 0 seconds
OK (3 tests, 3 assertions)
Generating report, this may take a moment.Figure 11.1 shows an excerpt from a Code Coverage report. Lines of code that were executed while running the tests are highlighted green, lines of code that are executable but were not executed are highlighted red, and "dead code" is highlighted grey. The number left to the actual line of code indicates how many tests cover that line.
Clicking on the line number of a covered line will open a panel (see Figure 11.2) that shows the test cases that cover this line.
The code coverage report for our BankAccount example
shows that we do not have any tests yet that call the
setBalance(), depositMoney(), and
withdrawMoney() methods with legal values.
Example 11.1
shows a test that can be added to the BankAccountTest
test case class to completely cover the BankAccount
class.
Example 11.1: Test missing to achieve complete code coverage
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase
{
// ...
public function testDepositWithdrawMoney()
{
$this->assertEquals(0, $this->ba->getBalance());
$this->ba->depositMoney(1);
$this->assertEquals(1, $this->ba->getBalance());
$this->ba->withdrawMoney(1);
$this->assertEquals(0, $this->ba->getBalance());
}
}
?>
Figure 11.3 shows
the code coverage of the setBalance() method with the
additional test.
The @covers annotation (see
Table B.1) can be
used in the test code to specify which method(s) a test method wants to
test. If provided, only the code coverage information for the specified
method(s) will be considered.
Example 11.2
shows an example.
Example 11.2: Tests that specify which method they want to cover
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase
{
protected $ba;
protected function setUp()
{
$this->ba = new BankAccount;
}
/**
* @covers BankAccount::getBalance
*/
public function testBalanceIsInitiallyZero()
{
$this->assertEquals(0, $this->ba->getBalance());
}
/**
* @covers BankAccount::withdrawMoney
*/
public function testBalanceCannotBecomeNegative()
{
try {
$this->ba->withdrawMoney(1);
}
catch (BankAccountException $e) {
$this->assertEquals(0, $this->ba->getBalance());
return;
}
$this->fail();
}
/**
* @covers BankAccount::depositMoney
*/
public function testBalanceCannotBecomeNegative2()
{
try {
$this->ba->depositMoney(-1);
}
catch (BankAccountException $e) {
$this->assertEquals(0, $this->ba->getBalance());
return;
}
$this->fail();
}
/**
* @covers BankAccount::getBalance
* @covers BankAccount::depositMoney
* @covers BankAccount::withdrawMoney
*/
public function testDepositWithdrawMoney()
{
$this->assertEquals(0, $this->ba->getBalance());
$this->ba->depositMoney(1);
$this->assertEquals(1, $this->ba->getBalance());
$this->ba->withdrawMoney(1);
$this->assertEquals(0, $this->ba->getBalance());
}
}
?>
It is also possible to specify that a test should not cover
any method by using the
@coversNothing annotation (see
the section called “@coversNothing”). This can be
helpful when writing integration tests to make sure you only
generate code coverage with unit tests.
Example 11.3: A test that specifies that no method should be covered
<?php
class GuestbookIntegrationTest extends PHPUnit_Extensions_Database_TestCase
{
/**
* @coversNothing
*/
public function testAddEntry()
{
$guestbook = new Guestbook();
$guestbook->addEntry("suzy", "Hello world!");
$queryTable = $this->getConnection()->createQueryTable(
'guestbook', 'SELECT * FROM guestbook'
);
$expectedTable = $this->createFlatXmlDataSet("expectedBook.xml")
->getTable("guestbook");
$this->assertTablesEqual($expectedTable, $queryTable);
}
}
?>
Sometimes you have blocks of code that you cannot test and that you may
want to ignore during code coverage analysis. PHPUnit lets you do this
using the @codeCoverageIgnore,
@codeCoverageIgnoreStart and
@codeCoverageIgnoreEnd annotations as shown in
Example 11.4.
Example 11.4: Using the @codeCoverageIgnore, @codeCoverageIgnoreStart and @codeCoverageIgnoreEnd annotations
<?php
/**
* @codeCoverageIgnore
*/
class Foo
{
public function bar()
{
}
}
class Bar
{
/**
* @codeCoverageIgnore
*/
public function foo()
{
}
}
if (FALSE) {
// @codeCoverageIgnoreStart
print '*';
// @codeCoverageIgnoreEnd
}
?>The ignored lines of code (marked as ignored using the annotations) are counted as executed (if they are executable) and will not be highlighted.
By default, all sourcecode files that contain at least one line of code that has been executed (and only these files) are included in the report. The sourcecode files that are included in the report can be filtered by using a blacklist or a whitelist approach.
The blacklist is pre-filled with all sourcecode files of PHPUnit itself as well as the tests. When the whitelist is empty (default), blacklisting is used. When the whitelist is not empty, whitelisting is used. Each file on the whitelist is added to the code coverage report regardless of whether or not it was executed. All lines of such a file, including those that are not executable, are counted as not executed.
When you set processUncoveredFilesFromWhitelist="true"
in your PHPUnit configuration (see the section called “Including and Excluding Files for Code Coverage”) then these files
will be included by PHP_CodeCoverage to properly calculate the number of
executable lines.
Note
Please note that the loading of sourcecode files that is performed when
processUncoveredFilesFromWhitelist="true" is set can
cause problems when a sourcecode file contains code outside the scope of
a class or function, for instance.
PHPUnit's XML configuration file (see the section called “Including and Excluding Files for Code Coverage”) can be used to control the blacklist and the whitelist. Using a whitelist is the recommended best practice to control the list of files included in the code coverage report.
For the most part it can safely be said that PHPUnit offers you "line based" code coverage information but due to how that information is collected there are some noteworthy edge cases.
Example 11.5:
<?php
// Because it is "line based" and not statement base coverage
// one line will always have one coverage status
if(false) this_function_call_shows_up_as_covered();
// Due to how code coverage works internally these two lines are special.
// This line will show up as non executable
if(false)
// This line will show up as covered because it is actually the
// coverage of the if statement in the line above that gets shown here!
will_also_show_up_as_coveraged();
// To avoid this it is necessary that braces are used
if(false) {
this_call_will_never_show_up_as_covered();
}
?>