Previously, we were using --pyargs to make sure pytest would run the test suite against the installed package, because our tests
were embedded inside the package and hence importable.
Now that they are placed in an external folder, that option is not needed.
pytest without options will run all the test modules from the default test paths as defined in setup.cfg, i.e Tests/ folder.
The tests will be run against the installed fontTools in the current tox environment.
We also pass {posargs} in the 'htmlcov' tox env, to allow passing extra options after `tox -e htmlcov -- {posargs}` to the `coverage html` command.
This makes sure we upload the same files to Github Releases and PyPI.
Currently we were building them twice, with the risk of different files
being uploaded to the two repositories.
We no longer use coverage.py through the pytest-cov plugin, as it does not
support --parallel-mode (or at least I wasn't able to figure it out).
We run pytest through `coverage run -m pytest` instead, which does the same
as `pytest --cov`.
Running `tox` locally by default will now run pytest and coverage on 2.7, 3.5 and 3.6; it will then
combine coverage data from all three, and finally generate a coverage report in htmlcov/index.html
We need tp temporarily install coverage.py from the mercurial repo instead of the latest release
because we require a patch that is not available yet with coverage 4.2:
https://bitbucket.org/ned/coveragepy/pull-requests/118/check-source-isdir-not-just-exists-there/diff
Finally, I replaced Coveralls with Codecov, as the latter is better designed and also supports Appveyor.
This is useful if one wants to run coverage tests locally, and get a nice HTML report while one is writing tests.
Just run:
$ tox -e coverage -- {optional pytest arguments}
And you get a "htmlcov/index.html" to view in your browser.
This will ensure we don't forget to add any data files to the MANIFEST.in (which means test data is not available to the inlined tests).
{posargs} allows to pass extra arguments to the test runner, when runing `tox -- {posargs}` from the command line.
With `--pyargs` option, pytest tries to interpret arguments as python package/module names, and if import fails as regular filesystem paths.
When running `tox` command without specifying any `-e` option, or when no $TOXENV environment variable is specified, the 'envlist' in tox.ini file determines the list of environment tox will operate on.
http://tox.readthedocs.io/en/latest/config.html#confval-envlist=CSV
I chose Python 2.7 and 3.5 because I assume they are the versions that other fonttols developers run their tests on locally, before pushing and let the CI run the tests on the rest of the pythons.
The glob pattern determines which classes are considered for test collection by pytest.
We already follow this pattern for the unittest.TestCase derived subclasses
(which are collected regardless of this option).
the glob pattern will restricts the search only for pytest's kind of tests; it has no effect on methods of unittest.TestCase derived class, since pytest still uses unittest's own collection framework to collect those.
The 'minversion' is required to support some of the latest advanced features.
The current py.test is 2.8.7. You can update it with `pip install --upgrade pytest`.
