'fontTools' is not a folder name, but the import name of the package
where we have some doctests to be discovered and run.
even though we have --pyargs option (which makes pytest first attempt
to treat any positional arguments as module import names), with the
latest pytest 4.0.0 our tests started to fail as if the new pytest
is evaluating the default 'testpaths' (as defined in setup.cfg, used
when no arguments are supplied to pytest) _before_ the --pyargs
option, and thus it thinks that 'fontTools' is a directory name
instead of a package name...
This a workaround. It also means that if one runs `pytest` without
arguments, only the tests inside `Tests` directory are going to be
run automatically, and not also the doctests inside fontTools package.
When running the test suite using tox, all tests are run, both
those in Tests folder and the doctests in fontTools.
Restrucutred Text is very picky. 3.32.0 release process failed because of some broken rst.
I have to stop doing [skip ci] when I update the changelog... :(
I pushed a NEWS.rst with broken indentation, the PyPI page will
display plan text garbage.
Next time I'll use docutils and do `python setup.py check
--restructuredtext` before pushing a new tag.
Also, I forgot to update .travis.yml to build the dist packages
on python3.6 instead of 3.5 after I modified the tox.ini, so
the automatic release from Travis failed.
This time I had to use twine tool to upload the zip and whl
packages from my local machine to PyPI.
Sorry the mess...
The {posargs} are global for the current tox process, and we already use them to pass additional arguments to the pytest command in the default testenv, so we can't at the same time also use them to pass options to coverage.py.
The 'htmlcov' tox environment produces the combined coverage.py report, and store it in htmlcov/index.html; this is listed in the tox.ini 'envlist', so it is run by default at every 'tox' invocation, when no explicit env list is passed via either -e option or $TOXENV variable.
The options to `coverage html` are not much interesting anyway; it's much more useful being able to, e.g., run tox for specific tests matching a string or expression:
$ tox -- -k "feaLib"
When running tox locally during development, there's no need to also test python 3.5.
2.7 and 3.6 are enough, and the CI takes care of the others (3.4, 3.5, pypy, etc.).
Previously, we were using --pyargs to make sure pytest would run the test suite against the installed package, because our tests
were embedded inside the package and hence importable.
Now that they are placed in an external folder, that option is not needed.
pytest without options will run all the test modules from the default test paths as defined in setup.cfg, i.e Tests/ folder.
The tests will be run against the installed fontTools in the current tox environment.
We also pass {posargs} in the 'htmlcov' tox env, to allow passing extra options after `tox -e htmlcov -- {posargs}` to the `coverage html` command.
This makes sure we upload the same files to Github Releases and PyPI.
Currently we were building them twice, with the risk of different files
being uploaded to the two repositories.
We no longer use coverage.py through the pytest-cov plugin, as it does not
support --parallel-mode (or at least I wasn't able to figure it out).
We run pytest through `coverage run -m pytest` instead, which does the same
as `pytest --cov`.
Running `tox` locally by default will now run pytest and coverage on 2.7, 3.5 and 3.6; it will then
combine coverage data from all three, and finally generate a coverage report in htmlcov/index.html
We need tp temporarily install coverage.py from the mercurial repo instead of the latest release
because we require a patch that is not available yet with coverage 4.2:
https://bitbucket.org/ned/coveragepy/pull-requests/118/check-source-isdir-not-just-exists-there/diff
Finally, I replaced Coveralls with Codecov, as the latter is better designed and also supports Appveyor.
This is useful if one wants to run coverage tests locally, and get a nice HTML report while one is writing tests.
Just run:
$ tox -e coverage -- {optional pytest arguments}
And you get a "htmlcov/index.html" to view in your browser.
This will ensure we don't forget to add any data files to the MANIFEST.in (which means test data is not available to the inlined tests).
{posargs} allows to pass extra arguments to the test runner, when runing `tox -- {posargs}` from the command line.
With `--pyargs` option, pytest tries to interpret arguments as python package/module names, and if import fails as regular filesystem paths.
When running `tox` command without specifying any `-e` option, or when no $TOXENV environment variable is specified, the 'envlist' in tox.ini file determines the list of environment tox will operate on.
http://tox.readthedocs.io/en/latest/config.html#confval-envlist=CSV
I chose Python 2.7 and 3.5 because I assume they are the versions that other fonttols developers run their tests on locally, before pushing and let the CI run the tests on the rest of the pythons.
The glob pattern determines which classes are considered for test collection by pytest.
We already follow this pattern for the unittest.TestCase derived subclasses
(which are collected regardless of this option).
the glob pattern will restricts the search only for pytest's kind of tests; it has no effect on methods of unittest.TestCase derived class, since pytest still uses unittest's own collection framework to collect those.
The 'minversion' is required to support some of the latest advanced features.
The current py.test is 2.8.7. You can update it with `pip install --upgrade pytest`.
