Pythonity Blog Pythonity Blog

Coveralls with Travis CI, Tox and pytest

We have a proper, lengthy post coming real soon, but I just quickly wanted to tell you how to add Coveralls support to your Python project that runs tests via some sort of combination of Travis CI, Tox and pytest.

I recently went through that process and had a bit of a trouble - let's just say Coveralls docs aren't very helpful. The only thing they do is point you to one of python-coveralls or coveralls-python. I know, the choice is so obvious.

I decided to go with the more popular coveralls-python, which also seems to be more feature packed and for instance doesn't require any Coveralls configuration (like explicitly setting repository token) when run on Travis CI which is nice. I also found pytest-cov which sounded like exactly what I was looking for, but long story short - it wasn't.

Okay, so let's just use directly, make it do its thing. I know, let's add it as a separate Tox environment, similarly to flake8. Wow, so nicely done, right? Nope. If you think about it some more it becomes obvious why - Tox runs its tests in separated virtualenvs and the files created while running them aren't available for coveralls-python to find in Travis build directory. Duh.

So finally I came to the right conclusion - Tox is out of the scope and all stuff needs to be invoked directly in .travis.yml:

$ cat .travis.yml
language: python

  - "2.7"
  - "3.4"
  - "3.5"
  - "3.6"

  - pip install -r requirements/dev.txt
  - pip install tox-travis

  - tox
  - coverage run --source icon_font_to_png -m py.test
  - coverage report -m

  - coveralls

and just to get the full picture:

$ cat tox.ini
envlist = py27,py34,py35,py36,flake8

commands = py.test -v
deps = -r{toxinidir}/requirements/dev.txt
passenv =

commands = flake8 .
deps = flake8

python =
  3.6: py36, flake8

addopts =
python_files = *.py
python_functions = test_

exclude =

So there you go. Not really that complicated but worth putting online for future reference and saving that precious half an hour.

As always - let me know if you have any questions.