How to set-up and use py.test in Pycharm
We asked our friend Miro to try using Pytest with PyCharm. Miro has background in test automation, Python, DevOps and metal music. Since he was new to the set-up he would be the ideal person to write a beginner's guide. This is his take.
I've been using Vim and terminal as a weapon of choice for years. I've had a good time with it, however, more and more people ask me why I'm using this setup. And honestly, I don't know the answer.
I'm aware that things can be done more efficiently and an IDE can help with a lot of things. I guess that my weak spot is the unit tests and testing my code in general. I'm not running my tests when on the coding spree, I'm breaking lots of stuff, and only when I think I'm finished, I'll do the fixing and make everything running green again.
Well, I would like to change that. And I'm also curious about trying out new ways of doing things. The obvious choice for programming in Python is the PyCharm. It's a nice IDE, supports many features that I like and most importantly, it can help with the testing. PyCharm can easily integrate with popular test frameworks and run the tests for me.
In this article, we'll take a look at using py.test in PyCharm.
For the sake of this article, I've prepared a small project called MasterMind. It's a simple, console-based game just for learning purposes.
Most of you are probably familiar with the rules. One person thinks of a number and the second person must guess this number. In this case, you'll be playing against the computer. It'll generate a random number that you have to guess.
You can download the project from here.
I would like to briefly describe the MasterMind project structure.
/mastermind <-------------- top-level dir /src <------------------- source codes / modules /mastermind <---------- main python module __init__.py __main__.py <-------- entry point (startup of game) exceptions.py <------ contains game exceptions game.py <------------ game logic heading.txt <-------- heading at the beginning of the game utils.py <----------- various helpers /tests <----------------- pytests test_mastermind.py <--- test suite setup.py <--------------- package setup script
Please notice that I'm using an
src folder to separate top-level Python modules
from the main directory. This is used to protect from accidental imports
and other unwanted effects. If you're interested in more details, check out
I'm bringing this up because we'll have to do some additional steps in PyCharm configuration.
So here we are, I'm firing up PyCharm and opening the mastermind folder as a new project. If you're following this tutorial, you can extract mastermind.zip and open the mastermind directory in PyCharm. This will become the root folder of our project in PyCharm.
We'll have to configure PyCharm, to use the Python3 interpreter that we've
brew install python3. You can find this setting under
preferences -> project interpreter. If there are no interpreters at all,
we'll have to add one. This will also create virtualenv.
Since we are using a clean virtual environment, we'll have to install
package in order to use it for running the tests. Just stay on the interpreter
preference page and install packages from there. Right under selected Python
interpreter is a list of installed packages (so far just
pip). To install new packages click add
+ sign and search for
package. Press install package to install it. This will be added only to our
However, if you prefer console, you can install Python packages using
command. Just open the console by pressing terminal at the left - bottom.
pytest with command
pip install pytest. This has the same effect as
installing with PyCharm interface. Using terminal you should see something
(venv)$ pip install pytest Collecting pytest Downloading https://files.pythonhosted.org/packages/5d/c3/54f607bc9817fd284073ac68e99123f86616f431f9d29a855474b7cf00eb/pytest-4.4.1-py2.py3-none-any.whl (223kB) |████████████████████████████████| 225kB 1.7MB/s ...[MORE PIP OUTPUT]... Requirement already satisfied: setuptools in ./venv/lib/python3.7/site-packages (from pytest) (41.0.1) Installing collected packages: pluggy, py, attrs, atomicwrites, more-itertools, six, pytest Successfully installed atomicwrites-1.3.0 attrs-19.1.0 more-itertools-7.0.0 pluggy-0.9.0 py-1.8.0 pytest-4.4.1 six-1.12.0
Now we can configure
pytest as default test runner in PyCharm. You can find
this setting under preferences -> Tools -> Python Integrated Tools. Set
Default test runner to
Running App / Tests
So far, we've been playing with PyCharm and configuration. Now we should try
out the app, if it's working and if we can proceed further. MasterMind can be
executed as a module. In the terminal, we would need to execute this command to start
the game. Please note, that the current working directory must be in
(venv)$ cd src (venv)$ python -m mastermind
When you execute just the module like that, Python will know to go to the
__main__.py file, which contains
main function and execute it.
So how to do this in PyCharm? We'll need to create Run Configuration.
In this configuration, we'll basically do the same thing, as in terminal. Just
execute module called
mastermind. Also, in the terminal, we've
src, which contains our
mastermind module. We have to
configure this in PyCharm as well. Otherwise, it would not know where to
find our module. You can do so, by right clicking on
src directory and marking
it as sources root
Now, it's time to run our tests for MasterMind. To create a test configuration,
we'll use IDE to set it up for us. Just right-click on
tests directory in
Project tool window and select run pytest in tests.
This will do 2 things. It'll run the tests and create run configuration. Test results are displayed in Test Runner Tab.
Also, note the test configuration in the top right corner. We can save this configuration as our default test run.
There are a couple of features that are useful for everyday workflow.
Execute Single Test
When you're writing new tests for your app, PyCharm makes it really easy to execute a single test. E.g. the one that you've just finished writing. You can do so by pressing the green play button, next to the test definition.
Of course, that's not the end of it. PyCharm lets you execute a whole class (containing multiple tests) or a single file (test suite). You can select these options from the context menu of a mouse right click.
From the official documentation:
In many cases, you can initiate a testing session from a context menu. For this purpose, the Run and Debug commands are provided in certain context menus. For example, these commands are available for a test class, directory, or a package in the Project Tool Window. They are also available for a test class or method you are currently working on in the editor.
Auto Test Rerun
I like to focus only on programming and don't worry about the tests (until it's too late :P ). So let's make PyCharm do the hard work for us. In the Test Runner Tab, we can toggle the automatic rerun of the tests on a code change. Pretty neat, right?
So for example, if I introduce an error, I'll know this immediately. You've probably heard that to fix defects later is much more expensive that early. This will give you rapid feedback!
There are a couple of disadvantages IMHO. It's great to have rapid feedback, however, PyCharm will trigger the tests on each file change. Even if you add a new line. Moreover, PyCharm autosaves your changes, which leads to even more repeatedly executed tests.
You could set the automatic rerun of a subgroup of the tests (different run configuration). For example, if you are working on particular functionality, you can set a smaller test suite (covering only what's needed) and set auto tests just for that configuration. However, you have to do this manually, and not to forget to change it back, when you're finished. Good luck with that!
In this article, we've tried integration of
py.test and PyCharm, it was
fairly easy and straightforward. After setting up an automatic rerun of tests
we could focus on programming and get rapid feedback from our test cases.
Another nice feature is to execute tests that are being currently developed. We can write the tests and see if everything works as expected. Just with the press of a button, we can get the feedback.
By the way, when I was setting up PyCharm, there was an option to enable ideavim plugin. I'll have to give it a try and see how well it's working. Nevertheless, it's a pleasant surprise for vim users such as myself.
I'll keep on using PyCharm and
py.test for a couple of weeks before I make
any conclusions, but I have to say I'm quite comfortable using it even after
a couple of days. If you hesitate to try a new IDE or tool, just go for
it. Maybe you'll find something that suits you better.