convert to pytest entirely

I'm always using pytest to run the tests. Thought the code is using unittest. Those are somewhat compatible, but we should probably convert to a full pytest implementation to speed things up and get some benefits (like dependencies between tests).

Cc: @FFY00, FYI, in case you have a little bit of time to spend :)