best practices for dev+test bot

6b. For some complex scenarios—such as testing behavior on a specific complex state to find an obscure bug—that may not be possible. Writing tests first really helps with this as it forces you to think about the behavior of your code and how you're going to test it before you write it.

Testing first encourages smaller, more modular units of code, which generally means better code. A good book for getting started with the "test first" approach is Test Driven Development by Example, by Kent Beck.

6b. For some complex scenarios—such as testing behavior on a specific complex state to find an obscure bug—that may not be possible. Writing tests first really helps with this as it forces you to think about the behavior of your code and how you're going to test it before you write it.

Testing first encourages smaller, more modular units of code, which generally means better code. A good book for getting started with the "test first" approach is Test Driven Development by Example, by Kent Beck.

7a. For unit tests (including test infrastructure tests) all code paths should be tested. 100% coverage is a good place to start. You can't cover all possible permutations/combinations of state (combinatorial explosion), so that requires consideration. Only if there is a very good reason should code paths be left untested.

28. Generally, particularly in tests, wait for a specific change rather than sleeping for an arbitrary amount of time. Voodoo sleeps are hard to understand and slow down your test suite.

22a. Smaller, more tightly scoped unit tests give more valuable information when they fail—they tell you specifically what is wrong. A test that stands up half the system to test behavior takes more investigation to determine what is wrong. Generally a test that takes more than 0.1 seconds to run isn’t a unit test.

22b. There’s no such thing as a slow unit test. With tightly scoped unit tests testing behavior, your tests act as a de facto specification for your code. Ideally if someone wants to understand your code, they should be able to turn to the test suite as "documentation" for the behavior. A great presentation on unit testing practices is Fast Test, Slow Test, by Gary Bernhardt: https://www.youtube.com/watch?v=RAxiiRPHS9k

22a. Smaller, more tightly scoped unit tests give more valuable information when they fail—they tell you specifically what is wrong. A test that stands up half the system to test behavior takes more investigation to determine what is wrong. Generally a test that takes more than 0.1 seconds to run isn’t a unit test.

15. The more you have to mock out to test your code, the worse your code is. The more code you have to instantiate and put in place to be able to test a specific piece of behavior, the worse your code is. The goal is small testable units, along with higher-level integration and functional tests to test that the units cooperate correctly.

6a. Unit tests test to the unit of behavior, not the unit of implementation. Changing the implementation, without changing the behavior or having to change any of your tests is the goal, although not always possible.

So where possible, treat your test objects as black boxes, testing through the public API without calling private methods or tinkering with state.

21a. Make code correct first and fast second. When working on performance issues, always profile before making fixes. Usually the bottleneck is not quite where you thought it was. Writing obscure code because it is faster is only worth it if you’ve profiled and proven that it’s actually worth it.

7b. Lack of time is not a good reason for not testing all code paths and ends up costing more time. Possible good reasons include: genuinely untestable (in any meaningful way), impossible to hit in practice, or covered elsewhere in a test. Code without tests is a liability. Measuring coverage and rejecting PRs that reduce coverage percentage is one way to ensure you make gradual progress in the right direction.

7a. For unit tests (including test infrastructure tests) all code paths should be tested. 100% coverage is a good place to start. You can't cover all possible permutations/combinations of state (combinatorial explosion), so that requires consideration. Only if there is a very good reason should code paths be left untested.

18. Don’t do work in object constructors, which are hard to test and surprising. Don’t put code in `__init__.py` (except imports for namespacing). `__init__.py` is not where programmers generally *expect* to find code, so it’s "surprising".

10. Write defensively. Always think about what can go wrong, what will happen on invalid input, and what might fail, which will help you catch many bugs before they happen.

1b. YAGNI: "You Aint Gonna Need It". Don't write code that you think you might need in future, but don't need yet.

The same is true for commenting-out code; if a block of commented code is going into a release, it shouldn't exist. If it is code that may be restored, make a ticket and reference the commit hash for the code delete. YAGNI is a core element of agile programming. The best reference for this is Extreme Programming Explained, by Kent Beck.

4. When it comes to API design (external facing and object API): *Simple things should be simple; complex things should be possible.* Design for the simple case first, with preferably zero configuration or parameterization, if that's possible. Add *options* or additional API methods for more complex and flexible use cases (as they are needed).