Test & Code

When you are teaching someone web development skills, when is the right time to start teaching code quality and testing practices?Karl Stolley believes it's never too early. Let's hear how he incorporates code quality in his courses.Our discussion includes:starting people off with good dev practices and toolslintinghtml and css validationvisual regression testingusing local dev servers, including httpsincorporating testing with git hookstesting to aid in css optimization and refactoringBackstopNightwatchBrowserStackthe tree legged stool of learning and progressing as a developer: testing, version control, and documentationKarl is also writing a book on WebRTC, so we jump into that a bit too.Links:BackstopNightwatchBrowserStackProgramming WebRTC: Build Real-Time Streaming Applications for the Web by Karl Stolley

Being productive is obviously a good thing. Can we measure it? Should we measure it? There's been failed attempts, like lines of code, etc. in the past. Currently, there are new tools to measure productivity, like using git metrics.Nick Hodges joins the show to discuss the good and the bad of developer and team productivity, including how we can improve productivity.

Django has a handful of console commands to help manage and develop sites. django-rich adds color and nice formatting. Super cool. In a recent release, django-rich also adds nice colorized tracebacks to the Django test runner. Links:django-rich · PyPIepisode 181: Boost Your Django DX - Adam Johnsondjango-crispy-formsPython Developers Survey 2020 Results

Twisted has been supporting asynchronous / event driven applications way before asyncio. Twisted, and Glyph, have also been encouraging automated tests for a very long time.Twisted uses a technique that should be usable by other applications, even those using asyncio or other event driven architectures.Links:Community Service Award Recipient Glyph Lefkowitz — The PSF article I was referring to early in the episode Twistedtwisted/kleintwisted/treqdjango/daphne — Django Channels HTTP/WebSocket serverScrapySo Easy You Can Even Do It in JavaScript: Event-Driven Architecture for Regular Programmers - YouTubeFeature test - twitter engineeringPython Testing with pytest — Some reading for Glyph if he wants to learn to love pytestpytest-twistedpygamePyodideBrython

Ryan Cheley joins me today to talk about some challenges of managing software teams, and how to handle them. We end up talking about a lot of skills that are excellent for software engineers as well as managers.Some topics discussed:handling code reviewsasking good questionsbeing honest about what you can't do with current resources and datadiscussing tradeoffs and offering solutions that can be completed faster than the ideal solutionbalancing engineering and managingmaking sure documentation happensremote teams encouraging collaborationencouraging non-work-related conversationswatching out for overworking

Don't you just love technical interviews, with someone who just saw your resume or CV 5 minutes ago asking you to write some code on a whiteboard. Probably code that has nothing to do with anything you've done before or anything you will do at the company.No? Neither does Nathan Aschbacher. So when he started building the team at his company, he decided to do things differently.Hiring is one of the essential processes for building a great team. However, it's a high noise, low signal process.Nathan Aschbacher has a relatively unorthodox tech hiring approach. He's trying to make it very humane, with a better signal to noise ratio. Nathan is not intereseted in bizarre interview processes where the interviewer doesn't know anything about the interviewee beforehand, all people are asked the same questions, and people are asked to code on white boards.Instead, he states "if the goal is to try to figure out if the person can do the work with your team, and your trying to build the team that you are adding this person to, they need to know what the team is like, and determine if they want to be part of the team, and the team needs to know what the person is like and if they would be addititve to the team.So what's Nathan's process:Screening resumes and CVs, looking for internal motivation to become an expert at something.Basic phone screen, very informal.A couple 2-3 hour pairings with someone on the team with whatever they are working on.Debriefing both the candidate and the team afterwords.Giving the candidate an opportunity for a second impression and following up on difficulties during the pairings.We discuss the process, and also:trying to remove the barriers to team integration treating people as humans And of course, there's the story of how Nathan ended up interviewing someone with Zoo experience an no technical experience for a technical role. Of course, it was a misunderstanding of a job requirement around experience with ZooKeeper. But it's a good story.

We talk with Adam Johnson about his new book, "Boost Your Django DX". Developer experience includes tools and practices to make developers more effective and efficient, and just plain make software development more fun and satisfying. One of the things I love about this book is that it's not just for Django devs. I'd guess that about half the book is about topics that all Python developers would find useful, from virtual environments to linters to testing. But of course, also tons of tips and tools for working with Django.Links:“Boost Your Django DX” Released — Adam's announcementBoost Your Django DX — The book

Lean TDD is an attempt to reconcile some conflicting aspects of Test Driven Development and Lean Software Development.I've mentioned Lean TDD on the podcast a few times and even tried to do a quick outline at the end of episode 162.This episode is a more complete outline, or at least a first draft.If you feel you've got a good understanding of TDD, and it's working awesome for you, that's great. Keep doing what you're doing. There are no problems.For me, the normal way TDD is taught just doesn't work. So I'm trying to come up with a spin on some old ideas to make it work for me. I'm hoping it works for you as well.I'm calling the new thing Lean TDD. It's inspired by decades of experience writing software and influence from dozens of sources, including Pragmatic Programmer, Lean Software Development, Test-Driven Development by Example, and many blog posts and wiki articles. The main highlights, however, come from the collision of ideas between Lean and TDD and how I've tried to resolve the seemingly opposing processes.Links:Lean TDD | pythontest.comepisode 162: Flavors of TDDWhat is Test Driven Development (TDD)? | Agile AllianceTest First Programming / Test First Development | pythontest.comepisode 7: The Waterfall Model and “Managing the Development of Large Software Systems”episode 51: Feature TestingThe testing renaissance — Twitter Engineering blog post about Feature TestingLean software development - Wikipedia

Exploratory testing is absolutely an essential part of a testing strategy. This episode discusses what exploratory testing is, its benefits, and how it fits within a framework of relying on automated tests for most of our testing.

"There are five practical reasons that we write tests. Whether we realize it or not, our personal testing philosophy is based on how we judge the relative importance of these reasons." - Sarah MeiThis episode discusses the factors.Sarah's order:Verify the code is working correctlyPrevent future regressionsDocument the code’s behaviorProvide design guidanceSupport refactoringBrian's order:Verify the code is working correctlyPrevent future regressionsSupport refactoringProvide design guidanceDocument the code’s behaviorThe episode includes reasons why I've re-ordered them.Links:Five Factor Testing - Sarah Mei

A recent Twitter thread by Simon Willison reminded me that I've been meaning to do an episode on the testing trophy. This discussion is about the distinction between unit and integration tests, what those terms mean, and where we should spend our testing time.Links:Simon Willison's Twitter ThreadThe Testing Trophy and Testing Classifications — Kent C DoddsWrite tests. Not too many. Mostly integration. — Kent C DoddsOn the Diverse And Fantastical Shapes of Testing — Martin Fowler

The idea of having a software as a service product sound great, doesn't it? Solve a problem with software. Have a nice looking landing page and website. Get paying customers. Eventually have it make enough revenue so you can turn it into your primary source of income. There's a lot of software talent out there. We could solve lots of problems. But going from idea to product to first customer is non-trivial. Especially as a side hustle. This episode discusses some of the hurdles from idea to first customer. Brandon Braner is building Released.sh. It's a cool idea, but it's not done yet. Brandon and I talk about building side projects:finding a target audiencelimiting scope to something doable by one personbuilding a great looking landing pagefinding time to work on thingsprioritizing and planninglearning while buildingeven utilizing third party services to allow you to launch fasterand last, but not least, having funSpecial Guest: Brandon Braner.Links:ReleasedTailwind CSS Tailwind UIFigmaHerokuGoogle App Engine

Who should do QA?How does that change with different projects and teams?What does "doing QA" mean, anyway?Answering these questions are the goals of this episode.Links:Test Automation - Who Should be Involved? | Thoughtworks

In this episode, I talk with Paul Ganssle about a fun workflow that he calls pseudo-TDD. Pseudo-TDD is a way to keep your commit history clean and your tests passing with each commit. This workflow includes using pytest xfail and some semi-advanced version control features.Some strict forms of TDD include something like this:write a failing test that demonstrates a lacking feature or defectwrite the source code to get the test to passrefactor if necessaryrepeatIn reality, at least for me, the software development process is way more messy than this, and not so smooth and linear.Pauls workflow allow you to develop non-linearly, but commit cleanly.Links:A pseudo-TDD workflow using expected failuresepisode 171: How and why I use pytest's xfail - Paul Ganssleepisode 165: pytest xfail policy and workflowepisode 162: Flavors of TDD

In the preface of "Python Testing with pytest" I list some reasons to use pytest, under a section called "why pytest?". Someone asked me recently, a different but related question "why NOT unittest?".unittest is an xUnit style framework. For me, xUnit style frameworks are fatally flawed for software testing.That's what this episode is about, my opinion of "Why NOT unittest?", or more broadly, "What are the fatal flaws of xUnit?"Links:Python Testing with pytest, Second Editionunittest docsunittest assert methodsxUnit - Wikipedia

A prototype is a a preliminary model of something, from which other forms are developed or copied. In software, we think of prototypes as early things, or a proof of concept. We don't often think of prototyping during daily software development or maintenance. I think we should. This episode is about growing better designed software with the help of a prototype mindset.Links:Selecting a programming language can be a form of premature optimization — Brett Cannon's blog post

Paul Ganssle, is a software developer at Google, core Python dev, and open source maintainer for many projects, has some thoughts about pytest's xfail. He was an early skeptic of using xfail, and is now an proponent of the feature. In this episode, we talk about some open source workflows that are possible because of xfail. Special Guest: Paul Ganssle.Links:How and why I use pytest's xfail — Paul's blog post mentioned in the episodeCraft Minimal Bug Reports — Matthew Rocklin's articleepisode 111: Subtests in Python with unittest and pytest - Paul Ganssleepisode 165: pytest xfail policy and workflowepisode 166: unittest expectedFailure and xfail

Prayson Daniel, a principle data scientist, discusses testing machine learning pipelines with pytest.Prayson is using pytest for some pretty cool stuff, including:unit tests, of coursetesting pipeline stagescounterfactual testingperformance testingAll with pytest. So cool.Special Guest: Prayson Daniel.Links:Python Bytes 250, with Prayson Daniel — Listen to this for more of an introduction to Prayson

Performance monitoring and error detection is just as important with services and microservices as with any system, but with added complexity. Omri Sass joins the show to explain telemetry and monitoring of services and of systems with services. Special Guest: Omri Sass.

To understand complex code, it can be helpful to remove abstractions, even if it results in larger functions. This episode walks through a process I use to refactor code that I need to debug and fix, but don't completely understand.

Paul has a tutorial on testing and TDD with React and TypeScript. We discuss workflow and the differences, similarities between testing with React/TypeScript and Python. We also discuss what lessons that we can bring from front end testing to Python testing.Special Guest: Paul Everitt.Links:React, TypeScript, and TDD — Paul Everitt's tutorialReact Testing Library

xfail isn't just for pytest tests. Python's unittest has @unittest.expectedFailure.In this episode, we cover:using @unittest.expectedFailurethe results of passing and failing tests with expectedFailureusing pytest as a test runner for unittestusing pytest markers on unittest testsDocs for expectedFailure: https://docs.python.org/3/library/unittest.html#skipping-tests-and-expected-failuresSome sample code. unittest only:import unittest class ExpectedFailureTestCase(unittest.TestCase): @unittest.expectedFailure def test_fail(self): self.assertEqual(1, 0, "broken") @unittest.expectedFailure def test_pass(self): self.assertEqual(1, 1, "not broken") unittest with pytest markers:import unittest import pytest class ExpectedFailureTestCase(unittest.TestCase): @pytest.mark.xfail def test_fail(self): self.assertEqual(1, 0, "broken") @pytest.mark.xfail def test_pass(self): self.assertEqual(1, 1, "not broken")

A discussion of how to use the xfail feature of pytest to help with communication on software projects.The episode covers:What is xfailWhy I use itUsing reason effectively by including issue tracking numbersUsing xfail_strictAdding --runxfail when transitioning from development to feature freezeWhat to do about test failuresHow all of this might help with team communication

An overview of the pytest flags that help with debugging. From Chapter 13, Debugging Test Failures, of Python Testing with pytest, 2nd edition.pytest includes quite a few command-line flags that are useful for debugging. We talk about thes flags in this episode.Flags for selecting which tests to run, in which order, and when to stop:-lf / --last-failed: Runs just the tests that failed last.-ff / --failed-failed: Runs all the tests, starting with the last failed.-x / --exitfirst: Stops the tests session afterEd: after?Author: yep the first failure.--maxfail=num: Stops the tests after num failures.-nf / --new-first: Runs all the tests, ordered by file modification time.--sw / --stepwise: Stops the tests at the first failure. Starts the tests at the last failure next time.--sw-skip / --stepwise-skip: Same as --sw, but skips the first failure.Flags to control pytest output:-v / --verbose Displays all the test names, passing or failing.--tb=[auto/long/short/line/native/no] Controls the traceback style.-l / --showlocals Displays local variables alongside the stacktrace.Flags to start a command-line debugger:--pdb Starts an interactive debugging session at the point of failure.--trace Starts the pdb source-code debugger immediately when running each test.--pdbcls Uses alternatives to pdb, such as IPython’s debugger with –-pdbcls=IPython.terminal.debugger:TerminalPdb.This list is also found in Chapter 13 of Python Testing with pytest, 2nd edition. The chapter is "Debugging Test Failures" and covers way more than just debug flags, while walking through debugging 2 test failures.Links:Python Testing with pytest — The fastest way to get up to speed on pytest.all pytest flags in pytest 6.2.x

pip : "pip installs packages" or maybe "Package Installer for Python" pip is an invaluable tool when developing with Python. A lot of people know pip as a way to install third party packages from pypi.org You can also use pip to install from other indexes (or is it indices?)You can also use pip to install a package in a local directory. That's the part I want to jump in and explore with Stéphane Bidoul. The way pip installs from a local directory is about to change, and the story is fascinating.Special Guest: Stéphane Bidoul.Links:The Odoo Community AssociationPEP 610 -- Recording the Direct URL Origin of installed distributions | Python.orgPEP 660 -- Editable installs for pyproject.toml based builds (wheel based) | Python.org — Bidoulpip install --no-index --find-links Solving issues related to out-of-tree builds · Issue #7555 · pypa/pippip list json format

What flavor of TDD do you practice? In this episode we talk about:Classical vs Mockist TDDDetroit vs London (I actually refer to it in the episode as Chicago instead of Detroit. Oh well.)Static vs BehaviorInside Out vs Outside InDouble Loop TDDBDDFDDTracer BulletsRules of TDDTeam StructureLean TDD This is definitely an episode I'd like feedback on. Reach out to me for further questions or if I missed some crucial variant of TDD that you know and love.Links:Mocks Aren't Stubs - Martin FowlerMockists Are Dead. Long Live Classicists.Double Loop TDDBDD: Behavior-driven developmentFDD: Feature-driven developmentMy reaction to “Is TDD Dead?” - pythontest.comTest First Programming / Test First DevelopmentHumorous list of TDD variants — BDD = Buzzword Driven Development, CDD = Calendar Driven Development, etc

Software development processes create value, and have waste, in the Lean sense of the word waste. Lean manufacturing and lean software development changed the way we look at value and waste. This episode looks at lean definitions of waste, so we can see it clearly when we encounter it.I'm going to use the term waste and value in future episodes. I'm using waste in a Lean sense, so we can look at software processes critically, see the value chain, and try to reduce waste.Lean manufacturing and lean software development caused people to talk about and examine waste and value, even in fields where we didn't really think about waste that much to begin with.Software is just ones and zeros. Is there waste? When I delete a file, nothing goes into the landfill.The mistake I'm making here is confusing the common English definition of waste when what we're talking about is the Lean definition of waste.This episode tries to clear up the confusion.Links:Big Design Up FrontLightweight MethodologiesManifesto for Agile Software DevelopmentExtreme programmingThe New MethodologyTest First Programming / Test First DevelopmentTest Driven DevelopmentThe Pragmatic ProgrammerSix SigmaDMAICLean software developmentLean manufacturingThe Toyota WayLean Six SigmaDefinition of Waste by Merriam-Webster

Should your code be DRY or DAMP or something completely different? How about your test code? Do different rules apply? Wait, what do all of these acronyms mean?We'll get to all of these definitions, and then talk about how it applies to both production code and test code in this episode. Links:The Pragmatic Programmer, 20th Anniversary EditionDon't repeat yourself - Wikipediaa-ha - Take On MeRule of three - WikipediaWhat does “DAMP not DRY” mean when talking about unit tests? - Stack Overflow

When learning data science and machine learning techniques, you need to work on a data set. Matt Harrison had a great idea: Why not use your own Twitter analytics data? So, he did that with his own data, and shares what he learned in this episode, including some of his secrets to gaining followers.In this episode we talk about:Looking at your own Twitter analytics data.Using Python, pandas, Jupyter for data cleaning and exploratory analysisData visualizationMachine learning, principal component analysis, clusteringModel drift and re-running analysisWhat kind of tweets perform wellAnd much moreLinks:Applied Pandas: Twitter Analytics — the course

Iterative processes that include writing test code and production code together, such as TDD, help make coding fun. All of us that care about developing quality code with the help of testing can learn from each other, regardless of programming language.Today we step outside our normal Python comfort zone and talk with Gio about TDD in Swift.Gio Lodi, author of TDD in Swift, joins the show to discuss Test Driven Development, software workflows, bottom up vs top down, rapid feedback, developer vs customer facing tests, and more.Links:TDD in Swift with SwiftUI and Combinemokacoding - Gio's blogXCTestSoftware Design: Tidy First? - Kent Beck on Substack

pre-commit started as a framework for running linters and code formatters during git actions via git hooks. It's grown and expanded and now supports an extensive list of languages and actions and manual running of actions. But even at it's core, it's great for letting computers nitpick about whitespace and formatting, so that code reviews can focus on architecture and design.Anthony Sottile discusses pre-commit, for using locally by developers, and pre-commit.ci, which can run actions during merge requests."Git hook scripts are useful for identifying simple issues before submission to code review. We run our hooks on every commit to automatically point out issues in code such as missing semicolons, trailing whitespace, and debug statements. By pointing these issues out before code review, this allows a code reviewer to focus on the architecture of a change while not wasting time with trivial style nitpicks." - pre-commit.com"Developers spend a fair chunk of time during their development flow fixing relatively trivial problems in their code. pre-commit.ci both enforces that these issues are discovered, which is opt in for each developer workflow via pre-commit, but also fixes the issues automatically, letting developers focus their time on more valuable problems." - A user of pre-commit.ciLinks:pre-commitpre-commit supported hookspre-commit.ci

Flake8 is a command-line tool for linting Python projects. By default, it includes lint checks provided Pyflakes, pycodestyle, and McCabe It's also a platform, and allows plugins to extend the checks. Flake8 will run third-party extensions if they are found and installed.But what does all of that mean?Anthony Sottile is a maintainer of flake8 and has kindly offered to explain it to us.Links:Flake8: Your Tool For Style Guide Enforcementpyflakespycodestylemccabepyflakes/messages.pyflake8 Error / Violation Codeswemake-python-styleguideflake8-typing-importstox automation projectNoxpre-commit

Tim Ottinger has four questions that work great in many situations, from doing homework, to cooking, to writing code, to entire software projects.They are actually awesome questions to ask during a software project.We discuss the questions, where they came from, and look at some uses in software.The questions:What is it that needs to be done?What do we need in order to do it?Where can we get what we need?How can we tell if we’re doing it right?Bonus question that can be swapped out for #1:What's the most important thing that it doesn't do yet?Links:The Four Questions

You need tests for your web app. And it has a database. What do you do with the database during testing? Should you use the real thing? or mock it? Jeff Triplett says don't mock it.In this episode, we talk with Jeff about testing web applications, specifically Django apps, and of course talk about the downsides of database mocking.Links:"🤔 In ~12 years, I have never seen a case where mocking a database was a good idea. ..." Django NewsMotoTest & Code 103: Django - Lacey Williams HenschelModel Bakery: Smart fixtures for better tests Faker

Playwright is an end to end automated testing framework for web apps with Python support and even a pytest plugin.Links:Playwright for PythonWhy Playwright?playwright-pytest

I always learn a lot when I talk to Brett, and this episode is no exception. We talk about the packaging workflow, tools, changes, pyproject.toml, flit, setuptools, and so much more. I hope you learn as much as I did in this great discussion.Links:pytest-srcpathsPython Packaging User Guidepypa/packaging: Core utilities for Python packagesflit and requires-pythonExtending and Embedding the Python Interpreter — Python 3.9.2 documentationFlitsetuptoolsPEP 518 -- Specifying Minimum Build System Requirements for Python Projects | Python.orgPEP 517 -- A build-system independent format for source trees | Python.orgFlit editable installs with --symlink or --pth-filePEP 440 -- Version Identification and Dependency Specification | Python.orgSpecifying requires-python with flitwheelUniversal 1 binaryUniversal 2 binaryCython: C-Extensions for PythonCFFIpybind11: Seamless operability between C++11 and PythonCondaconda environmentsconda-forgeVaultsOfParnassus

Adventure, or Colossal Cave Adventure, was written between 1975 and 1977 in Fortran. Brandon Rhodes ported it to Python 3, initial release in 2011, and still maintains it. We talk to Brandon about this wonderful game.YOU ARE STANDING AT THE END OF A ROAD BEFORE A SMALL BRICK BUILDING. AROUND YOU IS A FOREST. A SMALL STREAM FLOWS OUT OF THE BUILDING AND DOWN A GULLY. >>> east A bit later... IT IS NOW PITCH DARK. IF YOU PROCEED YOU WILL LIKELY FALL INTO A PIT. >>> light(lamp) YOUR LAMP IS NOW ON. YOU ARE IN A DEBRIS ROOM FILLED WITH STUFF WASHED IN FROM THE SURFACE. A LOW WIDE PASSAGE WITH COBBLES BECOMES PLUGGED WITH MUD AND DEBRIS HERE, BUT AN AWKWARD CANYON LEADS UPWARD AND WEST. A NOTE ON THE WALL SAYS ... What's happening is that I'm playing adventure, which you can pip install thanks to Brandon Rohdes. Adventure is a faithful port to Python 3 from the original 1977 FORTRAN code by Crowther and Woods that lets you explore Colossal Cave, where others have found fortunes in treasure and gold, ...In this episode, we talk with Brandon Rhodes about this marvelous game.Links:adventure · PyPIpython-adventure: Original Colossal Caves adventure game, but in Python 3Brandon Rhodes Personal SitePython Design Patternspyephem: Scientific-grade astronomy routines for Pythonpython-skyfield: Elegant astronomy for PythonAdventure in Python 3 - announcement blog post from 2012NetHackFTL: Faster Than LightPEP 517 -- A build-system independent format for source trees | Python.org

Coming up with a testing strategy doesn't have to be stressful. Prioritizing features to test, and generating test cases for each feature can be fairly quick and painless. This episode covers a strategy for both that can be applied to many types of software.

Corey Quinn is the Chief Cloud Economist at The Duckbill Group. He's also a podcaster and writes a newsletter. And he also automates things with Python. But he doesn't write tests. Let's find out why.Reason for the interview. Rough summary of a twitter conversation:Corey: What podcasts should I try to get an invite onto?ToonArmyCaptain: Python Bytes, Test & Code, Talk PythonCorey: But... I don't test my code, "crappy Python" is all I write, and I'd feel like a giant imposter. So yeah, I'd be game. linkSo here we are. This diagram is referenced in the show, the Last Week In AWS Newsletter Production Pipeline.Special Guest: Corey Quinn.Links:Last Week in AWS

How do you test installed packages using coverage.py? Also, a couple followups from last week's episode on using coverage for single file applications. Links:episode 147: Testing Single File Python Applications/Scripts with pytest and coverageSpecifying source files — Coverage.py documentationTesting & Packaging - Hynekack

Have you ever written a single file Python application or script? Have you written tests for it? Do you check code coverage?This is the topic of this weeks episode, spurred on by a listener question.The questions:For single file scripts, I'd like to have the test code included right there in the file. Can I do that with pytest?If I can, can I use code coverage on it?The example code discussed in the episode: script.pydef foo(): return 5 def main(): x = foo() print(x) if __name__ == '__main__': # pragma: no cover main() ## test code # To test: # pip install pytest # pytest script.py # To test with coverage: # put this file (script.py) in a directory by itself, say foo # then from the parent directory of foo: # pip install pytest-cov # pytest --cov=foo foo/script.py # To show missing lines # pytest --cov=foo --cov-report=term-missing foo/script.py def test_foo(): assert foo() == 5 def test_main(capsys): main() captured = capsys.readouterr() assert captured.out == "5\n" Suggestion by @cfbolz if you need to import pytest:if __name__ == '__main__': # pragma: no cover main() else: import pytest

Building any software, including web apps and APIs requires testing. There's automated testing, and there's manual testing. In between that is exploratory testing aided by automation tools. Michael Kennedy joins the show this week to share some of the tools he uses during development and maintenance.We talk about tools used for semi-automated exploratory testing. We also talk about some of the other tools and techniques he uses to keep Talk Python Training, Talk Python, and Python Bytes all up and running smoothly. We talk about:Postmanngroksitemap link testingscripts for manual processesusing failover servers during maintenance, redeployments, etcgitHub webhooks and scripts to between fail over servers and production during deployments automaticallyfloating IP addresses services to monitor your site: StatusCake, BetterUptimethe affect of monitoring on analyticscrash reporting: Rollbar, Sentryresponse timesload testing: LocusLinks:Python Bytes PodcastTalk Python To Me PodcastTalk Python TrainingPostmanngrokStatusCakeBetter UptimeRollbarSentryLocust12 requests per second in Python

A discussion about mocking in Python with the original contributor of unittest.mock, Michael Foord.Of course we discuss mocking and unittest.mock. We also discuss:testing philosophyunit testing and what a unit isTDDwhere Michael's towel is, and what colorMicheal was instrumental in the building of testing tools for Python, and continues to be a pragmatic source of honest testing philosopy in a field that has a lot of contradictory information.Links:unittest.mock - Python docsMocks Aren't Stubs - Martin Fowlerpytest-mockmock.patchAutospeccingArrange Act Asserttesting-in-python mailing listClassical and Mockist Testing — Classical and Mockist Testing Test First Programming / Test First Developmentepisode 102: Cosmic Python, TDD, testing and external dependencies - Harry Percivalepisode 132: mocking in Python - Anna-Lena Popkespytestunittest - Python docspytest assert usage30 best practices for software development and testing | Opensource.com

Test Driven Development, TDD, is not easy to incorporate in your daily development. Martin and Brian discuss TDD and testing and Martin's experience with testing, TDD, and using it for code involved with scientific research. We discuss lots of topics around this, including:What is TDD?Should research software be tested in order to be trusted?Time pressure and the struggle to get code done quickly. How do you make time for tests also?Is testing worth it for code that will not be reused?Sometimes it's hard to know how to test something.Maybe people should learn to test alongside learning how to code.A desire for a resource of testing concepts for non-CS people.Are the testing needs and testing information needs different in different disciplines? Biology, Physics, Astrophysics, etc. Do they have different testing needs?Do we need a "how to test" resource for each?Special Guest: Martin Héroux.Links:Joy Division Album Coverepisode 140: Testing in Scientific Research and Academia - Martin Héroux — Martin's previous episode.

Completely nerding out about pytest markers with Anthony Sottile.Some of what we talk about:Running a subset of tests with markers.Using marker expressions with and, or, not, and parentheses.Keyword expressions also can use and, or, not, and parentheses.Markers and pytest functionality that use mark, such as parametrize, skipif, etc.Accessing markers with itermarkers and get_closest_marker through item.Passing values, metadata through markers to fixtures or hook functions.Links:Registering markersslow marker example in pytest documentation — Control skipping of tests according to command line optionpytest-repeat · PyPIsource code for pytest-repeatWorking with custom markers — pytest documentationUsing -k expr to select tests based on their nameMarker revamp and iteration, Historical Notes — pytest documentation

MongoDB is possibly the most recognizable NoSQL document database. Mark Smith, a developer advocate for MongoDB, answers my many questions about MongoDB. We cover some basics, but also discuss some advanced features that I never knew about before this conversation.Special Guest: Mark Smith.Links:MongoDBEverything You Know About MongoDB is Wrong!Implementing Event Sourcing and CQRS pattern with MongoDB

Visual Testing has come a long way from the early days of x,y mouse clicks and pixel comparisons. Angie Jones joins the show to discuss how modern visual testing tools work and how to incorporate visual testing into a complete testing strategy. Some of the discussion:Classes of visual testing: problems with pixel to pixel testingDOM comparisons, css, html, etc.AI driven picture level testing, where failures look into the DOM to help describe the problem. Where visual testing fits into a test strategy.Combining "does this look right" visual testing with other test workflows."A picture is worth a thousand assertions" - functional assertions built into visual testing.Baselining pictures in the test workflow.Also discussed:automation engineerTest Automation UniversityLinks:Test Automation University

Scientists learn programming as they need it. Some of them learn it in college, but even if they do, that's not their focus. It's not surprising that sharing the software used for scientific research and papers is spotty, at best. And what about testing? We'd hope that the software behind scientific research is tested. But why would we expect that? We're lucky if CS students get a class or two that even mentions automated tests. Why would we expect other scientists to just know how to test their code?Martin works in research and this discussion is about software and testing in scientific research and academia.Special Guest: Martin Héroux.Links:Python Testing with pytest: Simple, Rapid, Effective, and ScalableTest Driven Development: By ExampleMy reaction to "Is TDD Dead?" - Python TestingMartinHeroux/pliffy: Plotting differences with PythonPyBites Code ChallengesPython MorselsMartin Héroux (@martin_heroux) / TwitterScientifically Sound‪Martin Héroux‬ - ‪Google Scholar‬spike2py · PyPIpytest-mpl · PyPI

Talking with Nalin Parbhu about the software evolution towards more test automation and the creation of Infuse and useMango.We talk a software development and "shift left" where automated tests and quality checks have moved earlier into the software lifecycle.Software approaches and where quality fits inShift leftTest automationRoles of software developers, SDETs (software development engineer in test), testers, QA, etc.Developers doing testing and devopsAutomated testing vs manual testingRegression testing, UI testing, black bock testingUnit testing, white box, API, end to end testingUser acceptance testing (UAT)Mullet Methodology (Agile at the front, Waterfall at the back)Waterwheel Methodology (Requirements -> iterative development -> QA)What's an agile team?Developer resistance to testingManifesto for agile software developmentIterative developmentAdapting to changeAgility: being able to change course quicklySpecial Guests: Nalin Parbhu and Ola Omiyale.

Your test suite tells you about the quality of your code under test. Mutation testing is a way to tell you about the quality of your test suite. Anders Hovmöller wrote mutmut for mutation testing in Python, and can be used with pytest, unittest, and others. In this episode, Anders explains mutation testing, how mutation testing with mutmut works, and good workflows.Special Guest: Anders Hovmöller.Links:mutmut · PyPIcosmic-ray · PyPIMutPy · PyPIparso · PyPImutmut documentationNed Batchelder article on mutmut