Test & Code

RCRCRC was developed by Karen Nicole Johnson. In this episode we discuss the mnemonic/heuristic and use it to prioritize tests for the cards application. Recent: new features, new areas of code Core: essential functions must continue to work, your products USPs (Unique Selling Propositions) Risk: some areas of an application pose more risk, perhaps areas important to customers but not used regularly by the development team. Configuration sensitive: code that’s dependent on environment settings or operating system specifics Repaired: tests to reproduce bugs, tests for functionality that has been recently repaired. Chronic: functionality that frequently breaks Links:A heuristic for regression testing, by Karen Nicole Johnson@karennjohnsoncards on githubcards on Travis CIcards on AppVeyor

This episode starts down the path of test strategy with the first tests to write in either a legacy system or a project just getting off it's feet. We cover: My approach to testing existing systems. Put names to strategies so we can refer to them later. Explain the strategies in general terms and explain why they are useful. Discuss how these strategies are used in an example project. (The code is available on github). Strategies covered today: Dog Fooding Exploratory Testing Tracer Bullet Tests Act Like A Customer (ALAC) Tests Manual Procedures Initial automated tests at 2 levels, API and UI.

Stephanie is a co-founder and graphics engineer at Binomial. She works on Basis, an image compressor, and has customers in games, video, mapping, and any application that has lots of image data. Stephanie has also been encouraging experienced engineers to open up their twitter DMs to questions from anyone, to help mentor people not only in technical questions, but in career questions as well. She also sets aside some time to mentor people through skype when written form just doesn't cut it. That's the primary reason I have Stephanie on today, to talk about mentoring and open office hours. But we also talk about Binomial image compression texture mapping the use of both manual and automated testing for complex systems sane work hours work life balance and how long hours have led her to the opinions she holds today Special Guest: Stephanie Hurlburt.Links:BinomialList of Engineers Willing to Mentor You — Stephanie HurlburtStephanie Hurlburt (@sehurlburt) | TwitterSlack - Python Testing / Test & Code — A super nice group of folks who are great at answering test related questions.

There are lots of ways to up your skills. Of course, I'm a big fan of learning through reading books, such as upping your testing skills by reading Python Testing with pytest. And then there are online learning systems and MOOCs. At the other end of the spectrum is a full blown university degree. One option kind of in the middle is continuing education programs available through some universities, such as University of Washington. To discuss this option with me in more depth, we've got Andrew Hoover, Senior Director, Program Strategy, University of Washington Continuum CollegeSpecial Guest: Andrew Hoover.Links:UW Professional & Continuing EducationUW Career Accelerator CertificatesCertificate in Data AnalyticsCertificate in Data ScienceCertificate in Machine LearningCertificate in Project ManagementCertificate in Python Programming

An in depth discussion of Test Driven Development (TDD) should include a discussion of Test First. So that's where we start. Why write tests first? How do you know what tests to write? What are the steps for test first? Isn't this just TDD? Functional Tests vs Unit Tests Links:Test First Programming / Test First Development - Python TestingMy reaction to "Is TDD Dead?" - Python TestingEpisode 23: Lessons about testing and TDD from Kent BeckTalk Python, Episode #145 2017 Python Year in ReviewPyCon 2018 in Cleveland, Ohio | May 9-17Python Bytes PodcastPython Testing with pytest: Simple, Rapid, Effective, and Scalable: Brian Okken: 9781680502404: Amazon.com: Books

A discussion with Katharine Jarmul, aka kjam, about some of the challenges of data science with respect to testing. Some of the topics we discuss: experimentation vs testing testing pipelines and pipeline changes automating data validation property based testing schema validation and detecting schema changes using unit test techniques to test data pipeline stages testing nodes and transitions in DAGs testing expected and unexpected data missing data and non-signals corrupting a dataset with noise fuzz testing for both data pipelines and web APIs datafuzz hypothesis testing internal interfaces documenting and sharing domain expertise to build good reasonableness intermediary data and stages neural networks speaking at conferences Special Guest: Katharine Jarmul.Links:@kjam on Twitter — Data Magic and Computer SorceryKjamistan: Data Sciencedatafuzz’s Python library — The goal of datafuzz is to give you the ability to test your data science code and models with BAD data.Hypothesis Python library — Hypothesis is a Python library for finding edge cases in your code you wouldn’t have thought to look for.

A wonderful discussion with David Hussman. David and Brian look back at what all we've learned in XP, TDD, and other Agile methodologies, where things have gone awry, how to bring the value back, and where testing fits into all of this. How to build the wrong thing faster Agile vs Agility Product vs Process Where testing fits into software development practices. "Integration tests, there's a name that needs to be refactored desperately." Integration tests are "story tests". They tell the story of the product. XP and TDD and the relationship with tests To test for design, use microtests, xUnit style. User Advocy tests are often lacking, but are needed to learn about the product. "I just keep writing tests until I'm not scared anymore." - Kent Beck Dude's Law: Value = Why/How People often focus so much on the how that they forget about why they are doing something. Subcutaneous Tests "The hardest part of programming is thinking." Refactoring vs Repaving Agility means being able to quickly change direction During experimentation and learning, what matters isn't how much you got done, but how much you learn. "The best way to get automation is to make developers do manual tests." Special Guest: David Hussman.Links:DevJamPNSQC 2015 How to Build the Wrong Thing Faster and Learn From ItPeople's Front of JudeaTDD on c2The waterfall modelTest First ProgrammingDude's LawSubcutaneous Test

What started as a twitter disagreement carries over into this civil discussion of software testing. Brian and Paul discuss testing practices such as the testing pyramid, TDD, unit testing, system testing, and balancing test effort. the Testing Pyramid the Testing Column TDD unit testing balancing unit with system tests, functional tests API testing subcutaneous testing customer facing tests Special Guest: Paul Merrill.Links:Episode 34 - Software and Testing Models with Guest Host Brian Okken - Reflection As A Service — Cross posted to RaaSSubcutaneous Test — I use subcutaneous test to mean a test that operates just under the UI of an application.The Forgotten Layer of the Test Automation Pyramid — At the base of the test automation pyramid is unit testing.The Dreyfus model of skill acquisition — The Five-Stage Model of Adult Skill Acquisition

M. Scott Ford is the founder and chief code whisperer at Corgibytes, a company focused on helping other companies with legacy code. Topics include: How M. Scott Ford got into forming a company that works on legacy code. Technical debt Process debt Software testing The testing pyramid iterative development kanban readable code and readable test code Special Guest: M. Scott Ford.

Kobiton is a service to test mobile apps on real devices. QASymphony offers software testing and QA tools.Special Guest: Josh Lieberman.

Today we have an interview with Casey Rosenthal of Netflix. One of the people making sure Netflix runs smoothly is Casey Rosenthall. He is the manager for the Traffic, Intuition, and Chaos teams at Netflix. He's got a great perspective on quality and large systems. We talk about Chaos Engineering Experimentation vs Testing Testing Strategy Visualization of large amounts of data representing Steady State Special Guest: Casey Rosenthal.

What is the difference between a unit test, an integration test, and a system test? Mahmoud Hashemi helps me to define these terms, as well as discuss the role of all testing variants in software development. What is the difference between a unit test, an integration test, and a system test? TDD testing pyramid vs testing column the role of testing in software development web frameworks listen to wikipedia hatnote the world’s largest photo competition Enterprise Software with Python Links: Mahmoud on twitter: @mhashemi Mahmoud on sedimental hatnote listen to wikipedia Montage, the web platform used to help judge the world’s largest photo competition clastic 10 Myths of Enterprise Python Enterprise Software with Python course Enterprise Software with Python blog post. Special Guest: Mahmoud Hashemi.

Interview with Sam Van Oort about pyresttest, "A REST testing and API microbenchmarking tool" pyresttest A question in the Test & Code Slack channel was raised about testing REST APIs. There were answers such as pytest + requests, of course, but there was also a mention of pyresttest, https://github.com/svanoort/pyresttest, which I hadn't heard of. I checked out the github repo, and was struck by how user friendly the user facing test definitions were. So I contacted the developer, Sam Van Oort, and asked him to come on the show and tell me about this tool and why he developed it. Here's the "What is it?" section from the pyresttest README: A REST testing and API microbenchmarking tool Tests are defined in basic YAML or JSON config files, no code needed Minimal dependencies (pycurl, pyyaml, optionally future), making it easy to deploy on-server for smoketests/healthchecks Supports generate/extract/validate mechanisms to create full test scenarios Returns exit codes on failure, to slot into automated configuration management/orchestration tools (also supplies parseable logs) Logic is written and extensible in Python Support Special thanks to my wonderful Patreon supporters and those who have supported the show by purchasing Python Testing with unittest, nose, pytest

Interview with Dave HuntdWe Cover:Selenium Driverpytestpytest plugins: pytest-seleniumpytest-htmlpytest-variablestoxDave Hunt’s “help wanted” list on githubMozillaAlso:fixturesxfailCI and xfail and html reportsCI and capturing pytest code sprintworking remotely for Mozilla

pytest is an extremely popular test framework used by many projects and companies. In this episode, I interview Raphael Aurich (@hackebrot), a core contributor to both pytest and cookiecutter. We discuss how Raphael got involved with both projects, his involvement in cookiecutter, pytest, "adopt pytest month", the pytest code sprint, and of course some of the cool new features in pytest 3.Links:pytest - http://doc.pytest.orgcookie cutter - https://github.com/audreyr/cookiecuttercookiecutter-pytest-plugin - https://github.com/pytest-dev/cookiecutter-pytest-plugin

Kent Beck's twitter profile says "Programmer, author, father, husband, goat farmer". But I know him best from his work on extreme programming, test first programming, and test driven development. He's the one. The reason you know about TDD is because of Kent Beck. I first ran across writings from Kent Beck as started exploring Extreme Programming in the early 2000's. Although I don't agree with all of the views he's expressed in his long and verbose career, I respect him as one of the best sources of information about software development, engineering practices, and software testing. Along with Test First Programming and Test Driven Development, Kent started an automated test framework that turned into jUnit. jUnit and it's model of setup and teardown wrapping test functions, as well base test class driven test frameworks became what we know of as xUnit style frameworks now, which includes Python's unittest. He discussed this history and a lot more on episode 122 of Software Engineering Radio. The episode is titled "The History of JUnit and the Future of Testing with Kent Beck", and is from Sept 26, 2010. http://www.se-radio.net/2010/09/episode-167-the-history-of-junit-and-the-future-of-testing-with-kent-beck/ I urge you to download it and listen to the whole thing. It's a great interview, still relevant, and applicable to testing in any language, including Python. What I've done in this podcast is take a handful of clips from the interview (with permission from IEEE and SERadio), and discuss the clips and my opinions a bit. The lessons are: You're tests should tell a story. Be careful of DRY, inheritance, and other software development practices that might get in the way of keeping your tests easy to understand. All test should help differentiate good programs from bad programs and not be redundant. Test at multiple levels and multiple scales where it makes sense. Differentiating between TDD, BDD, ATDD, etc. isn't as important as testing your software to learn about it. Who cares what you call it.

How do you convert manual tests to automated tests? This episode looks at the differences between manual and automated tests and presents two strategies for converting manual to automated.

A listener requested that I start covering some terminology. I think it's a great idea.Covered in this episode:Test FixturesSubcutaneous TestingEnd to End Testing (System Testing)I also discuss:A book rewriteProgress on transcriptsA story from the slack channel

I talk with Michael about:Episodes of his show having to do with testing.His transition from employee to podcast host and online training entrepreneur.His Python training courses.The Pyramid Web framework.Courses by MichaelExplore Python Jumpstart by Building 10 AppsExplore Write Pythonic Code Like a Seasoned DeveloperPython for EntrepreneursTesting related podcast Episodes from Talk Python To Me:episode 10: Harry Percival, TDD for the Web in Python, and PythonAnywherePythonAnywhereHarry's book, TDD with Pythonepisode 45: Brian Okken, Pragmatic testing and the Testing ColumnTalk Python To Me podcastepisode 63: Austin Bingham, Mutation Testing, Cosmic RayCosmic Ray episode 67: David MacIver, HypothesisHypothesis

Interview with Robert Collins, current core maintainer of Python's unittest module.Some of the topics coveredHow did Robert become the maintainer of unittest?unittest2 as a rolling backport of unittesttest and class parametrization with subtest and testscenariosWhich extension to unittest most closely resembles Pytest fixtures?Comparing Pytest and unittestWill unittest ever get assert rewriting?Future changes to unittestI've been re-studying unittest recently and I mostly wanted to ask Robert a bunch of clarifying questions.This is an intermediate to advanced discussion of unittest. Many great features of unittest go by quickly in this talk. Please let me know if there's something you'd like me to cover in more depth as a blog post or a future episode.Linksunittestunittest2pipmocktesttoolsfixturestestscenariossubunitpipserverdevpitestresourcesTIP (testing in python) mailing list

In this episode, I interview with Joe Stump, cofounder of Sprintly (https://sprint.ly), to give the startup perspective to development and testing.Joe has spent his career in startups. He's also been involved with hiring and talent acquisition for several startups.We talk about testing, continuous integration, code reviews, deployment, tolerance to defects, and how some of those differ between large companies and small companies and startups.Then we get into hiring. Specifically, finding and evaluating good engineers, and then getting them to be interested in working for you.If you ever want to grow your team size, you need to listen to this.

The Travis Foundation. Interview with Laura GaetanoLinks and things we talked about:Travis FoundationOpen Source GrantsThe Foundation's support of Katrina Owen from exercism.ioExercism.ioRails Girls summer of codeDiversity TicketsConference supportSpeakerinnenPrompt

This is a small episode. I'm changing the name from the "Python Test Podcast" to "Test & Code". I just want to discuss the reasons behind this change, and take a peek at what's coming up in the future for this podcast. Links The Waterfall Model and "Managing the Development of Large Software Systems" Josh Kalderimis from Travis CI

An introduction to Lean Software Development This is a quick intro to the concepts of Lean Software Development. I'm starting a journey of trying to figure out how to apply lean principles to software development in the context of 2016/2017. Links Lean Software Development book by Mary & Tom Poppendieck wikipedia entry for Lean Software Development Patreon supporters of the show Talk Python to Me Podcast Python Jumpstart by Building 10 Apps - video course pytest sprint pytest.org pytest/tox indiegogo campaign

Interview with Josh Kalderimis from Travis CI. Josh is a co-founder and Chief Post-It Officer at Travis CI. Topics What is Continuous Integration, CI What is Travis CI Some history of the company travis-ci.org vs travis-ci.com and merging the two Enterprise and the importance of security Feature questions Travis vs Jenkins Travis notification through Slack Reporting history of Travis results Dealing with pytest results status other than pass/fail Capturing std out and stderr logging from tests Build artifacts Tox and Travis Using Selenium What does a Chief Post-It Officer do Differentiation between Travis and other CI options Using Slack to keep remote teams communicating well Travis team Funding open source projects Travis Foundation Rails Girls Summer of Code Open source grants Mustaches and beards Shite shirts New Zealand What does Team Periwinkle do Links Jeff Knupp's Open Sourcing a Python Project the Right Way Sven's blog post when Travis started Sven's mustache and Josh's beard Travis CI for open source Travis CI for private repositories and enterprise Slack Travis Foundation Rails Girls Summer of Code Talk Python to Me Podcast

Testing apps that use requests without using mock. Interview with Ian Cordasco (@sigmavirus24) Topics: Betamax - python library for replaying requests interactions for use in testing. requests github3.py Pycon 2015 talk: Ian Cordasco - Cutting Off the Internet: Testing Applications that Use Requests - PyCon 2015 Pytest and using Betamax with pytest fixtures The utility (or uselessness) of teaching programming with Java (My own rant mainly) Rackspace and Ian’s role at Rackspace and OpenStack Python Code Quality Authority: flake8, pep8, mccabe, pylint, astroid, … Static code analysis and what to use which tool when. Raymond Hettinger - Beyond PEP 8 -- Best practices for beautiful intelligible code - PyCon 2015 Links: Testing Python-Requests with Betamax Cutting Off the Internet: Testing Applications that Use Requests - PyCon 2015 github3.py requests Rackspace Openstack Python Code Quality Authority and documentation GitLab Raymond Hettinger - Beyond PEP 8 -- Best practices for beautiful intelligible code - PyCon 2015 Other Betamax resources: Betamaxing Boto3 Using Betamax with pytest fixtures Isolated @memoize

In this episode I interview Ned Batchelder. I know that coverage.py is very important to a lot of people to understand how much of their code is being covered by their test suites. Since I'm far from an expert on coverage, I asked Ned to discuss it on the show. I'm also quite a fan of Ned's 2014 PyCon talk "Getting Started Testing", so I definitely asked him about that. We also discuss edX, Python user groups, PyCon talks, and more. Some of what's covered (pun intended) in this episode: coverage.py types of coverage Line coverage branch coverage Behavior coverage Data coverage How Ned became the owner of coverage.py Running tests from coverage.py vs running coverage from test runner. edX what is it what Ned's role is Ned's blog Ned's PyCon 2014 talk "Getting Started Testing" Teaching testing and the difficulty of the classes being part of unittest fixtures package some of the difficulties of teaching unittest because of it's class based system. the history of classes in unittest coming from java's jUnit implementation Boston's Python Group PyCon in Portland Ned to do a talk here "Machete mode debugging". Practicing PyCon talks at local group meetings. At the very least, practice it in front of a live audience. Links: Ned Batchelder Coverage Coverage documentation django-nose pytest-django edX open edX Boston Python User Group Portland Python User Group - I need to go to these PyCon 2016 - Planning on attending, it's in Portland. Yay! Getting Started Testing - Ned's 2014 Pycon talk Special Guest: Ned Batchelder.

How pytest, unittest, and nose deal with assertions. The job of the test framework to tell developers how and why their tests failed is a difficult job. In this episode I talk about assert helper functions and the 3 methods pytest uses to get around having users need to use assert helper functions.

Given-When-Then is borrowed from BDD and is my favorite structure for test case design. It doesn’t matter if you are using pytest, unittest, nose, or something completely different, this episode will help you write better tests. The Given-When-Then structure for test method/function development. How and why to utilize fixtures for your given or precondition code. Similarities with other structure discriptions. Setup-Test-Teardown Setup-Excercise-Verify-Teardown. Arrange-Act-Assert Preconditions-Trigger-Postconditions. Benefits Communicate the purpose of your test more clearly Focus your thinking while writing the test Make test writing faster Make it easier to re-use parts of your test Highlight the assumptions you are making about the test preconditions Highlight what outcomes you are expecting and testing against. Links discussed in the show: Mechanics of pytest, unittest, nose unittest fixture reference nose fixture reference pytest fixtures (series of posts starting here) pytest style fixtures pytest parameterized fixtures

Intro to Harry Percival, his background and story of how he got into TDD and ended up writing a bookComparing using unittest and pytest with applicability to testing django projects. Functional end to end testing with selenium.The django test client for middle level tests.test isolationdjango and isolated unit testsunit tests vs integration testsTesting done by the development team without an external QADouble loop TDD: Functional test first, then unit testsSpikes: investigations without testsHarry's experience with having a freely available web version of a book that is also intended to be sold.Update: Comment from Harry Percival on 19-Jan-2014 I might have been a bit down on unit tests vs functional tests in that "unit tests never fail comment". Not true at all, particularly as we've just been thru upgrading django on our core system, and the unit tests really saved our bacon on that one...LinksTest-Driven Development with PythonObey the Testing Goat - Harry's site dedicated to the book and related posts.Python Testing with unittest, nose, pytestGary Bernhardt's talk, Boundaries talk including a discussion of "Functional Core, Imperative Shell".Video of Boundaries talk on youtube

In today's podcast, I dodge the question of "What do you think of Agile?" by reading an essay from Dave Thomas

The waterfall model has been used and modified and changed and rebelled against since before I started programming. Waterfall such an important character in the story of software development that we should get to know it a better.

My experience with writing software comes from my experience: where I grew up, what eras I lived through, what my economical and geographical experiences have been, when I learned to code, and what projects I've worked on.

Setup and Teardown Benefits of Test Fixtures code reuse cleanup of resources errors vs failures focusing your thinking on what you are testing and what you are not scoping for efficiency Brief look at pytest named fixtures References pytest fixtures series pytest fixtures nuts & bolts pytest session scoped fixtures unittest fixtures nose fixtures mentioned in introduction nose fixture reference post how to run a single class

Setup and Teardown Benefits of Test Fixtures code reuse cleanup of resources errors vs failures focusing your thinking on what you are testing and what you are not scoping for efficiency Brief look at pytest named fixtures References pytest fixtures series pytest fixtures nuts & bolts pytest session scoped fixtures unittest fixtures nose fixtures mentioned in introduction nose fixture reference post

Answering a listener question. Why testing? What are the benefits? Why automated testing over manual testing? Why test first? Why do automated testing during development? Why test to the user level API? After describing my ideal test strategy and project, I list: Business related, practical benefits of testing Personal reasons to embrace testing Pragmatic, day to day, developer benefits of testing

I list my requirements for a framework and discuss how Pytest, Unittest, and Nose measure up to those requirements. Mentioned: pytest unittest nose delayed assert pytest-expect doctest I did the audio processing differently for this episode. Please let me know how it sounds, if there are any problems, etc.