VSCode/Docs/Python/Testing

• Testing Python in Visual Studio Code - #Py #Pytest Software/Testing

  logbook

  • Run Tests

    • #VSCode #Keyshort for running tests from VSCode/Command/Palette Cmd Shift P

      • What's the VSCode command and Keyshort to run all the discovered tests? #card

        • VSCode/Command/Test/Run/All Tests
          • Command: Test: Run All Tests
          • Keyshort: ⌘; A
          • #VSCode/Keymap
            • json

              {
                "key": "cmd+; a",
                "command": "testing.runAll"
              }

      • What's the VSCode Command and Keyshort to run the tests in open editor file? #card

        • Command: VSCode/Command/Test/Run/Tests in Current File
          • Command: Test: Run Tests in Current File
          • Keyshort: ⌘; f
          • VSCode/Keymap
            • json

              {
                "key": "cmd+; f",
                "command": "testing.runCurrentFile",
                "when": "editorTextFocus"
              }

          • #Mnemonic
            • ⌘; commonly used chord for making things happen in the UI
            • F - for File, as in current File

      • VSCode Command and Keyshort to run the test at the cursorpoint? #card

        • Command: VSCode/Command/Test/Run/Test at Cursor
          • Keyshort: ⌘; C
          • VSCode/Keymap
            • json

              {
                "key": "cmd+; c",
                "command": "testing.runAtCursor",
                "when": "editorTextFocus"
              }

          • #Mnemonic - C for Test at Cursor

    • #VSCode #Keyshort for running tests from VSCode/View/Test Explorer

      • How can you run just a few specific tests from the test explorer? #card #Mouse

        • 1. VSCode/Command/Testing/Focus on Test Explorer
        • 2. Cmd Click on more than one
        • 3. click on the Run Test button on any of them
        • TODO #Q - is there any way to do this with focus management? Certainly there must for A11y reasons

  • Run Tests in Parallel

    logbook
    • add pytest-xdist to your project's python requirements - Python/Library/pytest-xdist pytest-xdist · PyPI

  • Run tests with coverage

    logbook
    • See also VSCode/Docs/Editor/Testing/Test Coverage

      • Where and what is the icon for the test coverage button in VSCode/View/Test Explorer? #card

        • two places:
          • one on the top of the test explorer for getting all test coverage - two triangles with check in circle
          • one on each test - one triangle with a check in circle
        • 

      • What is the 1.) #VSCode #Keyshort and 2.) icon for the Show Inline Coverage button? #card

        • keyshort: ⌘; ⇧⌘I
        • icon: two circles with check
          • image.png
        • #Mnemonic - I for Inline
        • This shows the test coverage in the active file
          • 

  • VSCode/Docs/Python/Testing/Debug tests https://code.visualstudio.com/docs/python/testing#_debug-tests

    logbook
    • see also
      • Python debugging configurations VSCode/Docs/Python/Debugging
      • general VS Code Debugging  VSCode/Docs/Editor/Debugging
    • important note:

      • Note: Running or debugging a test does not automatically save the test file. Always be sure to save changes to a test before running it, otherwise you'll likely be confused by the results because they still reflect the previous version of the file!
        

    • VSCode Test Debugging Commands

      • VSCode/Command/Test/Debug/All Tests
        • Test: Debug All Tests - Launches the debugger for all tests in your workspace.
      • VSCode/Command/Test/Debug/Tests in Current File
        • Test: Debug Tests in Current File - Launches the debugger for the tests you have defined in the file you have open in the editor.
      • VSCode/Command/Test/Debug/Test at Cursor
        • Test: Debug Test at Cursor - Launches the debugger only for the method where you have your cursor focused on the editor. You can also use the Debug Test icons in Test Explorer to launch the debugger for all tests in a selected scope and all discovered tests.

    • You can change default in the test explorer from run to debug

      • by changing the testing.defaultGutterClickAction setting value to debug in your settings.json file
        

    • You can customize #VSCode global settings for #Debugging #Software/Testing and create a VSCode/launch.json for all VSCode/Command/Test/Debug commands by ... ? #card

      • See VSCode/Docs/Editor/Debugging/Launch configurations here
        • updating launch.json to include a "purpose": ["debug-test"]
          • Why? because you might want to set environment variables, etc. I was actually wondering how to set Python/Environment/Variables/PYTHONDONTWRITEBYTECODE or Python/Environment/Variables/PYTHONUNBUFFERED

          • This configuration will be used when you run Test: Debug All Tests, Test: Debug Tests in Current File and Test: Debug Test at Cursor commands.
            

          • {
              "name": "Python: Debug Tests",
              "type": "debugpy",
              "request": "launch",
              "program": "${file}",
              "purpose": ["debug-test"],
              "console": "integratedTerminal",
              "justMyCode": false
            }

        • see also Debugging configurations for Python apps in Visual Studio Code VSCode/Docs/Python/Debugging

  • VSCode/Docs/Python/Testing/Test Commands Test commands

    logbook

    • VSCode/Command and #Keyshort to Activate VSCode/Debugger on just the failed Test(s) in the VSCode/Editor/Test Explorer #card

      • VSCode/Command/Test/Debug/Failed Tests with ⌘; ⌘E
        • Test: Debug Failed Tests

          Debug tests that failed in the most recent test run.
          

        • VSCode/Keymap
          • json

            {
              "key": "cmd+; cmd+e",
              "command": "testing.debugFailTests"
            }

        • #Mnemonic
          • ⌘; - commonly used chord prefix for shifting focus, making things appear, changing ui
          • ⌘E
            • the letter E kind of looks like a call stack
            • TEst: Debug FailEd TEsts
          • How to remember that this is debug failed tests, not debug tests?
            • Maybe think of pronouncing FailED tests in a shakespearean way

    • VSCode/Command and #Keyshort to Activate VSCode/Debugger on the Tests that were run in the last run #card

      • Command: Test: Debug Last Run VSCode/Command/Test/Debug/Last Run
        • Debug tests that were executed in the most recent test run.
      • Keyshort: ⌘; ⌘L
        • #Mnemonic
          • ⌘; - commonly used chordr prefix for activating focus or other things that "change the UI"
          • ⌘L - L for Last Run
      • VSCode/Keymap
        • json

          {
            "key": "cmd+; cmd+l",
            "command": "testing.debugLastRun"
          }

    • Commands if the VSCode/Editor/Peek/Error view is open

      • See also VSCode/Docs/User Guide/Code Navigation/Peek view here

      • VSCode/Command and #Keyshort to open and move VSCode/Editor/Peek/Error view of the previous or next #Test in the VSCode/View/Test Explorer that failed #card

        • VSCode/Command/Test/Go to/Previous/Test Failure

          • Test: Go to Previous Test Failure

          • If the error peek view is open, open and move to the peek view of the previous test in the explorer that has failed.
            

          • Keyshort: ⇧⌥F8
          • VSCode/Keymap
            • json

              {
                "key": "shift+alt+f8",
                "command": "testing.goToPreviousMessage",
                "when": "editorFocus && testing.isPeekVisible"
              }

          • #Mnemonic see Go to Next Test Failure below; this is a modifier of that one

        • VSCode/Command/Test/Go to/Next/Test Failure

          • Test: Go to Next Test Failure

          • If the error peek view is open, open and move to the peek view of the next test in the explorer that has failed.
            

          • Keyshort: ⌥F8
          • VSCode/Keymap
            • json

              {
                "key": "alt+f8",
                "command": "testing.goToNextMessage",
                "when": "editorFocus && testing.isPeekVisible"
              }

          • Mnemonic:
            • This is a modifier of F8, which is Go To Next Problem in Files
              • {

                "key": "f8",
                "command": "editor.action.marker.nextInFiles",
                "when": "editorFocus"
                }
                

            • similar to the VSCode/Debugger keyshorts; F9 is set breakpoint, F10 is step over, F11 is step in ... so F8 is kind of like "before debugging".

      • VSCode/Command and #Keyshort to open VSCode/Editor/Peek/Error for a #Test particular Method that failed #card

        • Command: Test: Peek Output - VSCode/Command/Test/Peek Output
          Opens the error peek view for a test method that has failed.
          

          • note that the command name doesn't have anything to do with errors
        • Keyshort ⌘; ⌘M
          • json

            {
              "key": "cmd+; cmd+m",
              "command": "testing.openOutputPeek"
            }

          • #Mnemonic
            • ⌘; - commonly used chord for changing focus or making things appear
            • ⌘M - M for Method

    • VSCode/Command and #Keyshort to refresh tests in VSCode/View/Test Explorer #card

      • VSCode/Command/Test/Refresh Tests: ⌘; ⌘R
      • VSCode/Keymap
        • json

          {
            "key": "cmd+; cmd+r",
            "command": "testing.refreshTests",
            "when": "testing.canRefresh"
          }

      • Mnemonic: R for Refresh

    • Focus the VSCode/View/Test Explorer with a VSCode/Command and my User-Defined #Keyshort #card

      • VSCode/Command/Testing/Focus on Test Explorer: ⌘; ⌘T
        • There is no default keyshort, I've made a VSCode/Keyshort/Chord/User
      • VSCode/Keymap
        • json

          {
            "key": "cmd+; cmd+t",
            "command": "workbench.view.testing.focus"
          }

      • Testing: Focus on Test Explorer View
        Open the Test Explorer view. Similar to Testing: Focus on Python View on versions prior to 2021.9.
        

    • note: there are about four more on the page I didn't import

  • #Defer TODO VSCode/Docs/Python/Testing/Django unit tests Django unit tests

  • VSCode/Docs/Python/Testing/Intellisense for Pytest #Pytest IntelliSense for pytest

    logbook
    • This section is about VSCode/Extension/Pylance
      • works with Pytest/Fixtures and Pytest/Parameterization
        • How to parametrize fixtures and test functions - pytest documentation Pytest/Docs/How To/Parameterize
        • Pylance will offer completions
          • from @pytest.mark.parameterize decorators
          • from conftest.py
        • Pylance also supports VSCode/Code Navigation actions for fixtures and parameterized tests
          • Go to Definition
          • Find All References
          • Rename Symbol Refactoring for these entities
            • rename symbol refactoring
          • example of autocompleting housing_data from fixture
            • 
        • pylance will show the type of the fixture on hover
          • 
        • pylance support VSCode/Code Actions - see VSCode/Docs/Editor/Refactoring
          • code actions can be used to add type annotations to test funcs with fixture parameters
          • example:
            • 
          • what is an VSCode/Term/Inlay Hint and what Pylance Test configuration is available? #Q #card

            • Inlay hints for inferred fixture parameter types can also be enabled by setting python.analysis.inlayHints.pytestParameters to true in your User settings.
              

            • via ChatGPT/Response here

              • An Inlay Hint in Visual Studio Code is a small piece of contextual information that appears inline in your code, often as gray, non-intrusive text. It provides additional details such as inferred types, parameter names, or other annotations, without actually altering the source code itself. These hints help you better understand the code and improve your workflow.
                

              • When you enable the setting python.analysis.inlayHints.pytestParameters by setting it to true, Pylance (the Python language server) will show inlay hints for inferred types of the parameters in pytest fixtures. This means you’ll see hints that indicate the expected data type of the fixture parameters in your test functions, helping you understand what type of data is being injected into the function.
                

              • For example, if you have a pytest fixture called housing_data that returns a tuple of Pandas DataFrames, enabling this setting might show a hint like this:
                

                • python

                  def cost_manager(housing_data):  # (train_df: DataFrame, test_df: DataFrame, target_feature: str)
                      train_df, test_df, target_feature = housing_data

              • The hint (# (train_df: DataFrame, test_df: DataFrame, target_feature: str)) appears inline and tells you that the housing_data parameter is inferred to be of a specific type.
                

  • VSCode/Docs/Python/Testing/Test configuration settings Test configuration settings

    logbook

    • What's the difference between the VSCode/Commands that begin with Testing and those that begin with Test #card

      • The settings that affect the UI of the testing features are provided by VS Code itself, and can be found in the VS Code Settings editor VSCode/Docs/Getting Started/Settings  when you search for "Testing".
        

    • General Python settings

      • autoTestDiscoverOnSaveEnabled - discover tests in each file on each save
      • cwd - working directory for tests
      • debugPort - port num used for debugging of unittest tests
      • promptToConfigure -

        • Specifies whether VS Code prompts to configure a test framework if potential tests are discovered.
          

        • I imagine this is for disabling if the prompts get annoying

    • TODO unittest configuration settings

      • skipped

    • pytest configuration settings #Pytest

      logbook
      • pytestEnabled - t/f
      • pytestPath - the path to the pytest executable - unclear if it's python path or not, probably not
      • pytestArgs - array - args to pass to pytest, See pytest command-line options.

    • IntelliSense settings

      • inlayHints.pytestParameters - we already covered this farther up.

        • Whether to display inlay hints for pytest fixture argument types. Accepted values are true or false.