gradema.test._reporter

Module Contents

Classes

CommandData

StdioCommandData

Represents what stdout and stderr should be set to on a command.

TestReporter

A TestReporter is passed to all tests. This exposes a public API for tests to use and report what they are doing.
class gradema.test._reporter.CommandData
stdout: TextIO

The stdout to pass to subprocess.run

stderr: TextIO

The stderr to pass to subprocess.run

class gradema.test._reporter.StdioCommandData

Represents what stdout and stderr should be set to on a command. stdout is represented by a file path because it is expected that the caller wants to be able to know how to read from that file after the command is executed.

stdout_file: pathlib.Path | None

The file that stdout should be directed to

stdout: TextIO

The stdout to pass to subprocess.run assuming that stdout_file is None

stderr: TextIO

The stderr to pass to subprocess.run

class gradema.test._reporter.TestReporter

Bases: abc.ABC

A TestReporter is passed to all tests. This exposes a public API for tests to use and report what they are doing. A TestReporter’s implementation may log what the test is doing to the console, or even perform an action depending on how the TestReporter is configured.

abstract log(message: str) → None

Log an informative message to the user.

Parameters:

message – The message to log. No formatting is support

abstract log_unexpected_exception(exception: BaseException) → None

Log an unexpected exception

Parameters:

exception – The exception to log

abstract resolve_command(command: Sequence[gradema.test.argument.Argument], test_identifier: str) → tuple[Sequence[str], Mapping[tuple[int, gradema.test.argument.Argument], gradema.test.argument.ResolvedArgumentInfo]]

Resolves a list of Arguments by returning a list of strings.

The returned value contains the resolved command and an argument info dictionary. The key of the argument info dictionary is a tuple of (index, Argument) where index is the index of the Argument in the command parameter.

If you are wondering why command’s type is Sequence[Argument] rather than a list[Argument], it is because the Sequence type is covariant. That means that you can assign a Sequence[str] to a Sequence[Argument], which is necessary to make sure mypy type checking is happy, and correctly documents how the command argument is used: it is not modified. More information here: https://mypy.readthedocs.io/en/stable/common_issues.html#variance

Parameters:

  • test_identifier – The identifier of the test being run

  • command – The command to resolve

Returns:

A tuple of (resolved command, argument info dictionary)

Raises:

ValueError – A ValueError may be risen if the parameters supplied to this function do not give enough information to resolve a particular Argument. This exception is currently not risen in any known implementation, but it’s possible that optional parameters will be added in the future that aid in resolving an Argument that is not created as of writing this docstring

abstract report_command(command: Sequence[str]) → CommandData

Reports that the given command WILL be run

Parameters:

command – The command that will be run

Returns:

A CommandData object containing parameters that determine where the stdout and stdin of the command should go.

abstract report_stdio_command(command: Sequence[str], test_identifier: str, input_file: pathlib.Path | None, stdout_as_output: bool) → StdioCommandData

Reports that a stdio test command WILL be run.

This is similar to report_command, but it is expected that the caller will run the command and direct its output to that specified by the returned object.

Parameters:

  • command – The base command that is being run (not including any I/O redirection)

  • test_identifier – An identifier for the test. This is not recommended to contain spaces or upper case characters

  • input_file – The input file that will be directed into the command’s stdin, or None if no specific input file is being used.

  • stdout_as_output – True to use the command’s stdout as an output, False to not

Returns:

A StdioCommandData that contains the file to direct stdout to and the TextIO to direct stderr to.

abstract maybe_launch_debugger(command: Sequence[str]) → None

Calling this function will launch the debugger by running the given command only if we are in debugging mode. The implementation will likely prompt the user to make sure they want to launch the debugger

Parameters:

command – The command to launch the debugger

abstract report_diff_result(test_identifier: str, goal_file: pathlib.Path, output_file: pathlib.Path, similarity: float, fuzzy: bool) → None

Reports the result of a diff and possibly opens a diff viewer depending on the configuration of this reporter.

Parameters:

  • test_identifier – The identifier for the test

  • goal_file – The goal file

  • output_file – The output file that is supposed to contain the same contents of the goal file

  • similarity – A number from 0 to 1 representing the similarity (1 is exactly the same). Non-fuzzy diffs should only pass 0 or 1.

  • fuzzy – True if fuzzy, false otherwise

abstract report_file_exists_test(file: pathlib.Path, expected: str) → None

Reports that a file existence test WILL BE performed.

Parameters:

  • file

  • expected

Returns:

abstract report_file_exists_test_result(file: pathlib.Path, expected: str, is_correct: bool, actual: str | None) → None

Reports that a file existence test has been performed and that the result is this.

Parameters:

  • file – The file being tested for its existence

  • expected – The expected type of the file (basically the output of file <file>

  • is_correct – True if the actual matches the expected

  • actual – The actual type of the file, or None if the file does not exist

Returns: