Skip to main content
Version: 7.5.5

Reporting

Test reporting is critical for DevOps. With so many releases and permutations, you’re probably sitting on a mountain of data. And that makes it hard to focus on what’s important — plus it slows you down.

Test Maker comes with powerful built-in reports and enable the developer to add their own reporters.

Built-in reporters in Test Maker:

  • Allure Reporter
  • Cucumber Reporter
  • Excel Reporter
  • Json Reporter
  • Microsoft Teams Reporter
  • Terminal Reporter
  • Text Reporter

Test Maker is detailed and offer surgical information about the whole process like:

  • Screenshots
  • Video Recording
  • Features Executed
  • Scenarios Executed
  • Steps Executed
  • Sub Steps Executed
  • Client Information
  • Duration of each Step and whole task
  • Number of Features
  • Number of Passed Features
  • Number of Failed Features
  • Number of Skipped Features
  • Detailed Error log that shows:
    • Stack Trace
    • Original Source code of error
    • Original Source code of error Line Number
    • Original Source code of error Column Number
    • Original Source code of error in Spec Line Number
    • Original Source code of error in Spec Column Number

By default Test Maker has the Terminal Reporter enabled but developer can chose to enable multiple reporters at the same time.

The following command enables both Terminal and Allure reporters.

This is how the same thing can be done through the configuration file.

NameTypeDescriptionDefault Value
errorobjectOptions for the order and the level of your error reporting.
hudobjectA hud visible in the browser window displaying info related to your test execution.
nativeDialogsHintobjectOptions for a native dialog hint to be displayed in your browser window when a native dialog has been trigered in your test.
notificationobjectOptions for notifications to appear on your desktop at different step of your test execution.
pathstringOptions for the path of your generated reports."./dist/reports"
reportersobject/string/ TestMakerReporterOptions for the different reporters that you want to use either built-in reporters or custom.["terminal"]
screenshotsobjectOptions for your screenchots.
videosobjectOptions for your videos.

error

NameTypeDescriptionDefault Value
allCodeFramesbooleanDetermines whether all code frames should be included in the error reporting.true
codeFrameOrder"topToBottom" | "bottomToTop" | undefinedSpecifies the order in which code frames should be displayed."topToBottom"
stackTraceOrder"topToBottom" | "bottomToTop" | undefinedSpecifies the order in which stack traces should be displayed."topToBottom"
stackFilter(params: { errorInfo?: ErrorInfo; stackFrames: CodeStackFrame[]; }) => Promise<CodeStackFrame[]> | undefinedA function to filter the stack frames based on certain conditions.undefined

Details:

  • allCodeFrames: Determines whether all code frames should be included in the error reporting. By default, it's set to true.
  • codeFrameOrder: Specifies the order in which code frames should be displayed, either from top to bottom, bottom to top, or left as undefined.
  • stackTraceOrder: Specifies the order in which stack traces should be displayed, either from top to bottom, bottom to top, or left as undefined.
  • stackFilter: A function that can be used to filter the stack frames based on certain conditions. This function takes an object parameter with optional errorInfo and required stackFrames, and returns a Promise of filtered CodeStackFrame array. By default, it's undefined.

hud

NameTypeDescriptionDefault Value
enabledbooleanEnable the presence of a test detailing hud during test execution .false
opacitynumberDetermine the opacity degree of the hud.1

Test Maker HUD

Test Maker HUD

nativeDialogsHint

NameTypeDescriptionDefault Value
alertbooleanIf true, Test Maker simulates an alert dialog by showing you a pop-up inside the browser window when an alert dialog is triggered.false
closeTimeOutnumber4000
confirmbooleanIf true, Test Maker simulates a confirm dialog by showing you a pop-up inside the browser window when an alert dialog is triggered.false
onbeforeunloadbooleanIf true, Test Maker simulates an onbeforeunload dialog by showing you a pop-up inside the browser window when an alert dialog is triggered.false
positionstringDefine the location on the page of the hint pop-up.top
promptbooleanIf true, Test Maker simulates an onbeforeunload dialog by showing you a pop-up inside the browser window when a prompt dialog is triggered.false

notification

NameTypeDescriptionDefault Value
onFeatureStartbooleanIf true, triggers a desktop notification upon reaching the start of a feature.false
onScenarioDonebooleanIf true, triggers a desktop notification upon reaching the end of a scenario.false
onScenarioStartbooleanIf true, triggers a desktop notification upon reaching the start of a scenario.false
onStepDonebooleanIf true, triggers a desktop notification upon reaching the end of a step.false
onStepStartbooleanIf true, triggers a desktop notification upon reaching the start of a step.false
onTaskDonebooleanIf true, triggers a desktop notification upon reaching the end of a task.false
onTaskStartbooleanIf true, triggers a desktop notification upon reaching the start of a task.false
onFeatureDonebooleanIf true, triggers a desktop notification upon reaching the end of a feature.false

reporters

NameTypeDescriptionDefault Value
namestringSpecify name of a reporter.
optionsundefinedDefine the options of a reporter

Basic Usage

For some reporters, some options are ready to be used but they need to be imported. See example below.

import { TerminalReporterOptions } from "test-maker";
[
{
...reporters.terminal,
...{
options: <TerminalReporterOptions>{
ignoreLogLevel: false,
feature: { start: true, done: true },
scenario: { start: true, done: true },
step: { start: true, done: true },
subStep: { start: false, done: true },
hook: { start: false, done: true },
controllerAction: { start: true, done: true },
selectorAction: { start: true, done: true, retries: false },
assertionAction: { start: false, done: true, retries: false },
scoreBase: 'feature',
},
},
},
]

TestMakerReporter

Basic Usage

reporting:{
reporters:[
reporters.allure,
reporters.cucumber,
reporters.excel,
reporters.json,
// reporters.microsoftTeams, require options to define microsoft team group.
reporters.terminal,
reporters.text
]
}

screenshots

NameTypeDescriptionDefault Value
enabledbooleanIf true, enable the taking of screenshots.true
enabledInobjectObject containing options that set when a screenshot has to be taken.
failedOnlybooleanIf true, screenshots will be taken and saved in case of a failure in the test execution.true
fullPagebooleanIf true, the screenshots will display the entirety of the website page.false
pathstringThe path where you want your screenshots to be located."./dist/reports/assets/screenshots"
pathPatternstring""

enabledIn

NameTypeDescriptionDefault Value
AfterEachFeaturebooleanIf true, a screenshot is taken after the end of a feature.true
AfterEachScenariobooleanIf true, a screenshot is taken after the end of a scenario.true
AfterEachStepbooleanIf true, a screenshot is taken after the end of a step.true
AfterFeaturebooleanIf true, a screenshot is taken after the end of a feature or after the end if of a feature hook if defined.true
AfterScenariobooleanIf true, a screenshot is taken at the end of a scenario or after the end if of a scenario hook if defined.false
BeforeEachFeaturebooleanIf true, a screenshot is taken just before the start of a feature.false
BeforeEachScenariobooleanIf true, a screenshot is taken just before the start of a scenario.false
BeforeEachStepbooleanIf true, a screenshot is taken just before the start of a step.false
BeforeFeaturebooleanIf true, a screenshot is taken just before the start of a feature or of a feature hook if defined.false
BeforeScenariobooleanIf true, a screenshot is taken just before the start of a scenario or of a scenario hook if defined.false
FeaturebooleanIf true, a screenshot is taken at the end of a feature.false
ScenariobooleanIf true, a screenshot is taken at the end of a scenario.false
StepbooleanIf true, a screenshot is taken at the end of a step.false

video

NameTypeDescriptionDefault Value
enabledbooleanIf true, enable the recording of videos.false
failedOnlybooleanIf true, videos will be recorded in case of a failure in the test execution.true
ffmpegobject
pathstringThe path where you want your videos to be located."./dist/reports/assets/videos"
pathPatternstring
singleFilebooleanfalse
sizeobjectObject defining the video format.

ffmpeg

NameTypeDescriptionDefault Value
encodingOptionsobjectDefine the encoding option of the recorded videos.
pathstring""

encodingOptions

size

NameTypeDescriptionDefault Value
heightnumberRepresents the height parameter of the recorded videos768
widthnumberRepresents the width parameter of the recorded videos1024