You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm setting up an automated performance testing pipeline using Unity and Jenkins but I'm running into the issue where even the slightest regression is marked as a failure. Is there a way to adjust the regression threshold? I noticed in the 1.0 manual this was once possible but has since been removed for some reason.
The text was updated successfully, but these errors were encountered:
I know this is a fairly old post but you can safely add the Threshold property back to SampleGroup to allow setting the threshold.
Copy the Performance Testing Extension package contents from the package cache into your project's Packages folder, and change the PTE entry in manifest.json to embedded. (Docs)
Edit Runtime/SampleGroup.cs and add the Threshold property:
/// <summary>/// The threshold, as a percentage of the aggregated sample group value, to use for regression detection./// </summary>/// <remarks>/// This value is not used directly in the Performance Testing Extension, but recorded for later use in a reporting tool (such as the Unity Performance Benchmark Reporter) to determine whether or not a performance regression has occurred when used with a baseline result set./// NOTE: Removed sometime around 2.0 of the Unity package without a note, added back in by in our custom version of the package./// </remarks>publicdoubleThreshold=0.15;
(Optional) Add threshold to the SampleGroup constructor:
publicSampleGroup(string name,SampleUnit unit = SampleUnit.Millisecond,double threshold =0.15,bool increaseIsBetter = false)
The Threshold property will now be serialized in the test results JSON and picked up by the Performance Benchmark Reporter!
I'm setting up an automated performance testing pipeline using Unity and Jenkins but I'm running into the issue where even the slightest regression is marked as a failure. Is there a way to adjust the regression threshold? I noticed in the 1.0 manual this was once possible but has since been removed for some reason.
The text was updated successfully, but these errors were encountered: