Passing a threshold for screenshot tests in Flutter
Since this was much harder than expected, I figured I should document it for future reference, and also in order to refine the solution as I get used to it.
The regular way to screenshot tests is to write:
await expectLater(find.byType(MaterialApp), matchesGoldenFile('goldens/render_$description.png'));
However this does not let you set a threshold. So if you run the screenshot tests on different platforms, let's say MacOS from your laptop, and Linux on GitHub actions, it might work on one platform and still fail on the other by a tiny margin:
After browsing tickets and PRs on GitHub, I ended up finding a solution that involves making your own comparison function at https://github.com/Piinks/cocoon/blob/f427f28acf6933ab693b44f9d74f52a4906405f2/app_flutter/test/utils/golden.dart.
The function additionally prepends the
goldens prefix, so usage becomes:
await expectGoldenMatches( find.byType(MaterialApp), 'render_$description.png');
You can find the tweaked file and its usage here: