engineering

February 04, 2020   |   9min read

Development Life Cycle—Utilizing the Flutter Unit Test Framework

So, the question is how to optimize the development life cycle? For sure, code must be changed, and—at least for now—we cannot abandon that stage. But the next two stages look promising once they are fully proceeded by build tools. The most obvious idea is to try to speed up the building and deploying processes, but every optimization has a limit. Thus, the best approach is just to purge those two stages.

It is such a brilliant idea that it has already been implemented, and anyone who has ever used JRebel, knows its benefits. I used to be a backend developer. I used to work on a monolith Java web application. The app was exposed to thousands of users each day and connected to dozens of external services. Verifying, even the smallest change in the code required recompilation of the app and (auto)deploying it on a local server. With JRebel, all code changes were immediately visible in the deployed app—without any rebuilding, deploying. It was a breathtaking experience to see how fast the development process could be.

Disclaimer: of course, JRebel does some building and deploying. It takes some time as well, but significantly less than a full build. Moreover, JRebel focuses on building as a small part of the project as possible, and then injects a new version of a class to running class loader—so the app has not to be restarted.

Let’s get back to our development cycle. Two stages left. “Changing the code” and “Testing.” What else can we do? Removing one more stage sounds like a brilliant idea. Everyone likes coding, so we are gonna remove “testing.”

Ok, maybe not literally removing. But if only we could make a computer to do it for us.

Flutter

Flutter is a cross-platform framework developed by Google. Apart from a lot of advantages, Flutter is provided with a great testing framework that covers tests on three levels:

  • unit test—to test a single method or class, some specific part of the code,
  • widget test—to test a single widget, well-defined component,
  • integrations test—to test a whole app as an isolated instance.

Unit test

Unit tests have few significant advantages, especially if writing tests are a part of the Test-driven Development (TDD) approach. But this time, I am going to focus only on the testing framework itself.

Flutter provides a ready-to-use unit test framework. The good news is that if you have ever used Mocha for JS or Spek for Kotlin or Spec for Ruby, then you are familiar with the concept of the Flutter Unit Test Framework.

Once you add dependency to test package,

dev_dependencies:
  test: <latest_version>

you are ready to create your first unit test in the test directory placed next to lib. Thus, let’s analyze a piece of a unit test file.

void main() {

  setUp(() {
  ...
   });

    group("Connection", () {
      DisconnectedBleDeviceMock disconnectedBleDevice;
      DeviceDetailsBloc deviceDetailsBloc;

       setUp(() {
       ...
        });

       test("on startup should connect to the device from repository", () async {
   //given
  when(flutterBlueMock.connect(any)).thenAnswer((_) => Observable.never());

   //when
   deviceDetailsBloc.init();

   //then
   await untilCalled(disconnectedBleDevice.connect());
 });
…….

The story starts with a definition of main() function—it is nothing surprising for a Flutter developer. I want to emphasize that any other structural part of the test is a function as well. It has some significant consequences, e.g., it determines the way of writing of parameterized tests. But let’s get back to the structure of the unit test. There are 3 basic elements:

  • setup function could be used to share code between tests. setUp() callback will run before every test in a group or test suite, in contrast to tearDown() callback that will run after, even if a test fails.
  • test() function allows declaring a single test. The first parameter of the function is the name of the test, the second is the body of a test.
  • group() lets you gather tests into groups. Each group’s description is added to the beginning of its test’s descriptions.

Android Studio provides great integration for Flutter Unit Tests. IDE can recognize the structure of a test case and shows results accordingly to the structure.

text

Test control

The Flutter Unit Test Framework contains advanced controlling of test execution. Moreover, it is aligned with a vision of running Flutter applications of all the platforms: mobile, web, and desktop. Skipping or changing timeouts could be defined individually for a platform that allows handling unexpected quirks of specific systems or browsers. Some of the control options fit better to a widget or functional test, but they are useful from a unit test perspective. Take a look at some examples of controlling test behavior:

(const Duration(seconds: 45))
   import "package:test/test.dart";
void main() {
// …
}
import "package:test/test.dart";
void main() {
  group("slow tests", () {
    // ...
    test("even slower test", () {
      // ...
    }, timeout: new Timeout.factor(2))
  }, timeout: new Timeout(new Duration(minutes: 1)));
}
(const {
  "windows": const Timeout.factor(2)
})
  import "package:test/test.dart";
void main() {
  test("do a thing", () {
    // ...
  }, onPlatform: {
    "safari": new Skip("Safari is currently broken (see #1234)")
  });
}

Sometimes, there is more than one way to define exactly the same behavior. The timeout could be set globally for the whole file or individually for each test. It introduces great flexibility and allows us to tailor rules of execution of test suite, group of tests, or a single test to an environment.

BLoC Test

Business Logic Component (BLoC) architecture is widely used in Flutter applications and is promoted by creators of the framework. There are a lot of great resources about BLoC, so I will not explain the architecture here and focus only on testing that kind of approach.

The architecture, or to be specific—the idea for API of the Business Logic Component—is based on streams. So in the test, we needed a bunch of utils methods that allow us to verify what has happened on a stream through the test or wait for some specific object or sequence of objects on the stream. The Flutter Test Framework is equipped with all those methods out of the box. It means that a developer can focus on testing the behavior of a BloC without bothering oneself with internal implementation.

test("should emit ble device in all states determined by connection state", () {
 //given
 StreamController<BleDevice> devicesStreamController = StreamController<BleDevice>();
 when(flutterBlueMock.connect(any)).thenAnswer((_) => Stream.fromIterable([BluetoothDeviceState.connecting, BluetoothDeviceState.connected]));

 //then
 expectLater(devicesStreamController.stream, emitsInOrder([
   predicate((BleDevice bleDevice) => bleDevice.bluetoothDeviceState == BluetoothDeviceState.connecting),
   predicate((BleDevice bleDevice) => bleDevice.bluetoothDeviceState == BluetoothDeviceState.connected),
 ]));

 //when
 disconnectedBleDevice.connect().pipe(devicesStreamController);
});

Parametrization

If you have ever tried to parametrize a test in JUnit 4, you know it is a nightmare. Tons of boilerplate must be written to run the same test with various parameters that allow verifying the behavior of a testing piece of code in all edge cases. Each test case in Flutter is a simple method call, which means the method could be called in a loop, with a predefined list of parameters.

Unit test summary

Flutter provides a unit framework that follows a widely used specification naming. It is fast, reliable, contains a well equipped assertion module and a build-in Mock module. Actually, the framework comes with all the required tools to write tests for any application.

The only thing that I do not like about the framework is the readability of parameterized tests. Placing a few parameterized tests in a group in a testsuite leads to a situation in which the actual test is hidden among thousands of braces and loops iterating through predefined values sets.

Parameterized tests are extremely useful and couldn’t be omitted because of poor readability. Thus, a developer must plan carefully how to implement those tests, so as not to lose one of the biggest value of unit test—defining requirements to the implemented business logic. Messy source code of the test makes it hardly possible to just straightforward read requirements from the testsuite.

Widget test

The basic concept that stands beside Flutter roots is to make all view elements separated and modular. All those root principles make all widgets testable in a very pleasant way. Flutter_test library allows for rendering the whole application without the need for running tests on a device or simulator. It speeds up tests execution time and makes it more stable, because it’s isolated from device disruptions. Moreover, we can benefit from making typical operations, such as end-to-end tests, for example, tapping on elements, entering text, or making assertions.

Let’s have a look at how it works in a real project (official sample from Flutter).

text

Below you can find an example of the test described in file test/widget_test.dart for a simple counter app, as shown here:

 testWidgets('Counter increments smoke test', (tester) async {
   // Build our app, provide it with a model, and trigger a frame.
   await tester.pumpWidget(
     ChangeNotifierProvider(
       create: (context) => Counter(),
       child: RepaintBoundary(child:MyApp(),
     )),
   );

   // Verify that our counter starts at 0.
   expect(find.text('0'), findsOneWidget);
   expect(find.text('1'), findsNothing);

   // Tap the '+' icon and trigger a frame.
   await tester.tap(find.byIcon(Icons.add));
   await tester.pump();

   // Verify that our counter has incremented.
   expect(find.text('0'), findsNothing);
   expect(find.text('1'), findsOneWidget);

   // Verify layout.
   await expectLater(
   find.byType(MyApp),
   matchesGoldenFile('counter_1.png'),
 );
 });

Within this test, it is checked whether the value of the counter has changed after tap operation on the icon element. Test execution triggered with flag --update-goldens stores layout snapshot from the very final moment of this test. Next run of a test without additional flag will compare reference image with the actual one and creates diff image as below (in this example I tapped icon more than 100 times so counter space was extended, and test framework marked the difference with gray boxes):

text

This test execution happens without any emulator running in the background. It boosts the time of execution and keeps away from device instabilities.

Flutter Driver Test

If you would like to know how the application behaves on various OS configurations, you can benefit from Flutter Driver tests. This framework comes with end-to-end testing possibilities running on various devices.

Flutter Driver provides useful API, which allows for most types of operations, which real user performs, like tapping, providing text, scrolling, and a lot of useful matchers—checks displayed texts, visibility, etc.

On the following screencast, you can see an example of how this type of test performs:

text

You can find a more detailed tutorial on how to conquer UI automation with Flutter in one of our recent blog posts.

Summary

Flutter and Dart teams definitely did their homework. The framework is shipped with solutions that cover the whole test pyramid.

text

It is a tautology that automatic tests are important and could save lots of time on money.

A complicated setup or unfriendly environment of a test framework demotivates the writing of tests in the first place. Starting writing tests in the middle of a project is even less likely, so there is a high probability that the project ends without any automatic test. When using Flutter, developers are provided with an easy to set up, fast, and well-equipped test framework.

At Polidea, we found automatic tests to be an important part of the development process. That is why we have created a new opportunity. The Blemulator allows covering those parts of an app that so far were unreachable with automatic functional tests—all the screens and features that require active connection with a BLE device.

Paweł Byszewski

Lead Software Engineer

Adam Stasiak

Senior Test Engineer

Did you enjoy the read?

If you have any questions, don’t hesitate to ask!