Contents

    Automated Tests in Flutter: Native Tests or Cross-platform Solutions

    When Surf first turned its hand to Flutter development, we wondered what could be automated with Flutter alone. After all, the apps it creates are cross-platform, which means the automated tests should be as well… However, a stable package for E2E and widget tests was added to the Flutter framework only a short while ago, namely, at the beginning of December 2020. So, the subject is worth discussing.

    Today we’ll talk about automated tests in the Flutter framework in the context of mobile applications. Read more about app testing in Surf in the article.

    Automated tests in Flutter

    With Flutter, automated tests can be written natively using Dart. The framework offers:

    • Unit tests to check a specific module of a system For example, to make sure that a component controller sets the right state. Since it doesn’t test the interface, there’s no need to emulate the app.
    • Widget tests. All Flutter apps consist of widgets: Widget tests help emulate widgets and test them as needed. A widget can be as big as an entire screen or as small as its specific elements: fields, buttons, checkboxes, etc.
    • End-to-end (E2E) are the tests where we load an entire app and simulate user actions within a real infrastructure with real services, APIs, etc.

    At Surf, all unit tests are written by developers because they tend to be much more familiar with an app. Read here how we hire the Flutter developers.

    All widget and E2E tests are written by QA automation engineers; both types of tests are closely connected with the finished app and test plans.

    Implementing and using automated tests in Flutter. The strategy

    Thorough testing needs to have both widget and E2E tests.

    Widget tests. These are great to run on pull requests. They work on mock data and are run a few times faster.

    E2E tests. Testing your app with comprehensive user scenarios and an actual server is a must. Otherwise, it’s hard to be sure that the system is fully functional. At the same time, it’s pointless to run them on pull requests and things like that: E2E tests are slow and unstable. Their instability is due to the infrastructure rather than the way they are written; at any time, you may be left with a server that went down, a request that failed, or a real device that is showing a system dialog. Any of those will crash a test.

    You can get the most out of automated tests by running them on stable features as often as possible.

    A simple authorization scenario consisting of three separate screens

    We cover a business scenario of both successful and unsuccessful authorization with several negative and positive test plans using E2E tests. To check if separate modules of the system are functioning, we need widget tests. At least one for each screen covered by the E2E scenario, as well as for each widget element, if necessary.

    The new package

    At the end of 2020, Flutter updated the flutter_test package. Before that, Flutter handled widget tests and E2E tests separately. 

    Widget components had a class called WidgetTester. With it, you could load a widget and test it. E2E had a class called FlutterDriver. It loaded the app and allowed you to imitate interface interactions. Now, we only have WidgetTester: Flutter Driver is deprecated.

    The first thing that catches your eye is everything’s become simpler.  E2E tests now use the same API as widget test — inside the flutter_test package using the WidgetTester. As a result, implementing support for both widget and E2E tests became easier. 

    How Surf structures automated tests in a Flutter project

    .
    ├── build.yaml                             
    ├── dart_test.yaml                        
    ├── integration_test                      
    │   ├── credentials                        
    │   │   ├── profiles                             
    │   │   │   └── *_profile.dart                  
    │   │   ├── credentials.dart                     
    │   │   └── texts.dart                           
    │   ├── features                                 
    │   │   └── *.feature                                
    │   ├── gherkin       
    │   │   └── reports                              
    │   │       └── gherkin_reports.json             
    │   ├── hooks                                    
    │   │   └── *_hook.dart                          
    │   ├── step_definitions                             
    │   │   └── *_steps.dart                         
    │   ├── worlds
    │   │   └── *_world.dart                         
    │   ├── gherkin_suite_test.dart                       
    │   ├── gherkin_suite_test.g.dart                  
    │   └── step_definitions_library.dart            
    ├── lib                                         
    │   ├── ui
    │   │   └── res
    │   │       └── test_keys.dart                   
    │   └── main.dart                                
    ├── test 
    │   ├── widget_test                              
    │   └── tests                                       
    ├── test_screens                                 
    │   ├── screens
    │   │   └── *_screen.dart                        
    │   ├── utils                                    
    │   │   └── *_utils.dart                         
    │   ├── screens_library.dart                     
    │   └── test_delays.dart                         
    ├── pubspec.yaml                                     
    └── test_driver                                       
        └── integration_test.dart  

    build.yaml — contains configurations for build_runner. It’s mainly used to add folders and files visible to a code generator.

    dart_test.yaml — contains unit- and widget-test configurations. For example, tagsto filter tests. 

    _profile.dart — contains correlations between accounts in scenarios and actual accounts. Makes it easier to change accounts in scenarios and to have various profiles, if needed.

    credentials.dart — contains account data in profiles. You can state any data needed for a scenario.

    texts.dart — contains various texts from the app. For instance, error messages from snack bars for their future use in testing. If we have several languages in the app, we create a separate object for each.

    .feature files — contain scenarios written in Gherkin. Each step of these scenarios is implemented and run during testing.

    At Surf, we write scenarios for E2E tests in a human-readable language — Gherkin. Thanks to that, any member of our team can understand what is being tested and how it is being tested — even if they know nothing about automated tests.

    To get an idea of the approximate code coverage, you can draw parallels with test cases. Scenarios can be borrowed from test cases. If a test fails, testers will have no trouble checking it: they will simply use a scenario to reproduce it.

    That’s why it was crucial for us to make sure that automated tests in Flutter also support Gherkin docs.

    gherkin_reports — the report.

    _hook.dart — hooks that flutter_gherkin uses when it runs scenarios. There are several types of them, which are run depending on the specific state of an application. 

    _steps.dart — files containing implementation of the steps from a Gherkin scenario. Those are grouped into separate files according to their content. Then they are assembled together by step_definitions_library.dart. The file contains a Step class with a list of defined steps. 

    _world.dart — objects storing the state of a current scenario in the flutter_gherkin framework. In the  _world.dart files, we can add or redefine the object logic. For example, allow to pass information between the steps of a single scenario.

    gherkin_suite_test.dart — a file where flutter_gherkin is configured and the tests are launched.

    gherkin_suite_test.g.dart — a file generated by a build_runner package based on scenarios in feature files. The reason is that, in order to work, integration_test currently needs all code to be generated before the tests are run. Editing the content is not advisable.

    step_definitions_library.dart — we need this file to incorporate the lists of step implementations into a single list, which is then passed to the stepDefinitions parameter of a configuration.

    test_keys — files containing widget component identification keys.

    main — the file allowing you to launch an app.

    test — the folder containing unit and widget tests.

    screen.dart) — is used to store locators in a single location and group them according to screens and features.It also helps reuse code in widget and Е2Е tests: finders have the same syntax. The screen file can also contain context parameters or gestures used in steps where testers have to swipe.

    _utils.dart — files containing various methods externalized to be reused in E2E testing or Widget testing steps. Mostly they are extensions of the WidgetTester class. Such utilities almost always require a WidgetTester instance. So, instead of passing it as a parameter, you can write an extension. It will be clearer and easier to use. 

    screens_library.dart — a file to group screens. It simplifies importing in steps and widget tests.

    test_delays.dart — various delays for requests and interactions.

    pubspec.yaml — contains dependencies for a Flutter project.

    integration_test.dart — mostly, this file contains code needed in order to generate an accurate report in the Cucumber-json format. The most important part is in the main function. It sets a pathway for the report and returns a driver for tests.

    Gherkin scenario

    There already are some ready-made frameworks to work with Gherkin. We were choosing between two packages: flutter_gherkin and ogurets_flutter. In the end, we picked flutter_gherkin. It had the most active contributors: you could tell that people were working on it. Spoiler alert: we were right and are still using it to this day.  

    #language: ru
    Feature: Authorization
      @auth
      Scenario: Auto: Authorization with a correct OTP
        When I launch the app
        And I go to the Library tab
        And I tap the Log in button
        And I enter a random phone number
        And I tap the Next button
        And I enter the OTP “12345”
        Then I see the Library tab of the authorized user
    
    < . . . >

    In new E2E tests, the code for these steps has to be generated before the run. Parsing Gherkin in real time doesn’t work anymore. After Gherkin scenarios are edited, you need to run codegen, which generates a scenario file in Dart code. This poses some problems, e.g., all the scenarios given in the report get into a single feature.

    It may be fixed soon, though: flutter_gherkin receives intensive contributions and upgrades.

    Implementing an E2E scenario

    class AuthSteps {
    when<ContextualWorld>(
             RegExp(r'I tap the Log in button'),
             (context) async {
               final tester = context.world.appDriver.rawDriver;
               await tester.implicitTap(AuthScreen.loginBtn);
               await tester.pumpAndSettle();
    
             },
           ),
    when1<String, ContextualWorld>(
             RegExp(r'I enter the OTP {string}$'),
             (code, context) async {
               final tester = context.world.appDriver.rawDriver;
       await tester.pumpUntilVisible(AuthScreen.otpCreateScreen);
               await tester.enterOtp(code, AuthScreen.otpFieldNoError);
               await tester.pumpUntilVisible(AuthScreen.otpRetryScreen);
             },
           ),
    < . . . >

    A step contains a string or a regular expression that refers it to a step in a Gherkin file in addition to a function to be executed in the step. 

    A digit you see after the name of the key word in the function stands for the number of arguments in this step. For instance, when1. We pass those together with the context for a function in the body of a step.

    Nota Bene: Let’s say we have several steps that look similar. One step contains the entire text of another step: “I tap the button” and “I double tap the button.” This can be problematic: the only step implementation defined is the one that was found first. In this case, it’s important to use regular expressions (just like in any other cross-platform tool). RegExp with a symbol $ at the end — RegExp(‘I press a button$’) — helps avoid conflicts and clearly specifies the end of the step. We no longer find ourselves in a situation where a matcher chooses the wrong step definition.

    Implementing a Widget scenario

    testWidgets('Input field limit is 11 symbols, (WidgetTester tester) async {
    // given
          await tester.pumpWidget(GeneralWidgetInit.defaultWrapper(
            const AuthLoginScreen(),
          ));
          await tester.pumpAndSettle();
          await tester.doEnterText(AuthScreen.phoneFld, randomNumber(12));
    // then
          final field = tester.widget<TextField>(AuthScreen.phoneFld);
          expect(field.controller.text.length, 11);
        });
    < . . . >
     }); // group('Phone-based authentication')
    }
    < . . . >

    A while ago, if you needed to access an element, you couldn’t define it with a locator so that it could be used in both widget tests and E2E tests. Now it can be done automatically.

    The cool thing is that you can now take an entire widget property, such as length, and compare it to the expected result. To check the input limit, we enter more than 11 symbols. When the result returns, we check if we got no more than 11 symbols.

    Reusing components and methods

    Reusable components

    Taking out reusable components is now as easy as anything. Both types of tests use the same class, WidgetTester. That’s why selectors and common functions cause no issues. We can, say, take each of them out and the job is done.

    Bottom line: selectors (Finder class), functions, and gestures can be reused between widgets and E2E tests.

    class AuthScreen {
     // PIN code input screen at authorization
     static Finder pinScreen = find.byKey(AuthTestKeys.pinScreen);
    
     // “Create PIN code” screen
     static Finder pinCreateScreen = find.byKey(AuthTestKeys.pinCreateScreen);
    
     // PIN keyboard element on the PIN code input screen at authorization
     static Finder pinNumberLogin(String number) =>
         find.descendant(of: pinScreen, matching: find.byKey(AuthTestKeys.pinKeyboardBtn(number)));
    
     // Log in field on the authorization screen
     static Finder loginField = find.byKey(AuthTestKeys.loginField);
    
    < . . . >
    }

    Keys

    abstract class AuthTestKeys
    {
      /// key for a login field on the authorization screen
    static const Key loginField = Key('fld_auth_login');
    }
    
    class AuthScreen {
      /// Log in field on the authorization screen
      static Finder loginField = find.byKey(AuthTestKeys.loginField);
    }

    It’s easier to store all the keys in the app folder. That way, developers and testers always know what key to use for which purpose. If it’s not a test key, it must have a real purpose.

    Pros:

    • Code readability has improved.
    • Autocomplete helps with locators.
    • The keys contain no mistakes or typos: they refer to a variable and are unique.

    There is no need to initialize the classes of screens and keys: they have no physical meaning.

    Reusable methods

    Even though the reusable methods mirror screens, they provide methods for specific steps so as not to replicate the code. In utils you almost always need access to WidgetTester, but passing it as a parameter doesn’t really look neat, so we use extension.

    extension AuthExtendedWidgetTester on WidgetTester {
    /// PIN code input method [pin]. The screen is set [pinNumber]  
    /// because Finder of a button depends on both the screen AND pin digits, hence we get the Finder in this method
    
    
      Future<void> enterPin(String pin, Finder Function(String) pinNumber) async {
        final streamPin = Stream.fromIterable(pin.split(''));
        await pumpForDuration(const Duration(milliseconds: 500));
        await for (final String ch in streamPin) {
          await implicitTap(pinNumber(ch));
        }
      }
    }

    We’re calling enterPin via the tester. That way, it’s clear what’s happening.  

    Gestures

    Offset scrollDown = Offset(0, -120)
    
    abstract class GeneralGestures {
      /// scroll direction - which way the list moves
      /// flick/swipe/etc direction - which way a finger moves
      // gestures to scroll the screens down
      static const Offset scrollDown = Offset(0, -120);
      static const Offset flickUp = Offset(0, -600);
    
      // gestures to scroll the screens up
      static const Offset scrollUp = Offset(0, 120);
      static const Offset flickDown = Offset(0, 600);
    }

    Tests often require swiping and scrolling. Writing const Offset scrollDown = Offset(0, -120) every time is too much. And changing something is too cumbersome. That’s why we have gesture files that we call in specific steps.

    For example, the gestures to scroll the screens in common gestures.

    Functions

    We added new functions to help us work more efficiently with automated tests in Flutter.

    For example,

    pump

    In widget and E2E tests, you need to call pump in between the actions so as to constantly keep the app going round and round. You have to specifically tell the test engine to rebuild a widget. If you’re not pumping, the app is not running. 

    flutter_test has the following default methods:

    • pump() — a method that starts processing a frame after a specified delay.
    • pumpAndSettle()—waits until timeout for new frames to finish rendering. This method will go round and round if the animation doesn’t stop.

    You could make do with these methods, but in some situations they just don’t work. That’s why we created several backup methods for the sake of convenience.

    pumpUntil

    /// A function that pumps until a Widget is found
    Future<bool> pumpUntilVisible(Finder target,
        {Duration timeout = _defaultPumpTimeout, bool doThrow = true}) async {
      bool condition() => target.evaluate().isNotEmpty;
      final found = await pumpUntilCondition(condition, timeout: timeout);
      // ignore: only_throw_errors
      if (!found && doThrow) throw TestFailure('Target was not found ${target.toString()}');
      return found;
    }
    /// A method that pumps until a certain [condition] occurs
    Future<bool> pumpUntilCondition(bool Function() condition,
        {Duration timeout = _defaultPumpTimeout}) async {
      final times = (timeout.inMicroseconds / _minimalPumpDelay.inMicroseconds).ceil();
      for (var i = 0; i < times; i++) {
        if (condition()) {
          await pumpForDuration(_minimalInteractionDelay);
          return true;
        }
        await pump(_minimalPumpDelay);
      }
      return false;
    }

    This method pumps until a specific condition or timeout occurs.

    You may have noticed the _minimalPumpDelay variable. Once, we found ourselves in a situation where tests started seeing buttons from the next screen even though the animated screen transition had not yet finished. If we call pump() in a cycle searching for a widget, it may jam the entire flow and the app will start lagging. Tests take a long time to run and sometimes get flaky. The logic doesn’t have enough time to execute because it’s blocked by complex locators.

    Solution: we added a 50 ms delay to pump in cycles. It worked. Thus, you need to add _mininalPumpDelay in order for the animated transitions to finish after a widget is found.

    implicitTap 

    Future<void> implicitTap(Finder finder, {Duration duration}) async {
        final found = await pumpUntilVisible(finder, duration: duration ?? _defaultPumpTimeout);
        if (!found) {
          // ignore: only_throw_errors
          throw TestFailure(finder.toString());

    implicitTap is a method that uses convenient pumpUntil* methods. implicitTap is, unsurprisingly, an implicit tap. It will wait for its Finder through pumpUntilVisible and tap it. At timeout, it shows a test error.

    Context

    class ContextualWorld extends FlutterWidgetTesterWorld {
      Map<String, dynamic> scenarioContext = <String, dynamic>{};
    
      @override
      void dispose() {
        super.dispose();
        scenarioContext.clear();
      }
    
      T getContext<T>(String key) {
        return scenarioContext[key] as T;
      }
    
      void setContext(String key, dynamic value) {
        scenarioContext[key] = value;
      }
    
      Future<void> attachScreenshot() async {
        final bytes = await appDriver.screenshot();
        attach(base64Encode(bytes), 'image/png');
      }
    }

    Certain scenarios require storing data. Let’s say we opened a flypage of a plastic card, added it to hidden cards, and now want to find it in the “Hidden” section. To do that, the app needs to store the product ID, or in other words, the search data.

    We wanted to be able to store any parameters, so we created our own World  — ContextualWorld. To aid the process, there are two helper methods:  

    • setContext sets a value for a key in the context.
    • getContext returns the value. Make sure you set Generic for getContext so as not to write something like “as String” in every step. The context can be dynamic, so you need to cast types. With this method, you can simply state the type <String> in <T>.

    The parameters you need in order to reuse it are stored in screen files. That way, you can use readable parameters, such as AuthParams.user, and not have to write the key strings each time.

    That’s how we rewrote authorization in scenarios. There’s a step “Suppose I’m using the “account” account”, which stores user. All steps that handle authorization use this parameter from the context. There’s no need to write “enter password for user “user””, “enter OTP for user “user”” and stuff like that. 

    Hooks 

    Hooks — are interceptors that help manage the lifecycle of a widget component.

    Resetting the application state

    Some large projects use GetIt, a global state storage. It’s initialized once when an app is launched. However, tests can launch the app several times per session, leading to conflicts. The onBeforeScenario hook type resets GetIt, and the app doesn’t fail to launch.

    onAfterScenarioWorldCreated resets the entire app storage before a new scenario, because a scenario may fail, and then the hook will not be executed.

    Screenshots of test failure

    There is a hook that makes a screenshot when a test fails. That’s handy when you run the whole test routine and then check the report.

    Restarting

    when<ContextualWorld>(
          RegExp(r'I am restarting the app$'),
              (context) async {
            final navigator = devGetIt<GlobalKey<NavigatorState>>().currentState;
            unawaited(navigator.pushAndRemoveUntil(SplashScreenRoute(), (_) => false));
            final tester = context.world.appDriver.rawDriver;
            await tester.pumpUntilVisibleAny([AuthScreen.loginField, AuthScreen.pinScreen]);
          },
        ),

    Another issue is having to restart an app in the middle of a scenario. For example, when you set a PIN code or get authorized with one.

    We found quite an elegant solution: we receive the current navigator and return it to the Route-splash-screen, the very first screen. We don’t actually restart an app, but it’s faster and the app acts like it’s been restarted.

    Steps

    The framework processes steps as a list of StepDefinitionGeneric entities. Therefore, you have to either write all the steps in a single list, or specify each of them individually, put them together in a list, and so on.

    We split steps into screens. Each file with steps has a class, which in turn has a parameter called steps: that’s where we put the steps. If the steps follow the same pattern, such as “I see the flypage of a debit card” and “I see the flypage of a credit card”, we take them out into a separate private variable and only then merge them into the list. All steps for all screens are then merged into a list in the step_definitions_library.dart file and are imported into the configuration.

    static Iterable<StepDefinitionGeneric> get steps => [
            ..._SeeProductDetails.steps,
            <otherStepsHere>,
    ]
    ---
    ///The “I see” steps on flypages
    class _SeeProductDetails {
     static final Iterable<StepDefinitionGeneric> steps = [
       then<ContextualWorld>(
         RegExp(r'I see...

    Coverage

    You can measure code coverage as follows:

    • with widget elements in the code. Here you can use services like Coverage. 
    • Similarly to manual cases (component and scenario ones).

    Reports

    To get reports, we use Cucumber-html-reporter. The framework finishes the test run and creates a json file in the Cucumber format. Using a third party application, you can get an interactive HTML report: it’s easy to view and it can be used as an artefact. All you have to do is put the file into the same folder as the Node application, launch it, and it will create a file.

    Test plans

    Every test starts with a scenario. If you can’t understand the exact process of writing a scenario, you can hardly see the whole benefit of it. Why do we use both types of tests? Why are both types of automated tests written by QA engineers, not developers?

    We have two types of test plans:

    • business scenarios, which we use to cover the critical flows of an app with,
    • component tests, which we use to check that each element of an app functions.

    Business scenarios

    Launch app-> App shows Authorization screen -> We have an account, Enter correct login and password -> We are successfully authorized -> App shows catalog screen -> Open flypage of any book -> Add book to cart -> “Add to cart” icon changes into “Go to cart” button -> Go to cart -> Cart contains book we added -> Proceed to checkout -> Choose payment at receipt, enter address -> Checkout successfully -> Order created, book disappeared from cart, but “My orders” screen has new entry

    Business scenarios allow us to run thorough regression tests and check the parts of the app frequently visited by users.

    Example:

    We have a book store app where you can order books when authorized. In this case, the scenarios are as follows:

    -> get authorized successfully and unsuccessfully (e.g., by entering the wrong password),

    -> choose a book (via the search or in the catalog), 

    -> go to the cart, order the book, and pay for it. 

    Plus various combinations reflecting the critical flows of the app.

    Component tests

    What we do is take each screen of a feature or an element of the screen and treat it as an individual object. We fully cover it with test plans: e.g., if we’re dealing with an input field, we need to check that it is displayed, that the number of symbols you can enter is limited, that the information is validated, etc.

    In a nutshell, here’s how to split a feature into separate components.

    Mobile apps consist of screens, curtains, pop-ups, input fields, buttons, check boxes, switchers, etc. That’s what we built upon in the first place: we decided to split all the parts into elements and their locations. Then, we delved into the client-server app architecture (these days, it’s a must). 

    The data we see on the screen almost always comes from a request. It can return a correct or incorrect response. Screens should be tested separately and in direct interaction with a request. The same goes for elements.

    Here’s a landing page giving a detailed explanation about types of testing we use in Surf.

    In Flutter, everything is a Widget, that’s why tests for Widget elements are mapped to component tests, which cover each element or screen in detail. An entire user scenario can be easily mapped to E2E tests.

    Hopefully, now you see why it’s easier to task QA with writing both types of tests: all the scenarios are needed in manual testing. Widget and E2E tests can make manual testing so much easier, e.g., in regression and full testing.

    First Flutter banking app in Europe for Rosbank developed by Surf

    More Flutter case studies developed by Surf 

    The pros of our strategy

    • Stable and quick widget tests. Our strategy helps run them on pull-requests and prevents bugs from emerging in test builds. They are easy to maintain; you can always tell when a test gets broken as a result of code adjustments in a mobile app.
    • E2E tests cover user scenarios. You can run those at night and make the life of a manual tester so much easier, having covered sanity and smoke tests.
    • Component and integration tests. These two types of tests help handle both mock data and a real server within the same project. This approach helps us measure the coverage of our code, scenarios, and technical specs.

    Pros and cons of this approach to automated tests

    Pros

    • The tests are native and cross-platform at the same time. The framework has active contributors.
    • Access to architecture and entities allows us to set locators for specific elements directly. 
    • Automation engineers can always detect file changes in a mobile app. Developers also see the component and scenario tests and are able to make timely adjustments themselves.

    Cons

    • The new framework has its own update schedule, which doesn’t always get synchronized with native tools. Actually, you can avoid that only if you use the native tools for test automation. On the other hand, if you use Calabash or Appium, you can’t.

    It’s worth noting that the Flutter community is very friendly. They quickly respond and fix issues.

    • The tests are a part of a mobile app, therefore they depend on it entirely. If your app breaks down, so do the tests; e.g., if a version of a package is updated, the dependencies are lost.
    • Changes in code force you to constantly build new versions (for E2E).

    Kludges

    • If one test crashes, so do all of them. It’s clumsy and frustrating, and you have to write your own logic, which clearly is a kludge.
    • As for proxy, it’s not that easy in Flutter. Dart has its own client that handles network interactions. All requests go through that, so you have to enter your proxy settings in the app. And to do so, you have to write extra logic. It matters when you must work on a project through a VPN. In one of our projects, we connect a computer to a VPN, and the device, in turn, works via proxy.
    • GetIt is initialized only once when an app is launched, and it may lead to conflicts. Tests can “launch” your app multiple times. Hence, you have to reset GetIt so that it is initialized all over again.
    • No support for native alerts. Flutter tests can’t interact with native alerts, so, as a workaround, we can give specific permissions via adb or simctl in integration_test.dart when we initiate testing. There’s an issue report on that in the Flutter repository, so…
    Future<void> main() async {
      integration_test_driver.testOutputsDirectory = 'integration_test/gherkin/reports';
    
      /// Give permission to positioning to avoid problems
      await Process.run('xcrun', ['simctl', 'privacy', 'booted', 'grant', 'location-always', <app_packet>]);
      [
        'ACCESS_COARSE_LOCATION',
        'ACCESS_FINE_LOCATION',
      ].forEach((permission) async => Process.run('adb',
          ['shell', 'pm', 'grant', <app_packet>, 'android.permission.$permission']));
      return integration_test_driver.integrationDriver(
        timeout: const Duration(minutes: 120),
        responseDataCallback: writeGherkinReports,
      );
    }

    Other technologies. Comparison

    Green — pros, red — cons, yellow — neutral facts

    All frameworks that enable automation testing in Flutter have their pros and cons

    We didn’t have a chance to use Calabash for automated testing in Flutter, although we do have something to say about it. It’s feasible in theory, but you may have trouble identifying the element id.

    If you don’t opt for native tests, you have to decide whether, apart from Е2Е, you need widget tests, and if so, who is going to write them.

    Flutter has this cool feature that blows my mind: you can access a widget element whenever you want and take any information you want from it.

    We recommend automated tests in Flutter, if:

    • you are tempted by close integration with an app,
    • you are not scared of Dart,
    • you want to study a new technology,
    • you like creating something by yourself,
    • you are interested in contributing to Flutter and taking part in development besides just testing,
    • you must keep it all under control as part of a more extensive code coverage.

    If you need more information about our team and approaches, do not hesitate to contact us at [email protected] To learn more about our projects in Flutter read the article.