* Stop running the UI tests on the double localized pseudolanguage
* Fix UI tests signalling comms problem because the json key ordering wasn't defined and the messages wouldn't match.
* Remove UI tests that are now the same as preview tests (i.e. non flow based, only screen based), fixed the rest so they run on iOS 17.2, the iPhone 15 and iPad 10th gen simulators.
* Remove unnecessary gitignore instructions, change UI test snapshot naming convention to be more similar to the unit test ones
* Enable derived data and spm checkout caching for UI tests
* Switch UI tests back to the perf-only runner, moved the integration tests to the iPhone 15 Pro simulator.
* Address PR comments
* Update Prefire, setup multiple snapshot devices and languages
Squashed commits:
[28cb4ae6d] Switch to macos-14, iOS 17 and the iPhone 15 simulator
[080fc82cb] Update snapshots
[c874ca9c7] Bump Compound and Prefire, reduce perceptual precission to 0.98
[a59b8a54c] Update snapshots after adding iPad and pseudolanguage snapshotting
[f30cc3277] Setup multi-language snapshotting
[aeb11c7e7] Update prefire template, setup multiple snapshot devices
* Update ruby dependencies and fastlane plugins
* Bump Compound, switch back to the original prefire - version 2.0.4
* Remove preview test simulator version specifiers
* Use Xcode 15 in GitHub actions.
Use iOS 16.4 simulators from Xcode 14 as iOS 17.0.1 simulators are **super** slow.
* Fix failing preview snapshots.
Snapshot fix.
* Improve setup
* Regenerate snapshots that had a conflict.
---------
Co-authored-by: Alfonso Grillo <alfogrillo@gmail.com>
* Fix flakey emoji provider tests
* Fix flakey RoomScreenViewModel tests
* Fix flakey HomeScreenViewModel tests
* Fix flakey RoomMemberListScreen tests, problem with bindings getting overriden and deferFulfillment cancellable not getting stored
* Fix flakey RoomNotificationSettingsScreen tests and crashes
* Fix flakey RoomMemberDetailsScreen tests
* Deprecate old `deferFulfillment` and `nextViewState` methods
* Convert more files to the new `deferFulfillment`
* Converted the rest of the tests to the new deferFulfillment
* Removed now unused `nextViewState` and `deferFulfillment`
* Remove automatic retries from unit tests
* Reset analytics flag after running unit tests
* Address PR comments
* Introduce a new `deferFulfillment(publisher, keyPath, transitionValues)` method and use it where appropiate
* prefire basic implementation
force the unit tests to run on iPhone 14
skipping plugin validation
archive artifacts
better workflow
removed the OS restriction
lfs
custom stencil
new test file
deleting
* git lfs
* lfs
* test
* customised the stencil to support our internal protocol
* for now we can remove the OS check from the stencil
* added a delay
* tests have been moved
* improvement
* recommit
* perceptual precision lowered
* updated snapshot testing and selected also iOS version
* added ios version control
* Increase the sentry log level used in debug symbol uploading
* Set sentry max http retries to 5
* Revert: set sentry max http retries to 5 as it doesn't work
* Implement automatic sentry_upload_dif retries
- Randomise test order.
- Use deferred fulfilment in more places.
- Make sure to change something before awaiting the fulfilment.
- Build a fresh AppSettings after reset().
- Retry tests on failure.
* Add signposts to performance tests.
- Update flow to include support for the migration screen.
* If the welcome screen shows, click on the button.
* Ensure a clean simulator each run.
* Add accessibility identifier for migration screen if required.
* Handle walking into the room and back out again.
* use iphone 14 pro to match what's used in xcode.
* Remove ApplicationTests as duplicated in LoginTests.
We measure app startup time in LoginTests as part of the flow - we may as well avoid spending 60s doing only that measurement in ApplicationTests
* Sleep 10s, the ui is otherwise showing up in random order.
* Revert "Remove ApplicationTests as duplicated in LoginTests."
This reverts commit 8670710315bcd0d6c3c3046f534b32b4c728b837.
* Update script to parse out correct values from results file.
* Allow cancellation of password prompt in any order.
* Remove test timeout, performance tests will always take a while.
* Adjust parsing further
* Remove ApplicationTests.
* Move to a more elegant way to wait for something to disappear.
* Linting.
* Fix unit tests.
---------
Co-authored-by: Doug <douglase@element.io>
Co-authored-by: Doug <6060466+pixlwave@users.noreply.github.com>
* Fixed unstored weak reference warning
* Prevent the timeline from jumping whenever we access XCUIApplication properties
* Prevent UI tests from causing side effects on the timeline
* Update userSession reference screenshots
* Fix create room UI tests
* Update inviteUsers UI test reference screenshots
* Allow certain tests to enable timeline accessibility
* Fix roomLayout tests and update reference screenshots
* Disable the testTimelineLayoutInMiddle test because of flakiness
* Update screenshots following timestamp addition
* Another attempt
* Yet another attempt, replace XCUIScreen screenshots with XCUIApplication ones, decrease delay to 1 second
* Allow tests to retry up to 3 times before failing the run
* let's disable autocorrection
* waiting some more time in flaky tests and updated the content of a test that was failing
* controlled delay waited a bit moe for the timeline to settle
* try await
* trying with a bit more waiting time after the tap and a slow velocity for the swipe
* let's try waiting more time
* workflow for a single ui test
* las try
* slow swiping tests
* commenting out flaky CI tests, for now we'll just disable them
* better description
* flaky test improvement
* disabling animations that are not needed in UITests
* code that already prevents animations in UI tests
* code improv
* settting macOS to 13, and let's runa test of... the tests
* I want the artifacts of the test, might give me some more insight in Xcode
* archive artifact
* trying a thing with the xcode version
* push notifications alert makes this test fail
* test improvement
* let's try with a very long time
* let's disable autocorrection
* sleep
* 1 seconds fixed 90% of the issue, 2 should fix 100%
* waiting some more time in flaky tests and updated the content of a test that was failing
* updated a screenshot test that had a notification
* this integration test is a bit so flaky increased the timing
* controlled delay waited a bit moe for the timeline to settle
* try await
* MainActor everywhere
* milliseconds fix
* trying with a bit more waiting time after the tap and a slow velocity for the swipe
* let's try waiting more time
* created the row in the view and the alert, and added the new function to the RoomProxy
* fixed an issue with the alert function
* handling the navigation
* fixed a bug with the detail coordinators being dismissed incorrectly when inside a stack
* implementation completed
* replaced UI screenshots
* added a test for the fixed bug of the coordinators
* trying to increase the wait time for the expectation
* improved the test
* improved the buttons UI
* uploading artifacts for unit tests
* added result bundle true
* improved the tests
* added a new test
* pr suggestions
* updating mock
* PR suggestions
* improved tests
* fixed UI tests
* pr should be ready now
* removed testing code
* reduced complexity
* fixed test
* added a an assert to the new test case
* more tests and messages cases
* pr comments addressed
* completed
* BrowserStack support
* Add fastlane step to automatically upload builds to BrowserStack
* Add error log for when Application Groups are not available
* Use better applicationSupportDirectory API
* Remove status bars when running UITests
* Remove simulator OS versions from screenshot names, be less lenient with image diffing
* Rename UITests splash screen to onboarding
* Switch tests to Xcode 14.1, remove version names for used simulators and switched from iPhone 13 Pro Max to iPhone 14
* Fix ServiceLocator and fallbackLaguage for UI tests
* Drop snapshot diffing precision to 0.99 as 0.999 triggers false negatives
* Fix server selection UI tests after exposing the sliding sync proxy configuration
* Update reference screenshots
* Fix UI test simulator versions
* Pin the used Xcode version used by fastlane to `~> 14.0.1`, as simctl status_bar overrides don't work on the Xcode14.1 RC
* Update base screenshots, fix status bar script warning, fix settings screen test
* Fix Timeline on Xcode 14/iOS 16
Raise requirement to iOS 16+
Reduce pagination jumping.
Sonarcloud fixes.
Fix verification test.
Adopt if let optional { syntax.
* Remove unused ScrollViewReader
The ScrollViewReader didn't appear to change the behaviour.
* Fix warnings on Run Scripts.
Run script build phase 'SwiftLint' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase.
