* Disable image and document picker integration tests as they randomly fail to load and are flakey.
* Delete any pre-existing log files
* Various tracing tweaks and fixes:
- delete the custom tracing log levels as we can't control their ouput
- implement comparable on them
- change default levels only if the new chosen level increases their verbosity
* Make logging targets mandatory and fix their logging levels
* Switch back to using the `run_tests` reset simulator flag as `fastlane snapshot reset_simulators` was too generic and slow
* Switch all integration test taps to `tapCenter` (nee forceTap) after noticing missed taps on CI.
* Make the logging file prefix explicit, let the main app not use one.
* Rename tracing configuration `target` to `currentTarget`
* Update various packages.
Notable changes:
- DeviceKit now detects iPad Pro (M4), iPad Air (M2) and iPhones 16.
- KZFileWatchers isn't updated but now has a tag to pin to.
* Try using a raw destination for tests.
To fix an issue where CI isn't finding iPhone 16 when it's clearly listed by simctl.
* Replace Prefire with a very similar but simpler and more direct approach
* Move PreviewTest snapshots to their new location (snapshotting default, mirrors UITests)
* Update acknowledgements
* Switch snapshot configuration extensions to a custom version
* Exclude snapshots from the PreviewTests target
* Fix acknowledgements
* Rename snapshotting preference keys
* Remove `skipPackagePluginValidation`
- Create an app.yml with the base app variant.
- Override app.yml contents on Nightly/PR builds.
- Add AppHooks mechanism.
- Add Enterprise submodule.
- Conditionally import Enterprise/fastlane/Fastfile if available.
* Set up GitHub generated release notes.
* Remove Towncrier from the release script.
* Add changelog failures for Labels with Danger and stop requesting a changelog file.
* Update changelog documentation.
* Hardcode Xcode 15.2 as there are multiple versions available on the performance runner
* Remove the derived data caching step as it's not necessary on the performance runner.
* Stop running the UI tests on the double localized pseudolanguage
* Fix UI tests signalling comms problem because the json key ordering wasn't defined and the messages wouldn't match.
* Remove UI tests that are now the same as preview tests (i.e. non flow based, only screen based), fixed the rest so they run on iOS 17.2, the iPhone 15 and iPad 10th gen simulators.
* Remove unnecessary gitignore instructions, change UI test snapshot naming convention to be more similar to the unit test ones
* Enable derived data and spm checkout caching for UI tests
* Switch UI tests back to the perf-only runner, moved the integration tests to the iPhone 15 Pro simulator.
* Address PR comments
* Update Prefire, setup multiple snapshot devices and languages
Squashed commits:
[28cb4ae6d] Switch to macos-14, iOS 17 and the iPhone 15 simulator
[080fc82cb] Update snapshots
[c874ca9c7] Bump Compound and Prefire, reduce perceptual precission to 0.98
[a59b8a54c] Update snapshots after adding iPad and pseudolanguage snapshotting
[f30cc3277] Setup multi-language snapshotting
[aeb11c7e7] Update prefire template, setup multiple snapshot devices
* Update ruby dependencies and fastlane plugins
* Bump Compound, switch back to the original prefire - version 2.0.4
* Remove preview test simulator version specifiers
* Use Xcode 15 in GitHub actions.
Use iOS 16.4 simulators from Xcode 14 as iOS 17.0.1 simulators are **super** slow.
* Fix failing preview snapshots.
Snapshot fix.
* Improve setup
* Regenerate snapshots that had a conflict.
---------
Co-authored-by: Alfonso Grillo <alfogrillo@gmail.com>
* Fix flakey emoji provider tests
* Fix flakey RoomScreenViewModel tests
* Fix flakey HomeScreenViewModel tests
* Fix flakey RoomMemberListScreen tests, problem with bindings getting overriden and deferFulfillment cancellable not getting stored
* Fix flakey RoomNotificationSettingsScreen tests and crashes
* Fix flakey RoomMemberDetailsScreen tests
* Deprecate old `deferFulfillment` and `nextViewState` methods
* Convert more files to the new `deferFulfillment`
* Converted the rest of the tests to the new deferFulfillment
* Removed now unused `nextViewState` and `deferFulfillment`
* Remove automatic retries from unit tests
* Reset analytics flag after running unit tests
* Address PR comments
* Introduce a new `deferFulfillment(publisher, keyPath, transitionValues)` method and use it where appropiate
* prefire basic implementation
force the unit tests to run on iPhone 14
skipping plugin validation
archive artifacts
better workflow
removed the OS restriction
lfs
custom stencil
new test file
deleting
* git lfs
* lfs
* test
* customised the stencil to support our internal protocol
* for now we can remove the OS check from the stencil
* added a delay
* tests have been moved
* improvement
* recommit
* perceptual precision lowered
* updated snapshot testing and selected also iOS version
* added ios version control
* Increase the sentry log level used in debug symbol uploading
* Set sentry max http retries to 5
* Revert: set sentry max http retries to 5 as it doesn't work
* Implement automatic sentry_upload_dif retries
- Randomise test order.
- Use deferred fulfilment in more places.
- Make sure to change something before awaiting the fulfilment.
- Build a fresh AppSettings after reset().
- Retry tests on failure.
* Add signposts to performance tests.
- Update flow to include support for the migration screen.
* If the welcome screen shows, click on the button.
* Ensure a clean simulator each run.
* Add accessibility identifier for migration screen if required.
* Handle walking into the room and back out again.
* use iphone 14 pro to match what's used in xcode.
* Remove ApplicationTests as duplicated in LoginTests.
We measure app startup time in LoginTests as part of the flow - we may as well avoid spending 60s doing only that measurement in ApplicationTests
* Sleep 10s, the ui is otherwise showing up in random order.
* Revert "Remove ApplicationTests as duplicated in LoginTests."
This reverts commit 8670710315bcd0d6c3c3046f534b32b4c728b837.
* Update script to parse out correct values from results file.
* Allow cancellation of password prompt in any order.
* Remove test timeout, performance tests will always take a while.
* Adjust parsing further
* Remove ApplicationTests.
* Move to a more elegant way to wait for something to disappear.
* Linting.
* Fix unit tests.
---------
Co-authored-by: Doug <douglase@element.io>
Co-authored-by: Doug <6060466+pixlwave@users.noreply.github.com>
* Fixed unstored weak reference warning
* Prevent the timeline from jumping whenever we access XCUIApplication properties
* Prevent UI tests from causing side effects on the timeline
* Update userSession reference screenshots
* Fix create room UI tests
* Update inviteUsers UI test reference screenshots
* Allow certain tests to enable timeline accessibility
* Fix roomLayout tests and update reference screenshots
* Disable the testTimelineLayoutInMiddle test because of flakiness
* Update screenshots following timestamp addition
* Another attempt
* Yet another attempt, replace XCUIScreen screenshots with XCUIApplication ones, decrease delay to 1 second
* Allow tests to retry up to 3 times before failing the run
* let's disable autocorrection
* waiting some more time in flaky tests and updated the content of a test that was failing
* controlled delay waited a bit moe for the timeline to settle
* try await
* trying with a bit more waiting time after the tap and a slow velocity for the swipe
* let's try waiting more time
* workflow for a single ui test
* las try
* slow swiping tests
* commenting out flaky CI tests, for now we'll just disable them
* better description
* flaky test improvement
* disabling animations that are not needed in UITests
* code that already prevents animations in UI tests
* code improv
