go/sliding-sync
go/sliding-sync
1
0
Fork 0
mirror of https://github.com/matrix-org/sliding-sync.git synced 2025-03-10 13:37:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
sliding-sync/sync3/sort_test.go

390 lines
10 KiB
Go
Raw Normal View History

Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
package sync3
import (
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
"reflect"
"strings"
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
"testing"
Rename package -> github.com/matrix-org/sliding-sync
2022-12-15 11:08:50 +00:00
"github.com/matrix-org/sliding-sync/internal"
"github.com/matrix-org/sliding-sync/sync3/caches"
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
)
refactor: add RoomFinder and use it in InternalRequestLists This is part of a series of refactors aimed to improve the performance and complexity of calculating list deltas, which up until now exists in its current form due to organic growth of the codebase. This specific refactor introduces a new interface `RoomFinder` which can map room IDs to `*RoomConnMetadata` which is used by `ConnState`. All the sliding sync lists now use the `RoomFinder` instead of keeping their own copies of `RoomConnMetadata`, meaning per-connection, rooms just have 1 copy in-memory. This cuts down on memory usage as well as cuts down on GC churn as we would constantly be replacing N rooms for each update, where N is the total number of lists on that connection. For Element-Web, N=7 currently to handle Favourites, Low Priority, DMs, Rooms, Spaces, Invites, Search. This also has the benefit of creating a single source of truth in `InternalRequestLists.allRooms` which can be updated once and then a list of list deltas can be calculated off the back of that. Previously, `allRooms` was _only_ used to seed new lists, which created a weird imbalance as we would need to update both `allRooms` _and_ each `FilteredSortableRooms` to keep things in-sync. This refactor is incomplete in its present form, as we need to make use of the new `RoomDelta` struct to efficiently package list updates.
2022-08-26 10:09:41 +01:00
type finder struct {
rooms map[string]*RoomConnMetadata
roomIDs []string
}
perf: swap to pointers for s.allRooms The RoomFinder accesses s.allRooms and is used when sorting the room list, where we would expect many accesses. Previously, we returned copies of room metadata, which caused significant amounts of GC churn, enough to show up on traces. Swap to using pointers and rename the function to `ReadOnlyRoom(roomID)` to indicate that it isn't safe to write to this return value.
2023-02-02 11:26:08 +00:00
func (f finder) ReadOnlyRoom(roomID string) *RoomConnMetadata {
refactor: add RoomFinder and use it in InternalRequestLists This is part of a series of refactors aimed to improve the performance and complexity of calculating list deltas, which up until now exists in its current form due to organic growth of the codebase. This specific refactor introduces a new interface `RoomFinder` which can map room IDs to `*RoomConnMetadata` which is used by `ConnState`. All the sliding sync lists now use the `RoomFinder` instead of keeping their own copies of `RoomConnMetadata`, meaning per-connection, rooms just have 1 copy in-memory. This cuts down on memory usage as well as cuts down on GC churn as we would constantly be replacing N rooms for each update, where N is the total number of lists on that connection. For Element-Web, N=7 currently to handle Favourites, Low Priority, DMs, Rooms, Spaces, Invites, Search. This also has the benefit of creating a single source of truth in `InternalRequestLists.allRooms` which can be updated once and then a list of list deltas can be calculated off the back of that. Previously, `allRooms` was _only_ used to seed new lists, which created a weird imbalance as we would need to update both `allRooms` _and_ each `FilteredSortableRooms` to keep things in-sync. This refactor is incomplete in its present form, as we need to make use of the new `RoomDelta` struct to efficiently package list updates.
2022-08-26 10:09:41 +01:00
return f.rooms[roomID]
}
func newFinder(rooms []*RoomConnMetadata) finder {
m := make(map[string]*RoomConnMetadata)
ids := make([]string, len(rooms))
for i := range rooms {
ids[i] = rooms[i].RoomID
m[ids[i]] = rooms[i]
}
return finder{
rooms: m,
roomIDs: ids,
}
}
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
func TestSortBySingleOperation(t *testing.T) {
Fixup tests
2023-05-24 15:50:53 +01:00
const listKey = "my_list"
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
room1 := "!1:localhost"
room2 := "!2:localhost"
room3 := "!3:localhost"
room4 := "!4:localhost"
Use pointers to RoomConnMetadata
2022-08-25 15:30:07 +01:00
rooms := []*RoomConnMetadata{
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room1,
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
Add stub account_data extension; migrate UserCache/GlobalCache to caches pkg caches pkg is required to avoid import loops as sync3 depends on extensions as the extension type is embedded in the `sync3.Request`, but extensions will need to know about the caches for account data to function correctly.
2022-03-23 16:34:43 +00:00
UserRoomData: caches.UserRoomData{
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
HighlightCount: 3,
NotificationCount: 12,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "foo",
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 600},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room2,
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
Add stub account_data extension; migrate UserCache/GlobalCache to caches pkg caches pkg is required to avoid import loops as sync3 depends on extensions as the extension type is embedded in the `sync3.Request`, but extensions will need to know about the caches for account data to function correctly.
2022-03-23 16:34:43 +00:00
UserRoomData: caches.UserRoomData{
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
HighlightCount: 0,
NotificationCount: 3,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "koo",
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 700},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room3,
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
Add stub account_data extension; migrate UserCache/GlobalCache to caches pkg caches pkg is required to avoid import loops as sync3 depends on extensions as the extension type is embedded in the `sync3.Request`, but extensions will need to know about the caches for account data to function correctly.
2022-03-23 16:34:43 +00:00
UserRoomData: caches.UserRoomData{
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
HighlightCount: 2,
NotificationCount: 7,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "yoo",
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 900},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room4,
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
Add stub account_data extension; migrate UserCache/GlobalCache to caches pkg caches pkg is required to avoid import loops as sync3 depends on extensions as the extension type is embedded in the `sync3.Request`, but extensions will need to know about the caches for account data to function correctly.
2022-03-23 16:34:43 +00:00
UserRoomData: caches.UserRoomData{
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
HighlightCount: 1,
NotificationCount: 1,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "boo",
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 800},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
}
// name: 4,1,2,3
// recency: 3,4,2,1
// highlight: 1,3,4,2
// notif: 1,3,2,4
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
// level+recency: 3,4,1,2 as 3,4,1 have highlights then sorted by recency
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
wantMap := map[string][]string{
SortByName: {room4, room1, room2, room3},
SortByRecency: {room3, room4, room2, room1},
SortByHighlightCount: {room1, room3, room4, room2},
SortByNotificationCount: {room1, room3, room2, room4},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
SortByNotificationLevel + " " + SortByRecency: {room3, room4, room1, room2},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
}
refactor: add RoomFinder and use it in InternalRequestLists This is part of a series of refactors aimed to improve the performance and complexity of calculating list deltas, which up until now exists in its current form due to organic growth of the codebase. This specific refactor introduces a new interface `RoomFinder` which can map room IDs to `*RoomConnMetadata` which is used by `ConnState`. All the sliding sync lists now use the `RoomFinder` instead of keeping their own copies of `RoomConnMetadata`, meaning per-connection, rooms just have 1 copy in-memory. This cuts down on memory usage as well as cuts down on GC churn as we would constantly be replacing N rooms for each update, where N is the total number of lists on that connection. For Element-Web, N=7 currently to handle Favourites, Low Priority, DMs, Rooms, Spaces, Invites, Search. This also has the benefit of creating a single source of truth in `InternalRequestLists.allRooms` which can be updated once and then a list of list deltas can be calculated off the back of that. Previously, `allRooms` was _only_ used to seed new lists, which created a weird imbalance as we would need to update both `allRooms` _and_ each `FilteredSortableRooms` to keep things in-sync. This refactor is incomplete in its present form, as we need to make use of the new `RoomDelta` struct to efficiently package list updates.
2022-08-26 10:09:41 +01:00
f := newFinder(rooms)
Fixup tests
2023-05-24 15:50:53 +01:00
sr := NewSortableRooms(f, listKey, f.roomIDs)
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
for sortBy, wantOrder := range wantMap {
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
sr.Sort(strings.Split(sortBy, " "))
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
var gotRoomIDs []string
refactor: add RoomFinder and use it in InternalRequestLists This is part of a series of refactors aimed to improve the performance and complexity of calculating list deltas, which up until now exists in its current form due to organic growth of the codebase. This specific refactor introduces a new interface `RoomFinder` which can map room IDs to `*RoomConnMetadata` which is used by `ConnState`. All the sliding sync lists now use the `RoomFinder` instead of keeping their own copies of `RoomConnMetadata`, meaning per-connection, rooms just have 1 copy in-memory. This cuts down on memory usage as well as cuts down on GC churn as we would constantly be replacing N rooms for each update, where N is the total number of lists on that connection. For Element-Web, N=7 currently to handle Favourites, Low Priority, DMs, Rooms, Spaces, Invites, Search. This also has the benefit of creating a single source of truth in `InternalRequestLists.allRooms` which can be updated once and then a list of list deltas can be calculated off the back of that. Previously, `allRooms` was _only_ used to seed new lists, which created a weird imbalance as we would need to update both `allRooms` _and_ each `FilteredSortableRooms` to keep things in-sync. This refactor is incomplete in its present form, as we need to make use of the new `RoomDelta` struct to efficiently package list updates.
2022-08-26 10:09:41 +01:00
for i := range sr.roomIDs {
gotRoomIDs = append(gotRoomIDs, sr.roomIDs[i])
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
}
for i := range wantOrder {
if wantOrder[i] != gotRoomIDs[i] {
t.Errorf("Sort: %s got %v want %v", sortBy, gotRoomIDs, wantOrder)
}
}
}
}
func TestSortByMultipleOperations(t *testing.T) {
Fixup tests
2023-05-24 15:50:53 +01:00
const listKey = "my_list"
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
room1 := "!1:localhost"
room2 := "!2:localhost"
room3 := "!3:localhost"
room4 := "!4:localhost"
Use pointers to RoomConnMetadata
2022-08-25 15:30:07 +01:00
rooms := []*RoomConnMetadata{
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room1,
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
Add stub account_data extension; migrate UserCache/GlobalCache to caches pkg caches pkg is required to avoid import loops as sync3 depends on extensions as the extension type is embedded in the `sync3.Request`, but extensions will need to know about the caches for account data to function correctly.
2022-03-23 16:34:43 +00:00
UserRoomData: caches.UserRoomData{
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
HighlightCount: 1,
NotificationCount: 1,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "foo",
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 600},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room2,
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
Add stub account_data extension; migrate UserCache/GlobalCache to caches pkg caches pkg is required to avoid import loops as sync3 depends on extensions as the extension type is embedded in the `sync3.Request`, but extensions will need to know about the caches for account data to function correctly.
2022-03-23 16:34:43 +00:00
UserRoomData: caches.UserRoomData{
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
HighlightCount: 1,
NotificationCount: 5,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "koo",
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 700},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room3,
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
Add stub account_data extension; migrate UserCache/GlobalCache to caches pkg caches pkg is required to avoid import loops as sync3 depends on extensions as the extension type is embedded in the `sync3.Request`, but extensions will need to know about the caches for account data to function correctly.
2022-03-23 16:34:43 +00:00
UserRoomData: caches.UserRoomData{
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
HighlightCount: 0,
NotificationCount: 0,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "yoo",
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 800},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room4,
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
Add stub account_data extension; migrate UserCache/GlobalCache to caches pkg caches pkg is required to avoid import loops as sync3 depends on extensions as the extension type is embedded in the `sync3.Request`, but extensions will need to know about the caches for account data to function correctly.
2022-03-23 16:34:43 +00:00
UserRoomData: caches.UserRoomData{
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
HighlightCount: 0,
NotificationCount: 0,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "boo",
Add UserRoomData to filter checks and thread it through This will allow us to apply filters on user-specific room data such as if the user is invited or if the user has this room as a DM room or not.
2021-11-09 10:36:29 +00:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 900},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
},
}
testCases := []struct {
SortBy []string
WantRooms []string
}{
{
SortBy: []string{SortByHighlightCount, SortByNotificationCount, SortByRecency, SortByName},
WantRooms: []string{room2, room1, room4, room3},
},
{
SortBy: []string{SortByHighlightCount, SortByName},
WantRooms: []string{room1, room2, room4, room3},
},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
{
SortBy: []string{SortByNotificationLevel, SortByName},
WantRooms: []string{room1, room2, room4, room3},
},
{
SortBy: []string{SortByNotificationLevel, SortByRecency},
WantRooms: []string{room2, room1, room4, room3},
},
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
}
refactor: add RoomFinder and use it in InternalRequestLists This is part of a series of refactors aimed to improve the performance and complexity of calculating list deltas, which up until now exists in its current form due to organic growth of the codebase. This specific refactor introduces a new interface `RoomFinder` which can map room IDs to `*RoomConnMetadata` which is used by `ConnState`. All the sliding sync lists now use the `RoomFinder` instead of keeping their own copies of `RoomConnMetadata`, meaning per-connection, rooms just have 1 copy in-memory. This cuts down on memory usage as well as cuts down on GC churn as we would constantly be replacing N rooms for each update, where N is the total number of lists on that connection. For Element-Web, N=7 currently to handle Favourites, Low Priority, DMs, Rooms, Spaces, Invites, Search. This also has the benefit of creating a single source of truth in `InternalRequestLists.allRooms` which can be updated once and then a list of list deltas can be calculated off the back of that. Previously, `allRooms` was _only_ used to seed new lists, which created a weird imbalance as we would need to update both `allRooms` _and_ each `FilteredSortableRooms` to keep things in-sync. This refactor is incomplete in its present form, as we need to make use of the new `RoomDelta` struct to efficiently package list updates.
2022-08-26 10:09:41 +01:00
f := newFinder(rooms)
Fixup tests
2023-05-24 15:50:53 +01:00
sr := NewSortableRooms(f, listKey, f.roomIDs)
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
for _, tc := range testCases {
sr.Sort(tc.SortBy)
var gotRoomIDs []string
refactor: add RoomFinder and use it in InternalRequestLists This is part of a series of refactors aimed to improve the performance and complexity of calculating list deltas, which up until now exists in its current form due to organic growth of the codebase. This specific refactor introduces a new interface `RoomFinder` which can map room IDs to `*RoomConnMetadata` which is used by `ConnState`. All the sliding sync lists now use the `RoomFinder` instead of keeping their own copies of `RoomConnMetadata`, meaning per-connection, rooms just have 1 copy in-memory. This cuts down on memory usage as well as cuts down on GC churn as we would constantly be replacing N rooms for each update, where N is the total number of lists on that connection. For Element-Web, N=7 currently to handle Favourites, Low Priority, DMs, Rooms, Spaces, Invites, Search. This also has the benefit of creating a single source of truth in `InternalRequestLists.allRooms` which can be updated once and then a list of list deltas can be calculated off the back of that. Previously, `allRooms` was _only_ used to seed new lists, which created a weird imbalance as we would need to update both `allRooms` _and_ each `FilteredSortableRooms` to keep things in-sync. This refactor is incomplete in its present form, as we need to make use of the new `RoomDelta` struct to efficiently package list updates.
2022-08-26 10:09:41 +01:00
for i := range sr.roomIDs {
gotRoomIDs = append(gotRoomIDs, sr.roomIDs[i])
Implement all sort orders and allow them to mix
2021-11-02 11:09:26 +00:00
}
for i := range tc.WantRooms {
if tc.WantRooms[i] != gotRoomIDs[i] {
t.Errorf("Sort: %v got %v want %v", tc.SortBy, gotRoomIDs, tc.WantRooms)
}
}
}
}
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
// Test that if you remove a room, it updates the lookup map.
func TestSortableRoomsRemove(t *testing.T) {
Fixup tests
2023-05-24 15:50:53 +01:00
const listKey = "my_list"
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
room1 := "!1:localhost"
room2 := "!2:localhost"
Use pointers to RoomConnMetadata
2022-08-25 15:30:07 +01:00
rooms := []*RoomConnMetadata{
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room1,
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 1,
NotificationCount: 1,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "foo",
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 700},
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
},
{
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: room2,
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 2,
NotificationCount: 2,
bugfix: sorting by room name didn't work correctly when room names were updated Caused by us not updating the `CanonicalisedName` which is what we use to sort on. This field is a bit of an oddity because it lived outside the user/global cache fields because it is a calculated value from the global cache data in the scope of a user, whereas other user cache values are derived directly from specific data (notif counts, DM-ness). This is a silly distinction however, since spaces are derived from global room data as well, so move `CanonicalisedName` to the UserCache and keep it updated when the room name changes. Longer term: we need to clean this up so only the user cache is responsible for updating user cache fields, and connstate treats user room data and global room data as immutable. This is _mostly_ true today, but isn't always, and it causes headaches. In addition, it looks like we maintain O(n) caches based on the number of lists the user has made: we needn't do this and should lean much more heavily on `s.allRooms`, just keeping pointers to this slice from whatever lists the user requests.
2022-08-18 13:11:05 +01:00
CanonicalisedName: "foo2",
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 600},
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
},
}
refactor: add RoomFinder and use it in InternalRequestLists This is part of a series of refactors aimed to improve the performance and complexity of calculating list deltas, which up until now exists in its current form due to organic growth of the codebase. This specific refactor introduces a new interface `RoomFinder` which can map room IDs to `*RoomConnMetadata` which is used by `ConnState`. All the sliding sync lists now use the `RoomFinder` instead of keeping their own copies of `RoomConnMetadata`, meaning per-connection, rooms just have 1 copy in-memory. This cuts down on memory usage as well as cuts down on GC churn as we would constantly be replacing N rooms for each update, where N is the total number of lists on that connection. For Element-Web, N=7 currently to handle Favourites, Low Priority, DMs, Rooms, Spaces, Invites, Search. This also has the benefit of creating a single source of truth in `InternalRequestLists.allRooms` which can be updated once and then a list of list deltas can be calculated off the back of that. Previously, `allRooms` was _only_ used to seed new lists, which created a weird imbalance as we would need to update both `allRooms` _and_ each `FilteredSortableRooms` to keep things in-sync. This refactor is incomplete in its present form, as we need to make use of the new `RoomDelta` struct to efficiently package list updates.
2022-08-26 10:09:41 +01:00
f := newFinder(rooms)
Fixup tests
2023-05-24 15:50:53 +01:00
sr := NewSortableRooms(f, listKey, f.roomIDs)
bugfix: update the roomIDToIndex map when rooms are removed We weren't doing this previously, but things didn't blow up because we would almost always call resort() shortly afterwards which _would_ update the map with the new sort positions. In some cases this could cause the lists to be sorting with incorrect index positions, notably: - when an invite is retired. - when a room no longer meets filter criteria and is removed. This could be a major source of duplicate rooms.
2022-08-16 15:56:16 +01:00
if err := sr.Sort([]string{SortByRecency}); err != nil { // room 1 is first, then room 2
t.Fatalf("Sort: %s", err)
}
if i, ok := sr.IndexOf(room1); i != 0 || !ok {
t.Errorf("IndexOf room 1 returned %v %v", i, ok)
}
if i, ok := sr.IndexOf(room2); i != 1 || !ok {
t.Errorf("IndexOf room 2 returned %v %v", i, ok)
}
// Remove room 1, so room 2 should take its place.
rmIndex := sr.Remove(room1)
if rmIndex != 0 {
t.Fatalf("Remove: return removed index %v want 0", rmIndex)
}
// check
if i, ok := sr.IndexOf(room1); ok { // should be !ok
t.Errorf("IndexOf room 1 returned %v %v", i, ok)
}
if i, ok := sr.IndexOf(room2); i != 0 || !ok {
t.Errorf("IndexOf room 2 returned %v %v", i, ok)
}
}
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
// dedicated test as it relies on multiple fields
func TestSortByNotificationLevel(t *testing.T) {
Fixup tests
2023-05-24 15:50:53 +01:00
const listKey = "my_list"
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
// create the full set of possible sort variables, most recent message last
roomUnencHC := "!unencrypted-highlight-count:localhost"
roomUnencHCNC := "!unencrypted-highlight-and-notif-count:localhost"
roomUnencNC := "!unencrypted-notif-count:localhost"
roomUnenc := "!unencrypted:localhost"
roomEncHC := "!encrypted-highlight-count:localhost"
roomEncHCNC := "!encrypted-highlight-and-notif-count:localhost"
roomEncNC := "!encrypted-notif-count:localhost"
roomEnc := "!encrypted:localhost"
roomsMap := map[string]*RoomConnMetadata{
roomUnencHC: {
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
Encrypted: false,
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 1,
NotificationCount: 0,
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 1},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
roomUnencHCNC: {
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
Encrypted: false,
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 1,
NotificationCount: 1,
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 2},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
roomUnencNC: {
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
Encrypted: false,
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 0,
NotificationCount: 1,
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 3},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
roomUnenc: {
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
Encrypted: false,
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 0,
NotificationCount: 0,
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 4},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
roomEncHC: {
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
Encrypted: true,
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 1,
NotificationCount: 0,
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 5},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
roomEncHCNC: {
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
Encrypted: true,
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 1,
NotificationCount: 1,
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 6},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
roomEncNC: {
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: roomEncNC,
Encrypted: true,
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 0,
NotificationCount: 1,
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 7},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
roomEnc: {
RoomMetadata: internal.RoomMetadata{
Update test data
2023-05-23 17:25:35 +01:00
RoomID: roomEnc,
Encrypted: true,
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
UserRoomData: caches.UserRoomData{
HighlightCount: 0,
NotificationCount: 0,
},
Fixup tests
2023-05-24 15:50:53 +01:00
LastInterestedEventTimestamps: map[string]uint64{listKey: 8},
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
},
}
roomIDs := make([]string, len(roomsMap))
rooms := make([]*RoomConnMetadata, len(roomsMap))
i := 0
for roomID, room := range roomsMap {
room.RoomMetadata.RoomID = roomID
roomIDs[i] = roomID
rooms[i] = room
i++
}
t.Logf("%v", roomIDs)
f := newFinder(rooms)
Fixup tests
2023-05-24 15:50:53 +01:00
sr := NewSortableRooms(f, listKey, roomIDs)
add extensions for typing and receipts; bugfixes and additional perf improvements Features: - Add `typing` extension. - Add `receipts` extension. - Add comprehensive prometheus `/metrics` activated via `SYNCV3_PROM`. - Add `SYNCV3_PPROF` support. - Add `by_notification_level` sort order. - Add `include_old_rooms` support. - Add support for `$ME` and `$LAZY`. - Add correct filtering when `*,*` is used as `required_state`. - Add `num_live` to each room response to indicate how many timeline entries are live. Bug fixes: - Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm. - Send back an `errcode` on HTTP errors (e.g expired sessions). - Remove `unsigned.txn_id` on insertion into the DB. Otherwise other users would see other users txn IDs :( - Improve range delta algorithm: previously it didn't handle cases like `[0,20] -> [20,30]` and would panic. - Send HTTP 400 for invalid range requests. - Don't publish no-op unread counts which just adds extra noise. - Fix leaking DB connections which could eventually consume all available connections. - Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at _some_ point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever. Improvements: - Greatly improve startup times of sync3 handlers by improving `JoinedRoomsTracker`: a modest amount of data would take ~28s to create the handler, now it takes 4s. - Massively improve initial initial v3 sync times, by refactoring `JoinedRoomsTracker`, from ~47s to <1s. - Add `SlidingSyncUntil...` in tests to reduce races. - Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s. - Add trace task for initial syncs. - Include the proxy version in UA strings. - HTTP errors now wait 1s before returning to stop clients tight-looping on error. - Pending event buffer is now 2000. - Index the room ID first to cull the most events when returning timeline entries. Speeds up `SelectLatestEventsBetween` by a factor of 8. - Remove cancelled `m.room_key_requests` from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.
2022-12-14 18:53:55 +00:00
if err := sr.Sort([]string{SortByNotificationLevel, SortByRecency}); err != nil {
t.Fatalf("Sort: %s", err)
}
var gotRoomIDs []string
for i := range sr.roomIDs {
gotRoomIDs = append(gotRoomIDs, sr.roomIDs[i])
}
// we expect the rooms to be grouped in this order:
// HIGHLIGHT COUNT > 0
// ENCRYPTED, NOTIF COUNT > 0
// UNENCRYPTED, NOTIF COUNT > 0
// REST
// Within each group, we expect recency sorting due to SortByRecency
wantRoomIDs := []string{
roomEncHCNC, roomEncHC, roomUnencHCNC, roomUnencHC, // in practice we don't expect to see this as encrypted rooms won't have highlight counts > 0
roomEncNC,
roomUnencNC,
roomEnc, roomUnenc,
}
if !reflect.DeepEqual(gotRoomIDs, wantRoomIDs) {
t.Errorf("got: %v", gotRoomIDs)
t.Errorf("want: %v", wantRoomIDs)
}
}
Reference in New Issue Copy Permalink