512 Commits

Author SHA1 Message Date
Half-Shot
aa43549897 Try some concurrent requests 2024-02-27 09:51:05 +00:00
Half-Shot
393f122020 Small tidyups 2024-02-27 09:51:05 +00:00
Half-Shot
b98a109d0a Install to apps 2024-02-27 09:51:05 +00:00
Half-Shot
a38d5c218b Add new request function 2024-02-27 09:51:05 +00:00
Minecraftchest1
6c6cd6ee92
Change example Openid Connect callback url's to be correct. (#899)
* Change oauth urls in config sample

Signed-off-by: Minecraftchest1 <minecraftchest1@outlook.com>

* Update github docs oauth url

Signed-off-by: Minecraftchest1 <minecraftchest1@outlook.com>

* Fix default oauth2 callback url.

Signed-off-by: Minecraftchest1 <minecraftchest1@outlook.com>

* Create changelog docs.

Signed-off-by: Minecraftchest1 <minecraftchest1@outlook.com>

---------

Signed-off-by: Minecraftchest1 <minecraftchest1@outlook.com>
2024-02-26 08:45:30 +00:00
Will Hunt
c09f00e2a3
Fix Atom feed hashing format. (#901)
* Ensure atom feeds hash and prefix

* Add some fixing code for this.

* Remove feed fixer

* Refactor RSS feed tests a bit

* Add tests to ensure hashes do not change

* changelog
2024-02-21 16:13:36 +00:00
Will Hunt
387f7c1ce9
Backoff failing RSS feeds (#890)
* Backoff RSS requests if a url repeatedly fails.

* Increase max backoff time to a day

* Add backoff for failing feeds.

* Remove unused finally

* Add this.feedLastBackoff

* Rewrite in rust.

* linting

* pop

* Optimise backoff function further

* Drop only!

* fix test

* lint

* lint further

* Better comments

* Fix urls calculation

* Remove testing URL

* Add some variance to speed up while loop

* correct comment

* Follow the advice and use a VecDeque as it's slightly faster.

* Vastly better shuffle method

* Speed up checking for previous guids.

* fix hasher function

* lint

* Content doesn't need to be calculated twice.

* Slightly more efficient iteration

* Improve performance of backoff insertion

* Configure feed reader

* lint

* Ensure appending and removing from the queue works as expected.

* Ensure we do keep urls that have been removed.

* lint

* Inc/dec metrics as queue items are added/deleted.

* Add comment

* tidy up
2024-02-20 22:21:19 +00:00
Will Hunt
3ff87b7564
Retry failed RSS entries (#891)
* Retry event send for RSS.

* Fix large summary bridges.

* Remove unused import

* Add some comments
2024-02-06 16:15:13 +00:00
Will Hunt
90927a71f4
Add support for mentioning multiple assignees on GitHub issues (#889)
* Add support for multiple assignees on issues

* changelog
2024-02-06 10:09:21 +00:00
Twilight Sparkle
675c1c14d0
Add command to list feeds in JSON and YAML format (#876)
* Add command to list feeds in YAML format

* Add line breaks

* Add changelog

* Add a period in the help

* Add $URL to templates and add feed list format docs

* Use an optional arg on yaml list instead of a separate function

* Update changelog to include JSON format
2024-01-04 19:14:11 +00:00
Will Hunt
41ee467e2d
Fix logged errors not being properly formatted (and tone down the error verbosity a bit) (#874)
* Ensure we add the error handlers last

* Refactor error middleware to be less noisy

* changelog

* Add finalise
2024-01-02 15:58:25 +00:00
Will Hunt
c0bb71d553
Fix GitHub events not verifying (#875)
* Ensure we verify the raw payload.

* changelog

* Tidy up types

* Add test for GitHib

* Mock out GitHub API to allow tests to pass.

* Lint
2024-01-02 15:52:43 +00:00
Simon Quigley
3af2aa7032
sucessfully (#869)
* sucessfully

* Create 869.misc

---------

Co-authored-by: Will Hunt <will@half-shot.uk>
2023-12-29 16:24:49 +00:00
Will Hunt
5c5bdd546d
Fix widgets not working when bound to the same listener as webhooks. (#872)
* Bind listener in bridge.

* Add a test to confirm behaviour

* Fix test name

* changelog
2023-12-29 16:11:08 +00:00
Will Hunt
8e115b40ab
Add End to End testing (#868)
* Ensure connection state always explicitly states all keys, even if some are undefined.

* changelog

* Fix type

* fix test types

* Add support for E2E testing

* Add CI job for e2e test

* Ensure integration test only runs when regular tests complete

* Add homerunner image

* Disallow concurrent runs

* Add concurrency to other expensive steps

* changelog

* Fix mq test

* Cache rust deps

* Drop only

* Use a shared key
2023-12-28 15:04:03 +00:00
Will Hunt
46d198c2c0
Fix feeds template / notify state not reflecting in the UI (#866)
* Ensure connection state always explicitly states all keys, even if some are undefined.

* changelog

* Fix type

* fix test types
2023-12-28 14:27:47 +00:00
Will Hunt
76a4209f0d
Remove GoNEB migrator (#867)
* Remove GoNEB migrator.

* changelog

* Remove frontend pieces

* Remove unused statuscodes

* Linting

* Remove warning about ts-comment
2023-12-28 11:36:11 +00:00
Will Hunt
9cc402b066
Add commands for listing & removing webhooks (#864)
* Add commands for removing webhooks

* Fix command setting state twice

* changelog
2023-12-27 17:11:38 +00:00
Will Hunt
066d908b4f
Fix feeds config UI not allowing notifyOnFailure to be toggled. (#865)
* Fix feeds config UI not allowing notifyOnFailure to be toggled.

* changelog

* amend changelog to be cohierent

* Actual fixes.
2023-12-27 15:25:30 +00:00
Joshua Hoffmann
1db6f60776
Some smaller improvements to metrics.md (#856)
* Update metrics.md

Made descriptions of metrics more consistent and some smaller improvements.

* Update README.md (#855)

* Update README.md

Smaller improvements. Fixed a typo.

* Create 855.doc

---------

Co-authored-by: Will Hunt <will@half-shot.uk>

* Update setup.md (#857)

* Update setup.md

Smaller improvements such as fixes for typos, also added an example on how to pass requests from Nginx

* Create 857.docs

* Rename 857.docs to 857.doc

---------

Co-authored-by: Will Hunt <will@half-shot.uk>

* Update workers.md (#858)

* Update workers.md

Fixed typos

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de

* Create 858.doc

---------

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de
Co-authored-by: Will Hunt <will@half-shot.uk>

* Update encryption.md (#860)

* Update encryption.md

Consistently spell Hookshot with an upper-case H as the first letter

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de

* Create 860.doc

---------

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de
Co-authored-by: Will Hunt <will@half-shot.uk>

* Update widgets.md (#859)

* Update widgets.md

Consistently spell Hookshot with an upper-case H as the first letter

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de

* Create 859.doc

---------

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de
Co-authored-by: Will Hunt <will@half-shot.uk>

* Drop Node 18, start testing Node 21 & update dependencies. (#862)

* Major package upgrades

* Update rust deps and fix a few things

* Drop 18 testing

* Use node 20

* lint rust

* lint

* changelog

* Drop usage of SVGs, use compound elements.

* Update widget API

* Better support for dark mode in widgets (#863)

* Major package upgrades

* Update rust deps and fix a few things

* Drop 18 testing

* Use node 20

* lint rust

* lint

* changelog

* Drop usage of SVGs, use compound elements.

* Update widget API

* Drop usage of SVGs, use compound elements.

* Add dark mode for widgets

* changelog

* Remove yarn-error.log

* Commit metrics changes

* Cleanup

---------

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de
Co-authored-by: Will Hunt <will@half-shot.uk>
2023-12-27 15:01:00 +00:00
Will Hunt
b96b45d98c
Drop Node 18, start testing Node 21 & update dependencies. (#862)
* Major package upgrades

* Update rust deps and fix a few things

* Drop 18 testing

* Use node 20

* lint rust

* lint

* changelog

* Drop usage of SVGs, use compound elements.

* Update widget API
2023-12-22 15:36:49 +00:00
Will Hunt
95738ed844
Warn if the bot cannot send a message in the widget (#852)
* Add a warning when the room does not have permission to post messages.

* Update warning

* changelog
2023-12-07 10:28:16 +00:00
Will Hunt
2a4b06bdbd
Use node:20-slim for Docker (#849)
* Use node:20-slim

* Add curl

* Lint

* retrigger

* Update stale cargo deps

* Update axios

* Add pkg-config and remove stale comment

* changelog

* Add openssl dep

* Add libssl-dev
2023-12-06 10:28:52 +00:00
Will Hunt
f8b71ea8b2
Add ability to respond to webhooks. (#838)
* Add ability to respond to webhooks.

* Refactor hook handling code for simplicity

* Cleanup checkbox

* Ensure we can send a response even we send no content.

* Add docs

* changelog

* Reflect local and global state in the UI

* Revert CHANGELOG for now

* Emphasize Slack the service in webhooks docs

and not time slack

* Guard against falsey non-string values

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-11-20 12:41:06 +00:00
Will Hunt
a539ad5e54
Ensure sample config only defaults to bare minimum requirements (#826)
* Comment out parts of the sample config that do not need to be on by default

* Update config

* Create 826.misc
2023-10-11 10:26:23 +01:00
Will Hunt
c3cb131e4d
Fix top level return incompatibility (#818)
* Run transformation functions at top level

* changelog
2023-09-26 12:56:11 +00:00
Will Hunt
dc126afa6a
Replace vm2 with quickjs (#817)
* quickjs test

* Replace vm2 with quickjs

* initalise -> initialise

* Remove unused transformation timeout time

* Don't assume quickModule is set

Also use whether it's set as the indicator of whether transformation
functions are allowed, instead of checking the config

* Refactor GenericHookConnectionState validation

- Do it in the constructor instead of in callers
- Make hookId mandatory so as to not drop it on some state updates
- Conflate a state event's state key with a connection state's name,
  which was already the case in practice

* Refactor validateState

* Drop explicit any

Better to infer the type instead

* Always validate transformation fn

* Fix test

* Add changelog

* Fix disposal, validation, and printing

* Fix transformation error string formatting

Also refactor similar code

* Let invalid transformations run & fail

instead of pretending that one was never set

* Restore transformation timeout time

* Don't execute transformation fn when validating it

Instead, only compile it

* Revert unrelated changes

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-09-25 14:55:15 +00:00
Will Hunt
d744974493
Ensure all forms of request failure on a feed are not treated as a parsing failure by metrics (#816)
* Only upload feed items when newGuids is non-zero

* Log other request failures as HTTP errors

* changelog

* Rust lint
2023-09-25 08:23:07 +00:00
Andrew Ferrazzutti
ee1b95678d
Remove Sled crypto store, use SQLite by default (#798)
* Remove Sled crypto store, use SQLite by default

on account of it being removed from the crypto-sdk

* Add changelog

* Don't start if Sled store config is set
2023-09-11 12:41:00 +00:00
Tadeusz Sośnierz
a64a561698
Bridge Gitlab comment replies as Matrix threads (#758)
* Bridge Gitlab comment replies as Matrix threads

* Persistently store Gitlab Discussion-Thread mapping

* Remove leftover debug line

* Denoise comment descriptions when they happen in Matrix threads

* Make comment debouncing time configurable

* Add some tests for Gitlab comments

* De-only Gitlab comment tests

* Linting

* Changelog

* Map multiple Gitlab discussions to a single Matrix thread

We debounce Gitlab comments, so multiple discussions can end up in one thread.
This ensures that replies to *any* of these discussions end up in the same thread.

* Add tests for the many-to-one reply case

* Move SerializedGitlabDiscussionThreads to Types

* Update changelog.d/758.feature

Co-authored-by: Will Hunt <will@half-shot.uk>

* Fix instructions for validating your config using Docker (#794)

* Fix instructions for validating your config using Docker

Fixes GH-787

* Changelog

---------

Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>

* Add more icons to GitHub messages (#795)

* Add more icons to GitHub messages

* Add merged icon

* Lint

* Add changelog

* Bump word-wrap from 1.2.3 to 1.2.4 (#799)

* Bump word-wrap from 1.2.3 to 1.2.4

Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add changelog

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

* Update matrix-appservice-bridge to 9.0.1 (#800)

* Bump semver from 5.7.1 to 5.7.2 (#797)

Bumps [semver](https://github.com/npm/node-semver) from 5.7.1 to 5.7.2.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/v5.7.2/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v5.7.1...v5.7.2)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Will Hunt <will@half-shot.uk>

* 4.4.0

* 4.4.1

* Set the default commentDebouncMs for Gitlab in its Config

* Rename `approvalState` to something more fitting

* Update sample config

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
Co-authored-by: Will Hunt <will@half-shot.uk>
Co-authored-by: Connor Davis <mail@connordav.is>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-08-14 12:58:21 +00:00
Connor Davis
269d8bb0e1
Add more icons to GitHub messages (#795)
* Add more icons to GitHub messages

* Add merged icon

* Lint

* Add changelog
2023-07-11 13:42:05 +00:00
Tadeusz Sośnierz
1c54506f6e
Sort feed list alphabetically in bot command response (#791)
* Sort feed list alphabetically in bot command response

Fixes GH-790

* Changelog

* Linting

---------

Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
2023-07-03 12:24:58 +00:00
Charlotte Van Petegem
4351a37131
Try to find an HTML-type link first when parsing atom feeds (#784)
Signed-off-by: Charlotte Van Petegem <charlotte@vanpetegem.me>
Co-authored-by: Will Hunt <will@half-shot.uk>
2023-06-28 15:35:06 +00:00
Will Hunt
3217b9eecf
Performance improvements to Feeds (#786)
* Various smaller changes

* Drop account data entirely

* Use max feed items

* Commit known working improvements

* Better status handlingh

* changelog

* Update changelog

* Add a note on Redis.

* Add proper HTTP tests

* Linty lint

* Tweaks

* New metrics woah

* Tweaks
2023-06-28 15:29:54 +00:00
Will Hunt
172ae30a73
Enable feeds to be polled concurrently (#779)
* Add support for concurrent RSS feed reading

* Add feed concurrency polling

* Actually load all workers

* Ensure feed just sleeps if there are no urls to poll

* Fix map

* Update default config

* Update src/feeds/FeedReader.ts

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

* Update src/feeds/FeedReader.ts

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-06-14 12:41:14 +00:00
Justin Carlson
96c398ce74
Fix crash from exceptions in handlers (#771)
* Fix crash from exceptions in handlers

* Fix withScope usage

* Add changelog
2023-06-05 16:25:19 +00:00
Will Hunt
6ca2b51893
Don't match issue comment notices on issue event type (#757)
* Don't match issue comment notices on issue event type

* changelog
2023-05-24 13:28:28 +00:00
Justin Carlson
0c4a493c56
Add support for uploading avatars (#767)
* Add uploading configured avatar images

* Add avatar assets

* Update docs and sample config

* Add changelog

* Fix sample config

* Add removing avatars when none configured
2023-05-24 13:16:37 +00:00
Christian Paul
00771d88c3
Fix #598; setup issues when the bot has PL 0 and room default isn't 0 (#755)
* Fix #598

If the bot has PL 0, the bot incorrectly used the default power level.

* Create 755.bugfix
2023-05-18 19:18:15 +00:00
Marcel
b91ca94249
fix: Make sure that the blockquote markdown for github issue comments are valid commonmark (#746)
* fix: Make sure that the blockquote mardkown for github issue comments are valid commonmark

* Add changelog file
2023-05-18 19:18:04 +00:00
Will Hunt
466ab6d5e3
Allow sanitized HTML from feed summaries (#738)
* Add a function to sanitize html output

* Allow some HTML in feed messages

* Update cargo

* changelog

* lint

* Saved a ton of code by just using ruma.

* Reverse order of rs:apply

* Lint

* Update src/Connections/FeedConnection.ts

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

* Apply suggestions from code review

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

* Add a test for partial html

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-05-18 18:59:58 +00:00
Will Hunt
b9313cd140
Add Sentry support (#754)
* Add sentry support

* Add basic listening for errors on express.

* Additional event types

* Add a default config

* changelog

* Improve wording

* Add docs
2023-05-18 11:05:23 +00:00
Will Hunt
af90e840e4
Add support for Gitlab individual review notifications (#736)
* Add support for individual reviews

* Document support + add frontend support

* changelog

* Remove redundant checks
2023-05-18 10:52:19 +00:00
Will Hunt
d772a4050c
Run Clippy in CI (#753)
* Clippy fixes

* Refactor to make rust linter happy

* Enable warnings since we have none

* Tidy up tests

* changelog
2023-05-18 11:38:59 +01:00
Marcel
409a8b83a7
Apply cargo clippy suggestions to reduce not needed allocations (#750)
* Apply cargo clippy suggestions to reduce not needed allocations

* Add changelog entry
2023-05-17 14:53:42 +01:00
Will Hunt
7223a86b2b
Fix message format for feed items with no title or link (#737)
* Allow empty title on feed item

* Allow bare URL

* Add tests for new format.

* changelog
2023-05-04 09:34:25 +01:00
Will Hunt
01159fa0d1
Update yaml dependency (#728)
* Update yaml dependency

* changelog
2023-04-25 15:52:57 +00:00
Will Hunt
43176adf7a
Fallback to parsing feeds as atom format if rss format fails. (#721)
* Support atom feeds in rust parser

* Add an apply linting command

* Add changelog

* Fixup

* Add tests for atom feeds + remove redundant code

* Remove unused rss-parser

* Tests for all formats.

* Move hashing logic into rust to save cross-context calls

* lint my rust

* Use a String::from

* Ensure guids are not repeated
2023-04-25 15:45:55 +00:00
Andrew Ferrazzutti
9bd46e4dc5
Fix hang in crypto reset script when E2EE is off (#723)
Ensure that the bridge storage is always disconnected
2023-04-21 23:57:40 +09:00
Andrew Ferrazzutti
f29fac7584
Allow usage of the SQLite-based crypto store (#714)
* Allow usage of the SQLite-based crypto store

Signed-off-by: Andrew Ferrazzutti <andrewf@element.io>

* Add changelog

* Log when crypto storage is initialized

* Add yarn script for resetting crypto state

Also document its usage & when it may be necessary to use it.

* Minor style improvements

- add trailing semicolons
- remove redundant `Promise.resolve`s

* Change logging in crypto reset script

- Move error object to parameter
- Rename logger

* Add method to close the Redis store

and call it to let the process exit cleanly

* Update matrix-bot-sdk

Use release with SQLite crypto store support

---------

Signed-off-by: Andrew Ferrazzutti <andrewf@element.io>
2023-04-21 23:18:27 +09:00