503 Commits

Author SHA1 Message Date
Twilight Sparkle
675c1c14d0
Add command to list feeds in JSON and YAML format (#876)
* Add command to list feeds in YAML format

* Add line breaks

* Add changelog

* Add a period in the help

* Add $URL to templates and add feed list format docs

* Use an optional arg on yaml list instead of a separate function

* Update changelog to include JSON format
2024-01-04 19:14:11 +00:00
Will Hunt
41ee467e2d
Fix logged errors not being properly formatted (and tone down the error verbosity a bit) (#874)
* Ensure we add the error handlers last

* Refactor error middleware to be less noisy

* changelog

* Add finalise
2024-01-02 15:58:25 +00:00
Will Hunt
c0bb71d553
Fix GitHub events not verifying (#875)
* Ensure we verify the raw payload.

* changelog

* Tidy up types

* Add test for GitHib

* Mock out GitHub API to allow tests to pass.

* Lint
2024-01-02 15:52:43 +00:00
Simon Quigley
3af2aa7032
sucessfully (#869)
* sucessfully

* Create 869.misc

---------

Co-authored-by: Will Hunt <will@half-shot.uk>
2023-12-29 16:24:49 +00:00
Will Hunt
5c5bdd546d
Fix widgets not working when bound to the same listener as webhooks. (#872)
* Bind listener in bridge.

* Add a test to confirm behaviour

* Fix test name

* changelog
2023-12-29 16:11:08 +00:00
Will Hunt
8e115b40ab
Add End to End testing (#868)
* Ensure connection state always explicitly states all keys, even if some are undefined.

* changelog

* Fix type

* fix test types

* Add support for E2E testing

* Add CI job for e2e test

* Ensure integration test only runs when regular tests complete

* Add homerunner image

* Disallow concurrent runs

* Add concurrency to other expensive steps

* changelog

* Fix mq test

* Cache rust deps

* Drop only

* Use a shared key
2023-12-28 15:04:03 +00:00
Will Hunt
46d198c2c0
Fix feeds template / notify state not reflecting in the UI (#866)
* Ensure connection state always explicitly states all keys, even if some are undefined.

* changelog

* Fix type

* fix test types
2023-12-28 14:27:47 +00:00
Will Hunt
76a4209f0d
Remove GoNEB migrator (#867)
* Remove GoNEB migrator.

* changelog

* Remove frontend pieces

* Remove unused statuscodes

* Linting

* Remove warning about ts-comment
2023-12-28 11:36:11 +00:00
Will Hunt
9cc402b066
Add commands for listing & removing webhooks (#864)
* Add commands for removing webhooks

* Fix command setting state twice

* changelog
2023-12-27 17:11:38 +00:00
Will Hunt
066d908b4f
Fix feeds config UI not allowing notifyOnFailure to be toggled. (#865)
* Fix feeds config UI not allowing notifyOnFailure to be toggled.

* changelog

* amend changelog to be cohierent

* Actual fixes.
2023-12-27 15:25:30 +00:00
Joshua Hoffmann
1db6f60776
Some smaller improvements to metrics.md (#856)
* Update metrics.md

Made descriptions of metrics more consistent and some smaller improvements.

* Update README.md (#855)

* Update README.md

Smaller improvements. Fixed a typo.

* Create 855.doc

---------

Co-authored-by: Will Hunt <will@half-shot.uk>

* Update setup.md (#857)

* Update setup.md

Smaller improvements such as fixes for typos, also added an example on how to pass requests from Nginx

* Create 857.docs

* Rename 857.docs to 857.doc

---------

Co-authored-by: Will Hunt <will@half-shot.uk>

* Update workers.md (#858)

* Update workers.md

Fixed typos

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de

* Create 858.doc

---------

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de
Co-authored-by: Will Hunt <will@half-shot.uk>

* Update encryption.md (#860)

* Update encryption.md

Consistently spell Hookshot with an upper-case H as the first letter

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de

* Create 860.doc

---------

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de
Co-authored-by: Will Hunt <will@half-shot.uk>

* Update widgets.md (#859)

* Update widgets.md

Consistently spell Hookshot with an upper-case H as the first letter

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de

* Create 859.doc

---------

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de
Co-authored-by: Will Hunt <will@half-shot.uk>

* Drop Node 18, start testing Node 21 & update dependencies. (#862)

* Major package upgrades

* Update rust deps and fix a few things

* Drop 18 testing

* Use node 20

* lint rust

* lint

* changelog

* Drop usage of SVGs, use compound elements.

* Update widget API

* Better support for dark mode in widgets (#863)

* Major package upgrades

* Update rust deps and fix a few things

* Drop 18 testing

* Use node 20

* lint rust

* lint

* changelog

* Drop usage of SVGs, use compound elements.

* Update widget API

* Drop usage of SVGs, use compound elements.

* Add dark mode for widgets

* changelog

* Remove yarn-error.log

* Commit metrics changes

* Cleanup

---------

Signed-off-by: Joshua Hoffmann joshua.hoffmann@b1-systems.de
Co-authored-by: Will Hunt <will@half-shot.uk>
2023-12-27 15:01:00 +00:00
Will Hunt
b96b45d98c
Drop Node 18, start testing Node 21 & update dependencies. (#862)
* Major package upgrades

* Update rust deps and fix a few things

* Drop 18 testing

* Use node 20

* lint rust

* lint

* changelog

* Drop usage of SVGs, use compound elements.

* Update widget API
2023-12-22 15:36:49 +00:00
Will Hunt
95738ed844
Warn if the bot cannot send a message in the widget (#852)
* Add a warning when the room does not have permission to post messages.

* Update warning

* changelog
2023-12-07 10:28:16 +00:00
Will Hunt
2a4b06bdbd
Use node:20-slim for Docker (#849)
* Use node:20-slim

* Add curl

* Lint

* retrigger

* Update stale cargo deps

* Update axios

* Add pkg-config and remove stale comment

* changelog

* Add openssl dep

* Add libssl-dev
2023-12-06 10:28:52 +00:00
Will Hunt
f8b71ea8b2
Add ability to respond to webhooks. (#838)
* Add ability to respond to webhooks.

* Refactor hook handling code for simplicity

* Cleanup checkbox

* Ensure we can send a response even we send no content.

* Add docs

* changelog

* Reflect local and global state in the UI

* Revert CHANGELOG for now

* Emphasize Slack the service in webhooks docs

and not time slack

* Guard against falsey non-string values

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-11-20 12:41:06 +00:00
Will Hunt
a539ad5e54
Ensure sample config only defaults to bare minimum requirements (#826)
* Comment out parts of the sample config that do not need to be on by default

* Update config

* Create 826.misc
2023-10-11 10:26:23 +01:00
Will Hunt
c3cb131e4d
Fix top level return incompatibility (#818)
* Run transformation functions at top level

* changelog
2023-09-26 12:56:11 +00:00
Will Hunt
dc126afa6a
Replace vm2 with quickjs (#817)
* quickjs test

* Replace vm2 with quickjs

* initalise -> initialise

* Remove unused transformation timeout time

* Don't assume quickModule is set

Also use whether it's set as the indicator of whether transformation
functions are allowed, instead of checking the config

* Refactor GenericHookConnectionState validation

- Do it in the constructor instead of in callers
- Make hookId mandatory so as to not drop it on some state updates
- Conflate a state event's state key with a connection state's name,
  which was already the case in practice

* Refactor validateState

* Drop explicit any

Better to infer the type instead

* Always validate transformation fn

* Fix test

* Add changelog

* Fix disposal, validation, and printing

* Fix transformation error string formatting

Also refactor similar code

* Let invalid transformations run & fail

instead of pretending that one was never set

* Restore transformation timeout time

* Don't execute transformation fn when validating it

Instead, only compile it

* Revert unrelated changes

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-09-25 14:55:15 +00:00
Will Hunt
d744974493
Ensure all forms of request failure on a feed are not treated as a parsing failure by metrics (#816)
* Only upload feed items when newGuids is non-zero

* Log other request failures as HTTP errors

* changelog

* Rust lint
2023-09-25 08:23:07 +00:00
Andrew Ferrazzutti
ee1b95678d
Remove Sled crypto store, use SQLite by default (#798)
* Remove Sled crypto store, use SQLite by default

on account of it being removed from the crypto-sdk

* Add changelog

* Don't start if Sled store config is set
2023-09-11 12:41:00 +00:00
Tadeusz Sośnierz
a64a561698
Bridge Gitlab comment replies as Matrix threads (#758)
* Bridge Gitlab comment replies as Matrix threads

* Persistently store Gitlab Discussion-Thread mapping

* Remove leftover debug line

* Denoise comment descriptions when they happen in Matrix threads

* Make comment debouncing time configurable

* Add some tests for Gitlab comments

* De-only Gitlab comment tests

* Linting

* Changelog

* Map multiple Gitlab discussions to a single Matrix thread

We debounce Gitlab comments, so multiple discussions can end up in one thread.
This ensures that replies to *any* of these discussions end up in the same thread.

* Add tests for the many-to-one reply case

* Move SerializedGitlabDiscussionThreads to Types

* Update changelog.d/758.feature

Co-authored-by: Will Hunt <will@half-shot.uk>

* Fix instructions for validating your config using Docker (#794)

* Fix instructions for validating your config using Docker

Fixes GH-787

* Changelog

---------

Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>

* Add more icons to GitHub messages (#795)

* Add more icons to GitHub messages

* Add merged icon

* Lint

* Add changelog

* Bump word-wrap from 1.2.3 to 1.2.4 (#799)

* Bump word-wrap from 1.2.3 to 1.2.4

Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add changelog

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

* Update matrix-appservice-bridge to 9.0.1 (#800)

* Bump semver from 5.7.1 to 5.7.2 (#797)

Bumps [semver](https://github.com/npm/node-semver) from 5.7.1 to 5.7.2.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/v5.7.2/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v5.7.1...v5.7.2)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Will Hunt <will@half-shot.uk>

* 4.4.0

* 4.4.1

* Set the default commentDebouncMs for Gitlab in its Config

* Rename `approvalState` to something more fitting

* Update sample config

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
Co-authored-by: Will Hunt <will@half-shot.uk>
Co-authored-by: Connor Davis <mail@connordav.is>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-08-14 12:58:21 +00:00
Connor Davis
269d8bb0e1
Add more icons to GitHub messages (#795)
* Add more icons to GitHub messages

* Add merged icon

* Lint

* Add changelog
2023-07-11 13:42:05 +00:00
Tadeusz Sośnierz
1c54506f6e
Sort feed list alphabetically in bot command response (#791)
* Sort feed list alphabetically in bot command response

Fixes GH-790

* Changelog

* Linting

---------

Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
2023-07-03 12:24:58 +00:00
Charlotte Van Petegem
4351a37131
Try to find an HTML-type link first when parsing atom feeds (#784)
Signed-off-by: Charlotte Van Petegem <charlotte@vanpetegem.me>
Co-authored-by: Will Hunt <will@half-shot.uk>
2023-06-28 15:35:06 +00:00
Will Hunt
3217b9eecf
Performance improvements to Feeds (#786)
* Various smaller changes

* Drop account data entirely

* Use max feed items

* Commit known working improvements

* Better status handlingh

* changelog

* Update changelog

* Add a note on Redis.

* Add proper HTTP tests

* Linty lint

* Tweaks

* New metrics woah

* Tweaks
2023-06-28 15:29:54 +00:00
Will Hunt
172ae30a73
Enable feeds to be polled concurrently (#779)
* Add support for concurrent RSS feed reading

* Add feed concurrency polling

* Actually load all workers

* Ensure feed just sleeps if there are no urls to poll

* Fix map

* Update default config

* Update src/feeds/FeedReader.ts

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

* Update src/feeds/FeedReader.ts

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-06-14 12:41:14 +00:00
Justin Carlson
96c398ce74
Fix crash from exceptions in handlers (#771)
* Fix crash from exceptions in handlers

* Fix withScope usage

* Add changelog
2023-06-05 16:25:19 +00:00
Will Hunt
6ca2b51893
Don't match issue comment notices on issue event type (#757)
* Don't match issue comment notices on issue event type

* changelog
2023-05-24 13:28:28 +00:00
Justin Carlson
0c4a493c56
Add support for uploading avatars (#767)
* Add uploading configured avatar images

* Add avatar assets

* Update docs and sample config

* Add changelog

* Fix sample config

* Add removing avatars when none configured
2023-05-24 13:16:37 +00:00
Christian Paul
00771d88c3
Fix #598; setup issues when the bot has PL 0 and room default isn't 0 (#755)
* Fix #598

If the bot has PL 0, the bot incorrectly used the default power level.

* Create 755.bugfix
2023-05-18 19:18:15 +00:00
Marcel
b91ca94249
fix: Make sure that the blockquote markdown for github issue comments are valid commonmark (#746)
* fix: Make sure that the blockquote mardkown for github issue comments are valid commonmark

* Add changelog file
2023-05-18 19:18:04 +00:00
Will Hunt
466ab6d5e3
Allow sanitized HTML from feed summaries (#738)
* Add a function to sanitize html output

* Allow some HTML in feed messages

* Update cargo

* changelog

* lint

* Saved a ton of code by just using ruma.

* Reverse order of rs:apply

* Lint

* Update src/Connections/FeedConnection.ts

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

* Apply suggestions from code review

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>

* Add a test for partial html

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2023-05-18 18:59:58 +00:00
Will Hunt
b9313cd140
Add Sentry support (#754)
* Add sentry support

* Add basic listening for errors on express.

* Additional event types

* Add a default config

* changelog

* Improve wording

* Add docs
2023-05-18 11:05:23 +00:00
Will Hunt
af90e840e4
Add support for Gitlab individual review notifications (#736)
* Add support for individual reviews

* Document support + add frontend support

* changelog

* Remove redundant checks
2023-05-18 10:52:19 +00:00
Will Hunt
d772a4050c
Run Clippy in CI (#753)
* Clippy fixes

* Refactor to make rust linter happy

* Enable warnings since we have none

* Tidy up tests

* changelog
2023-05-18 11:38:59 +01:00
Marcel
409a8b83a7
Apply cargo clippy suggestions to reduce not needed allocations (#750)
* Apply cargo clippy suggestions to reduce not needed allocations

* Add changelog entry
2023-05-17 14:53:42 +01:00
Will Hunt
7223a86b2b
Fix message format for feed items with no title or link (#737)
* Allow empty title on feed item

* Allow bare URL

* Add tests for new format.

* changelog
2023-05-04 09:34:25 +01:00
Will Hunt
01159fa0d1
Update yaml dependency (#728)
* Update yaml dependency

* changelog
2023-04-25 15:52:57 +00:00
Will Hunt
43176adf7a
Fallback to parsing feeds as atom format if rss format fails. (#721)
* Support atom feeds in rust parser

* Add an apply linting command

* Add changelog

* Fixup

* Add tests for atom feeds + remove redundant code

* Remove unused rss-parser

* Tests for all formats.

* Move hashing logic into rust to save cross-context calls

* lint my rust

* Use a String::from

* Ensure guids are not repeated
2023-04-25 15:45:55 +00:00
Andrew Ferrazzutti
9bd46e4dc5
Fix hang in crypto reset script when E2EE is off (#723)
Ensure that the bridge storage is always disconnected
2023-04-21 23:57:40 +09:00
Andrew Ferrazzutti
f29fac7584
Allow usage of the SQLite-based crypto store (#714)
* Allow usage of the SQLite-based crypto store

Signed-off-by: Andrew Ferrazzutti <andrewf@element.io>

* Add changelog

* Log when crypto storage is initialized

* Add yarn script for resetting crypto state

Also document its usage & when it may be necessary to use it.

* Minor style improvements

- add trailing semicolons
- remove redundant `Promise.resolve`s

* Change logging in crypto reset script

- Move error object to parameter
- Rename logger

* Add method to close the Redis store

and call it to let the process exit cleanly

* Update matrix-bot-sdk

Use release with SQLite crypto store support

---------

Signed-off-by: Andrew Ferrazzutti <andrewf@element.io>
2023-04-21 23:18:27 +09:00
Will Hunt
a9537c7961
Add option to enable/disable feed failure notices. (#716)
* Add new notifyOnFailure option

* Disable state buttons when updating is set

* Linting
2023-04-21 10:47:29 +00:00
Will Hunt
c6a04c2ebd
Fix case where GitHub repos aren't connectable if the GitHub App was manually approved (#718)
* Fix GitHub connections failing if the installation cache is stale

* changelog
2023-04-21 09:27:57 +00:00
Will Hunt
eda6fc8c26
Use rss crate for RSS feed parsing (#709)
* Refactor annoying type difference

* Add rss

* Add support for parsing RSS feeds

* Add rss lib

* Add Cargo project

* Lint rust

* changelog

* Add support for new fields
2023-04-17 11:48:03 +00:00
Tadeusz Sośnierz
bc57dbbc83
Don't process feed/item titles that aren't actually strings (#708)
* Don't process feed/item titles that aren't actually strings

Another case of empty-tag with attributes, these would occasionally parse
into non-string objects and make us crash when trying to stripHtml on them.

* Changelog

* Add tests for not-quite-empty <title> tags in feeds

---------

Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
2023-04-12 17:52:10 +02:00
Justin Carlson
cafe05b0a2
Add hookshot_ prefix on hookshot specific metrics (#701)
* Add hookshot_ prefix on all metrics

* Only add prefix to hookshot specific metrics

* Add old metric names as deprecated

* Fix feeds metrics names

* Update metrics docs

---------

Co-authored-by: Will Hunt <will@half-shot.uk>
2023-04-11 16:58:51 +00:00
Will Hunt
6345ad2347
Add support for templating feed messages (#702)
* Add support for specifying a template for feeds

* Remove unused params

* use the correct url

* make URL visible.

It was annoying me

* Update src/Connections/FeedConnection.ts

Co-authored-by: Tadeusz Sośnierz <tadzik@tadzik.net>

* Support migrations

* Hopefully support migrations now

* lint

---------

Co-authored-by: Tadeusz Sośnierz <tadzik@tadzik.net>
2023-04-11 15:24:00 +00:00
Will Hunt
c7ca2ba94d
Add support for showing GitHub issue comments (#697)
* Add support for issue/PR comments.

* changelog

* Quote message

* Truncate messages too
2023-04-04 13:28:51 +01:00
Will Hunt
7eef384563
Add support for using guid as a feed link (#700)
* Move statics above instance fields, and add new url praser

* uncomment out

* add comment

* Create 700.feature
2023-04-03 14:24:36 +00:00
Tadeusz Sośnierz
0ce06c4ea7
Allow users to import other people's go-neb services (#695)
* Allow room admins to import other people's go-neb services

This requires us to guess what these other people's MXIDs were,
so we scroll through the list of room members and make educated guesses
about which of them are Scalar+go-neb bots, and which users they were set up by.

* Relax our requirements for scraping others' go-neb connections

* Changelog

* Linting

---------

Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
2023-04-03 16:13:43 +02:00