Stabilize E2EE support (#989)

* Deprecate legacy sled store

* Add e2ee test

* Add support for e2ee testing in e2e environment

* Tidy up redis support

* Attempt to get test working

* cleanup test

* opportunistic lint

* tiny bit of cleanup

* remove ref

* tweak to homerunner

* switch to nightly images for Synapse (to test E2EE)

* use nightly

* newsfile.

* Update bot sdk to support authenticated media (now that Synapse requires it)

* fix typings

* MatrixError

* one more

* Graduate the encryption property to stable.

* update test config

* Update encryption docs.

* fix some old config bits
This commit is contained in:
Will Hunt 2024-11-26 16:04:56 +00:00 committed by GitHub
parent 46b0004581
commit 052d42fca8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
21 changed files with 286 additions and 178 deletions

View File

@ -92,7 +92,7 @@ jobs:
homerunnersha: ${{ steps.gitsha.outputs.sha }}
steps:
- name: Checkout matrix-org/complement
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: matrix-org/complement
- name: Get complement git sha
@ -100,7 +100,7 @@ jobs:
run: echo sha=`git rev-parse --short HEAD` >> "$GITHUB_OUTPUT"
- name: Cache homerunner
id: cached
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: homerunner
key: ${{ runner.os }}-homerunner-${{ steps.gitsha.outputs.sha }}
@ -125,23 +125,28 @@ jobs:
needs:
- test
- build-homerunner
services:
redis:
image: redis
ports:
- 6379:6379
steps:
- name: Install Complement Dependencies
run: |
sudo apt-get update && sudo apt-get install -y libolm3
- name: Load cached homerunner bin
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: homerunner
key: ${{ runner.os }}-homerunner-${{ needs.build-synapse.outputs.homerunnersha }}
fail-on-cache-miss: true # Shouldn't happen, we build this in the needs step.
- name: Checkout matrix-hookshot
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
path: matrix-hookshot
# Setup node & run tests
- name: Use Node.js
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version-file: matrix-hookshot/.node-version
- uses: Swatinem/rust-cache@v2
@ -152,8 +157,9 @@ jobs:
timeout-minutes: 10
env:
HOMERUNNER_SPAWN_HS_TIMEOUT_SECS: 100
HOMERUNNER_IMAGE: ghcr.io/element-hq/synapse/complement-synapse:latest
HOMERUNNER_IMAGE: ghcr.io/element-hq/synapse/complement-synapse:nightly
NODE_OPTIONS: --dns-result-order ipv4first
REDIS_DATABASE_URI: "redis://localhost:6379"
run: |
docker pull $HOMERUNNER_IMAGE
cd matrix-hookshot

2
changelog.d/989.feature Normal file
View File

@ -0,0 +1,2 @@
Support for E2E Encrypted rooms is now considered stable and can be enabled in production. Please see the [documentation](https://matrix-org.github.io/matrix-hookshot/latest/advanced/encryption.html)
on the requirements for enabling support.

View File

@ -1,5 +1,11 @@
# This is an example configuration file
logging:
# Logging settings. You can have a severity debug,info,warn,error
level: info
colorize: true
json: false
timestampFormat: HH:mm:ss:SSS
bridge:
# Basic homeserver configuration
domain: example.com
@ -11,12 +17,6 @@ passFile:
# A passkey used to encrypt tokens stored inside the bridge.
# Run openssl genpkey -out passkey.pem -outform PEM -algorithm RSA -pkeyopt rsa_keygen_bits:4096 to generate
./passkey.pem
logging:
# Logging settings. You can have a severity debug,info,warn,error
level: info
colorize: true
json: false
timestampFormat: HH:mm:ss:SSS
listeners:
# HTTP Listener configuration.
# Bind resource endpoints to ports and addresses.
@ -143,10 +143,12 @@ listeners:
# # For encryption to work, this must be configured.
# redisUri: redis://localhost:6379
#queue:
# # (Optional) Message queue configuration options for large scale deployments.
# # For encryption to work, this must not be configured.
# redisUri: redis://localhost:6379
#encryption:
# # (Optional) Configuration for encryption support in the bridge.
# # If omitted, encryption support will be disabled.
# storagePath:
# # Path to the directory used to store encryption files. These files must be persist between restarts of the service.
# ./cryptostore
#widgets:
# # (Optional) EXPERIMENTAL support for complimentary widgets

View File

@ -1,27 +1,32 @@
Encryption
==========
<section class="warning">
Encryption support is <strong>HIGHLY EXPERIMENTAL AND SUBJECT TO CHANGE</strong>. It should not be enabled for production workloads.
For more details, see <a href="https://github.com/matrix-org/matrix-hookshot/issues/594">issue 594</a>.
<section class="notice">
Support for encryption is considered stable, but the underlying specification changes are not yet.
Hookshot supports end-to-bridge encryption via [MSC3202](https://github.com/matrix-org/matrix-spec-proposals/pull/3202), and [MSC4203](https://github.com/matrix-org/matrix-spec-proposals/pull/4203). Hookshot needs to be configured against a a homeserver that supports these features, such as [Synapse](#running-with-synapse).
Please check with your homeserver implementation before reporting bugs against matrix-hookshot.
</section>
Hookshot supports end-to-bridge encryption via [MSC3202](https://github.com/matrix-org/matrix-spec-proposals/pull/3202). As such, encryption requires Hookshot to be connected to a homeserver that supports that MSC, such as [Synapse](#running-with-synapse).
## Enabling encryption in Hookshot
In order for Hookshot to use encryption, it must be configured as follows:
- The `experimentalEncryption.storagePath` setting must point to a directory that Hookshot has permissions to write files into. If running with Docker, this path should be within a volume (for persistency). Hookshot uses this directory for its crypto store (i.e. long-lived state relating to its encryption keys).
- The `encryption.storagePath` setting must point to a directory that Hookshot has permissions to write files into. If running with Docker, this path should be within a volume (for persistency). Hookshot uses this directory for its crypto store (i.e. long-lived state relating to its encryption keys).
- Once a crypto store has been initialized, its files must not be modified, and Hookshot cannot be configured to use another crypto store of the same type as one it has used before. If a crypto store's files get lost or corrupted, Hookshot may fail to start up, or may be unable to decrypt command messages. To fix such issues, stop Hookshot, then reset its crypto store by running `yarn start:resetcrypto`.
- [Redis](./workers.md) must be enabled. Note that worker mode is not yet supported with encryption, so `queue` MUST **NOT be configured**.
If you ever reset your homeserver's state, ensure you also reset Hookshot's encryption state. This includes clearing the `experimentalEncryption.storagePath` directory and all worker state stored in your redis instance. Otherwise, Hookshot may fail on start up with registration errors.
If you ever reset your homeserver's state, ensure you also reset Hookshot's encryption state. This includes clearing the `storagePath` directory and all worker state stored in your redis instance. Otherwise, Hookshot may fail on start up with registration errors.
Also ensure that Hookshot's appservice registration file contains every line from `registration.sample.yml` that appears after the `If enabling encryption` comment. Note that changing the registration file may require restarting the homeserver that Hookshot is connected to.
## Running with Synapse
[Synapse](https://github.com/matrix-org/synapse/) has functional support for MSC3202 as of [v1.63.0](https://github.com/matrix-org/synapse/releases/tag/v1.63.0). To enable it, add the following section to Synapse's configuration file (typically named `homeserver.yaml`):
[Synapse](https://github.com/matrix-org/synapse/) has functional support for MSC3202 and MSC4203 as of [v1.63.0](https://github.com/matrix-org/synapse/releases/tag/v1.63.0). To enable it, add the following section to Synapse's configuration file (typically named `homeserver.yaml`):
You may notice that MSC2409 is not listed above. Due to the changes being split out from MSC2409, `msc2409_to_device_messages_enabled` refers to MSC4203.
```yaml
experimental_features:
@ -29,3 +34,4 @@ experimental_features:
msc3202_transaction_extensions: true
msc2409_to_device_messages_enabled: true
```

View File

@ -63,7 +63,7 @@
"jira-client": "^8.2.2",
"markdown-it": "^14.0.0",
"matrix-appservice-bridge": "^9.0.1",
"matrix-bot-sdk": "npm:@vector-im/matrix-bot-sdk@^0.7.0-specific-device-2",
"matrix-bot-sdk": "npm:@vector-im/matrix-bot-sdk@v0.7.1-element.6",
"matrix-widget-api": "^1.6.0",
"micromatch": "^4.0.8",
"mime": "^4.0.1",

View File

@ -26,37 +26,4 @@ describe('Basic test setup', () => {
// Expect help text.
expect((await msg).data.content.body).to.include('!hookshot help` - This help text\n');
});
// TODO: Move test to it's own generic connections file.
it('should be able to setup a webhook', async () => {
const user = testEnv.getUser('user');
const testRoomId = await user.createRoom({ name: 'Test room', invite:[testEnv.botMxid] });
await user.waitForRoomJoin({sender: testEnv.botMxid, roomId: testRoomId });
await user.setUserPowerLevel(testEnv.botMxid, testRoomId, 50);
await user.sendText(testRoomId, "!hookshot webhook test-webhook");
const inviteResponse = await user.waitForRoomInvite({sender: testEnv.botMxid});
await user.waitForRoomEvent<MessageEventContent>({
eventType: 'm.room.message', sender: testEnv.botMxid, roomId: testRoomId,
body: 'Room configured to bridge webhooks. See admin room for secret url.'
});
const webhookUrlMessage = user.waitForRoomEvent<MessageEventContent>({
eventType: 'm.room.message', sender: testEnv.botMxid, roomId: inviteResponse.roomId
});
await user.joinRoom(inviteResponse.roomId);
const msgData = (await webhookUrlMessage).data.content.body;
const webhookUrl = msgData.split('\n')[2];
const webhookNotice = user.waitForRoomEvent<MessageEventContent>({
eventType: 'm.room.message', sender: testEnv.botMxid, roomId: testRoomId, body: 'Hello world!'
});
// Send a webhook
await fetch(webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({text: 'Hello world!'})
});
// And await the notice.
await webhookNotice;
});
});

66
spec/e2ee.spec.ts Normal file
View File

@ -0,0 +1,66 @@
import { MessageEventContent } from "matrix-bot-sdk";
import { E2ESetupTestTimeout, E2ETestEnv } from "./util/e2e-test";
import { describe, it, beforeEach, afterEach } from "@jest/globals";
const CryptoRoomState = [{
content: {
"algorithm": "m.megolm.v1.aes-sha2"
},
state_key: "",
type: "m.room.encryption"
}];
describe('End-2-End Encryption support', () => {
let testEnv: E2ETestEnv;
beforeEach(async () => {
testEnv = await E2ETestEnv.createTestEnv({ matrixLocalparts: ['user'], enableE2EE: true });
await testEnv.setUp();
}, E2ESetupTestTimeout);
afterEach(() => {
return testEnv?.tearDown();
});
it('should be able to send the help command', async () => {
const user = testEnv.getUser('user');
const testRoomId = await user.createRoom({ name: 'Test room', invite:[testEnv.botMxid], initial_state: CryptoRoomState});
await user.setUserPowerLevel(testEnv.botMxid, testRoomId, 50);
await user.waitForRoomJoin({sender: testEnv.botMxid, roomId: testRoomId });
await user.sendText(testRoomId, "!hookshot help");
await user.waitForRoomEvent<MessageEventContent>({
eventType: 'm.room.message', sender: testEnv.botMxid, roomId: testRoomId,
});
});
it('should send notices in an encrypted format', async () => {
const user = testEnv.getUser('user');
const testRoomId = await user.createRoom({ name: 'Test room', invite:[testEnv.botMxid], initial_state: CryptoRoomState});
await user.setUserPowerLevel(testEnv.botMxid, testRoomId, 50);
await user.waitForRoomJoin({sender: testEnv.botMxid, roomId: testRoomId });
await user.sendText(testRoomId, "!hookshot webhook test-webhook");
const inviteResponse = await user.waitForRoomInvite({sender: testEnv.botMxid});
await user.waitForEncryptedEvent<MessageEventContent>({
eventType: 'm.room.message', sender: testEnv.botMxid, roomId: testRoomId,
body: 'Room configured to bridge webhooks. See admin room for secret url.'
});
const webhookUrlMessage = user.waitForEncryptedEvent<MessageEventContent>({
eventType: 'm.room.message', sender: testEnv.botMxid, roomId: inviteResponse.roomId
});
await user.joinRoom(inviteResponse.roomId);
const msgData = (await webhookUrlMessage).data.content.body;
const webhookUrl = msgData.split('\n')[2];
const webhookNotice = user.waitForEncryptedEvent<MessageEventContent>({
eventType: 'm.room.message', sender: testEnv.botMxid, roomId: testRoomId, body: 'Hello world!'
});
// Send a webhook
await fetch(webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({text: 'Hello world!'})
});
// And await the notice.
await webhookNotice;
});
});

View File

@ -5,13 +5,24 @@ import { BridgeConfig, BridgeConfigRoot } from "../../src/config/Config";
import { start } from "../../src/App/BridgeApp";
import { RSAKeyPairOptions, generateKeyPair } from "node:crypto";
import path from "node:path";
import Redis from "ioredis";
const WAIT_EVENT_TIMEOUT = 10000;
const WAIT_EVENT_TIMEOUT = 20000;
export const E2ESetupTestTimeout = 60000;
const REDIS_DATABASE_URI = process.env.HOOKSHOT_E2E_REDIS_DB_URI ?? "redis://localhost:6379";
interface Opts {
matrixLocalparts?: string[];
config?: Partial<BridgeConfigRoot>,
enableE2EE?: boolean,
useRedis?: boolean,
}
interface WaitForEventResponse<T extends object = Record<string, unknown>> {
roomId: string,
data: {
sender: string, type: string, state_key?: string, content: T, event_id: string,
}
}
export class E2ETestMatrixClient extends MatrixClient {
@ -55,13 +66,10 @@ export class E2ETestMatrixClient extends MatrixClient {
}, `Timed out waiting for powerlevel from in ${roomId}`)
}
public async waitForRoomEvent<T extends object = Record<string, unknown>>(
opts: {eventType: string, sender: string, roomId?: string, stateKey?: string, body?: string}
): Promise<{roomId: string, data: {
sender: string, type: string, state_key?: string, content: T, event_id: string,
}}> {
const {eventType, sender, roomId, stateKey} = opts;
return this.waitForEvent('room.event', (eventRoomId: string, eventData: {
private async innerWaitForRoomEvent<T extends object = Record<string, unknown>>(
{eventType, sender, roomId, stateKey, eventId, body}: {eventType: string, sender: string, roomId?: string, stateKey?: string, body?: string, eventId?: string}, expectEncrypted: boolean,
): Promise<WaitForEventResponse<T>> {
return this.waitForEvent(expectEncrypted ? 'room.decrypted_event' : 'room.event', (eventRoomId: string, eventData: {
sender: string, type: string, state_key?: string, content: T, event_id: string,
}) => {
if (eventData.sender !== sender) {
@ -73,21 +81,36 @@ export class E2ETestMatrixClient extends MatrixClient {
if (roomId && eventRoomId !== roomId) {
return undefined;
}
if (eventId && eventData.event_id !== eventId) {
return undefined;
}
if (stateKey !== undefined && eventData.state_key !== stateKey) {
return undefined;
}
const body = 'body' in eventData.content && eventData.content.body;
if (opts.body && body !== opts.body) {
const evtBody = 'body' in eventData.content && eventData.content.body;
if (body && body !== evtBody) {
return undefined;
}
console.info(
// eslint-disable-next-line max-len
`${eventRoomId} ${eventData.event_id} ${eventData.type} ${eventData.sender} ${eventData.state_key ?? body ?? ''}`
`${eventRoomId} ${eventData.event_id} ${eventData.type} ${eventData.sender} ${eventData.state_key ?? evtBody ?? ''}`
);
return {roomId: eventRoomId, data: eventData};
}, `Timed out waiting for ${eventType} from ${sender} in ${roomId || "any room"}`)
}
public async waitForRoomEvent<T extends object = Record<string, unknown>>(
opts: Parameters<E2ETestMatrixClient["innerWaitForRoomEvent"]>[0]
): Promise<WaitForEventResponse<T>> {
return this.innerWaitForRoomEvent(opts, false);
}
public async waitForEncryptedEvent<T extends object = Record<string, unknown>>(
opts: Parameters<E2ETestMatrixClient["innerWaitForRoomEvent"]>[0]
): Promise<WaitForEventResponse<T>> {
return this.innerWaitForRoomEvent(opts, true);
}
public async waitForRoomJoin(
opts: {sender: string, roomId?: string}
): Promise<{roomId: string, data: unknown}> {
@ -173,10 +196,11 @@ export class E2ETestEnv {
if (err) { reject(err) } else { resolve(privateKey) }
}));
const dir = await mkdtemp('hookshot-int-test');
// Configure homeserver and bots
const [homeserver, dir, privateKey] = await Promise.all([
createHS([...matrixLocalparts || []], workerID),
mkdtemp('hookshot-int-test'),
const [homeserver, privateKey] = await Promise.all([
createHS([...matrixLocalparts || []], workerID, opts.enableE2EE ? path.join(dir, 'client-crypto') : undefined),
keyPromise,
]);
const keyPath = path.join(dir, 'key.pem');
@ -193,6 +217,15 @@ export class E2ETestEnv {
providedConfig.github.auth.privateKeyFile = keyPath;
}
opts.useRedis = opts.enableE2EE || opts.useRedis;
let cacheConfig: BridgeConfigRoot["cache"]|undefined;
if (opts.useRedis) {
cacheConfig = {
redisUri: `${REDIS_DATABASE_URI}/${workerID}`,
}
}
const config = new BridgeConfig({
bridge: {
domain: homeserver.domain,
@ -201,7 +234,7 @@ export class E2ETestEnv {
bindAddress: '0.0.0.0',
},
logging: {
level: 'info',
level: 'debug',
},
// Always enable webhooks so that hookshot starts.
generic: {
@ -214,6 +247,12 @@ export class E2ETestEnv {
resources: ['webhooks'],
}],
passFile: keyPath,
...(opts.enableE2EE ? {
encryption: {
storagePath: path.join(dir, 'crypto-store'),
}
} : undefined),
cache: cacheConfig,
...providedConfig,
});
const registration: IAppserviceRegistration = {
@ -227,7 +266,8 @@ export class E2ETestEnv {
}],
rooms: [],
aliases: [],
}
},
"de.sorunome.msc2409.push_ephemeral": true
};
const app = await start(config, registration);
app.listener.finaliseListeners();
@ -255,6 +295,12 @@ export class E2ETestEnv {
await this.app.bridgeApp.stop();
await this.app.listener.stop();
await this.app.storage.disconnect?.();
// Clear the redis DB.
if (this.config.cache?.redisUri) {
await new Redis(this.config.cache.redisUri).flushdb();
}
this.homeserver.users.forEach(u => u.client.stop());
await destroyHS(this.homeserver.id);
await rm(this.dir, { recursive: true });

View File

@ -1,9 +1,10 @@
import { MatrixClient } from "matrix-bot-sdk";
import { MatrixClient, MemoryStorageProvider, RustSdkCryptoStorageProvider, RustSdkCryptoStoreType } from "matrix-bot-sdk";
import { createHash, createHmac, randomUUID } from "crypto";
import { Homerunner } from "homerunner-client";
import { E2ETestMatrixClient } from "./e2e-test";
import path from "node:path";
const HOMERUNNER_IMAGE = process.env.HOMERUNNER_IMAGE || 'ghcr.io/element-hq/synapse/complement-synapse:latest';
const HOMERUNNER_IMAGE = process.env.HOMERUNNER_IMAGE || 'ghcr.io/element-hq/synapse/complement-synapse:nightly';
export const DEFAULT_REGISTRATION_SHARED_SECRET = (
process.env.REGISTRATION_SHARED_SECRET || 'complement'
);
@ -41,7 +42,7 @@ async function waitForHomerunner() {
}
}
export async function createHS(localparts: string[] = [], workerId: number): Promise<ComplementHomeServer> {
export async function createHS(localparts: string[] = [], workerId: number, cryptoRootPath?: string): Promise<ComplementHomeServer> {
await waitForHomerunner();
const appPort = 49600 + workerId;
@ -61,25 +62,35 @@ export async function createHS(localparts: string[] = [], workerId: number): Pro
SenderLocalpart: 'hookshot',
RateLimited: false,
...{ASToken: asToken,
HSToken: hsToken},
HSToken: hsToken,
SendEphemeral: true,
EnableEncryption: true},
}]
}],
}
});
const [homeserverName, homeserver] = Object.entries(blueprintResponse.homeservers)[0];
// Skip AS user.
const users = Object.entries(homeserver.AccessTokens)
const users = await Promise.all(Object.entries(homeserver.AccessTokens)
.filter(([_uId, accessToken]) => accessToken !== asToken)
.map(([userId, accessToken]) => ({
userId: userId,
accessToken,
deviceId: homeserver.DeviceIDs[userId],
client: new E2ETestMatrixClient(homeserver.BaseURL, accessToken),
})
);
.map(async ([userId, accessToken]) => {
const cryptoStore = cryptoRootPath ? new RustSdkCryptoStorageProvider(path.join(cryptoRootPath, userId), RustSdkCryptoStoreType.Sqlite) : undefined;
const client = new E2ETestMatrixClient(homeserver.BaseURL, accessToken, new MemoryStorageProvider(), cryptoStore);
if (cryptoStore) {
await client.crypto.prepare();
}
// Start syncing proactively.
await client.start();
return {
userId: userId,
accessToken,
deviceId: homeserver.DeviceIDs[userId],
client,
}
}
));
// Start syncing proactively.
await Promise.all(users.map(u => u.client.start()));
return {
users,
id: blueprint,
@ -119,7 +130,7 @@ export async function registerUser(
.update(password).update("\x00")
.update(user.admin ? 'admin' : 'notadmin')
.digest('hex');
return await fetch(registerUrl, { method: "POST", body: JSON.stringify(
const req = await fetch(registerUrl, { method: "POST", body: JSON.stringify(
{
nonce,
username: user.username,
@ -127,8 +138,10 @@ export async function registerUser(
admin: user.admin,
mac: hmac,
}
)}).then(res => res.json()).then(res => ({
mxid: (res as {user_id: string}).user_id,
client: new MatrixClient(homeserverUrl, (res as {access_token: string}).access_token),
})).catch(err => { console.log(err.response.body); throw new Error(`Failed to register user: ${err}`); });
)});
const res = await req.json() as {user_id: string, access_token: string};
return {
mxid: res.user_id,
client: new MatrixClient(homeserverUrl, res.access_token),
};
}

View File

@ -171,13 +171,10 @@ export default class BotUsersManager {
// Determine if an avatar update is needed
if (profile.avatar_url) {
try {
const res = await axios.get(
botUser.intent.underlyingClient.mxcToHttp(profile.avatar_url),
{ responseType: "arraybuffer" },
);
const res = await botUser.intent.underlyingClient.downloadContent(profile.avatar_url);
const currentAvatarImage = {
image: Buffer.from(res.data),
contentType: res.headers["content-type"],
image: res.data,
contentType: res.contentType,
};
if (
currentAvatarImage.image.equals(avatarImage.image)

View File

@ -104,7 +104,7 @@ export class RedisStorageProvider extends RedisStorageContextualProvider impleme
}
public async addRegisteredUser(userId: string) {
this.redis.sadd(REGISTERED_USERS_KEY, [userId]);
await this.redis.sadd(REGISTERED_USERS_KEY, [userId]);
}
public async isUserRegistered(userId: string): Promise<boolean> {
@ -112,7 +112,7 @@ export class RedisStorageProvider extends RedisStorageContextualProvider impleme
}
public async setTransactionCompleted(transactionId: string) {
this.redis.sadd(COMPLETED_TRANSACTIONS_KEY, [transactionId]);
await this.redis.sadd(COMPLETED_TRANSACTIONS_KEY, [transactionId]);
}
public async isTransactionCompleted(transactionId: string): Promise<boolean> {

View File

@ -45,7 +45,7 @@ export function getAppservice(config: BridgeConfig, registration: IAppserviceReg
},
storage: storage,
intentOptions: {
encryption: !!config.encryption,
encryption: !!cryptoStorage,
},
cryptoStorage: cryptoStorage,
});

View File

@ -13,6 +13,7 @@ import { Logger } from "matrix-appservice-bridge";
import { BridgeConfigCache } from "./sections/cache";
import { BridgeConfigGenericWebhooks, BridgeConfigQueue, BridgeGenericWebhooksConfigYAML } from "./sections";
import { GenericHookServiceConfig } from "../Connections";
import { BridgeConfigEncryption } from "./sections/encryption";
const log = new Logger("Config");
@ -356,8 +357,6 @@ interface BridgeConfigBridge {
mediaUrl?: string;
port: number;
bindAddress: string;
// Removed
pantalaimon?: never;
}
interface BridgeConfigWebhook {
@ -376,10 +375,7 @@ interface BridgeConfigBot {
displayname?: string;
avatar?: string;
}
interface BridgeConfigEncryption {
storagePath: string;
useLegacySledStore: boolean;
}
export interface BridgeConfigServiceBot {
localpart: string;
@ -415,7 +411,11 @@ export interface BridgeConfigRoot {
bot?: BridgeConfigBot;
bridge: BridgeConfigBridge;
cache?: BridgeConfigCache;
experimentalEncryption?: BridgeConfigEncryption;
/**
* @deprecated Old, unsupported encryption propety.
*/
experimentalEncryption?: never;
encryption?: BridgeConfigEncryption;
feeds?: BridgeConfigFeedsYAML;
figma?: BridgeConfigFigma;
generic?: BridgeGenericWebhooksConfigYAML;
@ -443,9 +443,7 @@ export class BridgeConfig {
For encryption to work, this must be configured.`, true)
public readonly cache?: BridgeConfigCache;
@configKey(`Configuration for encryption support in the bridge.
If omitted, encryption support will be disabled.
This feature is HIGHLY EXPERIMENTAL AND SUBJECT TO CHANGE.
For more details, see https://github.com/matrix-org/matrix-hookshot/issues/594.`, true)
If omitted, encryption support will be disabled.`, true)
public readonly encryption?: BridgeConfigEncryption;
@configKey(`Message queue configuration options for large scale deployments.
For encryption to work, this must not be configured.`, true)
@ -500,6 +498,9 @@ export class BridgeConfig {
constructor(configData: BridgeConfigRoot, env?: {[key: string]: string|undefined}) {
this.logging = configData.logging || {
level: "info",
}
this.bridge = configData.bridge;
assert.ok(this.bridge);
this.github = configData.github && new BridgeConfigGitHub(configData.github);
@ -548,13 +549,11 @@ export class BridgeConfig {
}
}
this.encryption = configData.experimentalEncryption;
this.logging = configData.logging || {
level: "info",
if (configData.experimentalEncryption) {
throw new ConfigError("experimentalEncryption", `This key is now called 'encryption'. Please adjust your config file.`)
}
this.encryption = configData.encryption && new BridgeConfigEncryption(configData.encryption, this.cache, this.queue);
this.widgets = configData.widgets && new BridgeWidgetConfig(configData.widgets);
this.sentry = configData.sentry;
@ -640,37 +639,6 @@ export class BridgeConfig {
log.warn("The `widgets.openIdOverrides` config value SHOULD NOT be used in a production environment.")
}
if (this.bridge.pantalaimon) {
throw new ConfigError("bridge.pantalaimon", "Pantalaimon support has been removed. Encrypted bridges should now use the `experimentalEncryption` config option");
}
if (this.encryption) {
log.warn(`
You have enabled encryption support in the bridge. This feature is HIGHLY EXPERIMENTAL AND SUBJECT TO CHANGE.
For more details, see https://github.com/matrix-org/matrix-hookshot/issues/594.
`);
if (!this.encryption.storagePath) {
throw new ConfigError("experimentalEncryption.storagePath", "The crypto storage path must not be empty.");
}
if (this.encryption.useLegacySledStore) {
throw new ConfigError(
"experimentalEncryption.useLegacySledStore", `
The Sled crypto store format is no longer supported.
Please back up your crypto store at ${this.encryption.storagePath},
remove "useLegacySledStore" from your configuration file, and restart Hookshot.
`);
}
if (!this.cache) {
throw new ConfigError("cache", "Encryption requires the Redis cache to be enabled.");
}
if (this.queue) {
throw new ConfigError("queue", "Encryption does not support message queues.");
}
}
if (this.figma?.overrideUserId) {
log.warn("The `figma.overrideUserId` config value is deprecated. A service bot should be configured instead.");
}

View File

@ -15,9 +15,6 @@ export const DefaultConfigRoot: BridgeConfigRoot = {
port: 9993,
bindAddress: "127.0.0.1",
},
queue: {
redisUri: "redis://localhost:6379",
},
cache: {
redisUri: "redis://localhost:6379",
},
@ -154,6 +151,9 @@ export const DefaultConfigRoot: BridgeConfigRoot = {
sentry: {
dsn: "https://examplePublicKey@o0.ingest.sentry.io/0",
environment: "production"
},
encryption: {
storagePath: "./cryptostore"
}
};

View File

@ -0,0 +1,27 @@
import { ConfigError } from "../../errors";
import { configKey } from "../Decorators";
import { BridgeConfigCache } from "./cache";
import { BridgeConfigQueue } from "./queue";
interface BridgeConfigEncryptionYAML {
storagePath: string;
}
export class BridgeConfigEncryption {
@configKey("Path to the directory used to store encryption files. These files must be persist between restarts of the service.")
public readonly storagePath: string;
constructor(config: BridgeConfigEncryptionYAML, cache: unknown|undefined, queue: unknown|undefined) {
if (typeof config.storagePath !== "string" || !config.storagePath) {
throw new ConfigError("encryption.storagePath", "The crypto storage path must not be empty.");
}
this.storagePath = config.storagePath;
if (!cache) {
throw new ConfigError("cache", "Encryption requires the Redis cache to be enabled.");
}
if (queue) {
throw new ConfigError("queue", "Encryption does not support message queues.");
}
}
}

View File

@ -60,7 +60,7 @@ fn parse_channel_to_js_result(channel: &Channel) -> JsRssChannel {
.and_then(|i| i.permalink.then(|| i.value.to_string()))
}),
id: item.guid().map(|f| f.value().to_string()),
id_is_permalink: item.guid().map_or(false, |f| f.is_permalink()),
id_is_permalink: item.guid().is_some_and(|f| f.is_permalink()),
pubdate: item.pub_date().map(String::from),
summary: item.description().map(String::from),
author: item.author().map(String::from),
@ -106,7 +106,7 @@ fn parse_feed_to_js_result(feed: &Feed) -> JsRssChannel {
link: item
.links()
.iter()
.find(|l| l.mime_type.as_ref().map_or(false, |t| t == "text/html"))
.find(|l| l.mime_type.as_ref().is_some_and(|t| t == "text/html"))
.or_else(|| item.links().first())
.map(|f| f.href.clone()),
id: Some(item.id.clone()),

View File

@ -25,7 +25,7 @@ describe("IntentUtils", () => {
return;
}
expect(roomId).to.equal(ROOM_ID);
throw new MatrixError({ errcode: "M_FORBIDDEN", error: "Test forced error"}, 401)
throw new MatrixError({ errcode: "M_FORBIDDEN", error: "Test forced error"}, 401, { })
};
// This should invite the puppet user.
@ -46,7 +46,7 @@ describe("IntentUtils", () => {
// This should fail the first time, then pass once we've tried to invite the user
targetIntent.ensureJoined = () => {
throw new MatrixError({ errcode: "FORCED_FAILURE", error: "Test forced error"}, 500)
throw new MatrixError({ errcode: "FORCED_FAILURE", error: "Test forced error"}, 500, { })
};
try {
ensureUserIsInRoom(targetIntent, matrixClient, ROOM_ID);

View File

@ -38,9 +38,13 @@ describe("Config/BridgeConfig", () => {
expect(config.cache?.redisUri).to.equal("redis://bark:6379");
});
it("with monolithic disabled", () => {
const config = new BridgeConfig({ ...DefaultConfigRoot, queue: {
monolithic: false
}});
const config = new BridgeConfig({
...DefaultConfigRoot,
encryption: undefined,
queue: {
monolithic: false
}
});
expect(config.queue).to.deep.equal({
monolithic: false,
});
@ -49,9 +53,13 @@ describe("Config/BridgeConfig", () => {
});
describe("will handle the queue option", () => {
it("with redisUri", () => {
const config = new BridgeConfig({ ...DefaultConfigRoot, queue: {
redisUri: "redis://localhost:6379"
}, cache: undefined});
const config = new BridgeConfig({ ...DefaultConfigRoot,
encryption: undefined,
queue: {
redisUri: "redis://localhost:6379"
},
cache: undefined
});
expect(config.queue).to.deep.equal({
redisUri: "redis://localhost:6379"
});

View File

@ -310,7 +310,7 @@ describe("GenericHookConnection", () => {
return roomId;
}
expect(roomId).to.equal(ROOM_ID);
throw new MatrixError({ errcode: "M_FORBIDDEN", error: "Test forced error"}, 401)
throw new MatrixError({ errcode: "M_FORBIDDEN", error: "Test forced error"}, 401, { })
};
// This should invite the puppet user.
@ -333,7 +333,7 @@ describe("GenericHookConnection", () => {
// This should fail the first time, then pass once we've tried to invite the user
intent.ensureJoined = () => {
throw new MatrixError({ errcode: "FORCED_FAILURE", error: "Test forced error"}, 500)
throw new MatrixError({ errcode: "FORCED_FAILURE", error: "Test forced error"}, 500, { })
};
try {
// regression test covering https://github.com/matrix-org/matrix-hookshot/issues/625

View File

@ -39,7 +39,7 @@ export class MatrixClientMock {
throw new MatrixError({
errcode: 'M_NOT_FOUND',
error: 'Test error: No account data',
}, 404);
}, 404, { });
}
async setRoomAccountData(key: string, roomId: string, value: string): Promise<void> {

View File

@ -1027,14 +1027,6 @@
dependencies:
"@lezer/common" "^1.0.0"
"@matrix-org/matrix-sdk-crypto-nodejs@0.1.0-beta.11":
version "0.1.0-beta.11"
resolved "https://registry.yarnpkg.com/@matrix-org/matrix-sdk-crypto-nodejs/-/matrix-sdk-crypto-nodejs-0.1.0-beta.11.tgz#537cd7a7bbce1d9745b812a5a7ffa9a5944e146c"
integrity sha512-z5adcQo4o0UAry4zs6JHGxbTDlYTUMKUfpOpigmso65ETBDumbeTSQCWRw8UeUV7aCAyVoHARqDTol9SrauEFA==
dependencies:
https-proxy-agent "^5.0.1"
node-downloader-helper "^2.1.5"
"@matrix-org/matrix-sdk-crypto-nodejs@0.1.0-beta.6":
version "0.1.0-beta.6"
resolved "https://registry.yarnpkg.com/@matrix-org/matrix-sdk-crypto-nodejs/-/matrix-sdk-crypto-nodejs-0.1.0-beta.6.tgz#0ecae51103ee3c107af0d6d0738f33eb7cc9857e"
@ -1043,6 +1035,14 @@
https-proxy-agent "^5.0.1"
node-downloader-helper "^2.1.5"
"@matrix-org/matrix-sdk-crypto-nodejs@0.2.0-beta.1":
version "0.2.0-beta.1"
resolved "https://registry.yarnpkg.com/@matrix-org/matrix-sdk-crypto-nodejs/-/matrix-sdk-crypto-nodejs-0.2.0-beta.1.tgz#b696707ccfa944cfed3c96cf7e54799b0f1e3329"
integrity sha512-CgbOKORfD6dvYgQTPhfN73H1RbQknrFkMnRRwCIJMt15iL2AF1gEowgbrlGhkbG6gNng4CgPnKs1iHKCRrhvmA==
dependencies:
https-proxy-agent "^5.0.1"
node-downloader-helper "^2.1.5"
"@mdn/browser-compat-data@^5.2.34", "@mdn/browser-compat-data@^5.3.13":
version "5.5.1"
resolved "https://registry.yarnpkg.com/@mdn/browser-compat-data/-/browser-compat-data-5.5.1.tgz#cd480874a4ebb97010f488feb8204ac035a86332"
@ -1933,7 +1933,7 @@
"@types/range-parser" "*"
"@types/send" "*"
"@types/express@^4.17.13", "@types/express@^4.17.14", "@types/express@^4.17.20":
"@types/express@^4.17.13", "@types/express@^4.17.14", "@types/express@^4.17.21":
version "4.17.21"
resolved "https://registry.yarnpkg.com/@types/express/-/express-4.17.21.tgz#c26d4a151e60efe0084b23dc3369ebc631ed192d"
integrity sha512-ejlPM315qwLpaQlQDTjPdsUFSc6ZsP4AN6AlWnogPjQ7CVi7PYF3YVz+CY3jE2pwYf7E/7HlDAN0rV2GxTG0HQ==
@ -5949,13 +5949,13 @@ matrix-appservice@^2.0.0:
request-promise "^4.2.6"
sanitize-html "^2.8.0"
"matrix-bot-sdk@npm:@vector-im/matrix-bot-sdk@^0.7.0-specific-device-2":
version "0.7.0-specific-device-2"
resolved "https://registry.yarnpkg.com/@vector-im/matrix-bot-sdk/-/matrix-bot-sdk-0.7.0-specific-device-2.tgz#3682e14708979a6f24cc19f3103e3292ed08bbea"
integrity sha512-97h2tIlcK6/3wEuLN3x6/LM9TITVISnnbjUo/9nVbqkDvSQ2TNFURxfAqjUfFmgQwo0o3KnhvaS6dZITrBvj6A==
"matrix-bot-sdk@npm:@vector-im/matrix-bot-sdk@v0.7.1-element.6":
version "0.7.1-element.6"
resolved "https://registry.yarnpkg.com/@vector-im/matrix-bot-sdk/-/matrix-bot-sdk-0.7.1-element.6.tgz#d1f8a86d3bd60084d92d150f42a48b25199871e1"
integrity sha512-0KfyTpQV5eyY4vPUZW89t7EZf1YF0UyFkyYqpsxL/6S7XIlbTMC4onod7vx/QpKC0lSREmwIiXx2JSjExP6CIw==
dependencies:
"@matrix-org/matrix-sdk-crypto-nodejs" "0.1.0-beta.11"
"@types/express" "^4.17.20"
"@matrix-org/matrix-sdk-crypto-nodejs" "0.2.0-beta.1"
"@types/express" "^4.17.21"
another-json "^0.2.0"
async-lock "^1.4.0"
chalk "4"