Fix Docker builds hanging/OOMing (#535)

* Add network timeouts to Docker builds

* Split cargo build out

* Put cargo in the right place

* debug usage

* install time

* Fix docker build OOMing in CI for arm64 builds

Yoinked from synapse:
- https://github.com/matrix-org/synapse/pull/14173
- 424d1d28cc

* Update changelog

Co-authored-by: Half-Shot <will@half-shot.uk>

.github/workflows/docker-hub-latest.yml

@@ -34,4 +34,9 @@ jobs:
           platforms: ${{ env.PLATFORMS }}
           push: ${{ env.PUSH }}
           tags: |
-            ${{ env.DOCKER_NAMESPACE }}/matrix-hookshot:latest
+            ${{ env.DOCKER_NAMESPACE }}/matrix-hookshot:latest
+
+          # arm64 builds OOM without the git fetch setting. c.f.
+          # https://github.com/rust-lang/cargo/issues/10583
+          build-args: |
+            CARGO_NET_GIT_FETCH_WITH_CLI=true

.github/workflows/docker-hub-release.yml

@@ -36,3 +36,8 @@ jobs:
           push: true
           tags: |
             ${{ env.DOCKER_NAMESPACE }}/matrix-hookshot:${{ env.RELEASE_VERSION }}
+
+          # arm64 builds OOM without the git fetch setting. c.f.
+          # https://github.com/rust-lang/cargo/issues/10583
+          build-args: |
+            CARGO_NET_GIT_FETCH_WITH_CLI=true

Dockerfile

@@ -7,17 +7,23 @@ FROM node:16 AS builder
 RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal
 ENV PATH="/root/.cargo/bin:${PATH}"
 
+# arm64 builds consume a lot of memory if `CARGO_NET_GIT_FETCH_WITH_CLI` is not
+# set to true, so we expose it as a build-arg.
+ARG CARGO_NET_GIT_FETCH_WITH_CLI=false
+ENV CARGO_NET_GIT_FETCH_WITH_CLI=$CARGO_NET_GIT_FETCH_WITH_CLI
+
 # Needed to build rust things for matrix-sdk-crypto-nodejs
 # See https://github.com/matrix-org/matrix-rust-sdk-bindings/blob/main/crates/matrix-sdk-crypto-nodejs/release/Dockerfile.linux#L5-L6
-RUN apt-get update && apt-get install -y build-essential cmake
+RUN apt-get update && apt-get install -y build-essential cmake time
 
 WORKDIR /src
 
 COPY package.json yarn.lock ./
-RUN yarn --ignore-scripts --pure-lockfile
+RUN yarn --ignore-scripts --pure-lockfile --network-timeout 600000
 
 COPY . ./
 
+RUN /usr/bin/time -v cargo build --jobs 1
 # Workaround: Need to install esbuild manually https://github.com/evanw/esbuild/issues/462#issuecomment-771328459
 RUN node node_modules/esbuild/install.js
 RUN yarn build
@@ -31,7 +37,7 @@ WORKDIR /bin/matrix-hookshot
 COPY --from=builder /src/yarn.lock /src/package.json ./
 
 
-RUN yarn --production --pure-lockfile && yarn cache clean
+RUN yarn --network-timeout 600000 --production --pure-lockfile && yarn cache clean
 
 COPY --from=builder /src/lib ./
 COPY --from=builder /src/public ./public

changelog.d/535.misc

@@ -0,0 +1 @@
+Increase network timeout for Docker builds, and fix Docker build OOMing in CI for arm64 builds.