Compare commits

..

7 commits

Author SHA1 Message Date
dependabot[bot]
4734ab2568
Bump github.com/docker/docker
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 26.1.5+incompatible to 28.3.3+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v26.1.5...v28.3.3)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-version: 28.3.3+incompatible
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-13 16:07:11 +00:00
Kegan Dougal
3197b09989
v0.15.1 (#3632)
Some checks failed
Dendrite / WASM build test (push) Has been cancelled
Dendrite / Linting (push) Has been cancelled
Dendrite / Unit tests (push) Has been cancelled
Dendrite / Build for Linux (push) Has been cancelled
Dendrite / Build for Windows (push) Has been cancelled
Dendrite / Initial tests passed (push) Has been cancelled
Dendrite / Integration tests (push) Has been cancelled
Dendrite / Upgrade tests (push) Has been cancelled
Dendrite / Upgrade tests from HEAD-2 (push) Has been cancelled
Dendrite / Sytest (SQLite Cgo) (push) Has been cancelled
Dendrite / Sytest (PostgreSQL) (push) Has been cancelled
Dendrite / Sytest (SQLite native) (push) Has been cancelled
Dendrite / Complement (PostgreSQL) (push) Has been cancelled
Dendrite / Complement (SQLite native) (push) Has been cancelled
Dendrite / Complement (SQLite Cgo) (push) Has been cancelled
Dendrite / Integration tests passed (push) Has been cancelled
Dendrite / Update Docker images (push) Has been cancelled
To pull in https://github.com/element-hq/dendrite/pull/3630

Also pulls in a bunch of bug fixes on v12 rooms, which testing did not
catch.

`FAILURE: #655: Server rejects invalid JSON in a version 6 room` is an
expected fail now.
2025-08-13 17:05:44 +01:00
Kegan Dougal
13fb97c1ab
Bump GMSL to pull in v12 sorting bugfix (#3633)
Also keeps processing PDUs even if one of them was bad c.f
https://github.com/element-hq/synapse/issues/7543 suspect moderation
tooling is causing invalid PDUs to be sent, meaning this is now expected
to fail:
```
FAILURE: #655: Server rejects invalid JSON in a version 6 room
```
2025-08-13 13:56:34 +01:00
Vivianne
0945374736
Fix bad merge for prior PR which caused more stalls (#3630)
Some checks are pending
Dendrite / Unit tests (push) Waiting to run
Dendrite / WASM build test (push) Waiting to run
Dendrite / Linting (push) Waiting to run
Dendrite / Build for Linux (push) Waiting to run
Dendrite / Build for Windows (push) Waiting to run
Dendrite / Initial tests passed (push) Blocked by required conditions
Dendrite / Integration tests (push) Blocked by required conditions
Dendrite / Upgrade tests (push) Blocked by required conditions
Dendrite / Upgrade tests from HEAD-2 (push) Blocked by required conditions
Dendrite / Sytest (SQLite Cgo) (push) Blocked by required conditions
Dendrite / Sytest (SQLite native) (push) Blocked by required conditions
Dendrite / Complement (PostgreSQL) (push) Blocked by required conditions
Dendrite / Complement (SQLite native) (push) Blocked by required conditions
Dendrite / Sytest (PostgreSQL) (push) Blocked by required conditions
Dendrite / Update Docker images (push) Blocked by required conditions
Dendrite / Complement (SQLite Cgo) (push) Blocked by required conditions
Dendrite / Integration tests passed (push) Blocked by required conditions
Due to a bad merge, timeouts were not early-returning and so could cause
a stall in the roomserver. Fix the bad merge and early-out.

Minor fix to a bad merge of my change for #3484 

This also incorporates a suggested tweak to the prior PR #3588 to avoid
holding the lock unnecessarily.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off
below](https://element-hq.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Vivianne Langdon <puttabutta@gmail.com>`

---------

Signed-off-by: Vivianne Langdon <puttabutta@gmail.com>
2025-08-13 09:53:33 +00:00
Kegan Dougal
5e05f3090b
v0.15.0 (#3627)
Some checks are pending
Dendrite / WASM build test (push) Waiting to run
Dendrite / Linting (push) Waiting to run
Dendrite / Unit tests (push) Waiting to run
Dendrite / Build for Linux (push) Waiting to run
Dendrite / Build for Windows (push) Waiting to run
Dendrite / Initial tests passed (push) Blocked by required conditions
Dendrite / Integration tests (push) Blocked by required conditions
Dendrite / Upgrade tests (push) Blocked by required conditions
Dendrite / Upgrade tests from HEAD-2 (push) Blocked by required conditions
Dendrite / Sytest (SQLite Cgo) (push) Blocked by required conditions
Dendrite / Sytest (PostgreSQL) (push) Blocked by required conditions
Dendrite / Sytest (SQLite native) (push) Blocked by required conditions
Dendrite / Complement (PostgreSQL) (push) Blocked by required conditions
Dendrite / Complement (SQLite native) (push) Blocked by required conditions
Dendrite / Complement (SQLite Cgo) (push) Blocked by required conditions
Dendrite / Integration tests passed (push) Blocked by required conditions
Dendrite / Update Docker images (push) Blocked by required conditions
Also updates the README to:
 - highlight Dendrite is in maintenance mode
- remove references to being "scalable" as we eventually decided to
focus on smaller deployments not huge ones.
- remove the progress section as it was horribly outdated and in general
Dendrite is roughly feature complete with synapse (with the exception of
sliding sync and OIDC, which is also now mentioned explicitly)
2025-08-12 20:44:48 +01:00
Kegan Dougal
d4c5804ad2
bugfix: ensure we release the lock (#3628)
The `case nats.ErrConsumerDeleted, nats.ErrConsumerNotFound:` bit was
merge conflicted in https://github.com/element-hq/dendrite/pull/3588 so
it broke the locking order.
2025-08-12 20:08:30 +01:00
Vivianne
f24688af11
Adjust roomserver locks, don't unsubscribe if new event is inflight (#3588)
This should fix #3484 -- at the very least, it has resolved the issues
we've had on our instance. I've extended the lock so it surrounds the
unsubscribe as well as a new check if the latest sequential ID seen by
the ephemeral thread is newer than the sequential ID seen by the durable
thread. This solves a race where the unsubscribe happened while a new
message was inflight, and so the message was not handled.

This is a fix for a race condition that has been pretty unreliable to
reproduce manually, so I don't know if there's a good way to add a
reliable automated test for it. If you have any ideas I'm open to it.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off
below](https://element-hq.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Vivianne Langdon <puttabutta@gmail.com>`

---------

Signed-off-by: Vivianne Langdon <puttabutta@gmail.com>
Co-authored-by: Kegan Dougal <7190048+kegsay@users.noreply.github.com>
2025-08-12 14:06:42 +01:00
10 changed files with 103 additions and 54 deletions

View file

@ -1,5 +1,34 @@
# Changelog
## Dendrite 0.15.1 (2025-08-13)
### Bug fixes
- Fixed an issue which could cause Dendrite to become unresponsive for minutes at a time. (contributed by [viviicat](https://github.com/viviicat))
- Fixed an issue which prevented joining v12 rooms in some circumstances.
- Fixed an issue which prevented sending invites to v12 rooms.
- Fixed an issue which prevented some clients from syncing v12 rooms.
- Fixed an issue where a single badly formed PDU would block entire transactions of PDUs being processed. See https://github.com/element-hq/synapse/issues/7543 for the related issue on Synapse.
## Dendrite 0.15.0 (2025-08-12)
### ⚠ Important
This is a security release, adding support for [room version 12](https://matrix.org/blog/2025/08/security-release/).
### Features
- Add support for [MSC4163](https://github.com/matrix-org/matrix-spec-proposals/pull/4163).
- Add support for [MSC3967](https://github.com/matrix-org/matrix-spec-proposals/pull/3967).
- Add support for room version 12.
### Bug fixes
- Refactored NATS JetStream code to gracefully handle more potential errors. (contributed by [neilalexander](https://github.com/neilalexander))
- Refactored NATS startup and readiness checking. (contributed by [neilalexander](https://github.com/neilalexander))
- Updated NATS to 2.11.7. (contributed by [neilalexander](https://github.com/neilalexander))
- Fixed an issue which could cause Dendrite to become unresponsive for minutes at a time. (contributed by [viviicat](https://github.com/viviicat))
- Order events when backfilling to reduce the amount of unecessary `/state_ids` requests.
- Gracefully handle incorrect sync filter JSON.
- Fixed an issue which prevented device deletion working correctly. (contributed by [robinsdan](https://github.com/robinsdan))
## Dendrite 0.14.1 (2025-01-16)
### ⚠ Important

View file

@ -2,14 +2,14 @@
[![Build status](https://github.com/element-hq/dendrite/actions/workflows/dendrite.yml/badge.svg?event=push)](https://github.com/element-hq/dendrite/actions/workflows/dendrite.yml) [![Dendrite](https://img.shields.io/matrix/dendrite:matrix.org.svg?label=%23dendrite%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite:matrix.org) [![Dendrite Dev](https://img.shields.io/matrix/dendrite-dev:matrix.org.svg?label=%23dendrite-dev%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite-dev:matrix.org)
Dendrite is a second-generation Matrix homeserver written in Go.
It intends to provide an **efficient**, **reliable** and **scalable** alternative to [Synapse](https://github.com/matrix-org/synapse):
Dendrite is a second-generation Matrix homeserver written in Go. It is currently in maintenance mode,
meaning only security fixes are being applied, for example [room version 12](https://matrix.org/blog/2025/08/security-release/).
It intends to provide an **efficient** and **reliable** alternative to [Synapse](https://github.com/matrix-org/synapse):
- Efficient: A small memory footprint with better baseline performance than an out-of-the-box Synapse.
- Reliable: Implements the Matrix specification as written, using the
[same test suite](https://github.com/matrix-org/sytest) as Synapse as well as
a [brand new Go test suite](https://github.com/matrix-org/complement).
- Scalable: can run on multiple machines and eventually scale to massive homeserver deployments.
[same](https://github.com/matrix-org/sytest) test [suites](https://github.com/matrix-org/complement) as Synapse.
Dendrite is **beta** software, which means:
@ -31,6 +31,10 @@ If you have further questions, please take a look at [our FAQ](docs/FAQ.md) or j
- **[#dendrite-dev:matrix.org](https://matrix.to/#/#dendrite-dev:matrix.org)** - The place for developers, where all Dendrite development discussion happens
- **[#dendrite-alerts:matrix.org](https://matrix.to/#/#dendrite-alerts:matrix.org)** - Release notifications and important info, highly recommended for all Dendrite server admins
Dendrite does not currently support the following MSCs, which impacts the ability to use Element X with Dendrite servers:
- [MSC4186](https://github.com/matrix-org/matrix-spec-proposals/pull/4186): Simplified Sliding Sync
- [MSC3861](https://github.com/matrix-org/matrix-spec-proposals/pull/3861): Next-gen auth OIDC
## Requirements
See the [Planning your Installation](https://element-hq.github.io/dendrite/installation/planning) page for
@ -83,36 +87,6 @@ $ ./bin/create-account --config dendrite.yaml --username alice
Then point your favourite Matrix client at `http://localhost:8008` or `https://localhost:8448`.
## Progress
We use a script called "Are We Synapse Yet" which checks Sytest compliance rates. Sytest is a black-box homeserver
test rig with around 900 tests. The script works out how many of these tests are passing on Dendrite and it
updates with CI. As of January 2023, we have 100% server-server parity with Synapse, and the client-server parity is at 93% , though check
CI for the latest numbers. In practice, this means you can communicate locally and via federation with Synapse
servers such as matrix.org reasonably well, although there are still some missing features (like SSO and Third-party ID APIs).
We are prioritising features that will benefit single-user homeservers first (e.g Receipts, E2E) rather
than features that massive deployments may be interested in (OpenID, Guests, Admin APIs, AS API).
This means Dendrite supports amongst others:
- Core room functionality (creating rooms, invites, auth rules)
- Room versions 1 to 10 supported
- Backfilling locally and via federation
- Accounts, profiles and devices
- Published room lists
- Typing
- Media APIs
- Redaction
- Tagging
- Context
- E2E keys and device lists
- Receipts
- Push
- Guests
- User Directory
- Presence
- Fulltext search
## Contributing
We would be grateful for any help on issues marked as

2
go.mod
View file

@ -25,7 +25,7 @@ require (
github.com/matrix-org/dugong v0.0.0-20210921133753-66e6b1c67e2e
github.com/matrix-org/go-sqlite3-js v0.0.0-20220419092513-28aa791a1c91
github.com/matrix-org/gomatrix v0.0.0-20220926102614-ceba4d9f7530
github.com/matrix-org/gomatrixserverlib v0.0.0-20250811193806-b7e0e0824751
github.com/matrix-org/gomatrixserverlib v0.0.0-20250813150445-9f5070a65744
github.com/matrix-org/pinecone v0.11.1-0.20230810010612-ea4c33717fd7
github.com/matrix-org/util v0.0.0-20221111132719-399730281e66
github.com/mattn/go-sqlite3 v1.14.28

4
go.sum
View file

@ -241,8 +241,8 @@ github.com/matrix-org/go-sqlite3-js v0.0.0-20220419092513-28aa791a1c91 h1:s7fexw
github.com/matrix-org/go-sqlite3-js v0.0.0-20220419092513-28aa791a1c91/go.mod h1:e+cg2q7C7yE5QnAXgzo512tgFh1RbQLC0+jozuegKgo=
github.com/matrix-org/gomatrix v0.0.0-20220926102614-ceba4d9f7530 h1:kHKxCOLcHH8r4Fzarl4+Y3K5hjothkVW5z7T1dUM11U=
github.com/matrix-org/gomatrix v0.0.0-20220926102614-ceba4d9f7530/go.mod h1:/gBX06Kw0exX1HrwmoBibFA98yBk/jxKpGVeyQbff+s=
github.com/matrix-org/gomatrixserverlib v0.0.0-20250811193806-b7e0e0824751 h1:x1pC7Nt1Qb24q9WtPybMHWo2uVFTzCKtlUAzarju8bk=
github.com/matrix-org/gomatrixserverlib v0.0.0-20250811193806-b7e0e0824751/go.mod h1:b6KVfDjXjA5Q7vhpOaMqIhFYvu5BuFVZixlNeTV/CLc=
github.com/matrix-org/gomatrixserverlib v0.0.0-20250813150445-9f5070a65744 h1:5GvC2FD9O/PhuyY95iJQdNYHbDioEhMWdeMP9maDUL8=
github.com/matrix-org/gomatrixserverlib v0.0.0-20250813150445-9f5070a65744/go.mod h1:b6KVfDjXjA5Q7vhpOaMqIhFYvu5BuFVZixlNeTV/CLc=
github.com/matrix-org/pinecone v0.11.1-0.20230810010612-ea4c33717fd7 h1:6t8kJr8i1/1I5nNttw6nn1ryQJgzVlBmSGgPiiaTdw4=
github.com/matrix-org/pinecone v0.11.1-0.20230810010612-ea4c33717fd7/go.mod h1:ReWMS/LoVnOiRAdq9sNUC2NZnd1mZkMNB52QhpTRWjg=
github.com/matrix-org/util v0.0.0-20221111132719-399730281e66 h1:6z4KxomXSIGWqhHcfzExgkH3Z3UkIXry4ibJS4Aqz2Y=

View file

@ -134,6 +134,8 @@ func (t *TxnReq) ProcessTransaction(ctx context.Context) (*fclient.RespSend, *ut
}
event, err := verImpl.NewEventFromUntrustedJSON(pdu)
if err != nil {
/* Do not reject the entire transaction for a single bad PDU, that's dumb.
if _, ok := err.(gomatrixserverlib.BadJSONError); ok {
// Room version 6 states that homeservers should strictly enforce canonical JSON
// on PDUs.
@ -146,7 +148,7 @@ func (t *TxnReq) ProcessTransaction(ctx context.Context) (*fclient.RespSend, *ut
Code: 400,
JSON: spec.BadJSON("PDU contains bad JSON"),
}
}
} */
util.GetLogger(ctx).WithError(err).Debugf("Transaction: Failed to parse event JSON of event %s", string(pdu))
continue
}

View file

@ -17,7 +17,7 @@ var build string
const (
VersionMajor = 0
VersionMinor = 14
VersionMinor = 15
VersionPatch = 1
VersionTag = "" // example: "rc1"

View file

@ -101,9 +101,12 @@ type worker struct {
roomID string
subscription *nats.Subscription
sentryHub *sentry.Hub
ephemeralSeq uint64
// last seq we fully processed
durableSeq uint64
}
func (r *Inputer) startWorkerForRoom(roomID string) {
func (r *Inputer) startWorkerForRoom(roomID string, seq uint64) {
v, loaded := r.workers.LoadOrStore(roomID, &worker{
r: r,
roomID: roomID,
@ -112,6 +115,9 @@ func (r *Inputer) startWorkerForRoom(roomID string) {
w := v.(*worker)
w.Lock()
defer w.Unlock()
w.ephemeralSeq = seq
if !loaded || w.subscription == nil {
streamName := r.Cfg.Matrix.JetStream.Prefixed(jetstream.InputRoomEvent)
consumer := r.Cfg.Matrix.JetStream.Prefixed("RoomInput" + jetstream.Tokenise(w.roomID))
@ -226,7 +232,8 @@ func (r *Inputer) Start() error {
"", // This is blank because we specified it in BindStream.
func(m *nats.Msg) {
roomID := m.Header.Get(jetstream.RoomID)
r.startWorkerForRoom(roomID)
meta, _ := m.Metadata()
r.startWorkerForRoom(roomID, meta.Sequence.Stream)
_ = m.Ack()
},
nats.HeadersOnly(),
@ -264,7 +271,7 @@ func (w *worker) _next() {
})
msgs, err := w.subscription.Fetch(1, nats.Context(ctx))
switch err {
case nil, nats.ErrTimeout, context.DeadlineExceeded, context.Canceled:
case nil:
// Is the server shutting down? If so, stop processing.
if w.r.ProcessContext.Context().Err() != nil {
return
@ -272,14 +279,33 @@ func (w *worker) _next() {
// Make sure that once we're done here, we queue up another call
// to _next in the inbox.
defer w.Act(nil, w._next)
// If no error was reported, but we didn't get exactly one message,
// then skip over this and try again on the next iteration.
if len(msgs) != 1 {
case nats.ErrTimeout, context.DeadlineExceeded, context.Canceled:
// Is the server shutting down? If so, stop processing.
if w.r.ProcessContext.Context().Err() != nil {
return
}
// The context exceeded, so we've been waiting for more than a
// minute for activity in this room. At this point we will shut
// down the subscriber to free up resources. It'll get started
// again if new activity happens.
w.Lock()
// inside the lock, let's check if the ephemeral consumer saw something new!
// If so, we do have new messages after all, they just came at a bad time.
if w.ephemeralSeq > w.durableSeq {
w.Unlock()
w.Act(nil, w._next)
return
}
if err = w.subscription.Unsubscribe(); err != nil {
logrus.WithError(err).Errorf("Failed to unsubscribe to stream for room %q", w.roomID)
}
w.subscription = nil
w.Unlock()
return
case nats.ErrConsumerDeleted, nats.ErrConsumerNotFound:
w.Lock()
defer w.Unlock()
// The consumer is gone, therefore it's reached the inactivity
// threshold. Clean up and stop processing at this point, if a
// new event comes in for this room then the ordered consumer
@ -287,21 +313,20 @@ func (w *worker) _next() {
if err = w.subscription.Unsubscribe(); err != nil {
logrus.WithError(err).Errorf("Failed to unsubscribe to stream for room %q", w.roomID)
}
w.Lock()
w.subscription = nil
w.Unlock()
return
default:
// Something went wrong while trying to fetch the next event
// from the queue. In which case, we'll shut down the subscriber
// and wait to be notified about new room activity again. Maybe
// the problem will be corrected by then.
// atomically clear the subscription and unsubscribe
w.Lock()
logrus.WithError(err).Errorf("Failed to get next stream message for room %q", w.roomID)
if err = w.subscription.Unsubscribe(); err != nil {
logrus.WithError(err).Errorf("Failed to unsubscribe to stream for room %q", w.roomID)
}
w.Lock()
w.subscription = nil
w.Unlock()
return
@ -310,10 +335,19 @@ func (w *worker) _next() {
// Since we either Ack() or Term() the message at this point, we can defer decrementing the room backpressure
defer roomserverInputBackpressure.With(prometheus.Labels{"room_id": w.roomID}).Dec()
// If no error was reported, but we didn't get exactly one message,
// then skip over this and try again on the next iteration.
if len(msgs) != 1 {
return
}
// Try to unmarshal the input room event. If the JSON unmarshalling
// fails then we'll terminate the message — this notifies NATS that
// we are done with the message and never want to see it again.
msg := msgs[0]
meta, _ := msg.Metadata()
w.durableSeq = meta.Sequence.Stream
var inputRoomEvent api.InputRoomEvent
if err = json.Unmarshal(msg.Data, &inputRoomEvent); err != nil {
// using AckWait here makes the call synchronous; 5 seconds is the default value used by NATS

View file

@ -31,7 +31,11 @@ type QueryState struct {
}
func (q *QueryState) GetAuthEvents(ctx context.Context, event gomatrixserverlib.PDU) (gomatrixserverlib.AuthEventProvider, error) {
return helpers.GetAuthEvents(ctx, q.Database, event.Version(), event, event.AuthEventIDs())
authEventIDs := event.AuthEventIDs()
if gomatrixserverlib.MustGetRoomVersion(event.Version()).DomainlessRoomIDs() {
authEventIDs = append(authEventIDs, "$"+event.RoomID().String()[1:])
}
return helpers.GetAuthEvents(ctx, q.Database, event.Version(), event, authEventIDs)
}
func (q *QueryState) GetState(ctx context.Context, roomID spec.RoomID, stateWanted []gomatrixserverlib.StateKeyTuple) ([]gomatrixserverlib.PDU, error) {

View file

@ -404,7 +404,9 @@ func (p *PDUStreamProvider) addRoomDeltaToResponse(
// "state" section and kept in "timeline".
sEvents := gomatrixserverlib.HeaderedReverseTopologicalOrdering(
gomatrixserverlib.ToPDUs(removeDuplicates(delta.StateEvents, events)),
gomatrixserverlib.TopologicalOrderByAuthEvents,
// sorting by auth events is not stable unless we know the create and historical PL events,
// which we don't for deltas like this.
gomatrixserverlib.TopologicalOrderByPrevEvents,
)
delta.StateEvents = make([]*rstypes.HeaderedEvent, len(sEvents))
var skipped int

View file

@ -21,4 +21,8 @@ Guest users can accept invites to private rooms over federation
# Tests Synapse specific behavior
/state returns M_NOT_FOUND for an outlier
/state_ids returns M_NOT_FOUND for an outlier
/state_ids returns M_NOT_FOUND for an outlier
# this is a silly restriction as it basically stops servers from communicating, so we relax it
# see https://github.com/element-hq/synapse/issues/7543
Server rejects invalid JSON in a version 6 room