Assertions needed to catch zero NIDs

[neilalexander]

All NIDs (event, event type, state block, state snapshot, event state key, room, etc) always start from 1, but there are sometimes cases where bugs cause us to request 0 NIDs. We should put some assertion to catch these bugs as they happen.

[neilalexander]

Whilst running the sytests for #954 against SQLite, I am noticing that the monolith logs are showing a lot of failed AddPrevEventsToEvents calls when building events.

It appears to be because QueryLatestEventsAndState calls DB.LatestEventIDs, which in turn calls selectLatestEventNIDs and then the selectLatestEventNIDsSQL query returns zero NIDs.

@kegsay In SQLite these are wrapped in a transaction in LatestEventIDs—in Postgres they are not. Any thoughts as to what's going on here?

[neilalexander]

Actually it looks like this is also happening sometimes in Postgres mode, just in a different place, so that's fun.

Request panicked!
goroutine 6979 [running]:
runtime/debug.Stack(0xc000104d20, 0xc0005df590, 0xc000104e00)
	/goroot/src/runtime/debug/stack.go:24 +0x9d
github.com/matrix-org/util.Protect.func1.1(0xc00057c600, 0x102ce20, 0xc0003a8540)
	/gopath/pkg/mod/github.com/matrix-org/[email protected]/json.go:98 +0x15d
panic(0xd64a40, 0x1007480)
	/goroot/src/runtime/panic.go:679 +0x1b2
github.com/matrix-org/dendrite/roomserver/query.(*RoomserverQueryAPI).QueryLatestEventsAndState(0xc000035f40, 0x10306e0, 0xc000445140, 0xc000210390, 0xc00018ce00, 0x0, 0x0)
	/src/roomserver/query/query.go:135 +0x6ee
github.com/matrix-org/dendrite/common.AddPrevEventsToEvent(0x10306e0, 0xc000445140, 0xc0005e0178, 0x103d580, 0xc000035f40, 0xc00018ce00, 0x15df3e0, 0xc000182180)
	/src/common/events.go:81 +0x2a1
github.com/matrix-org/dendrite/common.BuildEvent(0x10306e0, 0xc000445140, 0xc0005e0178, 0xc000180000, 0xbf9b47247a83982c, 0x3fe1d3ac7d, 0x15dfd60, 0x103d580, 0xc000035f40, 0xc00018ce00, ...)
	/src/common/events.go:48 +0x95
github.com/matrix-org/dendrite/clientapi/routing.joinRoomReq.joinRoomUsingServers(0xc00055f300, 0xbf9b47247a83982c, 0x3fe1d3ac7d, 0x15dfd60, 0xc0004451a0, 0xc0002271d0, 0x28, 0xc000180000, 0xc00018eee0, 0xc000208ae0, ...)
	/src/clientapi/routing/joinroom.go:243 +0x178
github.com/matrix-org/dendrite/clientapi/routing.joinRoomReq.joinRoomByID(0xc00055f300, 0xbf9b47247a83982c, 0x3fe1d3ac7d, 0x15dfd60, 0xc0004451a0, 0xc0002271d0, 0x28, 0xc000180000, 0xc00018eee0, 0xc000208ae0, ...)
	/src/clientapi/routing/joinroom.go:156 +0x91a
github.com/matrix-org/dendrite/clientapi/routing.JoinRoomByIDOrAlias(0xc00055f300, 0xc0002e6460, 0xc00032e41d, 0x20, 0xc000180000, 0xc00018eee0, 0xc000208ae0, 0x103d580, 0xc000035f40, 0x1032420, ...)
	/src/clientapi/routing/joinroom.go:85 +0xd2b
github.com/matrix-org/dendrite/clientapi/routing.Setup.func3(0xc00055f300, 0xc0002e6460, 0xd62e00, 0x1006a10, 0xeb7c60, 0xc0001b71f0)
	/src/clientapi/routing/routing.go:103 +0x261
github.com/matrix-org/dendrite/common.MakeAuthAPI.func1(0xc00057c700, 0x0, 0x0, 0x0, 0x0)
	/src/common/httpapi.go:33 +0x325
github.com/matrix-org/util.(*jsonRequestHandlerWrapper).OnIncomingRequest(0xc000010218, 0xc00057c700, 0xc00038da10, 0x30, 0x30, 0xe441e0)
	/gopath/pkg/mod/github.com/matrix-org/[email protected]/json.go:79 +0x33
github.com/matrix-org/util.MakeJSONAPI.func1(0x102ce20, 0xc0003a8540, 0xc00057c700)
	/gopath/pkg/mod/github.com/matrix-org/[email protected]/json.go:139 +0x97
github.com/matrix-org/util.Protect.func1(0x102ce20, 0xc0003a8540, 0xc00057c600)
	/gopath/pkg/mod/github.com/matrix-org/[email protected]/json.go:103 +0xa1
net/http.HandlerFunc.ServeHTTP(...)
	/goroot/src/net/http/server.go:2007
github.com/matrix-org/dendrite/common.MakeExternalAPI.func1(0x102ce20, 0xc0003a8540, 0xc00057c500)
	/src/common/httpapi.go:46 +0x203
net/http.HandlerFunc.ServeHTTP(0xc000264d20, 0x102ce20, 0xc0003a8540, 0xc00057c500)
	/goroot/src/net/http/server.go:2007 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc000178300, 0x102ce20, 0xc0003a8540, 0xc00057c200)
	/gopath/pkg/mod/github.com/gorilla/[email protected]/mux.go:212 +0xe2
github.com/matrix-org/dendrite/common.WrapHandlerInCORS.func1(0x102ce20, 0xc0003a8540, 0xc00057c200)
	/src/common/httpapi.go:147 +0x340
net/http.HandlerFunc.ServeHTTP(0xc0003d69c0, 0x102ce20, 0xc0003a8540, 0xc00057c200)
	/goroot/src/net/http/server.go:2007 +0x44
net/http.(*ServeMux).ServeHTTP(0x15dfc20, 0x102ce20, 0xc0003a8540, 0xc00057c200)
	/goroot/src/net/http/server.go:2387 +0x1bd
net/http.serverHandler.ServeHTTP(0xc00045e000, 0x102ce20, 0xc0003a8540, 0xc00057c200)
	/goroot/src/net/http/server.go:2802 +0xa4
net/http.(*conn).serve(0xc000253360, 0x1030620, 0xc0000767c0)
	/goroot/src/net/http/server.go:1890 +0x875
created by net/http.(*Server).Serve
	/goroot/src/net/http/server.go:2928 +0x384
" func=github.com/matrix-org/util.Protect.func1.1 file="/gopath/pkg/mod/github.com/matrix-org/[email protected]/json.go:97

[neilalexander]

So roomserver_rooms in my local Dendrite SQLite database appears to have quite a few rows of zero NIDs in the last_event_sent_nid and state_snapshot_nid columns. I suspect they are not being updated properly as my Postgres one doesn't have this.