Race Condition when using Metrics #604

New Issue

derekprovance · 2020-07-02T15:04:07Z

derekprovance commented

2020-07-02 15:04:07 +00:00

Running into an issue where the server crashes due to a fatal error.

Configuration is

Redis Enabled
In-Memory Caching is enabled using Redis
Metrics are enabled

To recreate:
Click through lists, and all menu items in rapid succession on the front-end. Eventually the server will crash and front-end will report a 502.

Environment:

Ubuntu 20.04
Apache server behind an NGINX proxy
Vikunja 0.14.0

Stacktrace:

Jul  2 14:57:16 webserver vikunja[8712]: fatal error: concurrent map writes                                                                                                Jul  2 14:57:16 webserver vikunja[8712]: goroutine 200 [running]:                                                                                                          Jul  2 14:57:16 webserver vikunja[8712]: runtime.throw(0x120a510, 0x15)                                                                                                    Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc0003255f0 sp=0xc0003255c0 pc=0x43be72                                     Jul  2 14:57:16 webserver vikunja[8712]: runtime.mapassign_fast64(0x10c6840, 0xc00011a030, 0x1, 0x0)                                                                       Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/map_fast64.go:176 +0x30a fp=0xc000325630 sp=0xc0003255f0 pc=0x419e5a                                Jul  2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/metrics.SetUserActive(0x149dfa0, 0xc000a417c0, 0x149dfa0, 0xc000a417c0)                                   Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/metrics/active_users.go:71 +0xeb fp=0xc000325690 sp=0xc000325630 pc=0xc7d3fb                  Jul  2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/routes.updateActiveUsersFromContext(0x14d8a40, 0xc0006d8000, 0x11f52d2, 0x4)                              Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/routes/metrics.go:107 +0x70 fp=0xc0003256d0 sp=0xc000325690 pc=0xe3d370                       Jul  2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/routes.setupMetricsMiddleware.func1.1(0x14d8a40, 0xc0006d8000, 0x4, 0x10c0720)                            Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/routes/metrics.go:91 +0x42 fp=0xc000325728 sp=0xc0003256d0 pc=0xe43632                        Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.JWTWithConfig.func2.1(0x14d8a40, 0xc0006d8000, 0x10, 0x1060d00)                            Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/jwt.go:207 +0x3c0 fp=0xc0003257e8 sp=0xc000325728 pc=0xe3a0e0 Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4.(*Echo).add.func1(0x14d8a40, 0xc0006d8000, 0x1213208, 0x1b)                                           Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/echo.go:512 +0x8a fp=0xc000325828 sp=0xc0003257e8 pc=0xac0d4a            Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.CORSWithConfig.func1.1(0x14d8a40, 0xc0006d8000, 0x0, 0xc00006c9d8)                         Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/cors.go:121 +0x477 fp=0xc000325990 sp=0xc000325828 pc=0xe388d7Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.RecoverWithConfig.func1.1(0x14d8a40, 0xc0006d8000, 0x0, 0x0)                               Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/recover.go:78 +0x12a fp=0xc000325a10 sp=0xc000325990 pc=0xe3c8ea                                                                                                                                                                         Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1(0x14d8a40, 0xc0006d8000, 0x0, 0x0)                                Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/logger.go:117 +0x130 fp=0xc000325b90 sp=0xc000325a10 pc=0xe3be70                                                                                                                                                                         Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4.(*Echo).ServeHTTP(0xc0005b81e0, 0x14b42e0, 0xc0005c0380, 0xc000528100)                                Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/echo.go:623 +0x16c fp=0xc000325be8 sp=0xc000325b90 pc=0xab967c           Jul  2 14:57:16 webserver vikunja[8712]: net/http.serverHandler.ServeHTTP(0xc0005c0000, 0x14b42e0, 0xc0005c0380, 0xc000528100)                                             Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:2807 +0xa3 fp=0xc000325c18 sp=0xc000325be8 pc=0x793753                                   Jul  2 14:57:16 webserver vikunja[8712]: net/http.(*conn).serve(0xc000a41720, 0x14b7360, 0xc000a31bc0)                                                                     Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:1895 +0x86c fp=0xc000325fc8 sp=0xc000325c18 pc=0x78f1cc                                  Jul  2 14:57:16 webserver vikunja[8712]: runtime.goexit()                                                                                                                  Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc000325fd0 sp=0xc000325fc8 pc=0x46bf51                                   Jul  2 14:57:16 webserver vikunja[8712]: created by net/http.(*Server).Serve                                                                                               Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:2933 +0x35c                                                                              Jul  2 14:57:16 webserver vikunja[8712]: goroutine 1 [chan receive]:                                                                                                       Jul  2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/cmd.glob..func2(0x1ef3900, 0x1f9ad50, 0x0, 0x0)                                                           Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/cmd/web.go:65 +0x1dd
Jul  2 14:57:16 webserver vikunja[8712]: github.com/spf13/cobra.(*Command).execute(0x1ef3900, 0xc0000321d0, 0x0, 0x0, 0x1ef3900, 0xc0000321d0)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x29d
Jul  2 14:57:16 webserver vikunja[8712]: github.com/spf13/cobra.(*Command).ExecuteC(0x1ef3900, 0x44b07a, 0x1f0aba0, 0xc000000180)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x349
Jul  2 14:57:16 webserver vikunja[8712]: github.com/spf13/cobra.(*Command).Execute(...)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
Jul  2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/cmd.Execute()
Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/cmd/cmd.go:43 +0x31
Jul  2 14:57:16 webserver vikunja[8712]: main.main()
Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/main.go:22 +0x20
Jul  2 14:57:16 webserver vikunja[8712]: goroutine 23 [select]:
Jul  2 14:57:16 webserver vikunja[8712]: net.(*netFD).Read(0xc0007c0600, 0xc0003f76c1, 0x1, 0x1, 0xc0003999d8, 0xc00050a768, 0x48365c)                            [14/1116]Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/fd_unix.go:202 +0x4f                                                                                    Jul  2 14:57:16 webserver vikunja[8712]: net.(*conn).Read(0xc0003964e8, 0xc0003f76c1, 0x1, 0x1, 0x0, 0x0, 0x0)                                                             Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/net.go:184 +0x8e                                                                                        Jul  2 14:57:16 webserver vikunja[8712]: net/http.(*connReader).backgroundRead(0xc0003f76b0)                                                                               Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:678 +0x58                                                                                Jul  2 14:57:16 webserver vikunja[8712]: created by net/http.(*connReader).startBackgroundRead                                                                             Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:674 +0xd0                                                                                Jul  2 14:57:16 webserver vikunja[8712]: goroutine 188 [select]:                                                                                                           Jul  2 14:57:16 webserver vikunja[8712]: github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc000081ec0, 0xc00001bb00, 0xc00008a5a0)                          Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:621 +0xbf                                                Jul  2 14:57:16 webserver vikunja[8712]: created by github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher                                                               Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:618 +0xbe                                                Jul  2 14:57:16 webserver vikunja[8712]: goroutine 198 [select]:                                                                                                           Jul  2 14:57:16 webserver vikunja[8712]: github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc000080de0, 0xc0003db680, 0xc000bd2540)                          Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:621 +0xbf
Jul  2 14:57:16 webserver vikunja[8712]: created by github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:618 +0xbe
Jul  2 14:57:16 webserver vikunja[8712]: goroutine 199 [running]:
Jul  2 14:57:16 webserver vikunja[8712]: runtime.throw(0x120a510, 0x15)
Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc0003235f0 sp=0xc0003235c0 pc=0x43be72
Jul  2 14:57:16 webserver vikunja[8712]: runtime.mapassign_fast64(0x10c6840, 0xc00011a030, 0x1, 0x0)
Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/map_fast64.go:176 +0x30a fp=0xc000323630 sp=0xc0003235f0 pc=0x419e5a
Jul  2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/metrics.SetUserActive(0x149dfa0, 0xc000b49180, 0x149dfa0, 0xc000b49180)
Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/metrics/active_users.go:71 +0xeb fp=0xc000323690 sp=0xc000323630 pc=0xc7d3fb
Jul  2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/routes.updateActiveUsersFromContext(0x14d8a40, 0xc0005ba280, 0x11f52d2, 0x4)
Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/routes/metrics.go:107 +0x70 fp=0xc0003236d0 sp=0xc000323690 pc=0xe3d370
Jul  2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/routes.setupMetricsMiddleware.func1.1(0x14d8a40, 0xc0005ba280, 0x4, 0x10c0720)
Jul  2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/routes/metrics.go:91 +0x42 fp=0xc000323728 sp=0xc0003236d0 pc=0xe43632
Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.JWTWithConfig.func2.1(0x14d8a40, 0xc0005ba280, 0x10, 0x1060d00)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/jwt.go:207 +0x3c0 fp=0xc0003237e8 sp=0xc000323728 pc=0xe3a0e0
Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4.(*Echo).add.func1(0x14d8a40, 0xc0005ba280, 0x1213208, 0x1b)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/echo.go:512 +0x8a fp=0xc000323828 sp=0xc0003237e8 pc=0xac0d4a
Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.CORSWithConfig.func1.1(0x14d8a40, 0xc0005ba280, 0x0, 0xc000a2d9d8)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/cors.go:121 +0x477 fp=0xc000323990 sp=0xc000323828 pc=0xe388d7Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.RecoverWithConfig.func1.1(0x14d8a40, 0xc0005ba280, 0x0, 0x0)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/recover.go:78 +0x12a fp=0xc000323a10 sp=0xc000323990 pc=0xe3c8ea
Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1(0x14d8a40, 0xc0005ba280, 0x0, 0x0)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/logger.go:117 +0x130 fp=0xc000323b90 sp=0xc000323a10 pc=0xe3be70
Jul  2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4.(*Echo).ServeHTTP(0xc0005b81e0, 0x14b42e0, 0xc00078ea80, 0xc0004d8600)
Jul  2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/echo.go:623 +0x16c fp=0xc000323be8 sp=0xc000323b90 pc=0xab967c
Jul  2 14:57:16 webserver vikunja[8712]: net/http.serverHandler.ServeHTTP(0xc0005c0000, 0x14b42e0, 0xc00078ea80, 0xc0004d8600)
Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:2807 +0xa3 fp=0xc000323c18 sp=0xc000323be8 pc=0x793753
Jul  2 14:57:16 webserver vikunja[8712]: net/http.(*conn).serve(0xc000a41680, 0x14b7360, 0xc000455980)
Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:1895 +0x86c fp=0xc000323fc8 sp=0xc000323c18 pc=0x78f1cc
Jul  2 14:57:16 webserver vikunja[8712]: runtime.goexit()
Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc000323fd0 sp=0xc000323fc8 pc=0x46bf51
Jul  2 14:57:16 webserver vikunja[8712]: created by net/http.(*Server).Serve
Jul  2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:2933 +0x35c

Running into an issue where the server crashes due to a fatal error. Configuration is 1. Redis Enabled 2. In-Memory Caching is enabled using Redis 3. Metrics are enabled To recreate: Click through lists, and all menu items in rapid succession on the front-end. Eventually the server will crash and front-end will report a 502. Environment: * Ubuntu 20.04 * Apache server behind an NGINX proxy * Vikunja 0.14.0 Stacktrace: ``` Jul 2 14:57:16 webserver vikunja[8712]: fatal error: concurrent map writes Jul 2 14:57:16 webserver vikunja[8712]: goroutine 200 [running]: Jul 2 14:57:16 webserver vikunja[8712]: runtime.throw(0x120a510, 0x15) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc0003255f0 sp=0xc0003255c0 pc=0x43be72 Jul 2 14:57:16 webserver vikunja[8712]: runtime.mapassign_fast64(0x10c6840, 0xc00011a030, 0x1, 0x0) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/map_fast64.go:176 +0x30a fp=0xc000325630 sp=0xc0003255f0 pc=0x419e5a Jul 2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/metrics.SetUserActive(0x149dfa0, 0xc000a417c0, 0x149dfa0, 0xc000a417c0) Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/metrics/active_users.go:71 +0xeb fp=0xc000325690 sp=0xc000325630 pc=0xc7d3fb Jul 2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/routes.updateActiveUsersFromContext(0x14d8a40, 0xc0006d8000, 0x11f52d2, 0x4) Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/routes/metrics.go:107 +0x70 fp=0xc0003256d0 sp=0xc000325690 pc=0xe3d370 Jul 2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/routes.setupMetricsMiddleware.func1.1(0x14d8a40, 0xc0006d8000, 0x4, 0x10c0720) Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/routes/metrics.go:91 +0x42 fp=0xc000325728 sp=0xc0003256d0 pc=0xe43632 Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.JWTWithConfig.func2.1(0x14d8a40, 0xc0006d8000, 0x10, 0x1060d00) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/jwt.go:207 +0x3c0 fp=0xc0003257e8 sp=0xc000325728 pc=0xe3a0e0 Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4.(*Echo).add.func1(0x14d8a40, 0xc0006d8000, 0x1213208, 0x1b) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/echo.go:512 +0x8a fp=0xc000325828 sp=0xc0003257e8 pc=0xac0d4a Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.CORSWithConfig.func1.1(0x14d8a40, 0xc0006d8000, 0x0, 0xc00006c9d8) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/cors.go:121 +0x477 fp=0xc000325990 sp=0xc000325828 pc=0xe388d7Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.RecoverWithConfig.func1.1(0x14d8a40, 0xc0006d8000, 0x0, 0x0) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/recover.go:78 +0x12a fp=0xc000325a10 sp=0xc000325990 pc=0xe3c8ea Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1(0x14d8a40, 0xc0006d8000, 0x0, 0x0) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/logger.go:117 +0x130 fp=0xc000325b90 sp=0xc000325a10 pc=0xe3be70 Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4.(*Echo).ServeHTTP(0xc0005b81e0, 0x14b42e0, 0xc0005c0380, 0xc000528100) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/echo.go:623 +0x16c fp=0xc000325be8 sp=0xc000325b90 pc=0xab967c Jul 2 14:57:16 webserver vikunja[8712]: net/http.serverHandler.ServeHTTP(0xc0005c0000, 0x14b42e0, 0xc0005c0380, 0xc000528100) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:2807 +0xa3 fp=0xc000325c18 sp=0xc000325be8 pc=0x793753 Jul 2 14:57:16 webserver vikunja[8712]: net/http.(*conn).serve(0xc000a41720, 0x14b7360, 0xc000a31bc0) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:1895 +0x86c fp=0xc000325fc8 sp=0xc000325c18 pc=0x78f1cc Jul 2 14:57:16 webserver vikunja[8712]: runtime.goexit() Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc000325fd0 sp=0xc000325fc8 pc=0x46bf51 Jul 2 14:57:16 webserver vikunja[8712]: created by net/http.(*Server).Serve Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:2933 +0x35c Jul 2 14:57:16 webserver vikunja[8712]: goroutine 1 [chan receive]: Jul 2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/cmd.glob..func2(0x1ef3900, 0x1f9ad50, 0x0, 0x0) Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/cmd/web.go:65 +0x1dd Jul 2 14:57:16 webserver vikunja[8712]: github.com/spf13/cobra.(*Command).execute(0x1ef3900, 0xc0000321d0, 0x0, 0x0, 0x1ef3900, 0xc0000321d0) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x29d Jul 2 14:57:16 webserver vikunja[8712]: github.com/spf13/cobra.(*Command).ExecuteC(0x1ef3900, 0x44b07a, 0x1f0aba0, 0xc000000180) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x349 Jul 2 14:57:16 webserver vikunja[8712]: github.com/spf13/cobra.(*Command).Execute(...) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887 Jul 2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/cmd.Execute() Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/cmd/cmd.go:43 +0x31 Jul 2 14:57:16 webserver vikunja[8712]: main.main() Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/main.go:22 +0x20 Jul 2 14:57:16 webserver vikunja[8712]: goroutine 23 [select]: Jul 2 14:57:16 webserver vikunja[8712]: net.(*netFD).Read(0xc0007c0600, 0xc0003f76c1, 0x1, 0x1, 0xc0003999d8, 0xc00050a768, 0x48365c) [14/1116]Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/fd_unix.go:202 +0x4f Jul 2 14:57:16 webserver vikunja[8712]: net.(*conn).Read(0xc0003964e8, 0xc0003f76c1, 0x1, 0x1, 0x0, 0x0, 0x0) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/net.go:184 +0x8e Jul 2 14:57:16 webserver vikunja[8712]: net/http.(*connReader).backgroundRead(0xc0003f76b0) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:678 +0x58 Jul 2 14:57:16 webserver vikunja[8712]: created by net/http.(*connReader).startBackgroundRead Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:674 +0xd0 Jul 2 14:57:16 webserver vikunja[8712]: goroutine 188 [select]: Jul 2 14:57:16 webserver vikunja[8712]: github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc000081ec0, 0xc00001bb00, 0xc00008a5a0) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:621 +0xbf Jul 2 14:57:16 webserver vikunja[8712]: created by github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:618 +0xbe Jul 2 14:57:16 webserver vikunja[8712]: goroutine 198 [select]: Jul 2 14:57:16 webserver vikunja[8712]: github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc000080de0, 0xc0003db680, 0xc000bd2540) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:621 +0xbf Jul 2 14:57:16 webserver vikunja[8712]: created by github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:618 +0xbe Jul 2 14:57:16 webserver vikunja[8712]: goroutine 199 [running]: Jul 2 14:57:16 webserver vikunja[8712]: runtime.throw(0x120a510, 0x15) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc0003235f0 sp=0xc0003235c0 pc=0x43be72 Jul 2 14:57:16 webserver vikunja[8712]: runtime.mapassign_fast64(0x10c6840, 0xc00011a030, 0x1, 0x0) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/map_fast64.go:176 +0x30a fp=0xc000323630 sp=0xc0003235f0 pc=0x419e5a Jul 2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/metrics.SetUserActive(0x149dfa0, 0xc000b49180, 0x149dfa0, 0xc000b49180) Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/metrics/active_users.go:71 +0xeb fp=0xc000323690 sp=0xc000323630 pc=0xc7d3fb Jul 2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/routes.updateActiveUsersFromContext(0x14d8a40, 0xc0005ba280, 0x11f52d2, 0x4) Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/routes/metrics.go:107 +0x70 fp=0xc0003236d0 sp=0xc000323690 pc=0xe3d370 Jul 2 14:57:16 webserver vikunja[8712]: code.vikunja.io/api/pkg/routes.setupMetricsMiddleware.func1.1(0x14d8a40, 0xc0005ba280, 0x4, 0x10c0720) Jul 2 14:57:16 webserver vikunja[8712]: #011/go/src/code.vikunja.io/api/pkg/routes/metrics.go:91 +0x42 fp=0xc000323728 sp=0xc0003236d0 pc=0xe43632 Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.JWTWithConfig.func2.1(0x14d8a40, 0xc0005ba280, 0x10, 0x1060d00) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/jwt.go:207 +0x3c0 fp=0xc0003237e8 sp=0xc000323728 pc=0xe3a0e0 Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4.(*Echo).add.func1(0x14d8a40, 0xc0005ba280, 0x1213208, 0x1b) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/echo.go:512 +0x8a fp=0xc000323828 sp=0xc0003237e8 pc=0xac0d4a Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.CORSWithConfig.func1.1(0x14d8a40, 0xc0005ba280, 0x0, 0xc000a2d9d8) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/cors.go:121 +0x477 fp=0xc000323990 sp=0xc000323828 pc=0xe388d7Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.RecoverWithConfig.func1.1(0x14d8a40, 0xc0005ba280, 0x0, 0x0) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/recover.go:78 +0x12a fp=0xc000323a10 sp=0xc000323990 pc=0xe3c8ea Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1(0x14d8a40, 0xc0005ba280, 0x0, 0x0) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/middleware/logger.go:117 +0x130 fp=0xc000323b90 sp=0xc000323a10 pc=0xe3be70 Jul 2 14:57:16 webserver vikunja[8712]: github.com/labstack/echo/v4.(*Echo).ServeHTTP(0xc0005b81e0, 0x14b42e0, 0xc00078ea80, 0xc0004d8600) Jul 2 14:57:16 webserver vikunja[8712]: #011/srv/app/pkg/mod/github.com/labstack/echo/v4@v4.1.16/echo.go:623 +0x16c fp=0xc000323be8 sp=0xc000323b90 pc=0xab967c Jul 2 14:57:16 webserver vikunja[8712]: net/http.serverHandler.ServeHTTP(0xc0005c0000, 0x14b42e0, 0xc00078ea80, 0xc0004d8600) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:2807 +0xa3 fp=0xc000323c18 sp=0xc000323be8 pc=0x793753 Jul 2 14:57:16 webserver vikunja[8712]: net/http.(*conn).serve(0xc000a41680, 0x14b7360, 0xc000455980) Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:1895 +0x86c fp=0xc000323fc8 sp=0xc000323c18 pc=0x78f1cc Jul 2 14:57:16 webserver vikunja[8712]: runtime.goexit() Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc000323fd0 sp=0xc000323fc8 pc=0x46bf51 Jul 2 14:57:16 webserver vikunja[8712]: created by net/http.(*Server).Serve Jul 2 14:57:16 webserver vikunja[8712]: #011/usr/local/go/src/net/http/server.go:2933 +0x35c ```

derekprovance commented

2020-07-02 15:05:53 +00:00

I believe the issue to be the metrics system. It doesn't occur when metrics is disabled

konrad commented

2020-07-02 15:49:30 +00:00

The fatal error: concurrent map writes part seems to be the crux here.

I'll try to reproduce it locally.

The `fatal error: concurrent map writes` part seems to be the crux here. I'll try to reproduce it locally.

konrad commented

2020-07-02 19:20:29 +00:00

I've pushed a possible fix for it in 7141050f8b.
I was not able to reproduce it with that fix. Could you try to reproduce it with the updated version?

A new release should be ready for download (master) in about an hour.

If you can confirm it fixes it I'll release 0.14.1 afterwards.

I've pushed a possible fix for it in https://kolaente.dev/vikunja/api/commit/7141050f8b8cbb0d59008507529e3482c553edb1. I was not able to reproduce it with that fix. Could you try to reproduce it with the updated version? A new release should be ready for download ([master](https://dl.vikunja.io/api/master)) [in about an hour](https://drone.kolaente.de/vikunja/api/1988). If you can confirm it fixes it I'll release 0.14.1 afterwards.

konrad commented

2020-07-09 16:35:45 +00:00

Since I could not reproduce the bug with the fix and there was no response since, I'm closing this issue - feel free to reopen.

konrad closed this issue

2020-07-09 16:35:45 +00:00

konrad referenced this issue from a commit

2024-02-07 14:00:26 +00:00

Update dependency date-fns to v2.23.0 (#604)

Sign in to join this conversation.