Reminders being sent using UTC instead of configured timezone. #877

Closed
opened 2021-06-02 14:23:26 +00:00 by SteveDinn · 13 comments

My timezone is "America/Halifax" which is UTC-3.
I set up a reminder for a task for 8:00am (local time), but I got the email at 11:00am (local time), which would be 8:00am UTC.

I have to conclude that reminders are being sent using UTC instead of the configured timezone.

My timezone is "America/Halifax" which is UTC-3. I set up a reminder for a task for 8:00am (local time), but I got the email at 11:00am (local time), which would be 8:00am UTC. I have to conclude that reminders are being sent using UTC instead of the configured timezone.
Owner

I also noticed that.

The problem is all dates in the db (not just reminders, also due dates, start/end dates etc) are stored in utc. Then when you retrieve the tasks it will return them in the configured time zone which will then be used to show the dates in your browser.
Now, when the api collects these dates on the server it simply checks every minute if there are any reminders due in the next minute and it ignores the configured time zone in that case because it will just do select * from task_reminders where reminder between now() and now() + 1 minute (pseudocode).
The conversion of stored time zone (which is always utc) and returned time zone is handled by the orm and this seems to bypass this (Might be a bug in the orm?).

Ideally, there'd be a setting to set the time zone per user which would let multiple users get the reminders in their specific time zone.

I also noticed that. The problem is all dates in the db (not just reminders, also due dates, start/end dates etc) are stored in utc. Then when you retrieve the tasks it will return them in the configured time zone which will then be used to show the dates in your browser. Now, when the api collects these dates on the server it simply checks every minute if there are any reminders due in the next minute and it ignores the configured time zone in that case because it will just do `select * from task_reminders where reminder between now() and now() + 1 minute` (pseudocode). The conversion of stored time zone (which is always utc) and returned time zone is handled by the orm and this seems to bypass this (Might be a bug in the orm?). Ideally, there'd be a setting to set the time zone per user which would let multiple users get the reminders in their specific time zone.
Author

The time left before the due date is calculated correctly, i.e., "due in 30 minutes", is this done in the same manner?

The time left before the due date is calculated correctly, i.e., "due in 30 minutes", is this done in the same manner?
Author

The due date seems to be using my local time but the reminder seems to be using UTC.

The due date seems to be using my local time but the reminder seems to be using UTC.
Author

I think your idea is correct though: ideally, everything on the API side should deal with nothing but UTC except when a date is presented to a user (in which case it should be converted to that user's local timezone) or a user is allowed to enter a date (in which case it should be accepted in that user's local timezone and then converted to UTC for the API).

I think your idea is correct though: ideally, everything on the API side should deal with nothing but UTC except when a date is presented to a user (in which case it should be converted to that user's local timezone) or a user is allowed to enter a date (in which case it should be accepted in that user's local timezone and then converted to UTC for the API).
Owner

All dates you see in the browser are (or should, at least) based on the time zone in your browser.

The due date seems to be using my local time but the reminder seems to be using UTC.

That's interesting - it should always show the time in the time zone in the browser. Can you reproduce this on try? (try runs in Europe/Berlin, not America/Halifax but the conversion mechanics are the same)

it should be accepted in that user's local timezone and then converted to UTC for the API

The browser sends the date in an iso timestamp with the time zone, the api then does all the converting to utc before saving it into the db.

All dates you see in the browser are (or should, at least) based on the time zone in your browser. > The due date seems to be using my local time but the reminder seems to be using UTC. That's interesting - it should always show the time in the time zone in the browser. Can you reproduce this on [try](https://try.vikunja.io)? (try runs in `Europe/Berlin`, not `America/Halifax` but the conversion mechanics are the same) > it should be accepted in that user's local timezone and then converted to UTC for the API The browser sends the date in an iso timestamp with the time zone, the api then does all the converting to utc before saving it into the db.
Author

First, without setting an email address, how can I verify that the reminder time is being interpreted correctly?

Even without considering that, I tried to enter times in Berlin time, but the time until due is being calculated against my local time. (see attachments).

First, without setting an email address, how can I verify that the reminder time is being interpreted correctly? Even without considering that, I tried to enter times in Berlin time, but the time until due is being calculated against my local time. (see attachments).
Owner

without setting an email address, how can I verify that the reminder time is being interpreted correctly

Ah that was more to verify the issue with the due date being displayed in another time zone in your browser. Try does not send emails - and they would certainly arrive at the utc time anyway.

the time until due is being calculated against my local time

That seems correct.

> without setting an email address, how can I verify that the reminder time is being interpreted correctly Ah that was more to verify the issue with the due date being displayed in another time zone in your browser. Try does not send emails - and they would certainly arrive at the utc time anyway. > the time until due is being calculated against my local time That seems correct.
Author

No, it wasn't correct. Timezone stuff is always confusing...I'll try to break it down.

  • I set the times in Berlin time so that they would be in the near future in my local time. Just before 1pm my time, I set the Due Date to 6pm Berlin time (1pm Halifax time).
  • The times appeared exactly as I set them, so there was no conversion at that point.
  • The due date should have been about 6 minutes in the future.
  • It was showing as 5 HOURS in the future when viewed in the list -- This is the difference between Halifax and Berlin times. Something's not right there.
  • Would "try" have known that I'm in Halifax in my browser and tried to convert something? I would have expected everything to be on Berlin time.
No, it wasn't correct. Timezone stuff is always confusing...I'll try to break it down. * I set the times in Berlin time so that they would be in the near future in my local time. Just before 1pm my time, I set the Due Date to 6pm Berlin time (1pm Halifax time). * The times appeared exactly as I set them, so there was no conversion at that point. * The due date should have been about 6 minutes in the future. * It was showing as 5 HOURS in the future when viewed in the list -- This is the difference between Halifax and Berlin times. Something's not right there. * Would "try" have known that I'm in Halifax in my browser and tried to convert something? I would have expected everything to be on Berlin time.
Owner

The time you see in the browser should be in the time zone of your browser. Could be a bug though. The api returns the dates in berlin time, but as an iso time stamp. If I implemented the formatting method correctly, your browser should understand that and convert it to your local time when displaying. Assuming your browser and os are correctly configured to use your local time.

This is how it looks in my browser:
image

And this is what is saved in the db:
image
image

Did you set the date to 1pm or 6pm (from your browser)? Because it saved 9pm, which would be correct in utc if you'd set it to 6pm in your time but not 1pm in berlin time:
image

I start to feel like the whole "just let the browser convert the time zones" makes this more complicated overall.

It was showing as 5 HOURS in the future when viewed in the list -- This is the difference between Halifax and Berlin times. Something's not right there.

This could indicate the conversion only worked one way, but not when loading again - or something like that.

Now is a good time to view this Tom Scott video again 🙂

The time you see in the browser *should* be in the time zone of your browser. Could be a bug though. The api returns the dates in berlin time, but as an iso time stamp. If I implemented the formatting method correctly, your browser should understand that and convert it to your local time when displaying. Assuming your browser and os are correctly configured to use your local time. This is how it looks in my browser: ![image](/attachments/1f829976-da4c-4c9a-8fcd-a2a758499ad8) And this is what is saved in the db: ![image](/attachments/37ddeab8-1e37-4267-8e3b-5454083d5389) ![image](/attachments/e13c5821-28c9-4353-8c95-83993ad58906) Did you set the date to 1pm or 6pm (from your browser)? Because it saved 9pm, which would be correct in utc if you'd set it to 6pm in your time but not 1pm in berlin time: ![image](/attachments/61fbc893-a885-4d49-88ce-6160a82bf519) I start to feel like the whole "just let the browser convert the time zones" makes this more complicated overall. > It was showing as 5 HOURS in the future when viewed in the list -- This is the difference between Halifax and Berlin times. Something's not right there. This could indicate the conversion only worked one way, but not when loading again - or something like that. Now is a good time to view [this Tom Scott video](https://www.youtube.com/watch?v=-5wpm-gesOY) again 🙂
Author

Ha. I've watched that video. I think I got things wrong from the start when I tried to reproduce this with 'try'. I set my times to what I thougth was Berlin time, but I was actually setting them to my local time -- 5 hours in my future.

So I guess this begs the question: If all the timestamps are sent in UTC (whether explicitly or via a local time with offset), then what is the point of having a configuration value for TimeZone? It seems like this could be much simpler with the API always dealing with UTC and the frontend always converting it to the frontend's local time.

In the end, it does't really matter how it's working as long as I can understand it and adjust for it :)

Ha. I've watched that video. I think I got things wrong from the start when I tried to reproduce this with 'try'. I set my times to what I thougth was Berlin time, but I was actually setting them to my local time -- 5 hours in my future. So I guess this begs the question: If all the timestamps are sent in UTC (whether explicitly or via a local time with offset), then what is the point of having a configuration value for TimeZone? It seems like this could be much simpler with the API always dealing with UTC and the frontend always converting it to the frontend's local time. In the end, it does't really matter how it's working as long as I can understand it and adjust for it :)
Owner

If all the timestamps are sent in UTC (whether explicitly or via a local time with offset), then what is the point of having a configuration value for TimeZone? It seems like this could be much simpler with the API always dealing with UTC and the frontend always converting it to the frontend's local time.

I agree with that. I've added the time zone setting when migrating all the unix timestamps to iso dates. Back when I made that change I needed a time zone to return the dates in but in retrospect it does not make that much sense. I guess we could deprecate the setting and move the reminders over to a user setting - then at some point maybe also the formatting in the browser to use that same setting.

> If all the timestamps are sent in UTC (whether explicitly or via a local time with offset), then what is the point of having a configuration value for TimeZone? It seems like this could be much simpler with the API always dealing with UTC and the frontend always converting it to the frontend's local time. I agree with that. I've added the time zone setting when migrating all the unix timestamps to iso dates. Back when I made that change I needed a time zone to return the dates in but in retrospect it does not make that much sense. I guess we could deprecate the setting and move the reminders over to a user setting - then at some point maybe also the formatting in the browser to use that same setting.
konrad added the
kind/feature
kind/bug
labels 2021-07-07 12:00:40 +00:00
Owner

Closing this as I belive it has been solved, please ping if you're still having issues with it.

Closing this as I belive it has been solved, please ping if you're still having issues with it.
Author

Just confirmed this one. Thanks!

Just confirmed this one. Thanks!
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: vikunja/vikunja#877
No description provided.