Daylight Saving Time / Summer Time

[astlouys]

Eric,

I was making some tests with Summer Time (for European Union) and I realized that wiki says time change occurs at 1h00 UTC (not local time).

That means the actual time for hour change could be different from one country to another (if the time zone is different). Is that correct ?

For example, in Spain, what is the time of the change ?

[UltrashRicco]

Hi André!

In both France and Spain, the switch to winter time occurs on the last Sunday of October. At 3:00AM, the hour is set back one hour to 2:00AM. It seems consistent with your information, since Paris/Madrid is on the UTC+2 time zone.

Conversely, on the last Sunday of March, the switch to summer time happens at 2:00AM, one hour is added in order to move directly to 3:00AM.

I hope this helps.
Let me know whether I can help further!
Regards,

[astlouys]

Eric,

Sorry but I'm not familiar yet with the operational details of GitHub (yes, I confess that I still haven't read most of its documentation...)

That being said, are you able to "see" what has been done by FFF27 (sorry for the nickname, but I don't know the "real" name). Basically, he implemented a logic "case" for European countries that are 2 hours apart from UTC time. He gently asks for a pull request to make he change available for other users and I understand that I should merge the changes to my source code, but I hesitate since I'm almost done with the generic algorithm I've done to cover most countries listed in the wiki. This new algorithm, as mentioned, is generic and different than simply adding "cases" to the algorithm of Version 6.00... I suppose that I could merge and GitHub would simply manage source code differences of the next version without problem, but I prefer to play on the safe side. FFF27, if you read me, do not hesitate to comment and join us.

If you have access to FFF27's code, Eric, (either source or uf2), maybe this update would be good for you.

I continue my tests and keep you posted.

[UltrashRicco]

Hi André, I am not too familiar with GitHub either, given that I am not a developper.

I can see FFF27's pull request indeed, it seems like he did implement a new option for DTS in Europe indeed: fa1fff8

I understand that your algorithm solution is conceived to be more comprehensive/universal.

I am currently trying to download FFF27's source in order to customize it to my needs and test it. I think it should be safe for you to merge it into your code, but I do not know how your further modifications (new algorithm) would be taken into account/merged in the future.

[FFF27]

First of all: Hi, my name is Frank.

This is the most comprehensive wiki article, with TZ and DST in one table: https://en.wikipedia.org/wiki/Time_zone#List_of_UTC_offsets

As you see, there are 3 relevant time zones for Europe. Russia and Belarus don't count in this case because they completely stopped using DST a few years ago.

I didn't know that DST was being worked on. I never used the discussion section before. Didn't see this thread until I checked on the pull request. Closed it now since it's obsolete. Oh well.

The 'Issues' section is supposed to be the list of bugs and new feature requests. It also has the side bar you can use to communicate what is being worked on by whom.

[astlouys]

Hello Frank,

Thank you for joining the discussion group. As I mentioned above, I'm still not familiar with the GitHub environment, and I must confess that I spent more time working on the code than learning how to use GitHub... sorry about that, and thank you for the few guidelines you provided about GitHub. I'll go checking the "Issues" section.

The link you gave above is effectively very exhaustive and gives many details about time zones. I hope I have properly covered Europe in the DST algorithm. I don't know if you are in the same Time Zone as Eric (Spain, UTC +1) or Lukas (Germany, also UTC +1), but as long as UTC is properly set in the clock and DST setting is 5, it should work. By looking at the many different DST settings in Wiki, I thought a more generic algorithm would be nice, allowing to easily add more countries if required. That being said, I proceeded to some tests myself, but I'm curious to see if I covered everything...

[astlouys]

Eric, Lukas,

I released Firmware Version 7.00 with the updated handling of Daylight Saving Time / Summer Time.

You may proceed with testing as you want. There are some warnings in the User Guide, but I'll put some of them here that are important for DST testing:

1. First of all, you need to power-up the clock and adjust two settings: "ST" stands for "Summer Time". If you remember, it was called "DST" before and had two possible choices (0 and 1). Now, it goes from 0 to 12 to cover most countries of the world. (That's why I needed to remove one letter (changing "DST" to "ST") to give place for 2-digits numbers). The setting for Europe (Spain and / or Germany) is "5" (five).

2. Also in the clock settings, there is a new parameter: "UT" (stands for "Universal Time" or (UTC)). For most countries, "UT" is not required now, but since Europe uses it to decide when to perform hour change, you need to put the value for Spain and Germany. Don't forget that UTC "does not change". So, now that Summer Time is no more active in Spain and Germany, the difference with UTC is +1 hour. However, if ever you adjust "UT" during the summer, "UT" will be "+2", since you added one hour during Summer Time in Spain and Germany, while UTC didn't change... When the clock runs, it takes care of this change automatically (something else you should check during the tests), but it is not kept by the battery backed-up DS3231, so if the clock happens to be Off during DST change, "UT" will require manual setting.
   NOTE: "UT" is also a prerequisite and will need to be properly setup for all countries whenever NTP is implemented.

3. There are two parts to test:

• When the clock is powered-up, there is an algorithm that determines if we are during DST active or inactive. We need to make sure this algorithm is correct (according to my own tests it's OK, but I used North America setting more than other countries). It usually scrolls on the clock display if you haven't turned On the QUICK_START compile flag.
• Once the clock is up and running, there is another algorithm (sharing parts with the previous one above) that will automatically perform hour change according to the standard for the configured country (last Sunday of March and October for Europe).

4. You must keep in mind that the DST status ("active" or "inactive") is kept in flash by the Pico. So, if you try to test twice in a row the change, say, from Normal Time to Summer Time, you will see that the hour change does not occur the second time since the clock remembers that it is already in "Summer Time active" (the same behavior occurs when changing from Summer Time to Normal Time, mutatis mutandis)... So, testing requires to go through "Spring-Forward" change and then "Fall-Back" change (or the other way around, but not the same test twice in a row.

Feel free to test for other countries if your curious. For Chile and New Zealand, I'm not sure my comprehension of DST is good. I hope users from those countries can test it...

Let me know if you find something wrong.

[UltrashRicco]

Hello Lukas!

I thought just the same, why not have a "Time Zone" setting that would not change relative to the UTC time, and a separate "summer time" setting that would be +0 in winter, and +1 in summer. Currently, the UtcDelta setting combines both. But actually, UtcDelta does reflect the actual time difference relative to UTC. I I take western Europe as an example, the actual time is indeed UTC+1 in winter, and UTC+2 in summer. Anyway, I think it would make more sense to have 2 separate settings, for a better understanding: what is the timezone, and is it winter time or summer time.

I will try to consistently reproduce the 2-hour jump, and I'll shoot a video if I can. :)

Take care!

[astlouys]

Hi Eric, Lukas,

Following your tests and comments, Eric, I found a glitch in the DST algorithm, and also thought about a way to keep all clock parameters corrects even if the clock is left powered Off for a while (and during a DST status change). I'm almost done with the PWM controlling the clock display brightness (with great results !) and will update DST stuff after.

I read your comment about timezone, Lukas. I'm not sure I understood everything fine, but as far as I know, the timezone - by definition - is the difference between UTC Time and local Time (am I missing something here ?). So, basically, the concept of Timezone and "UTC delta" in the software refers to the same thing (you make me think that I should probably rename all references to "UTC delta" to "Timezone" instead. It would probably make things easier to follow. So, the Timezone does change (for Europe and Canada) between Summer Time and Winter Time. My comprehension is that if we would decide to make an "independent" Timezone that never changes with UTC, we would have to create a new table with all countries. Another word about Eric's comment. There are currently 2 different settings: one is UTC delta (that I will rename Timezone), and the other one is FlagSummerTime (that indicates if we are during Summer Time or not). Even if we create an "independent" Timezone that does not change, we will simply move the problem to determine the current status of UTC delta time, given the current time and date.

Your other comment is correct, I forgot to reset the SecondCounter from 50 to 0 before publishing the code. As you may imagine, since may operations in the firmware occur when the time changes, the waiting time is shorter (10 seconds) when I set the seconds to 50. Following your observation, I'll probably make it a conditional "Developer's" compile... That being said, I don't think it could be the source of this 2-hours jump but I haven't checked yet, to be honest.

I appreciate your comments. Give me a few more chances... my QA team here is working hard but may miss some "undocumented features" (smile).

[astlouys]

P. S. Don't forget that when you change a parameter that must be saved to flash (for example DST, TimeZone (previously UTC delta), or something else), it may take up to one minute for the flash to be updated. This is because interrupts must be disabled before programming the flash. So I decided to do it just before time change xxm59s, after blanking the display. I thought it would be less intrusive this way.

[Spart-aaa]

Hi André,
my thinking here is along the lines that technically timezone and DST are two different concepts which both at the same time affect the actual delta to UTC. Yes you are absolutely correct that a timezone is a delta to utc, it just is not necessarily the same as the actual technical delta variable that you need to apply to utc to arrive at the time as long as DST exists in addition. That is why in many devices (network equipment for example) you have three settings to make> Set an actual timesource (NTP, PTP or manual)>Set a timezone (The list here always is vast because often the names of capitals of countries are all mashed together that rely on the same timezone and also some half hour timezones exist)>Set if you live in an area that utilizes DST or not and if yes the DST mode which in turn defines the switchover dates.

So in these implementation you indeed often have two mapping tables, or only a timezone mapping table and a manual selector for the switchover dates. This would therefore eliminate the issue of saving the config with a a UTC delta of 1 and turning it back on when in fact it should be 2 because your timezone setting in the flash always says UTC+1 and then during runtime the DST algorithm determines dynamically as it already does now but the user is never confronted with a setting of him changing dynamically.

But as mentioned as long as you have to set the time manually this can lead to a totally different confusion when the user sets the time because people are accustomed to setting the time in their local timezone and not in UTC. However that could be solved by not committing the hour setting immediately to the RTC but only after a timezone and DST setting have also been set because from these three bits of information you can then infer the actual UTC which then sets the RTC. (Current time as input by user minus timezone and if DST active minus one equals UTC.)

No stress please, we all appreciate what you are doing a lot but I myself enjoy the exchange with you way to much to not comment despite the fact that the clock on my desk works perfectly for me personally 😄

PS: Yes I saw your delayed flashwrite in the code and was blown away by the ingenuity of the idea to "hide" it there 😂

[astlouys]

Hi Lukas,

You brought interesting points to the discussion...

I agree about the two different concepts of Timezone and DST (in fact, some timezone simply ignored DST and "keep the same time" all year round). We can see on a map that a timezone is - more or less - "a slice" from north to south. Following your comments, we can think of it as a "physical, horizontal distance" of this vertical slice from a distant point (originally Greenwich).

I didn't know network equipment were using this concept of the three different parameters (I must admit that I didn't work often on network developments, except for 3 or 4 very specific projects). And of course, I understand there are many ways to reach to a workable solution. Five developers will end up with five different solutions and it's always interesting to see other point of view...

Don't worry Lukas, I don't stress too much (smile). I worked enough with email and other "virtual exchanges" that I know it may seems so, since there is no voice intonation to place the conversation into context. Sorry if I gave this impression. I also appreciate those exchanges. In fact, following those, I completely rewrote the DST algorithm today, based on the experience spent to write the code so far, and following the potential flaws that you and Eric brought up in this discussion (no... I'm still not using three parameters, but it's not a judgment call, it's simply that it's easier for me to develop the algorithm the way I learned DST and timezone (smile again !). There are so many possible use case that it takes relatively a long time to test everything. Again, you help is appreciated.

P. S. Wait till you see what PWM makes for display auto-brightness.

[UltrashRicco]

Happy new year a,d best wishes, folks! May your project be met with success!

Sorry I have had zero time to tinker with the Pico Green Clock recently.

Take good care!

[astlouys]

Hi Eric,

Best wishes to you as well and many interesting projects for the coming year !

Don't worry about Green Clock testing. I guess that when people will find problems, they will join us to report them so that they can be fixed. To be honest, I had to work on other projects for some time and I also missed some time for the clock. In any case, I look forward to release version 8.00 in the coming days / weeks.

Regards.

André

[KaeroDot]

Hi
I did some summer time testing with version 9.0 and NTP. First I have setup my computer as a local NTP server, and compiled the clock firmware with NTP address to my server. This worked. Next I have set my NTP server to date 26. 3. 2023, time 1:30. Started clock, it synced (clock settings was ST 5, TZ 1, after start it says DST Europe, inactive).
At time 2:00 the clock correctly changed to 3:00 as should be (2:00 stayed on the display for about a second.)
So I can say the NTP and DST works correctly for this case.

I did more tests, but I was not able to concentrate fully (kids) so I could have made something incorrectly:
I have experienced issues when going back in time. I have set time 1:30 on my ntp server. After clock restart the display showed DST active(!) during boot, and time 1:30 on the display. After second clock restart the DST was inactive but shown time was 0:30, and TZ was 0(!). It seems to me DST change affects timezone somehow.

To summarize, I am able to do DST tests with local NTP. If better tests are needed, I need to know how often the clock syncs with NTP server and how often the clock setting is written into DS3231 chip? Maybe what tests should I focus on? Definitely I have yet to test DST change in 29. 10. 2023.

[astlouys]

Hi KaeroDot,

Here is a test plan if you want to test handling of Summer Time / Winter Time with NTP being enabled.

This test plan assumes that you didn't change default values in the Firmware, namely the timings for the date scroll.

This plan also assumes that you are in Czech Republic where time difference with UTC is now +1 (that is, your local time is one hour "ahead" or "later" than UTC time). So, it assumes that the change to Summer Time will occur on Sunday 26-MAR-2023 at 1h00 UTC time (2h00 local time) and the change to Winter Time will be on Sunday 29-OCT-2023 at 1h00 UTC (3h00 local time).

1. Set your NTP server with time and date somewhere in January (2023). This is during Winter Time and in current year (2023).

2. Power-up the clock and make sure time and date are properly setup by NTP (should be done as soon as you see the time on clock display, but in the worst case, a few seconds later).

3. Let the clock run until a full date scroll has been completed.

4. Enter the clock configuration and make sure "DST country" and "Timezone" are properly adjusted (they should be respectively "5" and "+1").

5. Let the clock run until a full date scroll has been completed.

6. Make a small change to your NTP server. Add a few days (still in January 2023) and add also a few hours and a few minutes.

7. Make a power-up cycle on the clock (turn it Off, then Turn it On). Make sure the time and date have been properly re-adjusted to what they are on your NTP server.

8. Enter the clock configuration and make sure the "DST country" and "Timezone" are still correct.

9. If either "DST country" of "Timezone" (or both) is incorrect, re-adjust it and roll-back to step 5 above.

10. Set your NTP server to 25-MAR-2023 at 23h45.

11. Let the clock cross day change (00h00) and let it run until a full date scroll has been completed.

12. Your NTP server should already be on 26-MAR-2023 now. Adjust the time to 00h41.

13. Before 00h45 on your NTP server, enter clock setup and confirm once again that "DST country" and "Timezone" are still OK.

14. Make sure nothing special occurs when crossing 01h00 (your Summer Time algorithm is based on UTC time whereas for most countries it is based on local time).

15. Let the clock complete a full date scroll.

16. Set your NTP server at 01h45.

17. Again, check that "DST country" and "Timezone" are OK on the clock.

18. Observe time change from 2h00m01s to 3h00m01s (I let a one second delay to prevent race condition between the "Green Clock time" and the "DS3231 time" that I observed a few times (in the order of usec or msec).

19. Enter clock setting and check "DST country" and "Timezone". Your local time should now be "+2 hours ahead" of UTC time (the parameter that is called "Timezone" on the Green Clock).

20. Let the clock complete a full date scroll.

21. Set you NTP server to 29-OCT-2023 at 01h41.

22. Make a power-up cycle on the clock. Make sure time, date, "DST country" and "Timezone" are all OK. Timezone should still indicate +2 hours.

23. Now, to make a valid test, you need to let the clock run until passed 3h00 twice. The algorithm "recognizes" the first time it crosses 2h00 and 3h00, and also the second time (since we decrease by one hour). The one-hour fall-back should occur only once. Trying to accelerate testing at this point will probably add more confusion and make identifying potential problems more difficult.

24. Have a look once more to "DST country" and "Timezone". Obviously should still be 5, but Timezone should be back to "+1".

Let me know how it goes and if you encounter any glitch, tell me at what step and what was the actual behaviour.

ASL

[KaeroDot]

Thanks for the testing plan.

I made no timing change in the firmware. I have only commented out lines 4870 to 4843 in Pico-Clock-Green.c to disable blinking dot (I do not like blinking things) and of course I have changed IP address of NTP server to my own one in file picow_ntp_client.c.

It would help to know how often the clock syncs with NTP server? And I suppose the clock setting is written into DS3231 chip every displayscroll?

The result of testing. My notes are in italics:

1. Set your NTP server with time and date somewhere in January (2023). This is during Winter Time and in current year (2023).
   NTP was set to Tue 10 Jan 16:58:02 CET 2023.
2. Power-up the clock and make sure time and date are properly setup by NTP (should be done as soon as you see the time on clock display, but in the worst case, a few seconds later).
   Both time and date of clock was set immediately and correctly.
3. Let the clock run until a full date scroll has been completed.
   Done.
4. Enter the clock configuration and make sure "DST country" and "Timezone" are properly adjusted (they should be respectively "5" and "+1").
   Checked. Clock settings: ST 5, TZ 1, DS ON. Startup shows DST Europe, inactive.
5. Let the clock run until a full date scroll has been completed.
   Done
6. Make a small change to your NTP server. Add a few days (still in January 2023) and add also a few hours and a few minutes.
   NTP was set to Fri 13 Jan 23:30:14 CET 2023
7. Make a power-up cycle on the clock (turn it Off, then Turn it On). Make sure the time and date have been properly re-adjusted to what they are on your NTP server.
   Both time and date of clock was set immediately and correctly.
8. Enter the clock configuration and make sure the "DST country" and "Timezone" are still correct.
   Both correct, ST was 5, TZ was 1.
9. If either "DST country" of "Timezone" (or both) is incorrect, re-adjust it and roll-back to step 5 above.
   Not needed.
10. Set your NTP server to 25-MAR-2023 at 23h45.
    NTP server was set to Sat 25 Mar 23:45:09 CET 2023
11. Let the clock cross day change (00h00) and let it run until a full date scroll has been completed.
    (I suppose the clock day change should happen later than NTP server day change.)
    During clock cross day change, the NTP clock was Sun 26 Mar 00:11:46 CET 2023.
    After clock cross day change, the clock shows Sat 14th Jan 00:00.
12. Your NTP server should already be on 26-MAR-2023 now. Adjust the time to 00h41.
    NTP was already on 26 Mar 2023. NTP server was set to Sun 26 Mar 00:41:05 CET 2023.
13. Before 00h45 on your NTP server, enter clock setup and confirm once again that "DST country" and "Timezone" are still OK.
    Clock was OK, the ST was 5, TZ was 1.
14. Make sure nothing special occurs when crossing 01h00 (your Summer Time algorithm is based on UTC time whereas for most countries it is based on local time).
    Now clock shows 14. Jan 23 00:18 and NTP shows 26 Mar 2023 CET 00:58.
    After NTP crossing clock shows 14 Jan 2023 00:29 and NTP shows 26 Mar 2023 CET 01:08.
    After clock crossing clock shows 01 Jan 2023 01:01 and NTP shows Sun 26 Mar 01:40:32 CET 2023.
    After some time (not sure when exactly), the clock changed time and date to Sunday 26 Mar 2023 01:50, NTP was Sun 26 Mar 01:48:25 CET 2023.
15. Let the clock complete a full date scroll.
    Done
16. Set your NTP server at 01h45.
    Already there just by waiting for clock to change through 01:00.
17. Again, check that "DST country" and "Timezone" are OK on the clock.
    Clock was OK, the ST was 5, TZ was 1.
18. Observe time change from 2h00m01s to 3h00m01s (I let a one second delay to prevent race condition between the "Green Clock time" and the "DS3231 time" that I observed a few times (in the order of usec or msec).
    Clock showed 2:00 and 3:00 right after that, and started scrolling immediately.
    Date was 26 Jan 2023.
    NTP was Sun 26 Mar 03:02:03 CEST 2023
19. Enter clock setting and check "DST country" and "Timezone". Your local time should now be "+2 hours ahead" of UTC time (the parameter that is called "Timezone" on the Green Clock).
    Clock ST was 5, TZ was 2.
20. Let the clock complete a full date scroll.
    Done. It was second scroll after time change, because the first one was immediately after clock have changed from 2:00 to 3:00.
21. Set you NTP server to 29-OCT-2023 at 01h41.
    NTP was set to Sun 29 Oct 01:41:01 CEST 2023
22. Make a power-up cycle on the clock. Make sure time, date, "DST country" and "Timezone" are all OK. Timezone should still indicate +2 hours.
    Clock reports DST Europe, DST active. ST was 5, TZ was 2.
23. Now, to make a valid test, you need to let the clock run until passed 3h00 twice. The algorithm "recognizes" the first time it crosses 2h00 and 3h00, and also the second time (since we decrease by one hour). The one-hour fall-back should occur only once. Trying to accelerate testing at this point will probably add more confusion and make identifying potential problems more difficult.
    Transition 01:59 to 02:00 happened without any issues. ST was 5, TZ was 2.
    After one hour the NTP changed from 02:59 to 02:00. The clock changed 02:59 to 03:00 and after a second the display changed to 02:00. ST was 5, TZ was 1.
    After another hour the clock changed from 02:59 to 03:00 and continued.
24. Have a look once more to "DST country" and "Timezone". Obviously should still be 5, but Timezone should be back to "+1".
    ST was 5, TZ was 1.

So it seems clock works as should. It took quite a long time to test, but I am now confident the clock will wake me up properly even during DST change!

[astlouys]

Hello KaeroDot,

I'm glad that you went through the test and are now more confident about Daylight saving time handling... and thank you for running it ! As of me, I will test it "real time" next week-end since this is Summer Time toggling for most of Canada. Nonetheless, I would suggest you have a backup wake up media if you have important appointment around the date of Summer Time / Winter Time change.

One thing is not clear, however: in step 18, you mention that clock date was 26-JAN-23... I assume you meant 26-MAR-23 (?) since you wrote in step 14 that the clock date was 26-MAR-23.

ASL

[KaeroDot]

Ah, yes, my keyboard mistake, step 18 should be March.
If some more DST & NTP testing will be needed, just tell me.

[astlouys]

Thank you for your time and your tests, KaeroDot. I hope many users will benefit ! ;-)