time: NewTimer firing later if computer sleeps, how to use wall clock? #35012

gabzim · 2019-10-19T20:03:49Z

What version of Go are you using (`go version`)?

go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/gabriel/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/gabriel/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/gj/lr5vqgfn7jj_431xw83k37qc0000gn/T/go-build372524712=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

go func() {
	timer := time.NewTimer(timeUntilAlarm)
	select {
	case <-timer.C:
		log.WithField("event", event.Summary).Info("Firing alarm for event")
		alarmFired <- event
	case <-ctx.Done():
		// alarm cancelled
		timer.Stop()
	}
}()

What did you expect to see?

This works until my macbook goes to sleep (macOS) I'm guessing that it's because of the monotonic clock stopping. Is there a way to create a Timer that fires using the wall clock?

I need the alarm to fire at the right time (after x amount of minutes/hours have passed) regardless of whether the computer went into sleep mode in between the time when the timer was created and when it was supposed to fire.

What did you see instead?

If macbook goes to sleep, timer clock stops and it fires way later than it was supposed to (depending on how long the computer slept)

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2019-10-19T20:19:20Z

This seems related to #24595 on GNU/Linux.

Looking at the runtime package code I'm slightly surprised that this doesn't already work as you expect. I'm not sure how to fix it.

networkimprov · 2019-10-20T02:31:42Z

Apparently there are two options: #24595 (comment)

Maybe there should be two kinds of timer in Go, absolute and actual?

cc @eliasnaur @zx2c4

odeke-em · 2019-10-20T03:08:55Z

A similar issue from last year 2018, that could use reading differences between the monotonic and wall clocks #29308

zx2c4 · 2019-10-20T07:16:18Z

I patch Go using this patch for WireGuard:

https://git.zx2c4.com/wireguard-apple/tree/wireguard-go-bridge/goruntime-boottime-over-monotonic.diff

It's a network protocol, and phones and laptops like to sleep a lot these days, so this winds up being rather important.

gabzim · 2019-10-20T19:44:03Z

I found a lot of information about time.Sub but since time.NewTimer() only receives a Duration I'm not even sure if this is possible currently in go.

zx2c4 · 2019-10-20T19:57:35Z

Yea for that patch above to be complete, I'll need to augment the semaphore code (and futex code on Linux), either doing the same trick as on Windows, or perhaps finding some special argument. On the linux front, I talked to tglx (kernel timer maintainer) a few weeks ago about making futex boottime aware, and so maybe we'll push that forward for future improvements, if nothing currently existing suffices. On the macOS front, I haven't researched yet.

as · 2019-10-21T22:34:36Z

I need the alarm to fire at the right time (after x amount of minutes/hours have passed) regardless of whether the computer went into sleep mode in between the time when the timer was created and when it was supposed to fire.

This is impossible if the computer is asleep during the time which time.Timer is intended to fire.

What would you like time.Timer to do in this scenario?

I find that allowing time.Timer to fire after an arbitrary duration into the future unsettling, but it is even more unsettling when the input argument is a wall clock time.

I believe the correct front-facing API solution to this potential problem is not to create a new timer, but to add a NewTimerAt package function which takes in a Time argument instead of a duration.

func NewTimer(d Duration) *Timer
func NewTimerAt(t Time) *Timer

That being written, I do not think this is a problem worth solving in the standard library due to its apparent implementation complexity.

mpx · 2019-10-22T00:24:18Z

I find that allowing time.Timer to fire after an arbitrary duration into the future unsettling

This is necessarily how the contract for timers/sleep/etc.. works. A program gets control some time after the delay you specified, ideally not too long afterwards. However, there are many reasons why a program may not get control "quickly":

CPU starvation / scheduling, memory paging,..
SIGSTOP / SIGCONT
Suspend / resume

The best we can hope for is that the OS/runtime doesn't unnecessarily delay providing control to the program. Programs need to be written to cope with this uncertainly otherwise they are likely buggy.

In this case, I don't think we need a new timer, we just need the runtime to provide control as soon as is practical. At least, that allows the program to delay further if it is more appropriate.

Afaict, having a timer that pauses during suspend is almost never what you want, and having a timer that fires as close to the intended wall clock duration is almost always what you want. If this is problematic, then the programs were likely buggy to begin with.

gabzim · 2019-10-22T00:50:02Z

I need the alarm to fire at the right time (after x amount of minutes/hours have passed) regardless of whether the computer went into sleep mode in between the time when the timer was created and when it was supposed to fire.

This is impossible if the computer is asleep during the time which time.Timer is intended to fire.

I understand this, of course I don't mean that I expect the timer to fire if the computer is asleep.

However something that has been happening to me is that I set a timer an hour from now, the computer sleeps for 15 mins within that hour (eg.) and the timer fires 1:15 hours later instead of 1 hour later. This is just a little counterintuitive.

It's one thing if the host is asleep during the time when the timer is supposed to fire but the monotonic clock pausing is something that makes it almost impossible to schedule any work reliably. I think the default should be a non pausing monotonic clock (eg CLOCK_BOOTIME, which I hope is supported in macos?) since it's just more common sense behavior.

as · 2019-10-22T03:38:11Z

The best we can hope for is that the OS/runtime doesn't unnecessarily delay providing control to the program.

If the process can sleep for t seconds as a result of a suspend event, why can't it sleep for t+c seconds as a result of a suspend event in addition to the remaining duration on the timer? We know that the user can suspend the machine for an arbitrary amount of time. They could even break the machine, suspending it for +inf seconds. Complex logic around minimizing the constant c is a best-effort attempt that guarantees nothing unless we know the specific duration the machine will be suspended.

mpx · 2019-10-22T09:57:03Z

If the process can sleep for t seconds as a result of a suspend event, why can't it sleep for t+c seconds as a result of a suspend event in addition to the remaining duration on the timer?

It could, but that's seems less useful. Timers can be delayed for many reasons (see above). Programs should be able to cope with that where possible. However, if the timer fires unnecessarily late there is no opportunity to correct it and the experience is likely worse.

Wall clock durations are well defined, and almost always what a programmer expects/assumes for timers. If a further delay is useful, it's easy enough to reschedule the timer.

Otherwise, it's unclear what time a programmer might want, and how to define it. CPU time? Time the OS is available to schedule processes? Time the OS is available to schedule a particular process (eg, handle CPU contention, SIGSTOP)? These seem to be proxies for "work done", or "work potential" which could be measured other ways. The need for timers like this isn't particularly common in my experience.

Consider a very common time event processing loop:

Process events expiring before now.
Find next event & calculate duration D to wall time T
Sleep D

Using wall clock durations it will process events as close to the intended clock as is practical. Using "OS runtime durations" it will be further delayed after suspend, even if it could have been triggered on time.

Complex logic around minimizing the constant c is a best-effort attempt that guarantees nothing unless we know the specific duration the machine will be suspended.

There are few guarantees with timers and program scheduling (see reasons above). If there is a choice between "wall clock durations" and "OS runtime durations", I'd pick the one that's more useful.
It doesn't need to be complicated.

zx2c4 · 2019-10-22T10:08:20Z

To be clear, I have no interest in implementing anything regarding the "wall clock". And I don't think @networkimprov's suggestions are viable.

My future participation in this general category will consist of moving go from CLOCK_MONOTONIC to CLOCK_BOOTTIME, which will solve the original poster's problem as well as some others.

networkimprov · 2019-10-22T10:24:37Z

As documented in the other thread, there are use cases for both kinds of timer (absolute and suspend-aware), and a lot of code that expects the current absolute kind. So the question should be whether to leave things as they are, or to add a suspend-aware timing API.

@zx2c4 if it's viable to change the current scheme to clock_boottime, how could it be non-viable to add a timer API which uses that?

as · 2019-10-22T16:01:20Z

Using wall clock durations it will process events as close to the intended clock as is practical. Using "OS runtime durations" it will be further delayed after suspend, even if it could have been triggered on time.

@mpx What happens if the wall clock's time is changed?

zx2c4 · 2019-10-22T16:13:21Z

Using wall clock durations it will process events as close to the intended clock as is practical. Using "OS runtime durations" it will be further delayed after suspend, even if it could have been triggered on time.

@mpx What happens if the wall clock's time is changed?

Disaster ensues. Hence my suggestion that we use CLOCK_BOOTTIME instead.

mpx · 2019-10-22T22:04:30Z

Sorry, it seems there is a misunderstanding. It doesn't matter if the wall clock time is changed, timer durations are unaffected (that's what happens now, provided you don't suspend/resume). What I'm refering to is "real time passed" vs "time system running passed". So yes, CLOCK_BOOTTIME is better on Linux.

Monotonic timers are paused during standby. Thus these timers won't fire after waking up. Fall back to periodic polling to detect too large clock jumps. See golang/go#35012 for a discussion of go timers during standby.

gabzim changed the title ~~time.NetTimer firing later if computer sleeps, how to use wall clock?~~ time.NewTimer firing later if computer sleeps, how to use wall clock? Oct 19, 2019

ianlancetaylor changed the title ~~time.NewTimer firing later if computer sleeps, how to use wall clock?~~ time: NewTimer firing later if computer sleeps, how to use wall clock? Oct 19, 2019

ianlancetaylor added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin labels Oct 19, 2019

ianlancetaylor added this to the Backlog milestone Oct 19, 2019

ianlancetaylor added the help wanted label Oct 19, 2019

networkimprov mentioned this issue Oct 22, 2019

runtime: use CLOCK_BOOTIME, not CLOCK_MONOTONIC, when possible #24595

Closed

This was referenced Nov 9, 2019

runtime: Timer buckets may get "stuck" for long periods of time after Windows 8/10 systems wake from sleep #31528

Closed

runtime: revert Windows change to boot-time timers #35482

Closed

networkimprov mentioned this issue Dec 14, 2019

time: add ExternalNow, etc for external time and timers #36141

Open

anjmao mentioned this issue Feb 20, 2020

Fix time.Sub after device resumes from sleep mysteriumnetwork/node#1749

Closed

WGH- mentioned this issue May 10, 2020

Clock is incorrent after suspend soumya92/barista#150

Closed

MichaelEischer mentioned this issue Sep 10, 2022

Strict repository lock handling restic/restic#3569

Merged

8 tasks

sasha-s mentioned this issue Dec 20, 2022

token has expired. pulumi/pulumi#7094

Closed

inteon mentioned this issue Sep 12, 2023

Add health probe that detects skew between system clock and monotonic go process clock cert-manager/cert-manager#6328

Merged

seankhliao mentioned this issue Apr 17, 2024

time: macOS monotonic time paused during sleep #66870

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

time: NewTimer firing later if computer sleeps, how to use wall clock? #35012

time: NewTimer firing later if computer sleeps, how to use wall clock? #35012

gabzim commented Oct 19, 2019

ianlancetaylor commented Oct 19, 2019

networkimprov commented Oct 20, 2019 •

edited

Loading

odeke-em commented Oct 20, 2019

zx2c4 commented Oct 20, 2019

gabzim commented Oct 20, 2019

zx2c4 commented Oct 20, 2019

as commented Oct 21, 2019

mpx commented Oct 22, 2019

gabzim commented Oct 22, 2019 •

edited

Loading

as commented Oct 22, 2019

mpx commented Oct 22, 2019

zx2c4 commented Oct 22, 2019

networkimprov commented Oct 22, 2019 •

edited

Loading

as commented Oct 22, 2019

zx2c4 commented Oct 22, 2019

mpx commented Oct 22, 2019

time: NewTimer firing later if computer sleeps, how to use wall clock? #35012

time: NewTimer firing later if computer sleeps, how to use wall clock? #35012

Comments

gabzim commented Oct 19, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

ianlancetaylor commented Oct 19, 2019

networkimprov commented Oct 20, 2019 • edited Loading

odeke-em commented Oct 20, 2019

zx2c4 commented Oct 20, 2019

gabzim commented Oct 20, 2019

zx2c4 commented Oct 20, 2019

as commented Oct 21, 2019

mpx commented Oct 22, 2019

gabzim commented Oct 22, 2019 • edited Loading

as commented Oct 22, 2019

mpx commented Oct 22, 2019

zx2c4 commented Oct 22, 2019

networkimprov commented Oct 22, 2019 • edited Loading

as commented Oct 22, 2019

zx2c4 commented Oct 22, 2019

mpx commented Oct 22, 2019

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

networkimprov commented Oct 20, 2019 •

edited

Loading

gabzim commented Oct 22, 2019 •

edited

Loading

networkimprov commented Oct 22, 2019 •

edited

Loading