Increase the time resolution beyond whole seconds for rate mode (argument -r)

[edgar-bonet]

The program torture.c outputs a waveform of amplitude 5 and period 0.72 s. Its peak derivative is therefore expected to be

5 × (2π ÷ 0.72) ≈ 43.6

However,

./torture | ./ttyplot -r

shows a waveform that peaks at about 0.5, which is off by two orders of magnitude. The reason is that data points arrive every 10 ms, and ttyplot “rounds” this value to a full second, effectively underestimating the rate by a factor 100.

This pull request fixes this behavior by handling time with microsecond resolution. It also moves the computation of rates to its own function for better code clarity.

[hartwork]

Hi @edgar-bonet I think there is a chance that there is a misunderstanding of what -r was meant to do. Maybe I'm wrong, that's okay, let's find out!

The way I understand the current -r is that it is meant to be used with accumulators, values that only ever go up. For example, let's take a data source like this:

# while grep numa_hit /proc/vmstat | awk '{print $2}' ; do sleep 0.25 ; done
113308336
113308915
113309576
113310145
113310700
113311263
113311825
113312415
113312971
113313544
^C

Now if we diff these values, we can see they are roughly 700 apart:

# a=0; while sleep 0.25; do b="$(grep numa_hit /proc/vmstat | awk '{print $2}')"; echo "$((b - a))"; a="${b}"; done
113417603
678
685
688
686
692
782
706
693
695
698
^C

When I pipe that as input to ./ttyplot -r now as…

# while grep numa_hit /proc/vmstat | awk '{print $2}' ; do sleep 0.25 ; done | ./ttyplot -r

…on master, ttyplot display values and average match that "roughly 700 apart" reality, seems like a match.

When I do the same now on branch rate(commit 0863f67) the values and average are way higher, more like 2400.

Therefore my impression is that we have two different scenarios here and that maybe we need an additional flag to support your scenario in addition under a new command line argument rather than replacing the current -r.

What do you think?

[edgar-bonet]

@hartwork: Replace sleep 0.25 with sleep 2. You will see that both master an this PR's branch compute the same thing: a rate of roughly 2400 hits/s.

[hartwork]

@hartwork: Replace sleep 0.25 with sleep 2.

@edgar-bonet … because the old code uses a timer with seconds resolution and therefore 0.25 is too fast for it then I suppose, interesting!

I guess then maybe this pull request is just doing the right thing? 😃

The variable naming x and y seem a bit unfortunate given that (1) both seem to be y coordinates and (2) the application uses x and y already for 2-dimensional coordinates in ncurses space. If you could pick a different pair of names, I'd likely be all for the new ones. What do you think?

[edgar-bonet]

@hartwork wrote:

The variable naming x and y seem a bit unfortunate

Indeed it is. I replaced them with v1 and v2, which are more consistent with the current naming. I also added const qualifiers as you suggested. Squashed and force-pushed.

[hartwork]

@edgar-bonet thanks for the adjustments 👍 Let's see how @tenox7 likes it.

[tenox7]

interesting, so basically if samples for rate calculator are sub second interval the data will be lost? I would love to fix this, but I will need to do a lot more testing, typically with network equipment/snmp where I use it the most

[tenox7]

also note that torture.c was never designed for rate testing, it would be cool if you could add it as well, maybe make the sleep/interval an arg/flag and option for rate (-r) ?

ideally torture -r should be able to generate a similar wave plot like without -r by manipulating rate (higher/lower)

[edgar-bonet]

@tenox7 wrote:

if samples for rate calculator are sub second interval the data will be lost?

Not exactly. Each number received on the input adds one bar to the plot, so no data is lost. The issue is that, in master, the values are incorrect. If the interval is large, the rates displayed are real rates (i.e. time derivatives), in counts per second. If the interval becomes smaller than one second, then ttyplot “rounds” it up to one full second, which results in overestimated rates.

The issue also impacts intervals that are longer (but not much longer) than one second. For example, the command

for n in $(seq 1 10 1000); do echo $n; sleep 1.5; done

creates a counter that increments by ten units every 1.5 seconds. This is a rate of 6.7 counts per second. Piping this to ttyplot in master shows a rate that alternates between 5 and 10. With this pull request, ttyplot shows a very steady rate of 6.7.

torture.c was never designed for rate testing, it would be cool if you could add it as well [...]

I would gladly do so. Within this PR or in a follow-up one?

[hartwork]

I would gladly do so. Within this PR or in a follow-up one?

My vote for merging to master what we have here and then doing a follow-up. Fewer bits in flight, baby steps, shorter-lived branches. Clear vote for a merge from my side.