Some items in the <pids> API carry a '_C' suffix which
implies such values include reaped children. So, we'll
now extend this practice to that new UTILIZATION item.
Reference(s):
. Mar, 2022 - tweak PIDS_UTILIZATION
commit 9c0e8e9429
. Feb, 2022 - added PIDS_UTILIZATION
commit c69104b2b8
Signed-off-by: Jim Warner <james.warner@comcast.net>
When preparing the ps program for a %CUU field, it was
revealed the PIDS_UTILIZATION item may sometimes yield
a result of 'nan' or 'inf' for that ps program itself.
Apparently, it was caused by the short lived nature of
such a one-shot program. And, while this anomaly could
be handled in ps, the solution belongs in the library.
So, I reworked an item's algorithm to avoid this oops.
Signed-off-by: Jim Warner <james.warner@comcast.net>
While we really do not need the extra precision, we'll
trade that recently introduced float type for a double
for efficiency on a 64 bit platform. Additionally, the
UTILIZATION algorithm was tweaked producing less code.
The net result is four fewer machine instructions with
a reduction of 14 total bytes in that function's size.
Reference(s):
. introduced 'real' type & 'utilization' item
commit c69104b2b8
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit introduces some new capabilities available
in libproc-2 under the <PIDS> interface. Along the way
errors impacting some item values have been corrected.
The following summarizes the major changes being made.
1. The PIDS_TIME_START item was represented as seconds
since system boot but really held tics since boot. And
some programs properly divided it by Hertz to actually
yield seconds while others acted as if it already was.
So, now we have a new PIDS_TICS_BEGAN field and all of
the 'TIME' fields properly reflect seconds. With those
'TIME' fields, the type was changed to 'float/real' so
one could convert it back to tics without loss of some
centiseconds reflected in the Hertz guy (usually 100).
2. The boot_seconds was established in procps_pids_new
meaning it was fixed/unchanging. As a result, one item
(PIDS_TIME_ELAPSED) was rendered useless. So now, each
of the three retrieval functions establishes a current
boot_seconds well before the set functions are called.
3. Added a PIDS_UTILIZATION item that will provide the
CPU usage over the life of a process, as a percentage.
4. Added PIDS_TIME_ALL_C for symmetry with the similar
item called PIDS_TICS_ALL_C (which reflects raw tics).
5. That 'derived from' notation has been added to some
additional header items to reflect their true origins.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Since gcc no longer warns of 'unreachable code', we'll
just deal with any such possibility ourselves I guess.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This started off with fixing the compilier warning:
proc/readproc.c: In function ‘simple_nextpid’:
proc/readproc.c:1373:38: warning: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 58 [-Wformat-truncation=]
1373 | snprintf(path, PROCPATHLEN, "/proc/%s", ent->d_name);
| ^~
proc/readproc.c:1373:3: note: ‘snprintf’ output between 7 and 262 bytes into a destination of size 64
1373 | snprintf(path, PROCPATHLEN, "/proc/%s", ent->d_name);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We know that ent->d_name will fit under 64 bytes because we check the
name starts with a digit. The first change was simple and changed the
printf to use tgid like the task function below it.
Is everything under /proc that starts with a digit a directory with a
PID only? Today, it is but there are no guarantees.
The entire function works ok if every non-pid directory doesn't start
with a number. We don't check for strtoul() having an issue nor
if the for loop just falls off the end. The moment the kernel guys
(or some module writer) think "/proc/12mykernelval" is a neat idea this
function is in trouble. We won't get buffer overflow as we are
using snprintf at least.
This change now:
We check if strtoul() actually came across a number
Process the pid directory as a conditional branch
Treat falling off the for loop as a not-found
Signed-off-by: Craig Small <csmall@dropbear.xyz>
[ you wouldn't believe how many back-and-forths were ]
[ involved in Craig convincing me there were several ]
[ inconsistencies. i am so dense sometimes (often?). ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch was prompted by Björn Fischer's merge #147
request referenced below. It has been generalized such
that it now embraces both of those 'pids_fetch' types.
[ and thanks to Björn for initiating this extension ]
Reference(s):
https://gitlab.com/procps-ng/procps/-/merge_requests/147
Prototyped-by: Björn Fischer <bf@CeBiTec.Uni-Bielefeld.DE>
Signed-off-by: Jim Warner <james.warner@comcast.net>
This just updates the copyright dates in the documents
where I was already represented. Others are unchanged.
Signed-off-by: Jim Warner <james.warner@comcast.net>
For some unknown reason all the 'info' structures were
declared between macros and function prototypes rather
than right after all the other structure declarations.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This change is really being made on behalf of the ps &
top programs. Besides, over 2 billion CPUs are plenty!
Signed-off-by: Jim Warner <james.warner@comcast.net>
Our new library's now well protected against potential
problems which arise when a multi-threaded application
opens more than one context within the same API at the
same time. However, with a single-threaded application
designed along those same lines, some problems remain.
So, to avoid potential corruption of some data (which
was classified as local 'static __thread') from those
single-threaded designs, we'll move several variables
to the info structure itself and remove the '__thread'
qualifier. Now they're protected against both designs.
[ we'll not be protected against some multi-threaded ]
[ application that shares a single context yet calls ]
[ that interface from separate threads. this is just ]
[ bad application design & no different than sharing ]
[ other modifiable global data between such threads! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
upminutes and uphours could both not get initialised in the
procps_uptime_sprint_short() function.
Error was probably introduced at the referenced commit.
References:
Coverity 240787 Uninitialized scalar variable
Coverity 240776 Uninitialized scalar variable
commit 0496b39876
Signed-off-by: Craig Small <csmall@dropbear.xyz>
This commit will place the 'file2strvec' function on a
par with the 'file2str' function if ENOMEM was raised.
Plus, just to keep coverity happy, we'll also clean up
after potential failures with the 'openproc' function.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This MR revisits a partial fix from 2018. The problem stems from incorrect
handling of unsigned 32-bit uid_ts and gid_ts as signed when values are
large - i.e. when the high bit is set. In that case, pgrep and pkill fail to
identify processes by uid. (They succeed when finding the same processes by
username.) The primary fix for this is to impliment the "FIXME" comment in
proc/readproc.h, the implementation of which allows the removal of the (int)
casts from the partial fix from 2018.
The other fixed code in this MR consists of tests in strict_atol() that
detects and errors out on overflows.
References:
Merge !146
In the commit referenced below, a '__thread' attribute
was added to numerous static variables to protect them
from concurrent access conflicts with multi-threading.
Unfortunately, that patch did not go quite far enough.
So, this commit adds a few more '__thread' qualifiers.
Reference(s):
commit 23cfb71366
Signed-off-by: Jim Warner <james.warner@comcast.net>
uptime -p would show empty output after 52 weeks of uptime. This commit
is largely the work of Ed but reformatted for newlib branch.
Signed-off-by: Craig Small <csmall@dropbear.xyz>
References:
procps-ng/procps!141procps-ng/procps#217
Even though we we had to abandon the master branch top
multi-thread effort and even though the newlib version
of a multi-threaded top provides no real benefit, that
whole exercise was not wasted. Rather, it has revealed
some deficiencies in our library which this addresses.
If two or more threads in the same address space tried
to use procps_loadavg or procps_uptime simultaneously,
there's a chance they would experience problems due to
thread-unsafe functions our library called internally.
So, this patch switches them for thread-safe versions.
[ along the way we will also make that procps_uptime ]
[ initialization of his 'up' & 'idle' variables mean ]
[ something by delaying the -ERANGE return a little. ]
Reference(s):
https://www.freelists.org/post/procps/a-few-more-patches,7
Signed-off-by: Jim Warner <james.warner@comcast.net>
Even though we we had to abandon the master branch top
multi-thread effort and even though the newlib version
of a multi-threaded top provides no real benefit, that
whole exercise was not wasted. Rather, it has revealed
some deficiencies in our library which this addresses.
If two or more threads in the same address space tried
to access the same api simultaneously, there is a good
chance some function-local static variables will yield
some of those renowned unpredictable results. So, this
patch protects them with the '__thread' storage class.
Reference(s):
https://www.freelists.org/post/procps/a-few-more-patches,7
Signed-off-by: Jim Warner <james.warner@comcast.net>
Ever since 2003, the 'listed_nextpid' routine has been
misrepresenting its duties. Far from finding processes
in a list given to openproc, it just inserted the next
pid in that list into the passed proc_t as BOTH a tgid
and tid. There was no attempt to validate such values.
The net result is that tid & tgid were valid only with
a thread group leader. When called with a pid for some
sibling thread, the resulting tgid would be incorrect.
With this commit, our little function will now attempt
to validate both the tid and tgid. If this should fail
then the fallback position will be the same as what we
inherited. So we're no worse off & likely much better.
[ note that calling the function with a thread's pid ]
[ likely stems from 2011 when a 'readeither' routine ]
[ was added which dealt with both tasks and threads! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
In the patch shown below, two lines involving the flag
PROC_UID were uncommented (enabled). However given the
construct of the readeither function, it is impossible
for the simple_readtask guy to be called when its TGID
leader has already been ignored. So, let's disable it.
[ it's only now true that the lines serve no purpose ]
[ after the commit shown below tweaked readeither to ]
[ access the base directory of the tgid leader. but, ]
[ before that, the 2 lines should have been enabled! ]
Reference(s):
. two lines uncommented
commit af34cc964a
. tweaked readeither
commit a375262609
Signed-off-by: Jim Warner <james.warner@comcast.net>
The patch referenced below corrected some flaws in the
procps_pids_select implementation. But, there remained
one flaw which this commit will now hopefully address.
Rather than assume callers wished to select only tasks
and not threads meant a command like 'top -H -p 10329'
works differently under newlib than release 3.3.17. It
fails to honor the '-H' (threads) switch under newlib.
So, to fix that oops, we'll allow that select function
to get threads or tasks depending on its 'which' parm.
Reference(s):
. Oct 2015, some flaws corrected
commit bc616b3615
Signed-off-by: Jim Warner <james.warner@comcast.net>
With that recent addition of the autogroup provisions,
it became apparent that '/proc/<pid>/task' information
is sometimes incomplete. In fact, there's no autogroup
file at all in any '/proc/<pid>/task/<pid>' directory!
As a result, when the top -H mode was invoked, all the
processes showed a -1 for AGID, even the group leader.
So, this commit will ensure that for every TGID leader
its basic '/proc/<pid>' directory will always be used.
The 'task' subdirectory is now only used for siblings.
[ and it's time that readeither prologue was updated ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
In the link referenced below there's an explanation of
the linux autogroup feature which has been around ever
since linux-2.6.38. With that explanation there's also
surprising (maybe shocking) revelations about the nice
and renice commands if CONFIG_SCHED_AUTOGROUP was set.
When autogroups are active, such programs are rendered
mostly useless because the nice value will only affect
scheduling priority relative to other processes in the
same autogroup. In order to accomplish what we thought
of as renice, that nice value in /proc/<pid>/autogroup
must be changed. Altering any single member of a group
will also affect every other member of that autogroup.
So, this commit will set the stage for users of newlib
to display autogroup identifiers plus their associated
nice values (now that their importance is understood).
Reference(s):
https://github.com/nlburgin/reallynice
Signed-off-by: Jim Warner <james.warner@comcast.net>
The 'PROC_FILL' flags, found in readproc.h, had become
almost unmanageable. The hex values were scattered all
over the map as new flags had been introduced. So this
commit resets all of them and will help ensure any new
flags don't duplicate some already existing hex value.
Signed-off-by: Jim Warner <james.warner@comcast.net>
When declaring a pointer there's usually a space after
the thing-pointed-to and no space between the asterisk
and the pointer-thingy itself. So this commit enforces
such conventions where needed on old library elements.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch just brings 'PIDS_IO' source into alignment
with the names being used in that /proc/<pid>/io file.
[ i had my chance to fix them in a whitespace change ]
[ made in the patch referenced below, but i blew it! ]
Reference(s):
commit 2dcbe71f3b
Signed-off-by: Jim Warner <james.warner@comcast.net>
That recent addition of USS to the ps program prompted
this change. Rather than have it (and soon top) add 2
separate items to yield the desired value, we will let
our new library perform the arithmetic when necessary.
Outside of a little extra storage, there is no runtime
costs for such an extension. There is, however, a real
benefit to having such code in the library. Now should
callers choose to sort on this new field, results will
be guaranteed to be what was expected (i.e. accurate).
Signed-off-by: Jim Warner <james.warner@comcast.net>
The two special hugetlbfs items were misnamed. The TBL
reference (table) should be TLB (transaction lookaside
buffer). Besides, I never liked their position anyway!
[ and one macro argument tweak is being snuck in too ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
A couple of people have suggested that smaps_rollup be
added to the ps program and/or top program. This patch
is intended to set the stage for just such extensions.
There are currently 20 displayable items in the rollup
file. And newlib sometimes uses sscanf when populating
the target, sometimes hsearch and one customized gperf
approach. None of these fit well with the smaps items.
Thus, an approach using a simple table lookup was used
and, by disabling 1 code line, it could be made immune
from changes to the items order (unlike a sscanf call)
and doesn't carry the greater cost of a hsearch/gperf.
Note: The next patch will allow top to display some of
these new fields. Then, it'll be possible to determine
the colossal costs of accessing the smaps_rollup file.
Here is a small preview of just what you will discover
when using the command 'time top/top -d0 -n1000' while
configured with just two fields: PID + 1 memory field.
------------------------------------ as a regular user
with only PID + RES (statm)
real 0m2.605s
user 0m1.060s
sys 0m1.377s
with only PID + RSS (smaps)
real 0m26.397s 10x more costly
user 0m1.253s
sys 0m24.915s
----------------- as a root (thus smaps for all tasks)
with only PID + RES (statm)
real 0m2.651s
user 0m1.177s
sys 0m1.286s
with only PID + RSS (smaps)
real 0m33.040s 12x more costly
user 0m1.256s
sys 0m31.533s
Reference(s):
. ps: expose shared/private memory separately
https://gitlab.com/procps-ng/procps/-/issues/201
. top/ps: add support for PSS reporting
https://gitlab.com/procps-ng/procps/-/issues/112
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit is strictly cosmetic. It was an attempt to
normalize/standardize/alphabetize those #define/#undef
statements. Some missing #undef's were added plus some
comments regarding sources corrected/standardized too.
Signed-off-by: Jim Warner <james.warner@comcast.net>
The commit referenced below was well done but needed a
small whitespace tweak to preserve existing alignment.
Reference(s):
. io accounting added
commit a7afe06e6f
Signed-off-by: Jim Warner <james.warner@comcast.net>
This is a modification of MR !122 by @renit1609 to fit the new
library.
Problem statement:
The procps library has no PROC_FILLIO flag to
fetch the proc field "/proc/[pid]/io" data
process-wise.
IO Accounting is not included as part of procps.
Requirement:
We have a requirement to fetch process wise
IO utilization which can be used for monitoring.
When looking through the procps library, I see
that IO Accounting (/proc/[pid]/io) is not being
included as part of procps. There is no such
flag like PROC_FILLIO being included in readproc.h .
Solution:
While looking at the implementation done for
other proc fields, I used the spare bits in app code.
I renamed PROC_SPARE_1 as PROC_FILLIO, the spare bit from
PROC_SPARE_* and used it for fetching /proc/[pid]/io
data as part of the procps library similar to other
fields. I moved the PROC_SPARE_* bits each by 1 bit
to retain the spare bits. Meanwhile added the IO fields
in proc_t structure.
References:
procps-ng/procps!122procps-ng/procps#184
nl_langinfo and CODESET are undefined in a musl system. Instead of
uncondionally including langinfo.h, this change includes include/nls.h
which has the tests and work-arounds for systems that don't have these
features. This is similar to how other programs within procps include
langinfo.h via nls.h
References:
procps-ng/procps!130
Signed-off-by: Craig Small <csmall@dropbear.xyz>
This simple two line code change fixes an intermittent
bug whereby %CPU for parent(s) with collapsed children
could be vastly understated from those displayed under
the current 3.3.17 publicly available top & libprocps.
If one started several top instances in the background
using very a small delay interval (zero?), then if the
shell under which they were running was collapsed, you
would see similar %CPU results for both the libraries.
However, when running a demanding 'make' like a kernel
compile (especially if backed by fast processors and a
SSD), then newlib would generally show only 1/3 to 1/2
of the collapsed %CPU values that appeared for 3.3.17.
Of course, now that the bug has been swatted with this
commit the disparities between those results is easily
explained. Since newly created tasks never contributed
tics during the interval where they were created, only
with many short lived tasks would differences surface.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Rather than run risks of more expensive and repetitive
address resolution, we will establish this local index
for a one time cost and avoid any potential gcc bloat.
[ this commit was made in pursuit of a bug involving ]
[ the distortion of elapsed task tics. but, it turns ]
[ out these changes had nothing to do with that bug. ]
[ however, the patch is being retained as desirable. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
That old library defined this field as 'unsigned int'.
However, here it was known as a 'signed int'. Thus for
consistency we'll now also treat it as 'unsigned int'.
[ this commit was made in pursuit of a bug involving ]
[ the distortion of elapsed task tics. but, it turns ]
[ out these changes had nothing to do with that bug. ]
[ however, the patch is being retained as desirable. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
After removing brackets from those 'derived' notations
I was surprised to discover that several origin/source
comments were wrong. So this patch fixes those errors.
[ along the way a couple enumerators were renamed to ]
[ better (i hope) reflect what they're representing. ]
[ that, in turn, also required a little rearranging. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
None of the other four new api headers use brackets on
derived items. With this patch we normalize the fifth.
[ it makes for a cleaner, less confusing, appearance ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This small change was a result of some experimentation
trading our current 'hsearch' hash scheme for 'gperf'.
I discovered that when the ':' character was a part of
each 'gperf' key, that generated search logic was more
complicated and thus slower. But without a ':', it was
a little cleaner/leaner and therefore slightly faster.
Assuming that the same trailing ':' *might* affect the
current 'hsearch' logic, to be safe we will remove it.
[ while the 'gperf' version will slightly outperform ]
[ an 'hsearch', too many ugly implementation details ]
[ were exposed which complicates future maintenance. ]
[ thus, we'll retain our current 'hsearch' approach. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch will condense some logic in those functions
associated with the file input operations. The changes
will not, for the most part, alter any generated code.
More significantly (though not very) was the change to
two 'strtoul' calls. Since the returned 'endptr' value
isn't exploited, when that parm is set to NULL, we can
save one instruction on each side of such calls (wow).
Signed-off-by: Jim Warner <james.warner@comcast.net>
When #define QUICK_THREADS was introduced, for copying
some task data for a child thread, one proc_t pad byte
was used to mark, then later identify, those children.
Later the QUICK_THREADS was recycled as FALSE_THREADS,
and used for a different purpose, but a conditional in
the header file erroneously remained. Now, it is gone!
Reference(s):
. Jul, 2016 - QUICK_THREADS become FALSE_THREADS
commit c546d9dd44
. Aug, 2011 - QUICK_THREADS intruduced
commit bb4f08ba29
Signed-off-by: Jim Warner <james.warner@comcast.net>
What to call the new library?
Keep using libprocps wouldn't do, its a very different library from
the programs' point of view. It would also mean we could have some
clashes around the packages (two package names, same library name).
The ancient procps used libproc or libproc-a.b.c where a.b.c was the
package version. Kept the revision numbers down (it was always 0.0.0)
but the name of the library changed.
So if we use libproc-2 is there a clash with an ancient procps?
procps v 2.0.0 was around in 1999 so it was 22 years ago, also the
name of the library would have been libproc-2.0.0.so not libproc-2.so
so we're fine with that.
libproc-2 seems to fit, our second major re-work of the procps
library.
Signed-off-by: Craig Small <csmall@dropbear.xyz>
Since 'procps_uptime' will access the /proc filesystem
the <pids> 'new' guy should should protect against the
possibility /proc isn't mounted when 'boot_seconds' is
established. A zero is better than the negative value.
[ the only distortion would be to PIDS_TIME_ELAPSED. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
