Commit Graph

724 Commits

Author SHA1 Message Date
Jim Warner
a375262609 library: ensure thread group leader retrieval accuracy
With that recent addition of the autogroup provisions,
it became apparent that '/proc/<pid>/task' information
is sometimes incomplete. In fact, there's no autogroup
file at all in any '/proc/<pid>/task/<pid>' directory!

As a result, when the top -H mode was invoked, all the
processes showed a -1 for AGID, even the group leader.

So, this commit will ensure that for every TGID leader
its basic '/proc/<pid>' directory will always be used.
The 'task' subdirectory is now only used for siblings.

[ and it's time that readeither prologue was updated ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-08-09 22:03:23 +10:00
Jim Warner
631e5d91f3 library: add provision for displaying autogroup values
In the link referenced below there's an explanation of
the linux autogroup feature which has been around ever
since linux-2.6.38. With that explanation there's also
surprising (maybe shocking) revelations about the nice
and renice commands if CONFIG_SCHED_AUTOGROUP was set.

When autogroups are active, such programs are rendered
mostly useless because the nice value will only affect
scheduling priority relative to other processes in the
same autogroup. In order to accomplish what we thought
of as renice, that nice value in /proc/<pid>/autogroup
must be changed. Altering any single member of a group
will also affect every other member of that autogroup.

So, this commit will set the stage for users of newlib
to display autogroup identifiers plus their associated
nice values (now that their importance is understood).

Reference(s):
https://github.com/nlburgin/reallynice

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-08-09 22:00:23 +10:00
Jim Warner
0360f64027 library: refactor all the readproc.h 'PROC_FILL' flags
The 'PROC_FILL' flags, found in readproc.h, had become
almost unmanageable. The hex values were scattered all
over the map as new flags had been introduced. So this
commit resets all of them and will help ensure any new
flags don't duplicate some already existing hex value.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-08-09 22:00:23 +10:00
Jim Warner
fca8bcbd40 library: standardize 'pointer-to-thing' whitespace use
When declaring a pointer there's usually a space after
the thing-pointed-to and no space between the asterisk
and the pointer-thingy itself. So this commit enforces
such conventions where needed on old library elements.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-07-27 21:17:18 +10:00
Jim Warner
e7acf3caf0 library: correct sources for IO accounting, <pids> api
This patch just brings 'PIDS_IO' source into alignment
with the names being used in that /proc/<pid>/io file.

[ i had my chance to fix them in a whitespace change ]
[ made in the patch referenced below, but i blew it! ]

Reference(s):
commit 2dcbe71f3b

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-06-23 18:23:49 +10:00
Jim Warner
3e94210fe4 library: add new derived smaps_rollup item, <pids> api
That recent addition of USS to the ps program prompted
this change.  Rather than have it (and soon top) add 2
separate items to yield the desired value, we will let
our new library perform the arithmetic when necessary.

Outside of a little extra storage, there is no runtime
costs for such an extension. There is, however, a real
benefit to having such code in the library. Now should
callers choose to sort on this new field, results will
be guaranteed to be what was expected (i.e. accurate).

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-06-17 16:42:58 +10:00
Jim Warner
bf0b3b6522 library: rename/reposition two enumerators, <pids> api
The two special hugetlbfs items were misnamed. The TBL
reference (table) should be TLB (transaction lookaside
buffer). Besides, I never liked their position anyway!

[ and one macro argument tweak is being snuck in too ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-05-02 20:08:38 +10:00
Jim Warner
12543b6c76 library: add support for smaps_rollup file, <pids> api
A couple of people have suggested that smaps_rollup be
added to the ps program and/or top program. This patch
is intended to set the stage for just such extensions.

There are currently 20 displayable items in the rollup
file. And newlib sometimes uses sscanf when populating
the target, sometimes hsearch and one customized gperf
approach. None of these fit well with the smaps items.

Thus, an approach using a simple table lookup was used
and, by disabling 1 code line, it could be made immune
from changes to the items order (unlike a sscanf call)
and doesn't carry the greater cost of a hsearch/gperf.

Note: The next patch will allow top to display some of
these new fields. Then, it'll be possible to determine
the colossal costs of accessing the smaps_rollup file.

Here is a small preview of just what you will discover
when using the command 'time top/top -d0 -n1000' while
configured with just two fields: PID + 1 memory field.

------------------------------------ as a regular user
    with only PID + RES (statm)
real       0m2.605s
user       0m1.060s
sys        0m1.377s
    with only PID + RSS (smaps)
real      0m26.397s                    10x more costly
user       0m1.253s
sys       0m24.915s

----------------- as a root (thus smaps for all tasks)
    with only PID + RES (statm)
real       0m2.651s
user       0m1.177s
sys        0m1.286s
    with only PID + RSS (smaps)
real      0m33.040s                    12x more costly
user       0m1.256s
sys       0m31.533s

Reference(s):
. ps: expose shared/private memory separately
https://gitlab.com/procps-ng/procps/-/issues/201
. top/ps: add support for PSS reporting
https://gitlab.com/procps-ng/procps/-/issues/112

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-04-29 21:42:44 +10:00
Jim Warner
b921776dad library: some miscellaneous tweaks before smaps_rollup
This commit is strictly cosmetic. It was an attempt to
normalize/standardize/alphabetize those #define/#undef
statements. Some missing #undef's were added plus some
comments regarding sources corrected/standardized too.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-04-29 21:42:44 +10:00
Jim Warner
2dcbe71f3b library: fix alignment with new 'io' types, <pids> api
The commit referenced below was well done but needed a
small whitespace tweak to preserve existing alignment.

Reference(s):
. io accounting added
commit a7afe06e6f

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-04-27 21:16:51 +10:00
Craig Small
a7afe06e6f library: adding IO accounting
This is a modification of MR !122 by @renit1609 to fit the new
library.

Problem statement:
The procps library has no PROC_FILLIO flag to
fetch the proc field "/proc/[pid]/io" data
process-wise.
IO Accounting is not included as part of procps.

Requirement:
We have a requirement to fetch process wise
IO utilization which can be used for monitoring.

When looking through the procps library, I see
that IO Accounting (/proc/[pid]/io) is not being
included as part of procps. There is no such
flag like PROC_FILLIO being included in readproc.h .

Solution:
While looking at the implementation done for
other proc fields, I used the spare bits in app code.
I renamed PROC_SPARE_1 as PROC_FILLIO, the spare bit from
PROC_SPARE_* and used it for fetching /proc/[pid]/io
data as part of the procps library similar to other
fields. I moved the PROC_SPARE_* bits each by 1 bit
to retain the spare bits. Meanwhile added the IO fields
in proc_t structure.

References:
 procps-ng/procps!122
 procps-ng/procps#184
2021-04-24 22:38:48 +10:00
Issam E. Maghni
6c36ab4fc7 escape.c: Fix missing nl_langinfo on certain configs
nl_langinfo and CODESET are undefined in a musl system. Instead of
uncondionally including langinfo.h, this change includes include/nls.h
which has the tests and work-arounds for systems that don't have these
features. This is similar to how other programs within procps include
langinfo.h via nls.h

References:
 procps-ng/procps!130

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-04-24 11:46:48 +10:00
Jim Warner
c082779769 library: fix an insidious bug affecting TICS_ALL_DELTA
This simple two line code change fixes an intermittent
bug whereby %CPU for parent(s) with collapsed children
could be vastly understated from those displayed under
the current 3.3.17 publicly available top & libprocps.

If one started several top instances in the background
using very a small delay interval (zero?), then if the
shell under which they were running was collapsed, you
would see similar %CPU results for both the libraries.

However, when running a demanding 'make' like a kernel
compile (especially if backed by fast processors and a
SSD), then newlib would generally show only 1/3 to 1/2
of the collapsed %CPU values that appeared for 3.3.17.

Of course, now that the bug has been swatted with this
commit the disparities between those results is easily
explained. Since newly created tasks never contributed
tics during the interval where they were created, only
with many short lived tasks would differences surface.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-04-24 11:11:03 +10:00
Jim Warner
0f691226ec library: lower addr resolution cost for TICS_ALL_DELTA
Rather than run risks of more expensive and repetitive
address resolution, we will establish this local index
for a one time cost and avoid any potential gcc bloat.

[ this commit was made in pursuit of a bug involving ]
[ the distortion of elapsed task tics. but, it turns ]
[ out these changes had nothing to do with that bug. ]
[ however, the patch is being retained as desirable. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-04-24 11:11:03 +10:00
Jim Warner
9b601307ff library: lessen chance of distortion on TICS_ALL_DELTA
That old library defined this field as 'unsigned int'.
However, here it was known as a 'signed int'. Thus for
consistency we'll now also treat it as 'unsigned int'.

[ this commit was made in pursuit of a bug involving ]
[ the distortion of elapsed task tics. but, it turns ]
[ out these changes had nothing to do with that bug. ]
[ however, the patch is being retained as desirable. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-04-24 11:11:03 +10:00
Jim Warner
39a8c08161 library: fix a few source/origin notations, <pids> api
After removing brackets from those 'derived' notations
I was surprised to discover that several origin/source
comments were wrong. So this patch fixes those errors.

[ along the way a couple enumerators were renamed to ]
[ better (i hope) reflect what they're representing. ]

[ that, in turn, also required a little rearranging. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-03-11 21:21:23 +11:00
Jim Warner
c16c05cce5 library: clean 'derived' origins in header, <pids> api
None of the other four new api headers use brackets on
derived items. With this patch we normalize the fifth.

[ it makes for a cleaner, less confusing, appearance ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-03-11 21:21:23 +11:00
Jim Warner
add01998e4 library: tweaked key used in hash table, <meminfo> api
This small change was a result of some experimentation
trading our current 'hsearch' hash scheme for 'gperf'.

I discovered that when the ':' character was a part of
each 'gperf' key, that generated search logic was more
complicated and thus slower. But without a ':', it was
a little cleaner/leaner and therefore slightly faster.

Assuming that the same trailing ':' *might* affect the
current 'hsearch' logic, to be safe we will remove it.

[ while the 'gperf' version will slightly outperform ]
[ an 'hsearch', too many ugly implementation details ]
[ were exposed which complicates future maintenance. ]

[ thus, we'll retain our current 'hsearch' approach. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-02-28 21:18:26 +11:00
Jim Warner
f4e9195149 library: normalized the 'read_failed' guys across APIs
This patch will condense some logic in those functions
associated with the file input operations. The changes
will not, for the most part, alter any generated code.

More significantly (though not very) was the change to
two 'strtoul' calls. Since the returned 'endptr' value
isn't exploited, when that parm is set to NULL, we can
save one instruction on each side of such calls (wow).

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-02-28 21:18:26 +11:00
Jim Warner
b0bda71ac0 library: eliminate a useless conditional in readproc.h
When #define QUICK_THREADS was introduced, for copying
some task data for a child thread, one proc_t pad byte
was used to mark, then later identify, those children.

Later the QUICK_THREADS was recycled as FALSE_THREADS,
and used for a different purpose, but a conditional in
the header file erroneously remained. Now, it is gone!

Reference(s):
. Jul, 2016 - QUICK_THREADS become FALSE_THREADS
commit c546d9dd44
. Aug, 2011 - QUICK_THREADS intruduced
commit bb4f08ba29

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-02-23 22:07:48 +11:00
Craig Small
9aa09d8862 library: Rename to libproc-2
What to call the new library?

Keep using libprocps wouldn't do, its a very different library from
the programs' point of view. It would also mean we could have some
clashes around the packages (two package names, same library name).

The ancient procps used libproc or libproc-a.b.c where a.b.c was the
package version. Kept the revision numbers down (it was always 0.0.0)
but the name of the library changed.

So if we use libproc-2 is there a clash with an ancient procps?
procps v 2.0.0 was around in 1999 so it was 22 years ago, also the
name of the library would have been libproc-2.0.0.so not libproc-2.so
so we're fine with that.

libproc-2 seems to fit, our second major re-work of the procps
library.

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-02-20 22:30:31 +11:00
Jim Warner
333a1bb6b8 library: tweak logic for /proc not mounted, <pids> api
Since 'procps_uptime' will access the /proc filesystem
the <pids> 'new' guy should should protect against the
possibility /proc isn't mounted when 'boot_seconds' is
established. A zero is better than the negative value.

[ the only distortion would be to PIDS_TIME_ELAPSED. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-02-18 18:35:32 +11:00
Jim Warner
0b689adde0 library: normalized 'extents_free_all' use across APIs
With the way those 'extents_free_all' guys were coded,
there's no real need to check for a NULL this->extents
before calling 'em. That's how <stat> already does it.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-02-18 18:35:32 +11:00
Jim Warner
fb0e4f3f68 library: update those source files for copyright dates
In addition to copyright date, the initial descriptive
line was changed from a generic statement to one which
reflects the specific portion of the proc/ filesystem.

[ such descriptions alternate between 'declarations' ]
[ (h files) & corresponding 'definitions' (c files). ]

Also, a few missing copyright attributions were added.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-02-09 16:40:52 +11:00
Jim Warner
3c87c46439 library: tweak 'other' user/group names for efficiency
This commit just ensures that the relatively expensive
ID to name conversions aren't performed unless they're
explicitly requested. It also internalizes those flags
that required the PROC_FILLSTATUS flag to also be set.

[ requiring a caller, in our case pids.c, to provide ]
[ two flags when a single field was the objective is ]
[ wrong & represents a future potential toe-stubber. ]

[ moreover, what's worse is that those two flags are ]
[ seemingly unrelated. but, without both, a SEGV can ]
[ can be expected when a result.str pointer is NULL. ]

[ by contrast, in the master branch those fields are ]
[ arrays which, when set to zeroes, produce an empty ]
[ string. So, there is no abend (but no name either) ]
[ when one of those two required flags were omitted. ]

[ and worth noting, in that branch it's not just one ]
[ caller required to observe a two flag requirement. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-02-09 16:40:52 +11:00
Craig Small
bc16fa87f0 misc: Update Craig's email 2021-01-21 17:37:48 +11:00
Jim Warner
423297c9db all: make buildable again for new 'misc.h' header file
With the 4 header files removed in the previous patch,
this commit just changes all those obsolete references
to that new consolidated 'misc.h' header file instead.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-01-21 17:30:25 +11:00
Jim Warner
1fdd7d4966 library: combine 4 files into new 'misc.h' header file
Prior to this patch, we had four separate header files
dealing with miscellaneous functions. Those four files
were documented in the single man page: procps_misc.3.

Now, we will have just a single header file documented
in a single man page (that is what's called symmetry).

[ and while we're at it, we will shorten that overly ]
[ long struct 'procps_namespaces' name to agree with ]
[ the function naming conventions, e.g. 'procps_ns'. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-01-21 17:30:24 +11:00
Jim Warner
ba9560ff02 library: reduce overhead for another 'escape' function
The preceding commit made that 'esc_all' function more
efficient by using direct pointer manipulation instead
of an indexed string reference approach within a loop.

This commit applies the same approach to the 'esc_ctl'
function. Now we'll save 12 more iterated instructions
while decreasing the function's code size by 43 bytes.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-01-21 17:30:24 +11:00
Jim Warner
031a08f2a7 library: improve performance for one 'escape' function
While this patch has some cosmetic whitespace changes,
more importantly it makes that 'esc_all' function more
efficient. By abandoning the indexed loop approach for
a direct pointer manipulation, we will save 9 iterated
machine instructions, for a total of 33 bytes of code.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-01-21 17:30:24 +11:00
Jim Warner
649e45482f library: refactor 'escape' logic for newlib (2nd time)
Much of what was represented in the commit message for
the reference shown below was revisited in this patch.
It also means that the assertion in the last paragraph
of that message will only now be true with LANG unset.

[ and forget all the bullshit about not altering any ]
[ kernel supplied data. sometimes we must to avoid a ]
[ corrupt display due to a string we can not decode. ]

And while this commit still avoids the overhead of the
'mbrtowc', 'wcwidth' 'isprint, & 'iswprint' functions,
we achieve all the benefits with simple table lookups.
Plus such benefits are extended to additional strings.

For example, both PIDS_EXE and PIDS_CMD fields are now
also subject to being 'escaped'. If a program name did
contain multibyte characters, potential truncation may
corrupt it when it's squeezed into a 15/63 byte array.

Now, all future users of this new library only need to
deal with the disparities between string and printable
lengths. Such strings themselves are always printable.

[ the ps program now contains some unnecessary costs ]
[ with the duplicated former 'escape' functions. But ]
[ we retain that copied escape.c code for posterity. ]
[ besides, in a one-shot guy it's of little concern. ]

Note: Proper display of some multibyte strings was not
possible at the linux console. It would seem a concept
of zero length chars (like a 'combining acute accent')
is not recognized. Thus the display becomes corrupted.
But if utf8 decoding is disabled (via LANG=), then all
callers will now see '?', restoring correct alignment.

Reference(s):
. Dec 2020, newlib 'escape' logic refactored
commit a221b9084a

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-01-02 15:42:10 +11:00
Jim Warner
a221b9084a library: refactor 'escape' logic for newlib essentials
This new library provides callers with pure strings or
string vectors. It is up to those callers to deal with
potential utf8 multibyte characters and any difference
between strlen and the corresponding printable widths.

So, it makes no sense for the library to go to all the
trouble of invoking those rather expensive 'mbrtowc' &
'wcwidth' functions to ultimately yield total 'cells'.

Thus, this patch will eliminate all the code and parms
that are involved with such possible multibyte issues.

[ Along the way we'll lose the ability to substitute ]
[ '?' for an invalid/unprintable multibyte sequence. ]
[ We will, however, replace ctrl chars with the '?'. ]

[ This presents no problem for that ps program since ]
[ it now duplicates all of the original escape code. ]
[ And, we'll no longer be executing that code twice! ]

[ As for the top program, it takes the position that ]
[ it is wrong to alter kernel supplied data. So with ]
[ potential invalid/unprintable stuff, he'll rely on ]
[ terminal emulators to properly handle such issues! ]

[ Besides, even using a proper multibyte string, not ]
[ all terminals generate the proper printable width. ]
[ This is especially true when it comes to an emoji. ]

[ And should callers chose not to be portable to all ]
[ locales by calling setlocale(LC_ALL, ""), they can ]
[ expect to see lots of "?", regardless of what this ]
[ library fixes in a faulty multibyte string anyway. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-12-29 08:48:23 +11:00
Jim Warner
e76c773561 library: fix a potential error relating to 'exe' logic
After 'errno' management was standardized, a couple of
fields were added to the <pids> api. Only 1 (PIDS_EXE)
involved dynamic memory and, unfortunately, it did not
conform to that expected normalized ENOMEM convention.

Reference(s):
. Jun 2018, added 'exe' to library
commit ad4269f118
. Nov 2017, standardized 'errno' management
commit 06be33b43e

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-12-29 08:48:23 +11:00
Jim Warner
fd64b91d09 library: extend ITEMTABLE_DEBUG just a tad, <PIDS> api
Under the above #define this commit now also addresses
2 additional possible toe stubbers involving 'select'.

If some readproc.h constants were uncoupled from their
pids.h enumerators a 'make check-lib' will now detect.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-12-29 08:48:23 +11:00
Jim Warner
07e4c08c62 misc: just a couple cosmetic changes for comments only
Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-12-29 08:48:23 +11:00
Jim Warner
eb83a1f821 library: a tweak to the UNREF_RPTHASH code, <pids> api
If a hash results report is output (via UNREF_RPTHASH)
a portion is devoted to occupied table entries ordered
by depth. There is a possibility that some depths will
not be found among existing occupied table entries and
to avoid any confusion probably should not be printed.

[ to illustrate the potential for confusion prior to ]
[ this patch, force a very small table size (like 8) ]
[ & then trigger the procps_pids_unref() eoj report. ]

So this patch ensures only 'in use' entries are shown.

[ admittedly, all of the remaining logic in the loop ]
[ could/should be subordinate to this new 'if' test, ]
[ but we will keep the change to a minimum. besides, ]
[ there's no harm subtracting/adding a zero numdepth ]
[ especially since the chance of a zero is very low. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-10-24 18:27:57 +11:00
Jim Warner
9eb930eebb library: tweak that history hash algorithm, <pids> api
This patch just raises the size of the hash table used
to calculate elapsed task stuff. The net result should
be less need for 'chaining' under pid hash collisions.

[ the hash scheme is intentionally kept as primitive ]
[ and, therefore, as fast as possible. it employs an ]
[ 'and' approach versus a 'mod' operation since both ]
[ yield similar distribution but the former approach ]
[ was 4 fewer cpu instructions in terms of overhead. ]

[ additionally, for hash collisions, 'chaining' uses ]
[ an array index rather than the usual pointer since ]
[ the HST_t guys may move when they are reallocated. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-10-11 17:53:22 +11:00
Jim Warner
e2b0f8b0af library: tweak the memory allocation logic, <pids> api
This patch separates the memory allocations into those
used initially from those used in later reallocations.
Thus, we can reduce that iterative realloc() overhead.

Additionally, we'll correct a long standing oops where
multiple history_info structures were created at 'new'
time when only one should have been allocated (jeeze).

[ originally the allocation was strangely based upon ]
[ number of 'items' (???) & later a #define constant ]

Reference(s):
. May, 2016 - subsequent bad history_info logic
commit 9ebadc1438
. Aug, 2015 - original faulty history_info code
commit 7e6a371d8a

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-10-11 17:53:22 +11:00
Jim Warner
ec21588be1 library: tweak those system cpu statistics, <stat> api
This patch may be a little misleading in terms of size
since most of the changes just reorder a little logic.
The most significant changes involve two GUEST values.

My original implementation excluded such tics from the
TOTAL calculation and, therefore, the BUSY figure too.

That decision was erroneously based on some code found
in ./kernel/sched/cputime.c which in hindsight applies
only to processes, not those system level cpu figures.

[ another likely oops classified STOLEN tics as IDLE ]

So, this patch attempts to bring those SUM values into
better agreement with the calculations performed for a
root cgroup (see ./kernel/cgroup/rstat.c source file).

[ we differ from those above in that we also include ]
[ the IDLE plus IOWAIT tics in our TOTAL calculation ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-09-13 10:00:49 +10:00
Jim Warner
bc55757231 library: eliminate unused 'FUNCTION' in private header
Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-09-13 10:00:49 +10:00
Jim Warner
2c1973d64c library: eliminate one potential source of human error
There's a huge toe-stubber awaiting future maintenance
as reflected in that commit below which deals with the
addition of new enumerators to the Item_table. Namely,
whenever the table is grown, one must remember to also
change that existing 'logical_end' enumerator's value.

Well, not anymore! Since that MAXTABLE macro was added
to the procps-private.h header we can now also exploit
it so a 'logical_end' automatically tracks table size.

This change also renders some code associated with the
ITEMTABLE_DEBUG #define unnecessary. So it's gone too.

Reference(s):
. 08/2016, add new enumerators
commit 09e1886c9e
. 08/2020, added MAXTABLE macro
commit c865b06c30
. 08/2020, introduced ITEMTABLE_DEBUG
commit 92d0297e1e

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-26 21:50:20 +10:00
Jim Warner
81db6664c9 library: fix a flaw in one 'new' function, <diskstats>
At 'new' time, the major API modules each ensure their
'info' parameter isn't NULL but what it pointed to was
except this single straggler, for some unknown reason.

So, this patch brings him into line with those others.

[ And, without going into the ugly details, this was ]
[ the reason I never experienced an abend originally ]
[ but Craig did. And, though related to stacks mgmt, ]
[ zero initialization was not a factor. Anyway, with ]
[ this patch, everybody would have experienced abend ]
[ under the original (faulty) test_Itemtables logic! ]

Reference(s):
https://www.freelists.org/post/procps/keep-on-patchin,13

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-24 21:37:52 +10:00
Jim Warner
0cddb06425 build-sys: proper fix for the 'test_Itemtables' module
Me thinks Craig had the right idea but perhaps not the
most correct solution. As currently structured, all of
the tests now go way too far by checking every 'unref'
and 'new' call when what we're trying for is survival.

In the final analysis, it doesn't matter who issues an
EXIT_FAILURE - that run_tests guy or an early Exit out
of a procps_new() function. They both will produce the
same end result of the desired "FAIL" test diagnostic.

[ and this patch once again allows the slabinfo test ]

Reference(s):
commit 4eeed6dcff

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-18 10:19:53 +10:00
Craig Small
4eeed6dcff build-sys: Check return values and NULL ctx
The referenced commit introduced a test program for the API
but it would often fail due to:
  The given pointer for _new() not being NULL
  The return value for _new not checked, so the subsequent _unref()
  would free() random memory
  slabinfo checks failing due to permission denied errors.

The first two are fixed, as well as returning a fail to the test
if they don't return correctly, with slabinfo waiting to see if there
is a way of initialising the structure without reading the slabinfo.

References:
 commit e616409aa4

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2020-08-17 22:47:39 +10:00
Jim Warner
e616409aa4 build-sys: added a 'test_Itemtables' to testing scheme
This change sets the stage for exploiting the recently
added ITEMTABLE_DEBUG #define. All tests are performed
in a single module (after trying 6 separate programs).

The chances of each test detecting errors is extremely
remote (at least while I'm maintaining these modules).
However, this single program approach has one flaw and
it relates to the response whenever an error is found.

Each of those six new API modules calls Exit() if they
detect an error. Otherwise, incorrect results would be
produced at the least or an abend encountered at most.

This means that multiple 'make check' invocations will
be needed if more than 1 module actually was in error.

All in all, it is a small price for a large assurance.

Reference(s):
https://www.freelists.org/post/procps/keep-on-patchin,7

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-17 21:49:14 +10:00
Jim Warner
92d0297e1e library: introduced 'Item_table' validation provisions
The recent work on updating the <meminfo> and <vmstat>
modules with some newly added linux fields reminded me
(again) of a need for some mechanism guaranteeing that
a header file agrees with the source file assumptions.

Sadly, in the past, if a table entry was omitted or if
the table and header are ordered differently, then the
library would silently return the wrong results values
or even potentially experience a SIGSEGV abnormal end.

This patch offers a much needed development assist for
ensuring that Item_table entries are synchronized with
header file enumerators in terms of number plus order.

It's intended solely for our use as libprocps evolves.

Now, by activating ITEMTABLE_DEBUG, either directly or
via ./configure CFLAGS='-DITEMTABLE_DEBUG', the number
and order will be verified. It is envisioned that this
feature will be used at least once prior to a release.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-13 21:08:58 +10:00
Jim Warner
c865b06c30 library: added MAXTABLE macro to proc/procps-private.h
This macro should help the following patch be a little
less prolix. Besides, this private header could/should
do just a little more to help with our newlib efforts.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-13 21:08:58 +10:00
Jim Warner
bb9e74c00b library: <diskstats> api, regularize macro conventions
This cosmetic change just prepares for the later patch
introducing validation of Item_tables and enumerators.

[ and, we'll now have better 'set' function names of ]
[ 'set_diskstats_ENUM' instead of the more redundant ]
[ current 'set_diskstats_DISKSTATS_ENUM' convention. ]

[ now our only exception is the <slabinfo> api where ]
[ a full enumerator identifier with 'SLAB' & 'SLABS' ]
[ prefixes are used, and 'SLABINFO_noop/extra' guys. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-13 21:08:58 +10:00
Jim Warner
9363bcca8f library: <vmstat> api, updated for current linux state
Since last visited, there have been several new fields
added to the /proc/vmstat file. Now, we are caught up.

[ references shown below represent linux git commits ]

Reference(s):
. 6/2020, added 'pgscan_...' & 'pgsteal_...'
commit 497a6c1b09902b22ceccc0f25ba4dd623e1ddb7d
. 4/2020, added 'nr_shadow_call_stack'
commit 628d06a48f57c36abdc2a024930212e654a501b7
. 4/2020, added 'thp_...fallback_charge'
commit 85b9f46e8ea451633ccd60a7d8cacbfff9f34047
. 4/2020, added 'thp_file_fallback'
commit dcdf11ee144133328664d90836e712d840d047d9
. 4/2020, added 'nr_foll_pin_...'
commit 1970dc6f5226416957ad0cc70ab47386ed3195a6
. 9/2019, added 'nr_file_...'
commit 60fbf0ab5da1c360e02b7f7d882bf1c0d8f7e32a

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-09 22:19:46 +10:00
Jim Warner
a9ea77e11c library: <meminfo> api, update for current linux state
Since last visited, there have been several new fields
added and one field deleted in the /proc/meminfo file.

[ references shown below represent linux git commits ]

Reference(s):
. 4/2020, added 'ShadowCallStack'
commit 628d06a48f57c36abdc2a024930212e654a501b7
. 9/2019, added 'FileHugePages' & 'FilePmdMapped'
commit 60fbf0ab5da1c360e02b7f7d882bf1c0d8f7e32a
. 9/2019, removed 'Quicklists'
commit 13224794cb0832caa403ad583d8605202cabc6bc

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-08-09 22:19:46 +10:00
Jim Warner
dbe34f6312 library: ensure 'XTRA_PROCPS_DEBUG' works in real life
This commit attempts to smooth some wrinkles impacting
any future libprocps user exploitation. The 2 problems
relate exclusively to our XTRA_PROCPS_DEBUG provision.

1. The 'xtra-procps-debug.h' header had an include for
'procps-private.h', which was not an installed header.
So the STRINGIFY macros will now be embedded directly.

2. Each of the new api headers referenced '<proc/...>'
rather than '<procps/...>' for the debugging #include.
So, we must drop that prefix in favor of a quoted file
name so that debugging builds work regardless of where
that 'xtra-procps-debug.h' header happens to be found.

Reference(s):
. Jun, 2020 - changed target install directory
commit d64755ea5f
. Sep, 2018 - #include migrated from procps.h
commit fab37662ef

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-07-29 19:01:50 +10:00
Jim Warner
c13ea57495 library: added some justification emphasis, <pids> api
I almost feel like I should apologize for this kind of
patch ( *almost* ). But, since this comment was unique
to the <pids> api and since it was especially designed
to align with the following comment and since the next
comment already carried the alignment emphasis, I will
refrain from issuing any apology and submit it anyway.

[ plus to prove that i am not totally anal-retentive ]
[ and can exercise some restraint there are two more ]
[ places where i COULD have added alignment emphasis ]

[ see lines 804-806 and lines 1360-1361 for my proof ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-07-29 19:01:50 +10:00
Jim Warner
a9ac3faa44 library: show allowable range in relative enum warning
This patch just trades the 'typestr' for the much more
meaningful 'valid range' when a relative enum has been
deemed invalid under that XTRA_PROCPS_DEBUG provision.

[ and we'll also make one comment a bit more generic ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-07-20 16:15:14 +10:00
Craig Small
8c115f6cf3 testsuite: Correct include
procps.h got removed in the referenced commit, the test program
needs to include the correct header.

References:
 commit 1aa8b16441
2020-07-05 21:18:36 +10:00
Jim Warner
605ea4a8f7 library: make that 'escape_str' private to our library
In that commit referenced below, a promise was made to
revisit an 'escape_str' function in efforts to make it
private to the library. The problem was it's needed by
both ps plus the library which is why it was exported.

So, in an effort to remove it from libprocps.sym, this
patch duplicates all the required code in ps/output.c.
Now, each version can be made private to their caller.

[ along the way we'll use this opportunity to remove ]
[ the 'restrict' qualifiers from function parameters ]
[ while swatting a compiler warning referenced below ]

Reference(s):
. April 2016, most escape functions made private
commit d916d5db86

proc/escape.c: In function `escape_command':
proc/escape.c:182:23: warning: initialization of `const char **' from incompatible pointer type `char **' [-Wincompatible-pointer-types]
  182 |     const char **lc = (char**)pp->cmdline;
      |                       ^

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-07-05 21:13:01 +10:00
Jim Warner
1aa8b16441 library: eliminated the questionable 'procps.h' header
There was a time when that procps.h file served a more
traditional role. Prior to the commit referenced below
it held just macros plus manifest constants. But, with
that change, such items were replaced with a series of
includes embracing all the library exported functions.

That approach was known to disguise errors which would
have otherwise yielded a compiler warning. And without
such a warning, there was no way to address the error.

So this patch will trade the all inclusive header file
approach for individual includes only where necessary.

Reference(s):
. April 2016, procps.h header file revamped
commit ccb6ae8de1
. Sept 2018, top abandoned use of procps.h
commit a6dfc2382e

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-07-05 21:13:01 +10:00
Jim Warner
70fd03e461 library: size two read buffers with manifest constants
The meminfo & vmstat file read buffers are now defined
through symbolic constants next to the symbolic names.

[ recent attention to the /proc/stat buffer size led ]
[ to this (hopefully) better approach to definitions ]

Reference(s):
commit 81f4a6acdf

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-07-05 21:13:01 +10:00
Jim Warner
81f4a6acdf library: raise amount by which the <stat> buffer grows
We won't go as far as the merge request shown below in
increasing buffer size, but we will reduce by half the
total iterations while growing it to achieve one read.

[ and this seems in line with what was recently done ]
[ to help top under a massively parallel environment ]
[ thru 2 abreast ('4') and combined cpus ('!') modes ]

Reference(s:
https://gitlab.com/procps-ng/procps/-/merge_requests/105

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-06-29 21:13:40 +10:00
Jim Warner
df8fb11a47 library: changes to some numa stuff in that <stat> api
Now that we may be getting serious with documentation,
that stat module was revisited with an eye toward user
friendliness. Heck, even this author puzzled over some
of the existing notes and naming conventions employed.

So, this patch will adjust some identifiers and expand
the notes to (hopefully) better serve potential users.

The most significant change was making the STAT_TIC_ID
always valid for numa nodes, even if any are inactive.

Thus the -22222 special STAT_NODE_INVALID constant now
is applied only to STAT_TIC_NUMA_NODE. It will be used
on the cpu summary and reaps with STAT_REAP_CPUS_ONLY.
And it will also mark any numa node that was inactive.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-06-25 20:08:22 +10:00
Jim Warner
52c847a019 library: rename a couple of 'reap' structs to 'reaped'
In preparation for a documentation attempt, this patch
renames 'struct reap' to 'struct reaped' in two of our
interfaces. That will bring all the non-pid interfaces
into agreement since the <stat> api already used that.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-06-23 19:09:05 +10:00
Jim Warner
356e9a0672 library: correct that 'escape_str_utf8' guy's behavior
Thanks to Konstantin for discovering 2 problems in the
issue referenced below. That 15+ year old logic went a
little too far overboard wrestling with a utf8 string.

Henceforth, we will not treat 'x9b' as special. And we
also will handle a 'combining acute accent' correctly.

Reference(s):
https://gitlab.com/procps-ng/procps/-/issues/176

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-06-23 19:09:05 +10:00
Jim Warner
46f7949845 library: misc accumulated cosmetic header file changes
Well gosh, I only accumulated 1 minor cosmetic change.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-05-28 19:49:40 +10:00
Dylan Swiggett
59e378642e Fixes small bug in struct proc_t documentation.
From http://man7.org/linux/man-pages/man5/proc.5.html:

(22) starttime  %llu
                        The time the process started after system boot.  In
                        kernels before Linux 2.6, this value was expressed
                        in jiffies.  Since Linux 2.6, the value is expressed
                        in clock ticks (divide by sysconf(_SC_CLK_TCK)).
2020-05-11 17:44:48 +10:00
Jim Warner
5cb46d3533 library: adapted to the latest lxc conventions (again)
Well, shit! With release 4.0 on March 25th the lxc/lxd
folks have stuck it to us once again. They changed the
cgroup lxc prefix used to identify the container name.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-04-20 22:07:43 +10:00
Jim Warner
12b6452135 library: allow negative system DELTA stats, <STAT> api
All TIC delta fields are checked for possible negative
results and set to zero when found. This is done so as
to protect against potential anomalies which depend on
kernel version and/or toggling cpus offline or online.

[ it's probably unnecessary with the latest kernels, ]
[ except for iowait. documentation suggests it might ]
[ decrease which would then create a negative delta. ]

The same approach is employed for most of the 'system'
deltas (ctxt, intr & procs_created). However, with two
of the fields (procs_blocked & procs_running) negative
results were allowed. But it now seems such a division
is unwise so this patch will allow all to go negative.

[ rather than force any 'system' delta value to show ]
[ what's logical, we'll now let all reflect reality. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-01-06 09:11:20 +11:00
Jim Warner
5c37e2d42d library: strictly cosmetic change involving whitespace
This patch just standardizes/normalizes the whitespace
employed within a couple of nearly identical #defines.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2020-01-06 09:11:20 +11:00
Jim Warner
11cf6cc005 library: removed that 'calloc/free' cruft where doable
Under newlib, the only caller of the readproc routines
is that pids module. And in every case, the address of
some static proc_t structure has always been provided.

As a result, there is no need for the logic supporting
calloc() for a possible NULL pointer which was present
in both of those readproc() and readeither() routines.

Additionally, that pids module takes ownership of most
dynamically acquired 'str' plus 'strv' memory whenever
assigning to a results structure. So, henceforth under
the free_acquired() guy we will only free those string
fields which might exist when not explicitly selected.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-10-27 22:27:49 +11:00
Jim Warner
281c37d1cd library: removed that 'simple_readtask' unused pointer
This patch just eliminates a parameter present for the
simple_readtask() function which is not needed nor has
it ever actually been used. It will make calls to that
function (via taskreader ptr) slightly more efficient.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-10-27 22:27:49 +11:00
Jim Warner
b5587b7b94 library: removed that 'did_fake' flag from the PROCTAB
When some cleanup was performed on the readproc module
in the commit shown below, some residual code involved
with the 'did_fake' flag remained. Since such logic no
longer served any real need, this patch will whack it.

Reference(s):
. cleanup of readproc functions
commit 887bb51016

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-10-27 22:27:49 +11:00
Jim Warner
97f7c7a549 library: removed that 'SIGNAL_STRING' conditional code
No libprocps user expects signal values to be returned
as 'long long' quantities. More importantly the <PIDS>
api only returns a 'str' result for signal categories.

So this patch eliminates all the conditional code that
depends on the absence of the #define 'SIGNAL_STRING'.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-10-27 22:27:49 +11:00
Jim Warner
11df806a7c library: correct one reference typo in that <PIDS> api
This patch will just correct an oops introduced in the
commit shown below. Thank goodness both 'str' & 'strv'
occupy the same storage location in that result union.

Reference(s):
. standardize 'errno' management
commit 06be33b43e

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-10-27 22:27:49 +11:00
Jim Warner
1d8d4be046 library: add a 'CPU_ZEROTICS' equivalent to <STAT> api
This patch just implements an equivalent to the master
branch 'CPU_ZEROTICS' provision. However, the original
impetus for that earlier implementation was ultimately
attributed to a likely kernel anomaly since corrected.

As a result, in this newlib implementation we take the
opposite approach to the default behavior. There is no
adjustment to TIC_SUM_DELTA values if fewer ticks than
expected are recorded, unless the define is activated.

The commit shown below explains why the 'CPU_ZEROTICS'
define was retained in spite of the fix to the kernel.

Reference(s):
. issue referencing CPU_ZEROTICS
https://gitlab.com/procps-ng/procps/issues/132
. master branch CPU_ZEROTICS summary
commit ee3ed4b45e
. lengthy thread leading to CPU_ZEROTICS
https://www.freelists.org/post/procps/CStates-handling-new-switch

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-07-04 20:46:18 +10:00
Jim Warner
40883a0b85 library: minimize the use of 'cleanup_stacks' routines
Some parts of our newlib implementation are the result
of functions which have been propagated from module to
module. In particular, those 'cleanup_stacks' routines
are all similar & likely originated in the <pids> api.

In that interface there was a need to free dynamically
acquired memory before the result structure was reused
to satisfy subsequent 'get', 'select' or 'reap' calls.
This, in turn, led to a concept of 'dirty' stacks with
the need to call one of two 'cleanup_stack' functions.

None of the remaining interfaces deal with such memory
yet they each had their own 'cleanup_stack' functions.
Those functions were responsible for resetting each of
the result unions to zero, excluding any 'noop' items.

The bottom line is that for all interfaces, repetitive
calls would require iterating through the stack(s) two
separate times: once to 'cleanup' another to 'assign'.

With this commit we will reduce iterations to just the
'assign' routine. A reset to zero will be accomplished
in the 'extra' item set routine (which is the only one
actually requiring any reset). All other items will be
reinitialized automatically by a new current set value
or upon reallocation when an items compliment changes.

In the <pids> interface, any freeing of dynamic memory
could have been accomplished by adding that 'freefunc'
check to the 'assign' function. However, that requires
an Item_table test with every item. Instead, we'll now
satisfy such needs as the very first step in those set
functions responsible for dynamically acquired memory.

[ the <pids> api retains 2 'cleanup_stack' functions ]
[ to accommodate stack(s) 'reset' & to serve 'unref' ]

Lastly, all the 'itemize_stack' functions were tweaked
by eliminating an unnecessary initialization of result
unions. That objective was already accomplished by the
calloc() in a 'stacks_alloc' function or the remaining
'cleanup_stack' routine found in the <pids> interface.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-07-04 20:46:18 +10:00
Jim Warner
22a3bcbd6d library: remove useless code for 2 'stacks_fetch' guys
These changes are an outgrowth of the research/testing
behind the previous commit. There is no commingling of
select/reap stacks in interfaces beyond the <pids> api
since there's no need to support any 'reset' function.

However, those <pids> changes prompted a review of all
interfaces offering that 'stacks_fetch' function, thus
revealing 2 instances of useless logic/wasted efforts.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-05-16 21:23:06 +10:00
Jim Warner
f87fc7dc83 library: tweak support for get/select/reap, <PIDS> api
When the <pids> api was refactored in the commit shown
below, one objective was enabling the simultaneous use
of 'get' & 'select/reap' functions. Unlike other 'get'
functions, this <pids> 'get' acts as an iterator where
successive calls will return successive tasks/threads.

However, that goal wasn't quite met since a stack used
by 'get' was commingled with the 'select/reap' stacks.
Such commingling supported the 'reset' function, again
a provision which was unique to this <pids> interface.

Unfortunately, some poor assumptions in 'stacks_fetch'
produced a SEGV whenever 'reap/select' followed 'get'.
Thus, this patch addresses those issues and guarantees
such commingled stacks (extents) will be accommodated.

Reference(s):
. standardize portions of interface, <PIDS> api
commit 9ebadc1438

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-05-16 21:23:06 +10:00
Jim Warner
a9bfb186eb library: correct 'vectorized' string logic, <PIDS> api
The commit referenced below addressed (some) anomalies
surrounding 'strv' pointers. However, there remained a
couple quirks involving a potential NULL return value.

Any NULL values returned from the old library readproc
guys would cause no real harm for newlib. But they did
produce the misleading "[ duplicate ENUM_ID ]" result.

The following all represent potential NULL results and
suggest shortcomings in testing of that earlier patch.

. kernel threads do not have cgroup, cmdline & environ
. even if present environ could require root to access

So, this patch reverts a portion of the earlier commit
and ensures when some vectored string is not available
a traditional dash ('-') is the 'strv' returned value.

[ and we'll also correct one typo in the header file ]

Reference(s):
. eliminated a final potential NULL, <PIDS> api
commit 09503dc597

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-05-16 21:23:06 +10:00
Jim Warner
6671a3a8b7 library: refactored some header file items and origins
This commit is intended as a refinement of the patches
mentioned below, where origins/sources of newlib items
were added to the header files for user documentation.

However, if those additions are to be truly effective,
along with kernel documentation (where available), the
following prerequisites must also have been satisfied:
. our identifiers closely align with linux field names
. our derived items are documented or self-documenting

Satisfying those prerequisites prompted this patch and
for these changes, kernel sources were emphasized over
available documentation (shame on me, it should always
have been so). And, while some 'new' fields were found
to be conditional, they were included unconditionally.

These changes appear more extensive than they actually
need be since I have attempted to enforce some spacing
conventions. So, I've summarize the significant things
in the sections that follow. For a proper perspective,
use: 'git diff --ignore-space-change' (good as alias).

___________________________________________ <PIDS> api
This api is unique in that there exists many different
file/directory origins subordinate to /proc/<pid>. And
our item identifiers are sometimes coerced so as to be
able to group related or similar enumerators together.

So, users needed more help relating our identifiers to
an actual documented field. Thus, we will now also add
the field names as with 'stat: delayacct_blkio_ticks'.

Each item ending with a '_C' now consistently includes
both the parent's count/time plus waited for children.

That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT
since its original name is an implementation artifact.

___________________________________________ <STAT> api
The only api change was to correct a typo ('dervied').

_________________________________________ <VMSTAT> api
Even ignoring white space, this interface received the
largest number of changes. Mostly, this was because of
deficiencies in the proc(5) documentation. Recall that
this documentation already sorely lacks any substance.
Usually, just kernel releases are noted, not contents.

When compared to kernel source, that proc(5) contained
many non-existent fields and also omitted many others.

________________________________________ <MEMINFO> api
Sadly, with this api many of the changes were simply a
correction of some earlier 'human error' where several
fields where hashed then tracked but never represented
with an item enumerator in this meminfo.h header file.

_______________________________________ <SLABINFO> api
The 'SLABS' (summary) & 'SLABNODE' items were reversed
since the former are derived from the separate caches.

More significantly, those 'SLABNODE' guys were renamed
to 'SLAB' since they concern individual caches and the
concept of 'nodes' is really an implementation detail.

Also, several enumerators were changed to more closely
agree with official slabinfo(5) documentation referred
to in what we're treating as a base document: proc(5).

Lastly, while those 'SLABS' items are solely a product
of our library and not represented in slabinfo(5), the
names attempt to parallel those found as 'SLAB' items.

______________________________________ <DISKSTATS> api
One enumeration identifier was changed so as to better
reflect its relationship to that actual documentation:
'Documentation/iostats.txt', as referenced in proc(5).

Reference(s):
. 12/2018, item origins added (and commit msg history)
commit 96d59cbf46
. 01/2019, <stat> origins tweaked
commit 201e816b26

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-26 19:55:30 +11:00
Jim Warner
47af06b52b library: repair any broken stuff found during refactor
Rather than offer three separate patches, they've been
consolidated in this single commit. All are related in
that they surfaced while preparing a subsequent patch.

------------------------------------------------------
library: correct a broken '#if define', <SLABINFO> api

It was introduced (embarrassingly) in the patch below.

Reference(s):
commit 97d078a9af

------------------------------------------------------
library: correct a broken 'GET' macro, <DISKSTATS> api

In the patch referenced below, which purported to make
all the 'GET' macros robust, the 'DISKSTATS_GET' macro
was broken. A necessary parameter wasn't passed to the
subsequently invoked function: procps_diskstats_get().

Reference(s):
commit bef8c7fb70

------------------------------------------------------
library: correct a broken 'sort' func, <DISKSTATS> api

In the commit shown below, an attempt to normalize the
errno handling, the sort function inadvertently lost 1
crucial line of code which produces a consistent SEGV.

Reference(s):
commit 06be33b43e

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-26 19:55:30 +11:00
Jim Warner
32f93b56b7 library: tweak that lxc container support a final time
Since the patch referenced below traded a compile-time
'sizeof' directive for a run-time 'strlen' call, there
is no need to declare lxc patterns as explicit arrays.

We'll also use the actual lxc patterns by omitting the
beginning slashes ('/') for both of those definitions.

And, looking to the future when most/all lxc users are
using the most recent lxc release, we will make things
slightly more efficient by reversing those two pattern
literals so the most recent pattern was checked first.

Of course, such a change only benefits tasks which are
running in a container. For the majority of processes,
both literals will be compared in that 'if' statement,
assuming the 'LXC' field is currently being displayed.

[ plus, a leftover parenthesis pair has been removed ]

Reference(s):
commit 288d759b8b

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-26 19:55:30 +11:00
Jim Warner
a4da552d4e library: eliminate an unnecessary #include, <STAT> api
That patch shown below should have also included this.

Reference(s):
commit 68d7f7a673

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-26 19:55:30 +11:00
Jim Warner
201e816b26 library: improve header file item comments, <STAT> api
This patch just polishes the 'origin' comments for the
<STAT> header file. In particular those derived/unique
items (the 'SUM' guys) will now be properly explained.

[ in order to employ the 'derived from above' phrase ]
[ with their DELTA versions, all SUM items had to be ]
[ relocated (and some renamed). in turn, that had an ]
[ impact on many portions of the .c source file too. ]

Reference(s):
. summary calculations introduced
commit 2c86c4984a
. origins added to header files
commit 96d59cbf46

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-01-22 22:49:24 +11:00
Jim Warner
68d7f7a673 library: remove one needless function call, <STAT> api
This small change is an outgrowth of the research into
the bug represented by that merge request shown below.

With the master branch, a real buglet was subsequently
addressed. In this newlib branch, no bug existed since
the <stat> API relies solely on just cpus reflected in
(and parsed from) the kernel's /proc/stat pseudo file.

[ since that procps_stat_new() priming read about to ]
[ be performed will value info->cpus.total, there is ]
[ no need to separately invoke a procps_cpu_count(). ]

Reference(s):
https://gitlab.com/procps-ng/procps/merge_requests/82

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-01-22 22:49:24 +11:00
Jim Warner
288d759b8b library: adapt readproc for the latest lxc conventions
The merge request shown below prompted (thankfully) an
examination of our lxc containers logic in readproc.c.

As it turns out, the lxc folks changed that eyecatcher
used to identify containers within a task cgroup file.

So this patch, with little extra cost, will enable the
libprocps lxc_containers() guy to handle both strings.

[ additionally, I was shocked to find lxc allows the ]
[ eyecatcher to be changed at ./configure time. such ]
[ a provision has always existed. unfortunately, the ]
[ changed value was only available to root, assuming ]
[ one wished to tackle that undocumented liblxc api. ]

Reference(s):
. what prompted lxc support reevaluation
https://gitlab.com/procps-ng/procps/merge_requests/82
. original lxc support introduced
commit 0557504f9c

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-01-22 22:49:24 +11:00
Jim Warner
96d59cbf46 library: add item origin (as comments) to header files
A lack of documentation seems to be the major obstacle
to releasing this new library. So, in an effort to get
the ball rolling again, this patch adds the origins of
each item as a comment to six of the new header files.

However, before reviewing how such changes may benefit
that documentation objective, it seemed appropriate to
first reflect on newlib's background & current status.

___________________________________________ BACKGROUND
Discussions about and work on a new library began back
in July 2012 but quickly died. After a lull of 2 years
those discussions were resumed in August 2014 but soon
died also (and no code survived the gitorious demise).

With those early discussions, the recommended approach
was to encapsulate all of the libprocps data offerings
in individual functions. When it came to extensibility
it was suggested we should rely on symbols versioning.

Unfortunately that approach would have made for a huge
Application Programming Interface virtually impossible
to master or even document. And, runtime call overhead
would have been substantial for ps and especially top.

So, an alternative design was sought but there were no
new suggestions/contributions via freelists or gitlab.
Thus, in spite of a lack of library design experience,
the procps-ng team (Craig & Jim) set out to develop an
alternative API, more concise and with lower overhead.

Reference(s):
. 07/01/2012, begin library design discussion
https://www.freelists.org/post/procps/Old-library-calls
. 08/12/2014, revival of library design discussion
https://www.freelists.org/post/procps/libprocs-redesign

_____________________________________ DESIGN EVOLUTION
Our newlib branch first appeared on June 14, 2015. And
our current API actually represents the 4th generation
during the past 3 years of evolution. First, there was
a basic 'new', 'get' and 'unref' approach, using enums
to minimize the proliferation of 'get' function calls.

Then, in anticipation of other programs like ps, where
multiple fields times multiple processes would greatly
increase the number of 'get' function calls, a concept
of 'chains' was introduced. This became generation #2.

Such 'chains' proved unnecessarily complex so 'stacks'
replaced them. This was considered the 3rd generation,
but too many implementation details were still exposed
requiring those users to 'alloc', 'read', 'fill', etc.

Finally, a 4th generation emerged representing several
refinements to standardize and minimize those exported
functions, thus hiding all implementation details from
the users. Lastly, handling of 'errno' was normalized.

Reference(s):
. 06/14/2015, revival of new API discussion
https://www.freelists.org/post/procps/The-library-API-again
. 06/24/2015, birth of the newlib branch
https://www.freelists.org/post/procps/new-library
. 06/29/2015, 2nd generation introduced 'chains'
https://www.freelists.org/post/procps/new-library,8
. 07/22/2015, 3rd generation introduced 'stacks'
https://www.freelists.org/post/procps/newlib-stacks-vs-chains
. 06/18/2016, 4th generation refinements begin
https://www.freelists.org/post/procps/newlib-generation-35
. 11/10/2017, 4th generation standardized 'errno'
https://www.freelists.org/post/procps/some-more-master-newlib-stuff

_______________________________________ CURRENT DESIGN
Central to this new design is a simple 'result' struct
reflecting an item plus its value (thanks to a union).
As a user option, these item structures can be grouped
into 'stacks', yielding many results with just 1 call.
Such a 'stack' can be seen as a variable length record
whose content/order is determined solely by the users.

Within that 'result' structure, the union has standard
C language types so there is never a doubt how a value
should be used in a printf statement. Given that linux
requires a least a 32-bit platform the only difference
in capacity surrounds 'long' integers. And, where such
types might be used, the 32-bit maximums are adequate.

The items themselves are simply enumerators defined in
the respective header files. A user can name any items
of interest then the library magically provides result
structure(s). The approach was proven to be extensible
without breaking the ABI (in commit referenced below).

The 6 major APIs each provide for the following calls:
. 'new' ---------> always required as the first call .
. 'ref' -------------------------> strictly optional .
. 'unref' --------> optional, if ill-behaved program .
. 'get' --------------------> retrieve a single item .
. 'select' ----------------> retrieve multiple items .

And the 'get' and 'select' functions provide for delta
results representing the difference between successive
get/select calls (or a 'new' then  'get/select' call).

For the <diskstats>, <pids>, <slabinfo> & <stat> APIs,
where results are unpredictable, a 'reap' function can
return multiple result structures for multiple stacks.

The <pids> API differs from others in that those items
of interest must be provided at 'new' or 'reset' time,
a function unique to this API. And the <pids> 'select'
function requires PIDs or UIDs which are to be fetched
which then operates as a subset of 'reap'. Lastly, the
'get' function is an iterator for successive PIDs/TIDs
returning items previously identified via 'new/reset'.

To provide assistance to users during development, the
special header 'proc/xtra-procps-debug.h' is available
to check type usage against library expectations. That
check is activated by including this header explicitly
or via build using: ./configure '-DXTRA_PROCPS_DEBUG'.

Reference(s):
. 08/05/2016, type validation introduced
https://www.freelists.org/post/procps/newlib-types-validation
commit e3270d463d
. 08/11/2016, extensibility while preserving ABI example
https://www.freelists.org/post/procps/new-meminfo-fields
commit 09e1886c9e

_________________________ INITIAL DOCUMENTATION EFFORT
The initial attempt, referenced below, dealt primarily
with the <pids> interface. Separate man pages for each
exported function were created. Plus there was another
document describing the items, among other miscellany.

Adopting such an approach encounters several problems:

1. In order to use these man pages, users are required
to already know how to use the library. Or alternately
one could randomly search each of them while trying to
ascertain which function call satisfies their need and
what exactly was the proper compliment/order required.

2. While we can explain what all of those <pids> items
represent, that certainly isn't true for all the APIs.
See the gaps in kernel documentation for <meminfo> and
complete lack of documentation with that <vmstat> API.

3. Our documentation effort should take pains to avoid
unnecessary implementation details. Here's an example:
. "The pointer to info will have memory"
. "allocated and a structure created."

Alternatively, the following conveys user requirements
while not offering any internal implementation detail:
. "You must provide the address of a NULL"
. "info structure pointer."

Reference(s):
. 01/04/2017, initial documentation offering
https://www.freelists.org/post/procps/Using-reap-and-get
commit 2598e9f2ce

___________________ RECOMMENDED DOCUMENTATION APPROACH
I recommend that the newlib documentation consist of 3
man pages only. The first would cover the 5 major APIs
and their common functions. The second would deal with
the <pids> API exclusively, explaining how it differs.
Any remaining exported libproc functions which are yet
to be included could be represented in a 3rd document.

For these new documents the following are are assumed:

1. Since we will not be able to document all items, we
shouldn't try to document any items. We should instead
rely on proc(5) or Documentation/filesystems/proc.txt.

2. Program development often involves referencing some
header file(s). So, make that an absolute requirement.

3. With the addition of item origins, represented with
this commit, and considering that 'types' were already
present, the header file might be all some users need.

4. And who knows, when a user of our libproc complains
about gaps in their documentation, it might prompt the
kernel folks to correct those long standing omissions.

To summarize, I suggest that we replace that libproc.3
document with a more general one explaining the basics
of accessing this new library and the common calls for
most of the major interfaces. We can then create a new
document (libproc-pids.3?), which explains differences
in using the <PIDS> application programming interface.
A final document (libproc-misc.3?) covers what's left.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-01-03 08:06:48 +11:00
Jim Warner
d9f88246f6 library: insert 'PIDS_ID_TID' for symmetry, <PIDS> api
This change is being made in anticipation of adding the
source origin of each item to the <pids.h> header file.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-01-03 08:06:48 +11:00
Jim Warner
3694a9a4c5 library: update for fields & latest doc, <MEMINFO> api
This patch will bring the <meminfo> API into line with
that proc(5) document. There were several undocumented
fields that were not noted and these two were omitted:

. 'MmapCopy' was conditional on the #define CONFIG_MMU
. 'Quicklists' depends on the #define CONFIG_QUICKLIST

And we're about to get the following new field in 4.20
which will be represented, at least, in that proc.txt:

. 'KReclaimable' will include SReclaimable plus others

Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-01-03 08:06:48 +11:00
Jim Warner
fab37662ef library: refactor the XTRA_PROCPS_DEBUG implementation
If we ever were to eliminate the procps.h header file,
as discussed in the thread referenced below, then that
would impair the current XTRA_PROCPS_DEBUG provisions.

The only remaining way to verify result types would be
to explicitly include that <proc/xtra-procps-debug.h>.

So, this commit will once again enable the ./configure
provision for defining the -DXTRA_PROCPS_DEBUG option.

Reference(s):
https://www.freelists.org/post/procps/newlib-Qualys-patches,6

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-09-12 21:12:06 +10:00
Jan Rybar
7febddabd8 Possible segfault in file2strvec introduced by latest CVE fix
'rbuf' used before allocated.
Revealed by static analysis

References:
 commit 36e066df52

Signed-off-by: Craig Small <csmall@enc.com.au>
2018-08-08 20:06:57 +10:00
Jim Warner
ad4269f118 library: expanded to provide for the 'executable path'
This patch is the first of three implementing a newlib
branch version of that Jan Rybar master merge request.

With this series we'll ultimately extend 'EXE' support
to both ps and top (plus, everyone else who wants it).

Reference(s):
. master branch merge request
https://gitlab.com/procps-ng/procps/merge_requests/66

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-26 21:25:18 +10:00
Jim Warner
50ae55ef10 library: avoid problems involving 'supgid' mishandling
Following that patch referenced below, the top SUPGRPS
field would produce a segmentation fault and ps SUPGRP
would often show "(null)". Such problems resulted from
some faulty logic in the status2proc() routine dealing
with 'Groups' (supgid) which served as a source field.

For many processes the original code produced an empty
string which prevented conversion to the expected "-".
Moreover, prior to release 3.3.15 such an empty string
will become 0 after strtol() which pwcache_get_group()
translates to 'root' yielding very misleading results.

So, now we'll check for empty '/proc/#/status/Groups:'
fields & consistently provide a "-" value for callers.

[ we'll also protect against future problems in that ]
[ new qualys logic by always ensuring valid 'supgrp' ]
[ pointers - logic which revealed our original flaw! ]

Reference(s):
. original qualys patch
0071-proc-readproc.c-Harden-supgrps_from_supgids.patch

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:20 +10:00
Jim Warner
80e210d38e library: refactor #define FALSE_THREADS dependent code
This refactor was done in response to the Qualys patch
referenced below, which deals with some 'readeither()'
flaws under the master branch. Under our newlib branch
those flaws mostly disappear since the function is now
private. But without a redesign the #define is broken.

When the #define FALSE_THREADS is active, some special
strings showing "[ duplicate ENUM ]" will appear under
each child thread. Note that the real reason for those
appearing isn't being exercised, only their mechanics.

In reality, they only show when a user duplicates such
enums in a results stack & only 1 instance can own it.

Reference(s):
. original qualys patch
0084-proc-readproc.c-Work-around-a-design-flaw-in-readeit.patch
. QUICK_THREADS became FALSE_THREADS
commit c546d9dd44

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:20 +10:00
Jim Warner
97d078a9af library: clean up some miscellaneous compiler warnings
Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:20 +10:00
Jim Warner
e051535686 library: adapt for increased (cmd) program name length
In the new library 'cmd' is dynamically allocated just
like 'cmdline'. This will align us with the ref below.

Reference(s):
. master branch increase to 64
commit 2cfdbbe897

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:20 +10:00
Qualys Security Advisory
48d118b81b 0084-proc/readproc.c: Work around a design flaw in readeither().
readeither() caches (in new_p) a pointer to the proc_t of a task-group
leader, but readeither()'s callers can do pretty much anything with the
proc_t structure passed to and/or returned by this function. For
example, they can 1/ free it or 2/ recycle it (by passing it to
readeither() as x).

1/ leads to a use-after-free, and 2/ leads to unexpected behavior when
taskreader()/simple_readtask() is called with new_p equal to x (this is
not a theoretical flaw: 2/ happens in readproctab3() when want_task()
returns false and p is a group leader).

As a workaround, we keep a copy of new_p's first member (tid) in static
storage, and the next times we enter readeither() we check this "canary"
against the tid in new_p: if they differ, we reset new_p to NULL, which
forces the allocation of a new proc_t (the new "leader", or reference).

This always detects 2/ (because free_acquired(x,1) memsets x and hence
new_p); always detects 1/ if freed via free_acquired() and/or freeproc()
(very likely, otherwise memory may be leaked); probably detects 1/ even
if freed directly via free() (because the canary is the first member of
proc_t, likely to be overwritten by free()); but can not detect 1/ if
free() does not write to new_p's chunk at all.

Moreover, accessing new_p->tid to check the canary in case 1/ is itself
a use-after-free, so a better long-term solution should be implemented
at some point (we wanted to avoid intrusive and backward-incompatible
changes in this library function, hence this imperfect workaround).

---------------------------- adapted for newlib branch
. adapted via 'patch' (rejected due to 'xcalloc' ref)
. with loss of both readproctab functions, most no longer true

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:19 +10:00
Qualys Security Advisory
b9a9fd4c74 0082-proc/readproc.c: Fix double-free()s in readtask().
If QUICK_THREADS is not defined (it is not by default, but most
distributions enable it) and task_dir_missing is true (only on very old
kernels), then readtask() forgets to reset some of the struct proc_t t's
members, which later results in double-free()s in free_acquired().

For now, we simply synchronized the list of members to be reset with the
list of members freed in free_acquired().

---------------------------- adapted for newlib branch
. now 'cmd' is also dynamic
. just synchronized with those freed in free_acquired
. QUICK_THREADS is now FALSE_THREADS, serving different purpose
. entire patch will be effectively reverted with upcoming refactor

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:19 +10:00
Qualys Security Advisory
990ea89ae9 0080-proc/readproc.c: Harden openproc().
Replace xmalloc() with xcalloc().

---------------------------- adapted for newlib branch
. trade xcalloc() for calloc()
. thus we must account for potential ENOMEM

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:19 +10:00
Qualys Security Advisory
c03e52786d 0078-proc/readproc.c: Harden simple_nextpid().
Replace memcpy+strcpy with snprintf.

---------------------------- adapted for newlib branch
. adapted via 'patch' (without rejections)

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:19 +10:00
Qualys Security Advisory
ed463c7d88 0077-proc/readproc.c: Harden fill_cgroup_cvt().
Check the return value of snprintf(), otherwise dst may point
out-of-bounds when it reaches the end of the dst_buffer (the snprintf()
always returns 1 in that case, even if there is not enough space left),
and vMAX becomes negative and is passed to snprintf() as a size_t.

---------------------------- adapted for newlib branch
. adapted via 'patch (without rejections)

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:19 +10:00
Qualys Security Advisory
1052091107 0076-proc/readproc.c: Harden vectorize_this_str().
This detects an integer overflow of "strlen + 1", prevents an integer
overflow of "tot + adj + (2 * pSZ)", and avoids calling snprintf with a
string longer than INT_MAX. Truncate rather than fail, since the callers
do not expect a failure of this function.

---------------------------- adapted for newlib branch
. logic is now in pids.c
. former 'vectorize_this_str' is now 'pids_vectorize_this'

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:19 +10:00
Qualys Security Advisory
d9c0a3e36f 0075-proc/readproc.c: Harden read_unvectored().
1/ Prevent an out-of-bounds write if sz is 0.

2/ Limit sz to INT_MAX, because the return value is an int, not an
unsigned int (and because if INT_MAX is equal to SSIZE_MAX, man 2 read
says "If count is greater than SSIZE_MAX, the result is unspecified.")

3/ Always null-terminate dst (unless sz is 0), because a return value of
0 because of an open() error (for example) is indistinguishable from a
return value of 0 because of an empty file.

4/ Use an unsigned int for i (just like n), not an int.

5/ Check for snprintf() truncation.

---------------------------- adapted for newlib branch
. adapted via 'patch (without rejections)

Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-06-09 21:35:19 +10:00