When this module was upgraded to 3rd generation in the
patch referenced below, numa node support was migrated
from the top program into newlib. The 'guest_nice' and
'guest' tics were overlooked as top did not need them.
So, this commit corrects that oversight and achieves a
proper symmetry between the cpu & numa jiffies counts.
Reference(s):
. 3rd gen redesign, numa support imported
commit abc71a46ad
Signed-off-by: Jim Warner <james.warner@comcast.net>
A priming read at 'new' time in that <slabinfo> module
was important so that permission problems are detected
early. Plus, it also had the potential of making delta
values valid when 'get' or 'select' were first called.
It is for that latter reason that such a read was also
incorporated in the <diskstats> module 'new' function.
No other module, however, employed such priming reads.
This patch just brings those potential benefits to all
of our other newlib modules with the exception of that
<pids> guy. That module is, of necessity, sufficiently
different from those others to justify such exclusion.
Not only are there precious few DELTA enums in <pids>,
but the costs of a priming read would be much greater.
[ otherwise, these newly added priming reads have no ]
[ measurable negative impact on performance/timings. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The <slabinfo> header provides 3 groups of enumerators
with prefixes of SLABINFO, SLABS & SLABNODE. The first
is strictly user oriented & isn't supported internally
by any structure. The other two, however, have structs
associated with 'em but, unfortunately, 1 is misnamed.
The 'struct slabs_node' is associated with 'nodes' and
supports the enumerators with the SLABNODE prefix. But
the 'struct slabs_hist' was associated with 'hist' yet
supports those enumerators with just the SLABS prefix.
We do not care very much what some structure is called
but we do care about an identifier used manipulate it.
This patch will trade the 'hist' identifier associated
with 'struct slabs_hist' for a more congruous 'slabs'.
[ it's awful when the author can't remember what the ]
[ true meaning of an identifier is after creating it ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
If, in fact, numa nodes are dynamic (like that current
total of on-line cpus) the existing logic was lacking.
It included an early return before checking the total.
So, this commit ensures that the nodes total is always
set or updated consistently in only a single function.
There's no need to set it at the time 'new' is called.
[ and since under our existing code this nodes total ]
[ could never possibly have been zero, the erroneous ]
[ test (with the early return) has now been whacked! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Still unhappy with a minor memory leak associated with
libnuma, I experimented with omitting the dlclose that
was issued at module's end. For some reason which will
remain a mystery, the valgrind leak then went bye-bye.
So this patch just omits one use of dlclose and relies
on whatever kernel magic is at work to free the memory
when each process ends. We kept, however, the original
code (now commented-out) to serve as a future caution.
There remains one potential (but unlikely) dlclose use
near the original dlopen. But there will be no leak as
that 'numa_node_of_cpu' will not yet have been called.
This seems to be the culprit that triggers such leaks.
None of this libnuma shit would likely have come close
to hitting our fan had the numa developers provided us
with 'new' and 'unref' functions like our newlib does.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Recent profiling and timings have resulted in improved
newlib performance. This patch completes that process.
It just normalizes naming conventions employed for all
allocations involving reaped stacks & history support.
The modules offering a 'reap' function will also offer
the now standardized corresponding STACKS_INCR define.
The modules which provide dynamic history support will
now have a separate #define called NEWOLD_INCR used in
allocations/reallocations. And, while values currently
are set equal to that STACKS_INCR value, in the future
some reason for divorcing those two may be discovered.
----------------------------- for future reference ---
In those modules which contain the STACKS_INCR #define
it is tempting to specify a large value so as to avoid
repeated calls to malloc/realloc. However, in doing so
an extra runtime price will be paid in 'cleanup_stack'
calls with any iterative programs like top or slabtop.
So, with the current values a balance has been sought.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Profiling revealed a large amount of time spent in the
'escape_str_utf8' function (escape.c) with both of our
NLS branches (newlib and master). That same result was
not seen under an ancient top-3.2.8 program & library.
Well, the 3.2.8 result was ultimately explained by the
absence of a 'setlocale', necessary under NLS support.
Thus, when that ancient library tested for locale, all
it got was 'ANSI_...' & assumed 'UTF-8' wasn't active.
But after a hack to that ancient code to place it on a
par with newlib/master, I still found cost differences
that led me to revisit an old change referenced below.
It turns out that 'iswprint' costs far more than would
a call of 'isprint', even with the extra support code.
So this commit just reverts that five year old change.
Reference(s):
commit 7b0fc19e9d
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch contains the following miscellaneous stuff:
. The pids_stacks_fetch() routine might call for newly
allocated stacks to be itemized. However, that job was
already tended to by the pids_stacks_alloc() function.
So, this patch just eliminates a redundant invocation.
------------------------------------------------------
. The concept of 'dirty_stacks' has not kept pace with
the evolving stacks implementation. Originally, stacks
were considered dirty only if free() of dynamic memory
was needed before refreshing any single result struct.
Later, with the introduction of the 'extra' item and a
promise to reset it to zero, 'dirty' was much broader.
So, this patch just treats the dirty flg as others do.
------------------------------------------------------
. Lastly, a word or three about performance & timings.
Tuning efforts concentrated on the <pids> API and top.
And unless an oldlib equivalent to the preceding patch
is applied (favoring stat vs. status), newlib top will
often outperform the oldlib version (obviously wrong).
So assuming /proc/stat is preferred in both libraries,
generally speaking, a cpu and elapsed time increase of
1-5% was found for this new stacks oriented interface.
Of course, there's also an increased memory footprint.
There are some occasions, however, when the newlib top
is at a substantial disadvantage. For example if WCHAN
or TTY is displayed, such items will be present in all
newlib reaped stacks (i.e. every process). But old top
would only incur such overhead with displayable tasks.
Thus, oldlib top could outperform newlib by up to 25%,
for example, if only fields requiring NO library flags
were displayed. However, such a scenario is not likely
since only GID, UID, PID, TGID & WCHAN would be shown.
In the usual case, that overhead associated with WCHAN
and/or TTY is overshadowed by other top runtime costs.
All in all a pleasing outcome I deem quite acceptable.
------------------------------------------------------
Signed-off-by: Jim Warner <james.warner@comcast.net>
Long ago, in a galaxy far away (when top was in charge
of library FILL flgs) /proc/status was to be preferred
over /proc/stat if a field could be satisfied by both.
This was done to avoid costly 64-bit math emulation in
a 32-bit application due to 'unsigned long long' data.
Well it's time to acknowledge the prevalence of 64-bit
platforms. And in such an environment the cost picture
has shifted significantly. It now costs 14 times (wow)
as much to access /proc/status compared to /proc/stat.
In other words, even with '%llu' variables, a sscanf()
call in stat2proc() beats the pants off that home brew
gperf based hashing employed by the status2proc() guy.
In fact, status2proc incurs higher costs than found in
the most expensive aspect of top's forest view option.
Here's a gprof extract to illustrate the costs. It was
produced with an rcfile requiring fields from both the
/proc/stat & /proc/status pseudo files (among others).
There were 5000 iterations in each of 4 separate gprof
runs subsequently merged into 1 gmon.sum for analysis.
% self self
time seconds calls us/call name
----- ------- ------- ------- -----------
28.65 4.10 4689423 0.87 status2proc
26.14 3.74 40000 93.50 forest_adds
...
01.96 0.28 4689427 0.06 stat2proc
[ since forest_adds is recursive, the calls value is ]
[ the non-recursive #, its 'call graph' shows totals ]
Anyway, now that such cost is known this patch becomes
what is euphemistically known as the usual no-brainer.
[ jeeze, was it really this long between profilings? ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch will begin some refinements associated with
gprof. Initially, functions names have been changed to
help in identifying potential bottlenecks. This effort
also included the obscure set, free and sort routines.
Plus the following additional modifications were made:
. the stacks_alloc prologue was generalized plus added
to a couple of modules where it had not yet propagated
. a couple of the '// end ...' comments were corrected
. some functions have been formally tagged as 'inline'
Signed-off-by: Jim Warner <james.warner@comcast.net>
The immediately prior commit demonstrated how our APIs
might be expanded in at some point in the future while
maintaining binary compatibility in previous programs.
However, since we've yet to release the 1st version of
our new library, there's no need to violate alphabetic
ordering just yet. So, this patch restores that order.
Signed-off-by: Jim Warner <james.warner@comcast.net>
With the 4.8 kernel, 2 new fields will be added to the
meminfo pseudo file. This commit, soon to be replaced,
is intended as an example of how such changes might be
incorporated plus still maintain binary compatibility.
This actually goes further than is strictly necessary,
by retaining meminfo_item ordering for 'set' functions
and the creation of hash table entries. However, there
is only 1 true requirement, that of Item_table entries
which must always agree exactly with item enumerators.
All of the other changes could be done alphabetically.
Ok, so what happens when an old program encounters the
new expanded meminfo items? Well, if it was thoroughly
tested against an old library, it won't even see those
new fields. On the other hand, if it somehow exceeds a
previous MEMINFO_logical_end, then it will just get an
extra result structure or two, with no real harm done.
[ this patch is being replace by the very next patch ]
[ so that our iniitial newlib release can maintain a ]
[ strict alphabetic ordering in all areas initially! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
When the numa stuff was imported from the top program,
that #include for dlopen() was not made conditional as
it should have been. Well, here it is being corrected.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Rather than return a 0 result for all VAL type errors,
return what would have been without validation active.
This will enable a program like pgrep to still print a
result even though it used some incorrect type member.
With this commit, our VAL macro validations logic will
behave in exactly the same way as the GET validations.
While warning messages may be issued, except for a bad
enumerator, values will always be returned to callers.
Signed-off-by: Jim Warner <james.warner@comcast.net>
One ought not to assume that random memory access will
always succeed or, when it does, that an obviously bad
item enumerator will always be found at that location.
Thus, this patch corrects some really poor assumptions
associated with the 'xtra_procps_debug.h' header file.
[ and it does so in somewhat contorted ways so as to ]
[ avoid several darn gcc -Wnonnull warning messages! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
When users call the native 'get' functions they have a
responsibility to check that the result struct address
was indeed returned. But when using those 'GET' macros
there was no protection for possible NULL dereference.
So this patch will add some protection for a potential
failure of an underlying 'get' function. And should it
occur then those 'GET' macros will just return a zero.
Plus, we'll also mirror that behavior in the debugging
header should the XTRA_PROCPS_DEBUG #define be active.
And, we might as well add a warning when invalid items
are passed to 'GET' macros, just like we do for 'VAL'.
[ lastly, we added the missing opening parens/braces ]
[ to 2 'GET' macros in that xtra-procps-debug.h file ]
[ which went unnoticed until the qa folks caught up. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
During development, we now have a means for validating
that a type referenced in application code matches the
actual type set by the library. The new feature can be
activated through either of the following two methods:
1) ./configure CFLAGS='-DXTRA_PROCPS_DEBUG' (all pgms)
2) an #include <proc/xtra-procps-debug.h> (single pgm)
[ in the future, one could add a formal configure.ac ]
[ provision. but for now a manual approach is safer. ]
Lastly, for any module which provides a sort function,
the handling for both 'noop' & 'extra' enumerators was
made consistent. Now, 'noop' is not sorted and 'extra'
will be sorted as that module's widest supported type.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch will set the stage for validating the types
referenced in the result union. For now, the parameter
representing that 'info' structure will remain unused.
[ and while we're at it, let us correct a faulty GET ]
[ macro in the diskstats header. that puppy missed a ]
[ parm which ain't so good if that guy is ever used! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit attempts to minimize the variety of types
currently used. Plus, the following were also changed:
. the MEM fields were switched to parallel the VM guys
. PIDS_MEM_VIRT -> PIDS_MEM_VIRT_PGS
. PIDS_MEM_VIRT_KIB -> PIDS_MEM_VIRT
. made NICE 's_int' so that it then parallels PRIORITY
. change RTPRIO & SCHED_CLASS from 'ul_int' to 's_int'
. removed Item_table 'oldflags' for an obsoleted field
. PIDS_WCHAN_ADDR
. added calculations like TICS_ALL_C for the following
. PIDS_TICS_USER_C
. PIDS_TICS_SYSTEM_C
. these three new 'TICS' fields have been incorporated
. PIDS_TICS_BLKIO - jiffies spent in block i/o
. PIDS_TICS_GUEST - jiffies spent as a guest
. PIDS_TICS_GUEST_C - as above, includes dead children
. that PIDS_TICS_DELTA was renamed PIDS_TICS_ALL_DELTA
( so it did not hide between TICS_BLKIO & TICS_GUEST )
( and to make clearer what's included: utime + stime )
. eliminated 'sl_int' entirely from that result struct
[ often, the <pids> module changes necessitated that ]
[ readproc header and source files had to change too ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
With this patch, all of the modules which offer a reap
function (pids, diskstats, slabinfo and stat too) will
now also provide for sorting whatever had been reaped.
It was easy to overlook a sort function for our <STAT>
guy given the paucity of CPUs on your typical personal
desktop or laptop. However, out in the world one might
find boxes with hundreds of CPUs plus many NUMA nodes.
Hey, who are we to disallow sorts on something another
person might see as useful under the above conditions?
And, there's always something to be said for symmetry.
[ of course, several minor tweaks were also included ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
. a more appropriate error return code was substituted
. a safer form of comparison is utilized in two places
. a STAT_VAL macro replaced by more proper MEMINFO_VAL
. several of the silly 'xerrx' formats had been missed
. a few whitespace changes have also been incorporated
Signed-off-by: Jim Warner <james.warner@comcast.net>
With this patch we've completed a progression toward a
standard approach to naming conventions which follows:
* Only functions will begin with that 'procps_' prefix
. ........................................... examples
. procps_vmstat_get ()
. procps_diskstats_select ()
- ----------------------------------------------------
* Exposed structures begin with the module/header name
. ........................................... examples
. struct pids_info
. struct stat_reaped
- ----------------------------------------------------
* Item enumerators begin like structs, but capitalized
. ........................................... examples
. VMSTAT_COMPACT_FAIL
. MEMINFO_DELTA_ACTIVE
[ slabinfo varies slightly due to some item variants ]
. SLABINFO_extra
. SLABS_SIZE_ACTIVE
. SLABNODE_OBJS_PER_SLAB
[ could cure with a prefix of SLABINFO, but too long ]
- ----------------------------------------------------
* Other enumerators work exactly like item enumerators
. ........................................... examples
. PIDS_SORT_ASCEND
. STAT_REAP_CPUS_AND_NODES
- ----------------------------------------------------
* Macros and constants begin just like the enumerators
. ........................................... examples
. #define SLABINFO_GET
. #define DISKSTATS_TYPE_DISK
- ----------------------------------------------------
Signed-off-by: Jim Warner <james.warner@comcast.net>
Many of our item enumerator identifiers are very long,
especially in that <VMSTAT> module. Additionally, they
all contain the exact same universal 'PROCPS_' prefix.
The origins for this are likely found in the desire to
avoid name clashes with other potential include files.
But with procps-ng newlib, we've probably gone way too
far. Did 'PROCPS_PIDS_TICS_SYSTEM' actually offer more
protection against clash than 'PIDS_TICS_SYSTEM' does?
I don't think so. Besides, no matter how big that name
becomes, one can never guarantee they'll never be some
clash. And, conversely, extremely short names will not
always create conflict. Of course, in either case when
some clash occurs, one can always #undef that problem.
Thus, this commit will eliminate that 'PROCPS_' prefix
making all of those enum identifiers a little shorter.
And, we'll still be well above some ridiculously short
(criminally short) names found in some common headers:
- - - - - - - - - - <term.h>
- 'tab', 'TTY', etc
- - - - - - - - - - - - - - - - <search.h>
- 'ENTER', ENTRY', 'FIND', etc
------------------------------------------------------
Finally, with this as a last of the wholesale changes,
we will have established the naming conventions below:
. only functions will begin with that 'procps_' prefix
. exposed structures begin with the module/header name
. item enumerators begin like structs, but capitalized
. other enumerators work exactly like item enumerators
. macros and constants begin just like the enumerators
------------------------------------------------------
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch attempts to standardize the naming of those
most important (declared not defined) context structs.
The present practice represents a hodge podge of names
only some of which reflect the source /proc file name.
And 2 of those file names embed a literal 'info' which
is likely the origin of that required parm identifier.
Now we'll append a universal '_info' to such structure
names, while including the names of those /proc pseudo
files where possible. In any case, that context struct
will *always* begin with the actual module/header file
name. And only the following two sound a little weird!
---------> 'meminfo_info' + 'slabinfo_info' <---------
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch will bring this interface up to our 3rd gen
standards. The following summarizes the major changes.
* New delta provisions have been added to most fields.
There are, of course, some fields for which a delta is
inappropriate. They include the identifying items such
as name, type, major and minor. Plus the io_inprogress
field which already acts, in effect, as a delta value.
* To provide delta support, dev_node historical values
have become persistent. By the same token, the library
must provide for future removal of disks/partitions. A
timestamp is used to detect 'stale' data which will be
deleted so as not to satisfy some get, select or reap.
* Such persistent support is provided by a linked list
which, by default, grows from the bottom down so as to
maintain compatibility with the /proc/diskstats order.
Initially, I was tempted to use the GNU tsearch (tree)
provisions until I discovered the overhead of building
that tree plus costs of a subsequent 'twalk'. Besides,
walking such a tree means retrieval order would differ
from an order required/expected by the vmstat program.
* The '/sys/block' directory is no longer scanned with
every refresh cycle. Rather, it's only accessed when a
node is first encountered. Then, that node's 'type' is
persistent for its lifetime like several other fields.
* A sort provision was included, at virtually no cost,
even though such a provision was not currently needed.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Where possible, libprocps files convey the name of the
actual source pseudo file under the '/proc' directory.
This brings diskstats into line with such an approach.
Signed-off-by: Jim Warner <james.warner@comcast.net>
[ plus we also play catch up on some earlier changes ]
[ that impacted skill.c, after using --enable-skill! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
A collection of miscellaneous code and comment tweaks.
[ such changes will stop when desk checking ends too ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
That #define QUICK_THREADS was impossible to implement
under the new <pids> interface. Plus it was also found
to distort some thread information (referenced below).
So, it's always been inactive under the newlib branch.
However, it will (with small changes) still serve some
useful purpose in our library. Now, when the redefined
FALSE_THREADS is active, those special strings showing
"[ duplicate ENUM ]" will appear for any child thread.
Note: the real reason for such strings appearing isn't
being exercised, only their mechanics. In actual usage
they are substituted when a user duplicates such items
in a results stack & only the 1st instance can own it.
With this patch, we are simply fooling the <pids> code
into thinking an item was duplicated via a NULL value.
Reference(s):
. from master branch
commit 25a6ecdbfb
Signed-off-by: Jim Warner <james.warner@comcast.net>
Yes, all of these changes are strictly cosmetic. It is
likely symptomatic of some deep-seated character flaw.
[ or, it might be because of a certain pride in this ]
[ new library and the desire to make it even better! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Ever since their introduction, plus continuing through
several evolutions, both the meminfo and vmstat 'read'
functions employed a 'do while' loop for /proc access.
However, that loop construct was wrong since identical
tests were already done (twice!) within each loop body
itself, then accompanied by its own 'break' statement.
So, we will now transform them both into forever loops
which will help us to emphasize such break statements.
[ plus, let's return an error should nothing be read ]
[ lastly, eliminate 1 erroneous PROCPS_EXPORT prefix ]
Reference(s):
. original meminfo introduction
commit a20e88e4e7
. original vmstat introduction
commit a410e236ab
Signed-off-by: Jim Warner <james.warner@comcast.net>
The procps library attempts to work out the tty of a process
through several methods. For things like /dev/tty123 or
/dev/foo it works fine.
For tty devices that put the minor number in a directory
of the major name this fails. So then we have to fallback
to stating things like the processes STDERR and try again.
Considering a lot of processes sit on ttys such as
/dev/pts/3 this is a lot of wasted time. At the point of
entering driver_name we know "/dev/pts" and we know "3"
we just didn't join them up the right way as this is old
code.
This change now looks for /dev/pts/3 as well. It does it
after looking for /dev/pts3 so the behaviour is the same.
References:
https://bugs.debian.org/770215
Signed-off-by: Craig Small <csmall@enc.com.au>
This commit brings all of those 'fetch' type functions
(supporting some form of 'reap') into closer alignment
with one another. The biggest impact is to be found in
the <stat> module, which now provides for the separate
copy of stack pointers which will be exposed to users.
The reason such a copy was not employed initially with
<stat>, unlike those for <pids> and <slabinfo>, is due
to the fact that such stacks were never sortable. Thus
the original raw consolidated extent pointers wouldn't
have been disturbed. But that meant no NULL delimiter.
So with this commit, all reap/fetch operations now use
pointer copies when returning results to callers. And,
all such arrays are now NULL delimited meaning callers
can choose their own access fencepost: totals or NULL.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit will consolidate the three separate copies
of the item enumerators currently supporting that reap
function (summary, cpus & nodes) into one shared copy.
That select function will continue to maintain its own
dedicated items copy, albeit via a new item structure.
[ and while we're at it, we'll move the 'select' guy ]
[ to its proper alphabetical place, after our 'reap' ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Unlike other cached string fields (such as USER name),
that lxcname field was defined as 'const char *'. This
required a custom 'set' function using a cast in order
to avoid a compiler warning regarding that assignment.
So this commit brings lxcname into line with all those
other cached string fields. And while we're at it, the
reason some string fields have no 'freefunc' will also
be explained with some new comments in the Item_table.
Signed-off-by: Jim Warner <james.warner@comcast.net>
As those 3rd generation newlib APIs evolved so too did
the extents_free_all() function. Most versions of this
function required the callers to first verify that the
extents anchor wasn't empty, which was poor etiquette.
This simple function should have been much more robust
and forgiving. With this commit, it fnally becomes so.
Signed-off-by: Jim Warner <james.warner@comcast.net>
With all our 3rd generation interfaces, we're now well
positioned to preserve binary compatibility should new
fields be added to any public structure (assuming that
the 'result' union already contains its largest type).
This remains true even for the <pids> interface, which
unlike the others, has one structure embedded within a
separate struct rather than declaring a pointer to it.
The counts struct was positioned after the stacks ptrs
array so as to preserve that ABI if ever new ints were
added. Logically, however, the counts (especially that
total) should precede the stacks array if we wished to
properly place a horse (total) before a cart (stacks).
So to enable relocating those counts we will no longer
embed that structure, but provide a pointer to it. And
this will make accessing syntax feel more natural too.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch represents the partial revert of the commit
referenced below. And, so that yours truly (the author
for goodness sake) doesn't shoot his tootsies again in
the future, a cautionary programmer comment was added.
Reference(s):
commit dea4033418
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch represents the following minor adjustments:
. old PROCTAB used with select & reap more descriptive
. reformatted PROCPS_PIDS_VAL macro like other 3rd gen
Signed-off-by: Jim Warner <james.warner@comcast.net>
The following commit message is shared with 4 patches.
------------------------------------------------------
Under the newlib interface most of our 'get' functions
represent a bit of a compromise in that the actual raw
values are coerced into one (probably ok) return type.
That approach creates the possibility of truncation at
best, and wouldn't serve future needs should something
other than numeric data be added to the 'get' results.
This commit trades the current compromise for a return
value guaranteed to satisfy all future needs, namely a
pointer to a particular api's specific results struct.
The impact on existing programs is minimal, especially
when using a new supplied macro. Otherwise, native 'C'
syntax could be used, but may feel somewhat unnatural.
[ as an aside, this new approach allows us to delete ]
[ all 'getsfunc' table entries & the supporting code ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The following commit message is shared with 4 patches.
------------------------------------------------------
Under the newlib interface most of our 'get' functions
represent a bit of a compromise in that the actual raw
values are coerced into one (probably ok) return type.
That approach creates the possibility of truncation at
best, and wouldn't serve future needs should something
other than numeric data be added to the 'get' results.
This commit trades the current compromise for a return
value guaranteed to satisfy all future needs, namely a
pointer to a particular api's specific results struct.
The impact on existing programs is minimal, especially
when using a new supplied macro. Otherwise, native 'C'
syntax could be used, but may feel somewhat unnatural.
[ as an aside, this new approach allows us to delete ]
[ all 'getsfunc' table entries & the supporting code ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The following commit message is shared with 4 patches.
------------------------------------------------------
Under the newlib interface most of our 'get' functions
represent a bit of a compromise in that the actual raw
values are coerced into one (probably ok) return type.
That approach creates the possibility of truncation at
best, and wouldn't serve future needs should something
other than numeric data be added to the 'get' results.
This commit trades the current compromise for a return
value guaranteed to satisfy all future needs, namely a
pointer to a particular api's specific results struct.
The impact on existing programs is minimal, especially
when using a new supplied macro. Otherwise, native 'C'
syntax could be used, but may feel somewhat unnatural.
[ as an aside, this new approach allows us to delete ]
[ all 'getsfunc' table entries & the supporting code ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The following commit message is shared with 4 patches.
------------------------------------------------------
Under the newlib interface most of our 'get' functions
represent a bit of a compromise in that the actual raw
values are coerced into one (probably ok) return type.
That approach creates the possibility of truncation at
best, and wouldn't serve future needs should something
other than numeric data be added to the 'get' results.
This commit trades the current compromise for a return
value guaranteed to satisfy all future needs, namely a
pointer to a particular api's specific results struct.
The impact on existing programs is minimal, especially
when using a new supplied macro. Otherwise, native 'C'
syntax could be used, but may feel somewhat unnatural.
[ as an aside, this new approach allows us to delete ]
[ all 'getsfunc' table entries & the supporting code ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
All our numa logic was well encapsulated under #ifndef
NUMA_DISABLE (as ./configure --disable-numa), with the
exception of 1 case label. The reason it didn't create
any problems is because the sole consumer (top) itself
responded to the presence of #define NUMA_DISABLE too.
But if that option wasn't used, an oops still awaited.
When experimenting with a 32-bit library plus a 64-bit
kernel, a bug was revealed should the numa library not
have been found. Our 'reap' function must tolerate the
potential of a 0 return from that stacks_fetch_tics().
Now, if numa is desired but not possible, we'll be ok.
[ and, we'll build numa history from fresher sources ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
On MacOS the system already has user_from_uid and group_from_gid.
These are renamed pwcache_get_user and pwcache_get_group.
They were also exported but did not appear in the symbol file
or used by any of the procps binaries. They are no longer exported.
References:
https://gitlab.com/procps-ng/procps/issues/34
Signed-off-by: Craig Small <csmall@enc.com.au>
When those items were made dynamic at 'new' time, some
other functions, previously assured of their presence,
failed to verify a 'reset' had acually been requested.
This commit just corrects that oversight and avoids an
attempt to 'assign_results' when no items are present.
Reference(s):
. when items/numitems became optional
commit 9ebadc1438
Signed-off-by: Jim Warner <james.warner@comcast.net>
Following is a summary of significant changes (if any)
to each of these now upgraded 3rd gen library modules.
<meminfo> ............................................
. eliminated duplicate decl of 'struct procps_meminfo'
. standardized/normalized results struct union members
. added 'std' & 'var' dividers in .c file, like <pids>
. how did i miss relocating all these friggin' #undefs
. cleanup 'get' return logic (remove a redundant 'if')
<pids> ...............................................
. repositioned the procps_pidsinfo structure in header
. removed the extra trailing comma from enum pids_item
. standardized/normalized results struct union members
<slabinfo> ...........................................
. corrected comment typo (jeeze, in an 'aligned' para)
. standardized/normalized results struct union members
. added 'std' & 'var' dividers in .c file, like <pids>
. removed an obsolete #undef from procps_slabinfo_sort
. cleanup 'get' return logic (remove a redundant 'if')
<stat> ...............................................
. how did i miss relocating all these friggin' #undefs
. corrected an initialization fencepost used with numa <=== see Craig, here's a bug fix
. removed the extra trailing comma from enum stat_item
. standardized/normalized results struct union members
. added 'std' & 'var' dividers in .c file, like <pids>
. strengthen those parm checks in procps_stat_get func
. cleanup 'get' return logic (remove a redundant 'if')
<vmstat> .............................................
. standardized/normalized results struct union members
. added 'std' & 'var' dividers in .c file, like <pids>
. cleanup 'get' return logic (remove a redundant 'if')
[ virtually all of these tweaks reflect the author's ]
[ continuing pursuit of an unreasonable goal -- that ]
[ of a 'perfect' (plus 'pretty') C language program! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
After reviewing the hsearch code in glibc, performance
will almost certainly benefit from abandoning a strcmp
approach in favor of hashing, just like that <vmstat>.
[ As an aside, now having struggled toward that goal ]
[ of opaqueness & making our API as user friendly as ]
[ possible, haven't we earned the rights to evaluate ]
[ other implementations? For example, GNU's hsearch? ]
[ We expose none of our 'info' struct details to the ]
[ users, but GNU exposes their 'hsearch_data' thingy ]
[ right there in <search.h>. But worse, they require ]
[ the user to zero it out before 1st use. Jeeze, you ]
[ mean that a function called hcreate_r could not do ]
[ its own memset? Aw, come on GNU! What's with that? ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
With the dust now settling on all those 3rd generation
upgrades, this patch tries to provide some consistency
among the separate modules involved. Someday we should
consider a 4th generation where all redundant code has
been removed and isolated in a new shared source file.
Following is a summary of significant changes (if any)
to each of these now upgraded 3rd gen library modules.
<meminfo> ............................................
. strictly formatting/comment changes, code unaffected
<pids> ...............................................
. replaced a local mkSTR macro with existing STRINGIFY
. added fetch narrative explaining duplicate addresses
<slabinfo> ...........................................
. rearranged some free logic for procps_slabinfo_unref
. added fetch narrative explaining duplicate addresses
<stat> ...............................................
. added #define ENFORCE_LOGICAL, just as in <slabinfo>
. replaced a local mkSTR macro with existing STRINGIFY
. alphabetized the function declarations in the header
<vmstat> .............................................
. made one coverity concession with read_vmstat_failed
[ several of these changes may reflect this author's ]
[ continuing pursuit of an unreasonable goal -- that ]
[ of a 'perfect' (plus 'pretty') C language program! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The 'procps.h' header includes all other headers which
contain exported (public) functions, while this header
has some (private) macros required for implementation.
If we mix the two, by including procps.h in this file,
we'll lose all hope of the compiler catching the kinds
of errors corrected in an immediately preceding patch.
[ we must be diligent, always constantly striving to ]
[ protect jimmy from ill effects due to copy & paste ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Needless to say, when one copies and then pastes, it's
always better when those results are refined somewhat.
The qsort callback declarations used the wrong results
type and 2 of their 3 input parameters were wrong too.
So this patch will fix that & generalize declarations.
[ plus add comment guidance about noop & extra enums ]
[ & provide for 'noop' sorting, as is done in <PIDS> ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Though all those callback's parameters are ignored and
qsort treats them as pointers to void, it's wrong when
3rd parm is 'enum pids_item', not 'struct sort_parms'.
So we will fix it in a way that lessens the likelihood
of another such a mistake when some new type is added.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit simply tries to keep naming plus formating
conventions on a par with the continuing climb up that
learning curve. These changes were suggested following
<slabinfo> sources upgrade from 2nd to 3rd generation.
Signed-off-by: Jim Warner <james.warner@comcast.net>
After normalizing the <SLABINFO> module, much of which
was based on this module, a structure which is crucial
to concurrent select/reap support is renamed the same.
Such a need for concurrent support was revealed during
slabtop testing. That program requests (select) global
data after calling for individual node data (reap) but
before those stacks have been accessed. Therefore, one
set of 'items'/'extents' could not possibly be shared.
So now we will know the former struct fetch_support as
ext_support. The former name was potentially confusing
since the struct was used by both 'select' and 'reap'.
However, only 'reap' ever called the 'fetch' function.
[ this <STAT> already accommodated concurrent usage. ]
[ in fact it contains 3 separate sets of items/exts. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Oops, the PROCPS_VMSTAT_logical_end enum reflected the
source state before all those DELTA enum's were added.
[ plus add a couple of overlooked #undef directives, ]
[ and whack an unneeded stacks_alloc assignment too! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch was prompted through work on the <slabinfo>
upgrade from 2nd gen to 3rd. And while this assignment
caused no real harm, it most certainly was misleading.
[ plus add a couple of overlooked #undef directives! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Before this major redesign, the slabs interface likely
was our messiest 2nd generation attempt at opaqueness.
Beyond the standard 'new', 'ref' & 'unref', there were
a total of 12 exported functions. Now, there are four.
The 1st step was to remove several of those functions.
These were quick to go since they were not used (yet):
. procps_slabnode_count
. procps_slabnode_getname
. procps_slabnode_getstack
Then, the following were internalized so users needn't
be burdened with implementation details in the future:
. procps_slabinfo_read (renamed: read_slabinfo_failed)
. procps_slabnode_stacks_alloc (renamed: stacks_alloc)
Still others evolved into the minimal interface we had
strived for in the other upgraded 3rd generation APIs:
. procps_slabnode_get -----------> procps_slabinfo_get
. separate stack_alloc/fill --> procps_slabinfo_select
. separate stacks_alloc/fill ---> procps_slabinfo_reap
. procps_slabnode_stacks_sort --> procps_slabinfo_sort
------------------------------------------------------
Beyond those reductions, the major modifications were:
. This API tries to be as forgiving as possible and as
such won't throw errors when a caller request makes no
sense. For example, if a 'get' or 'select' requested a
SLABNODE item (with no current means to id that node),
results will be zero. By the same token, should 'reap'
include a global SLABS item (meaning those values will
be duplicated in *every* node stack) it'll be allowed.
. If the above behavior is undesired, a new #define of
ENFORCE_LOGICAL can be used to restrict certain items.
. Permission problems will now be caught at 'new' time
thanks to a priming 'read' call. That read also serves
to make DELTA values potentially useful at 1st access.
. Separate slab/slabnode enumerators were consolidated
into one, simplifying validation & the results struct.
. Several internal parameter checks were relaxed since
they were already checked by the caller. Besides if we
cannot trust our own code we might as well hang it up.
. That sort provision was made more efficient and will
offer the ascending choice, in addition to descending.
------------------------------------------------------
Lastly, some additional thoughts regarding the future:
. It would not be difficult to expand 'select' to also
accept a nodeid, or to clone it as 'select_node'. And,
should the same be extended to 'get', a results struct
could be returned instead of signed long accommodating
the extra data type(s) like a node name (string data).
. The 'get' function is not currently affected by that
define ENFORCE_LOGICAL. However, at some future point
perhaps -EINVAL would be more appropriate than a zero.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Where possible, libprocps files convey the name of the
actual source pseudo file under the '/proc' directory.
This patch brings slab into line with such a standard.
Signed-off-by: Jim Warner <james.warner@comcast.net>
The original test did not NULL the unallocated info pointer. This
was incorrectly calling the procps_pids_new() function.
Signed-off-by: Craig Small <csmall@dropbear.xyz>
This interface represented a 2nd generation attempt at
the opaque newlib approach. In other words, it did not
involve the 1st generation 'chains'. Instead, 'stacks'
were employed. But the interface wasn't user friendly.
Users were required to create their own stacks, before
calling 'getstack' to retrieve multiple results with a
single call. Even worse, sometimes 'read' was required
before calling 'get' when working with single results.
So this commit represents the 3rd generation approach.
We eliminate the burden of 'read' and creating stacks.
Rather, beyond those standard 'new', 'ref' and 'unref'
functions, we'll offer just 'get' (single result) plus
a 'select' function (for multiple results in 1 stack).
And along the way, this commit vastly expands the data
extracted from /proc/vmstat. All values that currently
exist (and their delta equivalents) are now available.
Deltas were included for everything because there's no
real runtime costs beyond using a little extra memory.
The only problem is a lack of documentation for all of
those fields, as is reflected in the references below.
Oh well, maybe someday someone will dig through kernel
sources & finally plug that rather large document gap.
[ as an aside, rather than using a 'strcmp' approach ]
[ when parsing the /proc/vmstat file, as is found in ]
[ the <meminfo> module, we exploit those hash search ]
[ provisions that are found in the <search.h> header ]
Reference(s):
http://www.spinics.net/lists/linux-man/msg09096.htmlhttp://www.linuxinsight.com/proc_vmstat.html
Signed-off-by: Jim Warner <james.warner@comcast.net>
With an eye to the future, rather than managing a long
switch/case construct in the procps_stat_get function,
we'll adopt the approach used in the <meminfo> module.
By making 'get' processing table driven too, just like
'select' is already, that may ease future maintenance.
Along the way a rather large boo-boo was fixed dealing
with that /proc/stat 'procs_running' field which crept
in with the commit referenced below. It wasn't spelled
correctly and thus was never captured as PROC_RUNNING.
Reference(s):
commit abc71a46ad
Signed-off-by: Jim Warner <james.warner@comcast.net>
When this interface was normalized/standardized, under
the commit referenced below, the parameters were never
validated in the 'get' function. Let's plug that hole.
Reference(s):
commit 407f1b71de
Signed-off-by: Jim Warner <james.warner@comcast.net>
Update the pids tests for tweaked API. Unsure why the toomany
test doesn't work, may point to a problem in the change as it used
to trigger, or the library has flexibility.
In that reference below a specific systemd problem was
fixed in the commit shown. However lurking deep within
the <pids> interface was yet one final case where NULL
could be returned, involving 'strv' and the following:
. a user requested both a single string vector (always
returned as a normal string) & the vectorized version,
as with PROCPS_PIDS_CMDLINE and PROCPS_PIDS_CMDLINE_V.
. a user simply duplicated some vectorized enum items.
The root of that NULL problem is the fact those single
string vectors shared the same proc_t field with their
true vectorized version. So while multiple occurrences
for most strings could be satisfied with strdup versus
the normal ownership usurpation, those true vectorized
fields could not be quite so easily copied/duplicated.
Thus newlib chose to return a NULL result.strv pointer
under either of the above scenarios (which perhaps was
just a user boo-boo in the first place). In any event,
the NULL was a potential for true string vectors only.
Now, since newlib is the sole caller into the readproc
module, separate fields have been created for what are
just normal strings (never vectorized) and those which
remain the true vectorized versions. And, former flags
which only worked if combined, now act as stand alone.
Thus, both PROCPS_PIDS_CMDLINE & PROCPS_PIDS_CMDLINE_V
can be used simultaneously (as they should have been).
Also with this patch, items which a user duplicates in
the stack (beyond the first such item) will return the
the string "[ duplicate ENUM_ID ]". This practice will
apply to both single strings and true vectorized ones.
In addition to informing users of their error, it will
also mean potential NULLs need now never be a concern.
Reference(s);
http://www.freelists.org/post/procps/systemd-binary-vs-library
commit 0580a7b4c6
Signed-off-by: Jim Warner <james.warner@comcast.net>
Calls to free() have now been reintroduce in the new()
function to quiet coverity warnings. Those free's were
removed originally as that library 'new' was returning
with a fatal error and a caller should end abnormally.
Plus, it is virtually impossible to fail a malloc call
under linux. And lastly, they required braces with the
if statement making the code considerably less pretty.
[ commit also includes 2 unrelated whitespace tweaks ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
A potential conflict arises should should some program
attempt to access systemd data using our library built
without that support (--disable-systemd). And while we
could argue that all callers should check against NULL
string pointers, it's not expected of libprocps users.
So we'll guarantee 'em valid string consisting of "?".
Reference(s):
http://www.freelists.org/post/procps/systemd-binary-vs-library
Signed-off-by: Jim Warner <james.warner@comcast.net>
Rather than adopt an approach offered below, involving
changing all enumerator names to convey types, perhaps
this much more modest change is a worthwhile 1st step.
By just changing the comments to 'right justification'
it dramatically improves readability, at least it does
to my eye. Perhaps such an approach can postpone that
massive alternative effort for the foreseeable future.
Reference(s):
http://www.freelists.org/post/procps/newlib-drip-drip,4
Signed-off-by: Jim Warner <james.warner@comcast.net>
This is just beginning an effort to minimize/normalize
the sheer variety of results types in use for our API.
In taking these first baby steps, a few anomalies were
found. There'll no doubt be many more yet to discover.
. the _FLT_ (fault) fields were already signed long in
the proc_t (even though their sscanf format used %lu).
. although strtoul will alway return an unsigned long,
all of the _VM_ fields were made signed long just like
other memory fields (& signed sorts more efficiently).
Reference(s):
http://www.freelists.org/post/procps/newlib-drip-drip,4
Signed-off-by: Jim Warner <james.warner@comcast.net>
This represents a rather major interface redesign. The
following highlights most of the changes/enhancements.
. The 'read' interface (employed by pgrep & pidof) saw
the biggest change. The 'open', 'next' and 'shut' guys
all went bye-bye, replaced by a single 'get' function.
. The items specified at 'new' time no longer serve as
the maximum. In fact, items & numitems are now treated
as optional, should callers prefer to wait until later
when the 'reset' function would then become mandatory.
. Even at 'reset' time, the stacks are not tied to any
sort of maximum. They will grow dynamically as needed.
. The order of some parameters was changed to parallel
that found in our other APIs. Specifically, when items
& numitems are needed they're specified in that order.
. A user will no longer be prevented from concurrently
employing any accessor functions. In other words, that
'get' (old 'read') won't preclude 'reap' and 'select'.
. A duplicate enumerator was found dealing with locked
resident pages. So, the name VM_LOCK was eliminated in
favor of VM_RSS_LOCKED, which is way more descriptive.
. The struct address returned to callers following any
reap() or select() is now more sharable as pids_fetch.
. Some input parameter names were changed to make them
more descriptive of the intended purpose/requirements.
------------------------------------------------------
Internally, there were numerous implementation changes
made that did not directly impact any potential users.
. That #define FPRINT_STACKS was eliminated along with
the associated supporting function and its invocation.
. Addresses returned following 'reap' or 'select' will
now be NULL delimited, so one has the option of stacks
access via the total count or this new NULL fencepost.
. Input params were simplified and generalized in both
oldproc_open() & close() to enable more than 1 PROCTAB
to be open simultaneously, which was required for get.
. The PROCPS_PIDS_logical_end enum was relocated after
the Item_table making the need to keep it synchronized
more apparent (if the table expands it's right there).
. The 'Public function' section of the source file was
subdivided into 1) the three basic required functions;
and 2) functions that can sometimes vary between APIs.
Signed-off-by: Jim Warner <james.warner@comcast.net>
if (info->flags | PROC_UID)
Something OR a non-zero constant is always true.
Looks like it should be and'ed for the standard flag masking
pattern.
References:
Coverity #99118
Signed-off-by: Craig Small <csmall@dropbear.xyz>
A while loop used n >=0
A branch inside the loop had a test for n<0
That's never going to happen!
So the branch must go. There is similiar code outside
the branch. Yes, I miss pythons while else here.
Also cleaned up the ***-awful indentation this function had.
No wonder that branch lived for so long there.
References:
Coverity #99119
Signed-off-by: Craig Small <csmall@dropbear.xyz>
procps_slabnode_stacks_sort() would check if stacks was NULL
*after* the pointer was derefenced in the initialisation of
the variable p.
p is now assigned after the check.
References:
Coverity #99160
Because of the vast quantities of virtual memory which
may be allocated, it initially seemed like a good idea
to provide for a widest possible range through the use
of a 'ull_int' result type. However, on second thought
the implementation was a bit flawed for these reasons:
. that underlying meminfo_data variable 'VmallocTotal'
is 'unsigned long' not a required 'unsigned long long'
. there wasn't a convenient way to value it since each
variable was set with a strtoul() call, not strtoull()
So this patch will standardize on the 'ul_int' results
type (and reduce the associated delta to 's_int' too).
For now, we'll rely on protections under a 64-bit arch
where a 'ull_int' & 'ul_int' yield identical capacity.
Signed-off-by: Jim Warner <james.warner@comcast.net>
A recent buglet in the <meminfo> interface, where many
delta values could legitimately be negative, suggested
a review of this module where a minus was forced to 0.
As it turns out, there are two delta items that indeed
could be negative. They involve the current processes.
So henceforth, DELTA_PROC_BLOCKED & DELTA_PROC_RUNNING
will now be allowed to go negative. I believe that all
other items can only grow. But, if we find more later,
at least the adaptation approach has been established.
[ this whole business of checking for less than zero ]
[ harks back to an old kernel anomaly where negative ]
[ tics were sometimes experienced. top was affected. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
When the code for DELTA values (among other stuff) was
stolen from the recently revised <stat> interface, the
concept of ever growing values was propagated too. But
here we must manage both growing and shrinking values.
Thus former protections against a negative delta don't
have any place in this module and are hereby banished.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Thus function returns a pointer, not an int, so if there is
an error return NULL and not -errno.
proc/meminfo.c: In function ‘procps_meminfo_select’:
proc/meminfo.c:994:20: warning: return makes pointer from integer
without a cast [-Wint-conversion]
return -ENOMEM;
References:
commit 407f1b71de
This represents the refinement of this interface after
the <stat> API was redesigned. We now follow a pattern
of 'get' for single item retrieval & 'select' for when
multiple items are desired, with just 1 function call.
And again following the <stat> lead this interface now
provides for delta values encompassing most items. The
reason I went cuckoo nuts with those deltas is because
they are essentially free. At the cost of a little RAM
and just one memcpy there's no other price to be paid.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In the commit referenced below it was acknowledged the
procps_stat_get() function will intentionally restrict
returned values by excluding DELTA items among others.
Upon reflection that decision seemed rather arbitrary,
and so has now been lifted. And, while the initial use
of a DELTA will return 0, subsequent calls to that get
function will now return the difference (providing one
second, at least, has elapsed between any such calls).
Reference(s):
. reference to design decisions
commit abc71a46ad
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit is just my usual tweaking after an initial
submission, once the dust has settled & some more desk
checking was performed. Here are the changes included:
. get function need not check the 'stat_was_read' flag
( first time in save_sec == 0 so branch always taken )
. some table logic appeared outside the proper section
. cleanup_stack's loop made to work like assign_result
. eliminated an include of procps.h from stat.h header
. removed 'ext_numitems' from the stacks_extent struct
. changed the alignment of three prototypes for stat.h
. reorganized the pointers for 'stacks_extents' struct
. improved types indentation in header for readability
. lastly, some really gratuitous changes made to align
- comments within that private 'stacks_alloc' function
Signed-off-by: Jim Warner <james.warner@comcast.net
This commit represents a complete redesign of the stat
interface. Gone are the confusing 8 separate accessors
along with their 2 additional read functions. In their
place we have just 3 accessors, with no read required.
That old interface also suffered an inflexibility with
respect to structures. Now we deal with an unchanging
standard 'result' struct enabling future changes where
the binary interface will no longer need to be broken.
And gone is that former unnecessary typedef, used when
dealing with jiffies. Now the standard C type is used.
Our new API also adds some brand new functionality. If
a caller plans to employ successive 'select' or 'reap'
invocations, then delta values are available (which is
actually only what that top program is interested in).
At some future point a 'sort' function could be easily
introduced to complement the 'reap' function. However,
I saw no need for it at present and so it was omitted.
There were several design decisions which everyone may
not agree with. In support I'll offer these rationals:
. The 'get' function returns a signed long long result
which means a potential loss of some significance. But
I felt the ability to distinguish actual errors (minus
values) from true zero results were worth such a risk.
. The DELTA item enumerators were also made signed and
smaller than their parents. And they are intentionally
grouped as last so as to emphasize those distinctions.
. The SYS type items were excluded from the new 'reap'
function. It would not make sense to duplicate them in
each results stack. They're limited to 'get'/'select'.
. By the same token, some items (DELTA, etc.) will not
be allowed under that 'get' routine. That function was
already open to significant internal overhead (through
subsequent calls like in vmstat.c). That is why it has
been limited via 1 second between reads of /proc/stat.
Lastly, when we finally get around to documenting this
interface there's a real potential toe stubber when it
comes to the numa node portion. The libnuma.so doesn't
really provide any means to retrieve the active nodes.
Thus, any total reported by <stat> is just the highest
node number plus one, as reported by the numa library.
Any unused/inactive nodes are identified through these
. PROCPS_STAT_TIC_ID shows as PROCPS_STAT_NODE_INVALID
By the same token after the STAT_REAP_CPUS_ONLY 'reap'
. PROCPS_STAT_TIC_NUMA_NODE = PROCPS_STAT_NODE_INVALID
Reference(s):
http://www.freelists.org/post/procps/newlib-stat-interface
Signed-off-by: Jim Warner <james.warner@comcast.net>
During work on the new <stat> interface the typedef of
jiffs was eliminated along with the clang warning. The
exact same warning in sysinfo.h now goes bye-bye also.
Reference(s):
./proc/sysinfo.h:16:28: warning: redefinition of typedef 'jiff' is a C11 feature [-Wtypedef-redefinition]
typedef unsigned long long jiff;
Signed-off-by: Jim Warner <james.warner@comcast.net>
The previous commit did not take into account operator evaluation
order so always gave 0. Fixed the macro and added some checks for
the macro.
References:
commit 9abf7d879d
Linux release numbers are x.y.z we assumed but never protected
against x> 0x7fff and y,z > 0xff before.
Linux release in 1991, 1 major release per 6 years so we're fine
until 7452, unless they do way too many minor rels (max being 39)
procps_uptime, previously just plain uptime() used to put the
uptime (as a double) in the first argument and return uptime
(as an int).
It meant if you ran
myuptime2 = procps_uptime(&myuptime1, NULL);
You might get different results for myuptime1 and myuptime2 because
they are different types.
Most library calls use the return value to return the status,
procps_uptime was in the middle.
Until now.
This function will return 0 on success. If you want (for whatever
reason) uptime as an int then cast it.
All of the procps binaries didn't use the return value for uptime
except ps which set a variable to it but never used it anywhere.
While procps_diskstat_ref and procps_diskstat_unref were declared
they actually weren't implemented in the library. Linking some
binary that used these functions gave the following errors:
CCLD vmstat
vmstat.o: In function `disksum_format':
vmstat.c:719: undefined reference to `procps_diskstat_unref'
vmstat.o: In function `diskformat':
vmstat.c:591: undefined reference to `procps_diskstat_unref'
In an attempt to normalize the 'stat' interface, we'll
first shed any reminders of the old readproc interface
by changing file names to be more descriptive & brief.
Signed-off-by: Jim Warner <james.warner@comcast.net>
After the commit referenced below the potential exists
for a SEGV (resulting from an out-of-bounds Item_table
reference if PROCPS_PIDS_physical_end is encountered).
So this patch eliminates that PROCPS_PIDS_physical_end
as no longer necessary and completes the task of using
PROCPS_PIDS_logical_end as a sole necessary fencepost.
Reference(s):
commit e7585992d9
Without breaking either ABI or API just rearrange some
stuff to provide a little more consistent source file.
[ these were prompted by some work on the <stat> API ]
[ in preparation for him borrowing some of our code. ]
Reference(s):
Signed-off-by: Jim Warner <james.warner@comcast.net>
Generalised the library API tests and created a common
test-runner. Instead of copy and pasting the same code in
each time.
First cut of pids API test, for _new so far.
Remove a remnant of this new API's evolution. Here we
no longer fill stacks, rather we reap and select them.
Signed-off-by: Jim Warner <james.warner@comcast.net>
By co-mingling both external/internal identifiers with
actual implementation code, potential future additions
to our API would have been considerably more difficult.
So, this patch will now rely solely on internal/hidden
identifiers serving as fenceposts in validation logic.
And if the following convention is used for new fields
we will maintain that enumerator alphabetic ordering a
a little longer (even though 2 user fields now don't):
. PROCPS_PIDS_XTRA_FOO_A, PROCPS_PIDS_XTRA_FOO_B, etc.
Reference(s):
http://www.freelists.org/post/procps/me-too-newlib,7
Signed-off-by: Jim Warner <james.warner@comcast.net>
Unfortunately, reading that '/proc/sys/kernel/pid_max'
file returns a newline, which we will now account for.
[ also, we should use the existing dedicated buffer ]
Reference(s):
. original 'procps_pid_length' introduction
commit ccb6ae8de1
Signed-off-by: Jim Warner <james.warner@comcast.net>
escaped_copy(): only appears in ps, moved to ps/output.c
escape_strlist() only used in escape.c made static
escape_command() used in library, made internal
procps.h no longer includes escape.h
escape_str() used by library and ps so needs to be exported
definition put into procps.h including the odd define required.
Far from ideal to have it this way, will look at it another time
to have it all in, all out or split nicer so its not in the API;
perhaps a lib/ file?
readproc still had some of the old API hanging around that
was good while we were trying it out, but its time to say
goodbye:
readproc.h removed from public procps.h header file
enum ns_type - not used - removed
get_ns_name() - not defined - removed
get_ns_id() - not defined - removed
PROCTAB proc_t only used internal to library
readproctab() and 2,3 friends not used - removed
proc_data_t was used by readproctab23() - removed
readtask() - not used - removed
read_cmdline() - copy,renamed and made generic - remove original
freeproc() - not used - removed
get_proc_status - not used - removed, however there should be
a new function created that does this. Given a PID return data
about it instead of scanning the entire procfs. Maybe it already does.
Left as internal-to-library only functions:
readproc(), readeither(), look_up_our_self(), openproc(), closeproc()
Updated libprocps.sym to export only what we use.
Now that the conditional OOMEM_ENABLE has been removed
and, after reviewing current library support, it turns
out we've been using a deprecated /proc/<pid>/oom_adj.
What we should use instead is that more recent tunable
/proc/<pid>/oom_score_adj. This new field will provide
a range of -1000 thru +1000 (former was -17 thru +15).
Reference(s):
. removal of misguided OOMEM_ENABLE
commit 64238730fa
. linux, partial oom_adj revert (Nov, 2012)
commit fa0cbbf145aabbf29c6f28f8a11935c0b0fd86fc
. linux, removal oom_adj (Oct, 2012)
commit 01dc52ebdf472f77cca623ca693ca24cfc0f1bbe
Signed-off-by: Jim Warner <james.warner@comcast.net>
The includes used to define a lot of things a library include
should not. It was also a bit messy what was exposed in the library
and what was not.
get_pid_digits -> procps_pid_length and exported correctly
MALLOC attribute move into relevant .c files
NORETURN attribute moved to relevant .c, not used in library
PURE attribute removed, it wasn't used
KLONG/KLF/STRTOUKL were fixed for long, so now just use long
HIDDEN attribute removed. It was for 3 functions. The PROCPS_EXPORT
seems to do the same (opposite) thing.
likely/unlikely removed from most places, its highly debateable
this does anything useful as CPUs have gotten smarter about branches.
Re-arranged the includes, ALL external programs should just #include
<proc/procps.h> then proc/procps.h includes headers for files that
have exported functions. procps.h and the headers it includes should
not use items that are not exportable (e.g. hidden functions or
macros) they go in procps-private.h
Multiple scanf()s use the GNU-permitted %Lu. This is not supported in
other libraries and isn't to the POSIX specification. The L modifier
is only used for floats in POSIX.
Replacing %Lu with %llu is the same for GNU libc (scanf(3) says as much)
but means other libraries will work fine.
From master commit da715e3
References:
http://pubs.opengroup.org/onlinepubs/009695399/functions/fscanf.html
Beginning with linux-4.5, the following new fields are
being added under that /proc/<pid>/status pseudo file:
. RssAnon - size of resident anonymous memory
. RssFile - size of resident file mappings
. RssShmem - size of resident shared memory
This patch just represents the initial library and top
support, sharing a commit message with 2 more patches.
p.s. locked resident memory support was also added but
isn't directly related to the kernel 4.5 enhancements.
Reference(s):
commit 1f8e41d019
Signed-off-by: Jim Warner <james.warner@comcast.net>
Summarized below, miscellany addressed in this commit:
. deleted extraneous newline(s) for consistent spacing
Signed-off-by: Jim Warner <james.warner@comcast.net>
When reference counts were added to some string fields
the 3 true string vector fields were not duplicated as
were those other fields. Instead they were supposed to
disallow a duplicate stack reference beyond the first.
However, the actual implementation gave NULL for every
true vector field whenever such items were duplicated.
More importantly, such true string vector fields never
considered references to the shared proc_t source root
which would have forced the conversion of such vectors
into a single string form via the '_CVT' library flag.
So this commit restores the intended outcome with true
string vectors. There's only 1 valid reference allowed
and duplicates and converted fields will yield a NULL.
Signed-off-by: Jim Warner <james.warner@comcast.net>
While not changing generated code this commit corrects
one free reference from 'str' to a more proper 'strv'.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Ok, here is that rather major internal redesign hinted
at in the three previous commits. Its need was quickly
revealed after adapting top then attempting to display
newly added 'CGNAME' fields and an existing 'CGROUPS'.
That very quickly generated a SEGV. And the reason was
just as quickly recognized. Both fields relied on that
proc_t.cgroup member yet whichever result structure is
first in a stack is the one which assumes ownership of
of the vectored sting by resetting its cgroup to NULL.
So this commit introduces reference counting for a few
of the fields in the proc_t. Specifically there are 17
entries in the Item_table dealing with strings/vectors
where ownership is transferred to newlib. Now whenever
such fields are represented more than once in a stack,
the strings will be duplicated instead of transferred.
In this way we can generally remain optimized avoiding
string copies, yet still accommodate them when needed.
There's an exception to this scheme: those true string
vectors (CGROUP_V, CMDLINE_V and ENVIRON_V). When such
fields are duplicated in a stack the result structures
beyond the first will be set to NULL, which the caller
will (should) already be equipped to deal with anyway.
Signed-off-by: Jim Warner <james.warner@comcast.net>
The ps program was modified to print the control group
names, based on the library provided list of all those
control groups to which a process belongs. But this is
probably something the newlib should be doing for all.
So this commit borrows the ps approach to cg names and
thus will make that available to all future consumers.
[ but stay tuned! there is a commit coming soon that ]
[ represents a rather major internal redesign, which ]
[ was prompted by the ps and top adaptation testing. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The above function was the sole public function in the
<pids> interface to use the word 'stacks' in its name.
All of the others dealt exclusively with their duties,
So this commit normalizes that outlier by renaming it.
Signed-off-by: Jim Warner <james.warner@comcast.net>
The above function had been disabled via '#if 0' so as
to prevent a compiler warning. But it really should be
called by that 'procps_pids_read_shut' function rather
than it duplicating/reinventing the same logic itself.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Gosh, just because nobody uses some newlib provision I
guess, since it is being offered, it ought to actually
be tested at some point. Well, that point just arrived
and guess what? A surprise: some bugs were discovered.
The procps_pids_select function established a for loop
wherein readproc is called until the passed 'maxthese'
limit. Unfortunately this was incorrect for 2 reasons:
1. For PROCPS_FILL_PID results are limited by what the
oldlib finds, having established the pid list at open.
Total found already cannot exceed a passed 'maxthese';
2. With PROCPS_FILL_UID, returned results could exceed
a 'maxthese' thus making the for loop incorrect again.
[ plus yours truly neglected to forward the required ]
[ UIDs total to our old library, another oops biggie ]
In summary: the loop should have been forever, exiting
only when all those identified procs had been located.
So, while addressing those bugs, I've consolidated all
the retrieval code (initialize, iterate, summarize) in
a single helper function which will now serve both the
procps_pids_reap and select functions. And as a result
those guys were reduced to quite trivial housekeeping.
This patch, hopefully, completes the normalization for
reap/select (fill), which began with references shown.
Reference(s):
commit 0c953eccc5
commit 747dfc5987
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch contains the following minor modifications:
. When assigning to boot_seconds from a double, a cast
to 'unsigned long' was employed that in reality should
have been 'unsigned long long'. But, on second thought
it's probably best if the compiler makes the decision.
. Results for ID_TGID do not require an 'f_status' flg
since both tid and tgid will be available by virtue of
the 'simple_nextpid' function normal operations. Thus,
that flag has been removed from the <pids> Item_table.
. When the pids_stacks structure was eliminated with a
change to the alloc/dealloc functions in the reference
below, several casts became redundant and are removed.
Reference(s):
commit 747dfc5987.
Signed-off-by: Jim Warner <james.warner@comcast.net>
After simplifying that select/fill interface, there is
no longer a need for public 'alloc' & 'dealloc' stacks
functions. There is now only one instance of stacks as
an input parameter found in procps_pids_stacks_sort().
But sorting 'empty' stacks serves no possible purpose.
So this commit retains both functions, since they will
still be needed, but designates them private (static).
Additionally, with their demise we will eliminate that
pids_stacks structure from the header file, internally
using what always was the true 'stacks_extent' struct.
Signed-off-by: Jim Warner <james.warner@comcast.net>
After wrestling with the conversion of both top and ps
the differences between reap (all) & fill (select) has
become increasingly inconvenient. So this patch simply
normalizes that API making returned results identical.
The former procps_pids_stacks_fill identifier will now
be known as procps_pids_select which serves as logical
counterpart to the existing procps_pids_reap function.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Shoot, here's yet another bow to ps needs. But it's ok
because it makes a lot of sense. Rather than force all
users into their own calculations do but it once here.
As an aside this need arose during ps testing when the
sorts were using TIME_START or TICS_ALL. That was just
fine for almost every need except 'etime' plus 'time'.
That 'etime' was sorting the opposite of what's wanted
when using TIME_START (of course) while 'time' yielded
some weird ordering because TICS_ALL was too granular.
Signed-off-by: Jim Warner <james.warner@comcast.net>
While not documented in the man page, ps allows 'tty4'
as a valid output specifier complimenting 'tty8' & its
derivatives. So, in order to eliminate a dev_to_name()
call in the ps program the library will now offer this
abbreviated tty version (consisting of a number only).
Signed-off-by: Jim Warner <james.warner@comcast.net>
The 'PROCPS_PIDS_extra' enumerator already enjoys some
special place wherein it's zeroed with each iteration.
This patch simply extends the special handling to also
include support for sorting. It will be treated as the
'ull_int' data type, since that encompasses the entire
scope of that union within all pids_result structures.
[ plus, we've also corrected the actual 'extra' name ]
This change was prompted by the conversion of ps which
needs that enumerator to store the former 'pcpu' data,
so it will be available for sorting (not for display).
Signed-off-by: Jim Warner <james.warner@comcast.net>
The way that the passed sort order was validated would
allow the invalid 0 to fall between the sofa cushions.
So this patch will simply close that former oversight.
Signed-off-by: Jim Warner <james.warner@comcast.net>
As part of the push to remove the old API, 3 more old calls were
deleted from the library. Now all someone needs to do is fix ps
and we're largely done.
The values of PROCPS_SORT_ASCEND & PROCPS_SORT_DESCEND
were a tad unintuitive. This patch will just make them
a more natural +1 for ascending and -1 for descending.
[ plus it still allows that fast path multiplication ]
[ instead of a comparison for signed numbers/strings ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
After peeking at possible conversion of the ps program
it appeared that we may ultimately need the concept of
a 'static' pids_stack in suport of look_up_our_self().
In other words, one not impacted by procps_pids_reset.
This patch simply sets the stage for that possibility.
Signed-off-by: Jim Warner <james.warner@comcast.net>
With this patch the distinction between a 'long' KLONG
and a 'long long' KLONG is being abandoned in favor of
a consistent declaration as 'long' only. Plus we would
have also defined it as 'unsigned' except there exists
much code already explicitly specifying the qualifier.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Work on converting ps has revealed the desirability of
trading a void pointer for that ul_int type. There was
much arithmetic employed against such values and casts
would otherwise have been required. Even pmap needed a
cast on occasions when comparing an internal variable.
Besides, there is much to be said for reducing demands
on (and the complexity of) the result structure union.
[ we choose ul_int over ull_int since that former is ]
[ the exact same size and capacity as a void pointer ]
[ regardless of whether compiled as 32-bit or 64-bit ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Not sure how this one has gone unnoticed until now but
with valgrind's help it's going bye-bye lickety-split.
Reference(s):
==26533== Conditional jump or move depends on uninitialised value(s)
==26533== at 0x4E4082B: procps_meminfo_stack_fill (meminfo.c:408)
Signed-off-by: Jim Warner <james.warner@comcast.net>
[ but most definitely not all of them by a long shot ]
Reference(s):
proc/diskstat.c:186:17: warning: unused variable 'is_disk' [-Wunused-variable]
int retval, is_disk;
^
proc/namespace.c:110:1: warning: control may reach end of non-void function [-Wreturn-type]
}
^
proc/readproc.c:1131:50: warning: address of array 'ent->d_name' will always evaluate to 'true' [-Wpointer-bo
if(unlikely(unlikely(!ent) || unlikely(!ent->d_name))) return 0;
~~~~~~^~~~~~
proc/readproc.c:1158:50: warning: address of array 'ent->d_name' will always evaluate to 'true' [-Wpointer-bo
if(unlikely(unlikely(!ent) || unlikely(!ent->d_name))) return 0;
~~~~~~^~~~~~
proc/sysinfo.c:45:12: warning: unused variable 'stat_fd' [-Wunused-variable]
static int stat_fd = -1;
^
proc/sysinfo.c:49:12: warning: unused variable 'meminfo_fd' [-Wunused-variable]
static int meminfo_fd = -1;
^
proc/sysinfo.c:51:12: warning: unused variable 'vminfo_fd' [-Wunused-variable]
static int vminfo_fd = -1;
^
proc/sysinfo.c:53:12: warning: unused variable 'vm_min_free_fd' [-Wunused-variable]
static int vm_min_free_fd = -1;
^
proc/uptime.c:157:12: warning: unused variable 'realseconds' [-Wunused-variable]
time_t realseconds;
^
proc/uptime.c:158:16: warning: unused variable 'realtime' [-Wunused-variable]
struct tm *realtime;
^
vmstat.c:574:20: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat]
DSTAT(PROCPS_DISKSTAT_READ_TIME),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vmstat.c:578:20: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat]
DSTAT(PROCPS_DISKSTAT_WRITE_TIME),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
watch.c:230:7: warning: variable 'endptr' is uninitialized when used here [-Wuninitialized]
if (*endptr == '\0') set_ansi_attribute(0); /* [m treated as [0m */
^~~~~~
Signed-off-by: Jim Warner <james.warner@comcast.net>
OK, ok, this was kind of a huge omission. So please do
not select the TTY field for display in top quite yet,
at least until a next patch has been pushed to GitLab.
And to produce a correct sort order for this new field
the GNU 'strverscmp' routine was a necessary addition.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Uptime output for both w and uptime command were showing no
comma or space after days.
$ ./uptime
18:32:21 up 22 days7 min, 6 users, load average: 0.23, 0.46, 0.64
Minor tweak to fix this.
This commit is an outgrowth of the research into a bug
that recently surfaced with the 'w' program. And while
that program was just a victim several inconsistencies
were found in the handling of library flags during the
research. This patch just address such irregularities.
Reference(s):
http://www.freelists.org/post/procps/newlib-at-the-precipice,4
Signed-off-by: Jim Warner <james.warner@comcast.net>
I can't remember during what sleepy hour that enum was
added to pids.h, but it damn sure fell short of a full
implementation by approximately 1,000,000 miles or so.
Strangely it surfaced on Craig's system seemingly ggdb
related whereas on mine, a segmentation fault was only
seen when that -h (no header) argument was being used.
Oh well, the road to its resolution also offered us an
opportunity to cleanup some other items along the way.
Reference(s):
http://www.freelists.org/post/procps/newlib-at-the-precipice,4
Signed-off-by: Jim Warner <james.warner@comcast.net>
The former whattime logic used to suppress that 'days'
output when the system had been up less than 24 hours.
But since the refactor for newlib, '0 days' was always
output under those conditions. So this commit restores
the former expected behavior of suppressing that item.
[ plus an erroneous calculation of uphours was fixed ]
[ and the clang warnings shown below were also fixed ]
Reference(s):
proc/uptime.c:74:10: warning: unused variable 'buf' [-Wunused-variable]
char buf[256];
^
proc/uptime.c:131:58: warning: data argument not used by format string [-Wformat-extra-args]
pos += sprintf(upbuf + pos, "%d min, ", uphours, upminutes);
~~~~~~~~~~ ^
proc/uptime.c:175:15: warning: expression result unused [-Wunused-value]
comma +1;
~~~~~ ^~
Signed-off-by: Jim Warner <james.warner@comcast.net>
The presence of that PROCPS_PIDS_noop may yet see some
use in the future with its 'no alter' library promise.
However, when top used that item to reflect the forest
view nesting level, the unchanging nature of that item
became more of an inconvenience than benefit. For each
refresh top was forced to loop through all the stacks,
resetting that PROCPS_PIDS_noop result struct to zero.
So this commit will now offer users a choice between a
new re-initialized item (PROCPS_PIDS_extra) & the noop
invariant. Since the library already resets all those
result structures, top will now utilize it at no cost.
Signed-off-by: Jim Warner <james.warner@comcast.net>
A patch containing the following miscellaneous tweaks:
. make a supposedly robust parameter test truly robust
[ ensure the largest enum value used with validation ]
. remove duplicate item test in cleanup_stack function
[ is already subordinate to test of PROCPS_PIDS_noop ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Functions related to namespaces were half-in half-out of the
procps library and didn't fit the standard naming scheme.
While struct { long ns[x]} is a bit clunky, its the only way
to "lock in" x. The alternative is to use ns_* variables.
This work was needed before pgrep could be converted.
I was surprised to find that ol' gcc silently converts
a single (different) enum into an address where one or
more enums were expected to be dereferenced. Of course
this was just yet another way to generate an old SEGV.
So this commit will strengthen those parameter checks.
[ we will *not* blame Craig for a failure to consult ]
[ the documentation, since it doesn't even exist yet ]
Reference(s):
http://www.freelists.org/post/procps/newlib-ps-fix,8
Signed-off-by: Jim Warner <james.warner@comcast.net>
A patch containing the following miscellaneous tweaks:
. avoided distortions unique to PROCPS_PIDS_TICS_DELTA
. addressed several smatch warnings and/or suggestions
. ensured oldproc_close invoked should tally_proc fail
. keeping that namespace clean, added a missing #undef
. added 2 comments acknowledging pids_item as unsigned
. added/clarified comments regarding proc flags & strv
From smatch analysis:
. these were indeed boo-boos
pids.c:580 make_hist() warn: the 'Hr' macro might need parens
pids.c:1058 procps_pids_reap() warn: variable dereferenced before check 'info' (see line 1056)
. these were not errors (and we did double check)
pids.c:1067 procps_pids_reap() warn: double check that we're allocating correct size: 8 vs 128
pids.c:1068 procps_pids_reap() warn: double check that we're allocating correct size: 8 vs 128
Signed-off-by: Jim Warner <james.warner@comcast.net>
As an experiment a helper macro used to extract values
from a result stack has been added to the header file.
Don't force callers to reinvent that particular wheel.
Signed-off-by: Jim Warner <james.warner@comcast.net>
After experimenting with an adaptation of pidof to the
new pids interface, it became apparent that vectorized
versions of those command lines would be necessary. So
this commit adds that option and the strv result type.
And since the stage had been set, a vectorized version
of PROCPS_PIDS_ENVIRON & PROCPS_PIDS_CGROUP was added.
Lastly, any use of 'const' in the result structure was
removed so callers need not be bothered with casts and
compiler warnings. Hopefully, they'll respect a stack.
Signed-off-by: Jim Warner <james.warner@comcast.net>
To ease the transition to the new interface, for other
than that top program, individual read provisions have
been added to the <proc/pids.h> API. This represents a
refinement of a position stated in a post noted below.
Reference(s):
http://www.freelists.org/post/procps/newlib-ps-fix
Signed-off-by: Jim Warner <james.warner@comcast.net>
In my zeal to finalize the initial pids implementation
I omitted some quite important parameter checking from
the above function. Thank goodness top was kind to us.
Also, in anticipation of the additions of single stack
read and supporting functions some items were renamed.
Signed-off-by: Jim Warner <james.warner@comcast.net>
The patch below is where most proc_t fixed size arrays
became simple pointers to char. In that commit changes
to the above function were made so that dynamic memory
was freed which included the program name (cmd) field.
That change was prompted by a valgrind reported memory
leak during development that no longer seems to exist.
However, by keeping the look_up_our_self() changes the
ps command without args then fails to report anything.
So this patch just restores the expected old behavior.
Reference(s):
commit 3881a0844a
Signed-off-by: Jim Warner <james.warner@comcast.net>
It was probably always wrong to have a variable length
proc_t structure. This patch takes all remaining oomem
former suse only options and makes them unconditional.
Signed-off-by: Jim Warner <james.warner@comcast.net>
. traded a complex misaligned memory allocation scheme
in the make_hist function for a simple aligned scheme.
plus memory allocation increases are globally defined.
. changed 1 parameter for procps_pids_stacks_sort() to
better reflect the 'array of pointers', not an address
of a pointer as is used with guys such as 'new/unref'.
. the pids_reap struct was changed slightly to make it
more reflective of it's actual implementation details.
. the Item_table member .mustfree is now .needfree and
that .makehist was now made .needhist for consistency.
. reduced the number of separate 'return NULL;' source
statements in that primary procps_pids_reap() routine.
. ensured consistent reference to sizeof(void *) & not
occasional reference to sizeof(void*) without a space.
. rather than enable/disable validate_stacks via a #if
in the function body, it is now handled via a #define.
. some comments in the procps_pids_reset function were
adjusted to reflect this current implementation. shown
originally, they reflected an aborted attempt to avoid
a testing aberration not fully understood at the time.
. added a summary of the memory overhead cost of HST_t
processing to that UNREF_RPTHASH output at unref time.
. a 'PIDs at max depth:' portion of that UNREF_RPTHASH
enabled #define is now published only when the maximum
depth of hash table entry chains exceed depths of one.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit is the culmination of efforts to modernize
the library api. It should be treated as a first blush
attempt, especially since I have absolutely no library
design experience. But I did have a very strong desire
to lessen the new library's impact on the top program.
Under this new api, a 'stack' is the equivalent of the
old proc_t. It can be seen as a variable length record
whose contents & order is under complete user control.
That initial stack/record configuration is established
at procps_pids_new() time and will probably serve most
program needs. But, a dynamic & demanding program like
top will later change a stack via procps_pids_reset().
For programs like top & ps, procps_pids_reap() will be
the function that will retrieve all tasks and threads.
Any program that needs to filter / select only certain
processes or users have available other functions that
can be used: procps_pids_stacks_alloc, fill & dealloc.
This implementation attempts to maximize that existing
proven libprocps code base. As we gain more experience
such actual code can be migrated into the pids.c file.
Signed-off-by: Jim Warner <james.warner@comcast.net>
A few minor changes are being made to position the old
readproc logic for a transition to the newlib pid api.
These changes will not impact current users beyond the
the need to recompile such code. Hopefully this should
be very last version change to the deprecated library.
. most char arrays were replaced via char * to dynamic
memory. this was done so that newlib could just assume
ownership of such strings without using a strdup call.
. former user and group name arrays also became char *
but here the reason was because pwcache already cached
those names. so, copying to an array never made sense.
. the concept of QUICK_THREADS used to avoid duplicate
overhead for string data was disabled. it could not be
integrated with the newlib design, at least initially.
. any #define which influenced the size of that proc_t
was disable in the header. it was probably a poor idea
to approach optional features in such a manner anyway.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit just corrects the oversight wherein 'item'
was being employed when 'these' was actually intended.
Also, it trades some 'item' use for a more descriptive
input parameter which henceforth is known as a 'dest'.
And, there was one leftover 'next' pointer eliminated.
Finally, some logic was made a tad less dependent upon
enumerator names and a few comments were also updated.
Signed-off-by: Jim Warner <james.warner@comcast.net>
With the new perspective on potential uses of a 'noop'
enumerator (or whatever we decide to call it) there is
no longer a need to provide for any extra 'user' space
in the stack header structures used by slab & meminfo.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In addition to that text shown below the line which is
common to several commit messages, this patch contains
several minor changes with lessor impact upon the API:
. Standard copyright boilerplate was added in .c file.
. The #include header files are ordered alphabetically
now, with all those <sys/??> types separately grouped.
. The header file follows the conventions of indenting
(by 4 spaces) those parameters too lengthy for 1 line.
------------------------------------------------------
. The former 'chains' have now become 'stacks' without
the 'next' pointer in each result struct. The pointers
initially seemed to offer some flexibility with memory
allocations and benefits for the library access logic.
However, user access was always via displacement and a
a statically allocated chain was cumbersome to define.
. An enumerator ending in '_noop' will no longer serve
as a fencepost delimiter. Rather, it has become a much
more important and flexible user oriented tool. Adding
one or more such 'items' in any items list passed into
the library becomes the means of extending the 'stack'
to also include user (not just library) data. Any such
data is guaranteed to never be altered by the library.
. Anticipating PID support, where many different types
must be represented in a result structure, we'll adopt
a common naming standard. And, while not every results
structure currently needs to reflect disparate types a
union will be employed so the same dot qualifier ('.')
can be used consistently when accessing all such data.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In addition to that text shown below the line which is
common to several commit messages, this patch contains
several minor changes with lessor impact upon the API:
. A call to procps_stat_read_jiffs() has been added to
those jiffs functions carrying the 'fill' nomenclature
to parallel like functions in some of our other files.
. The #include header files are ordered alphabetically
now, with all those <sys/??> types separately grouped.
. Standard copyright boilerplate was added in .c file.
. The header file follows the conventions of indenting
(by 4 spaces) those parameters too lengthy for 1 line.
------------------------------------------------------
. The former 'chains' have now become 'stacks' without
the 'next' pointer in each result struct. The pointers
initially seemed to offer some flexibility with memory
allocations and benefits for the library access logic.
However, user access was always via displacement and a
a statically allocated chain was cumbersome to define.
. An enumerator ending in '_noop' will no longer serve
as a fencepost delimiter. Rather, it has become a much
more important and flexible user oriented tool. Adding
one or more such 'items' in any items list passed into
the library becomes the means of extending the 'stack'
to also include user (not just library) data. Any such
data is guaranteed to never be altered by the library.
. Anticipating PID support, where many different types
must be represented in a result structure, we'll adopt
a common naming standard. And, while not every results
structure currently needs to reflect disparate types a
union will be employed so the same dot qualifier ('.')
can be used consistently when accessing all such data.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In addition to that text shown below the line which is
common to several commit messages, this patch contains
several minor changes with lessor impact upon the API:
. A 'read' was added to function procps_slabnode_count
(but only when necessary, i.e. info->nodes_used == 0).
. The #include header files are ordered alphabetically
now, with all those <sys/??> types separately grouped.
------------------------------------------------------
. The former 'chains' have now become 'stacks' without
the 'next' pointer in each result struct. The pointers
initially seemed to offer some flexibility with memory
allocations and benefits for the library access logic.
However, user access was always via displacement and a
a statically allocated chain was cumbersome to define.
. An enumerator ending in '_noop' will no longer serve
as a fencepost delimiter. Rather, it has become a much
more important and flexible user oriented tool. Adding
one or more such 'items' in any items list passed into
the library becomes the means of extending the 'stack'
to also include user (not just library) data. Any such
data is guaranteed to never be altered by the library.
. Anticipating PID support, where many different types
must be represented in a result structure, we'll adopt
a common naming standard. And, while not every results
structure currently needs to reflect disparate types a
union will be employed so the same dot qualifier ('.')
can be used consistently when accessing all such data.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In addition to that text shown below the line which is
common to several commit messages, this patch contains
the following additional change without an API impact:
. The #include header files are ordered alphabetically
now, with all those <sys/??> types separately grouped.
------------------------------------------------------
. The former 'chains' have now become 'stacks' without
the 'next' pointer in each result struct. The pointers
initially seemed to offer some flexibility with memory
allocations and benefits for the library access logic.
However, user access was always via displacement and a
a statically allocated chain was cumbersome to define.
. An enumerator ending in '_noop' will no longer serve
as a fencepost delimiter. Rather, it has become a much
more important and flexible user oriented tool. Adding
one or more such 'items' in any items list passed into
the library becomes the means of extending the 'stack'
to also include user (not just library) data. Any such
data is guaranteed to never be altered by the library.
. Anticipating PID support, where many different types
must be represented in a result structure, we'll adopt
a common naming standard. And, while not every results
structure currently needs to reflect disparate types a
union will be employed so the same dot qualifier ('.')
can be used consistently when accessing all such data.
Signed-off-by: Jim Warner <james.warner@comcast.net>
While meminfo does implement only the singular version
of 'xxxx_chain_fill', it was wrong to limit the 'read'
to the plural version (only) within our new slabs API.
This kind of inconsistency will only spell trouble for
future users of the new libprocps API. So, this commit
will mean that any form of xxxx_chain(s)_fill function
also includes a 'read', whereas xxx_getchain does not.
Reference(s):
commit aab537bc13http://www.freelists.org/post/procps/newlib-interfaces
Signed-off-by: Jim Warner <james.warner@comcast.net>
With a little help from smatch, this commit eliminates
some inappropriate code. Also some programmer comments
were (barely) improved (i hope) in some small measure.
Reference(s):
smatch: 406 procps_meminfo_chain_fill() warn: variable dereferenced before check 'chain' (see line 403)
Signed-off-by: Jim Warner <james.warner@comcast.net>
With this patch, we will be close to an implementation
which will be needed when accommodating tasks/threads.
The following explanation was from an earlier message:
The slabtop requirements are similar to those of PIDs.
One must accommodate the variable number of slab nodes
(PIDs) while also accepting different data (char * and
unsigned long). Furthermore, some generalized means to
sort all that acquired stuff must somehow be provided.
------------------------------------------------------
So this patch expands the API to provide dynamic chain
allocation plus allow sorting of those dynamic chains.
While specific to slab needs (nodes, not global stats)
it is not too early to begin to think of newlib chains
as the opaque replacement for a deprecated old proc_t.
Better yet, any newlib chain is inherently variable in
length, something the old proc_t couldn't claim to be.
Of course, as we get to PIDs we'll want to grow/shrink
chains (easily accomplished with a special item enum).
And we'll want to grow/shrink those **head arrays too.
But these minor details don't seem insurmountable now.
Signed-off-by: Jim Warner <james.warner@comcast.net>
An earlier approach to meminfo chaining, referenced in
the patch shown below, represents the first baby steps
toward the goal of some generalized approach with PIDs
processing. However, statically allocating a chain for
each task or thread is totally impractical. And, while
a single chain could serve all PIDs, that would mean a
separate call to our library for each running process.
This commit is intended as the next evolutionary step,
dynamically allocating some 'result' chains to contain
as many or as few 'items' as a caller wishes. In other
words, holding only those 'items' of current interest.
This is the kind of service useful for both top and ps
programs if we finally get around to /proc/<PID> data.
Reference(s):
commit c3fd7473c5
Signed-off-by: Jim Warner <james.warner@comcast.net>
This was Craig's original patch, referenced below, but
it was never pushed to newlib. It has now been rebased
on top of some diskstat stuff to serve as a beginning.
The original effort was perfectly serviceable (after a
memory leak was fixed) but the approach would not have
served future PID needs when that proc_t went bye bye.
The slabtop requirements are similar to those of PIDs.
One must accommodate the variable number of slab nodes
(PIDs) while also accepting different data (char * and
unsigned long). Furthermore, some generalized means to
sort all that acquired stuff must somehow be provided.
So I wanted to try a different approach that seemed to
hold potential for satisfying future top and ps needs.
Subsequent commits will make that attempt, building on
Craig's original patch whose commit msg appears below.
------------------------------------------------------
All of the /proc/slabinfo related calls have been changed
here. They follow the same procps_slabinfo_* format.
Made both the slabtop and vmstat programs use the new
API as one was using the old one and one was just sort
of trying to do its own thing.
Sorting of slabnodes is also possible via the library.
Reference(s):
http://www.freelists.org/post/procps/Sorting-slabsprocesses,3http://www.freelists.org/post/procps/library-rework-slabinfo-calls
Signed-off-by: Craig Small <csmall@enc.com.au>
Signed-off-by: Jim Warner <james.warner@comcast.net>
The calls for reading diskstat have been moved out of
sysinfo and into new files diskstat.[ch] These new
library calls follow the standard pattern for the
new libprocps.
vmstat is updated to use the new API and also got
the weighted IO time added.
vmstat -p previously would only show partitions, not
disks. There does not appear to be any good reason to
artifically deny a user to use this command on a disk,
rather than a partition so this restriction was lifted.
I also realised using int for devid means you can send
the library negative numbers, the index uses unsigned int.
Other similiar calls will need to be fixed too.
Signed-off-by: Craig Small <csmall@enc.com.au>
This is actually a systemcall getpagesize(2) or it is defined
in configure using a variety of methods, including a default
hard coded value as a last resort.
There is no need to have this in libprocps
The new library meminfo & vmstat modules use structure
names for their context which exactly mirror the names
of the very /proc/ files whose particulars they yield.
The one exception to this rule was the readstat module
whose struct was named statinfo yet the file was stat.
This commit simply renames that structure (only) so as
to hopefully establish such a naming convention as our
standard going forward. And, it's makes good symmetry.
[ this module's name itself is just perfect as it is ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The earlier attempt at protecting these functions from
already freed memory worked just fine until the memory
was, in fact, reused by the OS. At that point, the ref
count would most likely fail an existing a test for 0.
So this commit will take control of the 'info' pointer
and force it to NULL when a reference count reaches 0.
Plus, since it makes little sense returning an address
that a caller already has, henceforth we will return a
reference count out of the 'ref' and 'unref functions.
Reference(s):
commit 74beff80ff
Signed-off-by: Jim Warner <james.warner@comcast.net>
Procps library previously held functions that were about either
listing or finding signal names. These are not really the right
location for a library about reading procfs.
This patch handles signal related functions in two ways:
For functions purely found in skill, these have been moved back
into this binary as they are used nowhere else.
For functions used across the binaries, these have been moved
into include/signals.h and lib/signals.c. Besides formatting,
these functions are largely the same.
To assist the skill functions, two functions to access the
signal map array have been added to lib/signals.c
There was not a way I could see to support top's needs
for cpu information with the new 'chained' provisions.
The sheer quantity of such data plus the unpredictable
number of potential processors suggested a totally new
approach was warranted while keeping internals opaque.
So this patch introduces two new structures solely for
use by potential callers (as seen in the API). They're
responsible for providing them to the library which is
then responsible for filling them with requested data.
The top program will continue to provide numa support,
without involving the library in any of those details.
[ not to mention all the 'dl' and 'stderr' numa crap ]
With this transfer of the cpu tics duty to our library
the provision associated with the CPU_ZEROTICS #define
could not initially be migrated. The commit referenced
below suggests it may have lost its importance. In any
case such logic may yet be incorporated in the future.
But for now, that #define has been completely removed.
Reference(s):
commit ee3ed4b45e
Signed-off-by: Jim Warner <james.warner@comcast.net>
Since we are not using a higher level standard C fopen
all of the read requests were made signal sensitive as
that can result in a 'temporarily' failed i/o request.
Also, protection against some user calling the 'unref'
function on already free memory has been incorporated.
This will protect us from some nasty 'Abort' surprise.
Signed-off-by: Jim Warner <james.warner@comcast.net>
If a caller chooses to reduce the overhead of repeated
function calls, this commit provides for acquiring all
the desired information in just a single library call.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch mostly just eliminates darn tab characters.
Plus the library function declarations and definitions
have been standardized. Most visibly, the input params
now have all been indented on their own separate line.
The following names were changed to more closely match
meminfo.c or provide a certain symmetry. Unfortunately
that also impacted some other pgms which were updated.
. 'procps_stat_get' evolved into 'procps_stat_get_sys'
. 'procps_stat_info' is now known as 'procps_statinfo'
[and just a little trailing whitespace was eliminated]
Signed-off-by: Jim Warner <james.warner@comcast.net>
If a caller chooses to reduce the overhead of repeated
function calls, this commit provides for acquiring all
the desired information in just a single library call.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch fills in some missing fields which have top
dependencies. Additionally, I've tried to mirror those
calculations Jaromir added for release 3.3.10. The one
calculation that remains missing is 'available' memory
for some kernels. For this API, we'll use a fall-back.
Lastly the lxc safeguards which were recently added to
the old procps library were incorporated here as well.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch mostly just eliminates darn tab characters.
Plus the library function declarations and definitions
have been standardized. Most visibly, the input params
now have all been indented on their own separate line.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Created new set of functions for meminfo related calls. Liked the
format of that better so changed vmstat around so the look similar.
Missed the makefile change for uptime so added it in now.
Removed the printf_uptime, binaries can do printf easily enough.
sprint_uptime split into two as there wasn't a lot of common
code
sprint_uptime(): old style uptime line
sprint_uptime_short(): short new style "uptime -p"
Hertz_hack needed this, no sane system uses the code (I think)
so just assume 100 like we do in FreeBSD.
Use the standard libc declarations.
For protecting the headers for C++ procps used to have its
own defines, this change makes them use the standard libc ones.
getstat() -> procps_stat_*
vminfo() -> procps_vmstat_*
These two components of the library now use the newer version of
the API with less exposed global variables. The old methods are
there for now.
Signed-off-by: Craig Small <csmall@enc.com.au>
wish folks (craig) would use these in their .gitconfig
[core]
whitespace = trailing-space, space-before-tab, blank-at-eof
[apply]
whitespace = warn
Signed-off-by: Jim Warner <james.warner@comcast.net>
Added function procps_linux_version() which used to be an
exported integer instead. Also changed the method of obtaining
the linux version (more correctly the os release) to use a specific
procfs entry. This works for both Linux and FreeBSD.
This patch will bring three of our man pages into line
with the recent refactor of the libprocps wchan logic.
[ and also eliminates more damn eol whitespace which ]
[ snuck in our repo with the commit referenced below ]
Reference(s):
http://www.freelists.org/post/procps/WCHAN,11
commit cf4788c28d
Signed-off-by: Jim Warner <james.warner@comcast.net>
Several Debian based distributions were recently found
to have omitted a kernel configuration option that had
the effect of rendering /proc/#/stat and /proc/#/wchan
useless for providing any 'sleeping in function' info.
That problem also prompted a reevaluation of the whole
approach to wchan matters which had grown increasingly
complex as our library evolved over the last 13 years.
The net result was a decision to rely on /proc/#/wchan
which arrived along with the 2.5 kernel. This then let
us vastly simplify the internal code plus the external
interface which will benefit both the top and ps pgms.
Reference(s):
http://www.freelists.org/post/procps/WCHAN,11https://lkml.org/lkml/2008/11/6/12https://bugs.debian.org/711592
Signed-off-by: Jim Warner <james.warner@comcast.net>
It doesn't make any sense to have the binary version strings
embedded into the library. The version strings are defined
already either in the Makefile or in include/c.h
This commit adds a lxc container name to every proc_t.
If a process is not running in a container, then a '-'
will be provided, making such a field always sortable.
Unlike other proc_t character pointers, lxc containers
will find many duplicate shared values. So rather than
strdup 'em (with a later free required upon reuse), we
try to keep track of those already seen and share that
address among all tasks running within each container.
We rely on the lines in the task's cgroup subdirectory
which may initially seem somewhat unsophisticated. But
the lxc library itself uses a similar approach when it
is called to list active containers. In that case, the
/proc/net/unix directory is parsed for the '/lxc' eye-
catcher, with potential complications from hashed path
and names that are too long (something we don't face).
[ too bad docker abandoned lxc - our commit won't do ]
[ anything for the users of those kind of containers ]
Reference(s):
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1424253https://bugs.launchpad.net/ubuntu/+source/procps/+bug/1424253
Signed-off-by: Jim Warner <james.warner@comcast.net>
Under a lxc container, the /proc/meminfo 'MemFree' and
'MemAvailable' amounts will be equal, unless memory is
being limited via cgroups in which case 'MemAvailable'
could exceed that for 'MemTotal'. And when a container
has been nested, there exist additional memory quirks.
A program might then display used or available amounts
greater than total memory (assuming unsigned honored),
or negative values (should a signed cast be employed).
This anomaly primarily impacted the top and free pgms.
Thus, two simple sanity checks have been introduced to
avoid any illogical kb_main_available or kb_main_used.
( Busybox top & free also display anomalous although )
( different results when running in a lxc container. )
Reference(s):
https://bugzilla.redhat.com/show_bug.cgi?id=1153817
Signed-off-by: Jim Warner <james.warner@comcast.net>
This will be required for subdir-objects, otherwise automake will have
problems with more than one Makefile.am having rules to build the same
files.
Tested that it builds and both `make check` and `make distcheck` work.
Tested `make install` and compared the tree with the one installed
before this commit, both installed the binaries to the same locations.
The binaries are also in the same location in the build tree (for
instance, ps/pscommand is still there.)
Checked the binaries for the correct libraries linked into them. Binary
sizes matched before and after this change.
Signed-off-by: Filipe Brandenburger <filbranden@google.com>
This is required for out-of-tree build to work, since many source files
include e.g. proc/*.h which is not under the include/ directory.
Tested that `make distcheck` starts working after this patch.
Signed-off-by: Filipe Brandenburger <filbranden@google.com>
Let's not report zero for kb_main_available when older
kernels don't have MemAvailable. Instead, if we simply
duplicate the 'free' amount we can avoid all ancillary
problems, such as those involving top's graphing mode.
Reference(s):
http://www.freelists.org/post/procps/kb-main-available-etc,3
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit just ensures recalculation of some amounts
for iterative processes, like top. It also trades some
repeated runtime calls to sysconf for a one time cost.
Reference(s):
http://www.freelists.org/post/procps/systemd-support-to-library,7
. fall-back calculations
commit b779855cf1
Signed-off-by: Jim Warner <james.warner@comcast.net>
[ plus remove just a little darn trailing whitespace ]
Reference(s):
. systemd migrated to library
commit 9d8ad6419f
. added library documentation
commit a74fb8fade
Signed-off-by: Jim Warner <james.warner@comcast.net>
The stderr message regarding ELF notes appears on some
systems (openSUSE-13.1 for example) but I have not yet
isolated why. Since at startup we go on to determine a
Hertz value the old fashion way, this patch just turns
off the useless message until the cause is understood.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit adds support for fallback calculation
of the MemAvailable field if not exported by the
kernel. The MemAvailable field appeared in kernel
3.14, but it's possible to calculate it from other
fields since 2.6.27 (splitLRU changes).
Nowadays the usernames can be 32 characters long
(typically OpenShift usernames use the whole length)
and the old limit was preventing us from processing
them correctly.
The macro change affects the proc_t structure size.
This commit adds a new switch -a/--available that
appends a new column called 'available' to the
output. The column displays an estimation
of how much memory is available for starting
new applications, without swapping. Unlike the data
provided by the 'cached' or 'free' fields, this
field takes into account page cache and also that
not all reclaimable memory slabs will be reclaimed
due to items being in use.
The subtraction was marked as reinforcing the misconception,
that memory in the page cache can be considered free.
The Cached value is not a sum of page cache and tmpfs,
as the tmpfs memory lives in the page cache and therefore
it's an inseparable part of it.
tmpfs has become much more widely used since distributions use it for
/tmp (Fedora 18+). In /proc/meminfo, memory used by tmpfs is accounted
into "Cached" (aka "NR_FILE_PAGES",
http://lxr.free-electrons.com/source/mm/shmem.c#L301 ).
The tools just pass it on, so what top, free and vmstat report as
"cached" is the sum of page cache and tmpfs.
free has the extremely useful "-/+ buffers/cache" output. However, now
that tmpfs is accounted into "cached", those numbers are way off once
you have big files in /tmp.
Fortunately, kernel 2.6.32 introduces "Shmem", which makes tmpfs memory
usage accessible from userspace (
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4b02108ac1b3354a22b0d83c684797692efdc395 ).
This patch substracts Shmem from Cached to get the actual page cache
memory. This makes both issues mentioned above disappear. For older
kernels, Shmem is not available (hence zero) and this patch is no-op.
Additionally:
* Update the man pages of free and vmstat to explain what is happening
* Finally drop "MemShared" from the /proc/meminfo parser, it has been
dead for 10+ years and is only causing confusion ( removed in kernel
2.5.54, see
https://git.kernel.org/cgit/linux/kernel/git/tglx/history.git/commit/?id=fe04e9451e5a159247cf9f03c615a4273ac0c571 )
vmstat -d or vmstat -p would crash mysteriously under different
circumstances. The problem was eventually tracked down to /sys not
being mounted which meant is_disk() always returned false.
The partition would then be attempted to be linked to a non-existent
disk causing a segfault.
vmstat will now not link to a disk if none exists.
The change in testing will skip those tests when /sys/block doesn't
exist.
Many thanks to Daniel Schepler for his analysis and suggestions.
Bug-Debian: http://bugs.debian.org/736628
Under some circumstances the ksh shell doesn't fork new processes
when executing scripts and the script is interpreted by the
parent process. That makes the execution faster, but it means
ksh needs to reuse the /proc/PID/cmdline for the new script name
and arguments while the file length needs to stay untouched.
The fork is skipped only when the new cmdline is shorter than
the parent's cmdline and the rest of the file is filled
with '\0'. This is perfectly ok until we try to read the cmdline
of such process. As the read_unvectored() function replaces
all zeros with chosen separator, these trailing zeros are replaced
with spaces in case of the ps tool. Consequently it appends
multiple spaces at the end of the arguments string even when these
zeros do not represent any separators and therefore shouldn't
be replaced.
With this commit the read_unvectored() function skips the
replacement of trailing zeros and separates valid content only.
Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1057600
While 'invisible' thread subdirectories are accessible
under /proc/ with stat/opendir calls, they have always
been treated as non-existent, as is true with readdir.
This patch trades the /proc/#/ns access convention for
the more proper /proc/#/task/#/ns approach when thread
access is desired. In addition some namespace code has
been simplified and made slightly more efficient given
the calloc nature of proc_t acquisition and its reuse.
Reference(s):
commit a01ee3c0b3
Signed-off-by: Jim Warner <james.warner@comcast.net>
Previously the shared memory column was always zero
for 2.6 series kernels (and later) due to the fact,
that the value was taken from the MemShared entry
that disappeared with 2.6 series kernels.
Later a new Shmem entry appeared in the /proc/meminfo
file and the 'shared' column now displays either
the MemShared or the Shmem value (depending on their
presence - the presence is mutually exclusive).
If none of the two entries is exported by the kernel,
then the column is zero.
An unpleasant thing happened when I comitted the shmem support
for the 'free' tool. We already had a merge request from
Adrian Brzezinski in the queue, doing exactly the same.
As Adrian deserves credits, I'm reverting the change
and re-applying with the next commit in order to make
him a part of the project history.
Previously the shared memory column was always zero
for 2.6 series kernels (and later) due to the fact,
that the value was taken from the MemShared entry
that disappeared with 2.6 series kernels.
Later a new Shmem entry appeared in the /proc/meminfo
file and the 'shared' column now displays either
the MemShared or the Shmem value (depending on their
presence - the presence is mutually exclusive).
If none of the two entries is exported by the kernel,
then the column is zero.
It seems in some cases procs_running field of /proc/stat can contain 0 even if vmstat itself is running. At least this can be reproduced on Linux 3.9.3 compiled with BFS scheduler.
Since getstat() decrements value of procs_running by 1, we get overflow:
$ vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 667732 918996 57376 911260 21 30 36 40 98 45 14 82 4 1
4294967295 0 667728 916716 57376 911264 8 0 8 0 1958 3733 28 7 65 1
0 0 667700 915996 57376 911416 24 0 152 0 1735 3600 23 5 71 1
4294967295 0 667700 915872 57376 911392 0 0 0 0 1528 3165 21 4 76 0
Function getbtime() currently makes the assumption that btime==0 equals
btime not being present in /proc/stat. This is not quite accurate, as
timestamp 0 is, in fact, also a valid time (Epoch), and /proc/stat may
report it as such.
We introduce a flag to indicate whether btime was found in /proc/stat.
In this way, btime==0 becomes a valid case, provided /proc/stat
actually reports this as the boot time.
procps can still detect the case of btime actually not being reported
by the kernel.
Signed-off-by: Markus Mayer <mmayer@broadcom.com>
One recent patch to dynamic buffer management involved
over-allocating the buffer increase to lessen calls to
xrealloc. That was successful, but the actual increase
amount did not attempt to optimize size or alignments.
With this commit, we'll copy an approach recently used
by the top program and round up buffer sizes to 1 KiB.
More importantly, while buffers are quickly reaching a
KiB optimum multiple, no memcpy will ever be employed!
To illustrate just how effective top's algorithm would
be, just change the initial and subsequent allocations
from the current 1024 bytes to just a single byte then
add an fprintf. Those one byte reallocations while on
the way to optimum buffer size will be a one-time cost
and won't represent any recurring performance penalty.
( gosh, that top program *must be* one fart smeller, )
( or was that a smart feller, i can't remember which )
Reference)s):
commit 6d605f521c
commit a45dace4b8
Signed-off-by: Jim Warner <james.warner@comcast.net>
Each process in Linux has a /proc/<pid>/ns directory which contains
symbolic links to pipes that identify which namespaces that process
belongs to. This patch adds support for ps to display that information
optionally.
Signed-off-by: Aristeu Rozanski <arozansk@redhat.com>
An earlier commit attempted to cleanse our environment
of all useless trailing whitespace. But the effort did
not catch 'empty' lines with a single space before ^J.
This commit hopefully finishes off the earlier effort.
In the meantime, let's pray that contributors' editors
are configured so that such wasted crap is disallowed!
Reference(s):
commit fe75e26ab6
Signed-off-by: Jim Warner <james.warner@comcast.net>
When utility buffers were introduced for file2str read
requests, a subtle change was inadvertently introduced
such that a read of zero no longer returns a -1 value.
This commit ensures that zero bytes read returns a -1.
And although the solution differs from a merge request
submitted by sergey.senozhatsky@gmail.com, a thank you
is offered for revealing this potential abend problem.
References(s):
commit a45dace4b8http://gitorious.org/procps/procps/merge_requests/11
Signed-off-by: Jim Warner <james.warner@comcast.net>
Signed-off-by: Craig Small <csmall@enc.com.au>
When dynamic buffers were recently introduced for read
of the status, stat and statm subdirectories one extra
call to read() was required for end-of-file detection.
This patch avoids most all such extra calls to read().
Additionally, the frequency of memory reallocations is
reduced by overallocating each increase more than 25%.
Reference)s):
commit a45dace4b8
Signed-off-by: Jim Warner <james.warner@comcast.net>
Signed-off-by: Craig Small <csmall@enc.com.au>
Internal changes to libproc means the revision number
is incremented. This does not mean an ABI or API change has
occured, we just do the stuff under the covers better or in this
case reduce the compile warnings mainly.
See Jim, I do read the commit messages :)
readproc.c: In function 'stat2proc' :
readproc.c:516: warning: use of assignment suppression and length modifier together in gnu_scanf format
readproc.c:516: warning: use of assignment suppression and length modifier together in gnu_scanf format
Signed-off-by: Gilles Espinasse <g.esp@free.fr>
A recent Debian bug report, dealing with release 3.2.8
and its even more restrictive buffer sizes (1024) used
in stat, statm and status reads via file2str calls, is
a reminder of what could yet happen to procps-ng. Size
needs are determined by kernel evolution and/or config
options so that bug could resurface even though buffer
size is currently 4 times the old procps-3.2.8 limits.
Those sizes were raised from 1024 to 4096 bytes in the
patch submitted by Eric Dumazet, and referenced below.
This patch makes libprocps immune to future changes in
the amount of stuff that is ultimately found in a proc
'stat', 'statm' or 'status' subdirectory. We now trade
the former static buffer of 4096 bytes for dynamically
allocated buffers whose size can be increased by need.
Even though this change is solely an internal one, and
in no way directly affects the API or the ABI, libtool
suggests that the LIBprocps_REVISION be raised. I hope
Craig remembers to do that just before a next release.
We don't want a repeat of the procps-ng-3.3.4 boo-boo,
but with no API/ABI impact that probably can't happen.
p.s. A big thanks to Jaromir Capik <jcapik@redhat.com>
who reviewed my original version and, of course, found
some of my trademark illogic + unnecessary code. After
his coaxing, he helped make this a much better commit.
Reference(s):
. procps-3.2.8
http://bugs.debian.org/702965
. allow large list of groups
commit 7933435584
Signed-off-by: Jim Warner <james.warner@comcast.net>
Reviewed by: Jaromir Capik <jcapik@redhat.com>
The entire tree's polluted with inappropriate trailing
whitespace. This commit rids our environment of all of
those useless keystrokes. Unfortunately, it sure ain't
a permanent solution and requires every contributor to
instruct their editor(s) to prevent or eliminate them.
Plus it's strongly recommended we all insert something
like what's shown below to our '.gitconfig' file so as
to provide at least some warnings when we try to apply
any patches (git am) that do contain the #@!%& things!
References(s):
~/.gitconfig excerpt ---------------------------------
[core]
whitespace = trailing-space, space-before-tab, blank-at-eof
[apply]
whitespace = warn
--------------------------------- ~/.gitconfig excerpt
Signed-off-by: Jim Warner <james.warner@comcast.net>
freeproc was missing from the libproc API which meant while you could
allocate proc structures, you couldn't free them!
Bug-Debian: http://bugs.debian.org/681653
Fixes error which did not happen always. Changes of being affected by
the bug where greater the more there where pids defined as pmap argument.
The debian bug referral can almost certainly reproduce the problem,
especially when tried multiple times in row.
pmap: malloc.c:3096: sYSMALLOc: Assertion `(old_top == (((mbinptr)
(((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct
malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >=
(unsigned long)((((__builtin_offsetof (struct malloc_chunk,
fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) ==
0)' failed.
Reported-by: lee <lee@yun.yagibdah.de>
Bug-Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=688180
Signed-off-by: Sami Kerola <kerolasa@iki.fi>
