That #define QUICK_THREADS was impossible to implement
under the new <pids> interface. Plus it was also found
to distort some thread information (referenced below).
So, it's always been inactive under the newlib branch.
However, it will (with small changes) still serve some
useful purpose in our library. Now, when the redefined
FALSE_THREADS is active, those special strings showing
"[ duplicate ENUM ]" will appear for any child thread.
Note: the real reason for such strings appearing isn't
being exercised, only their mechanics. In actual usage
they are substituted when a user duplicates such items
in a results stack & only the 1st instance can own it.
With this patch, we are simply fooling the <pids> code
into thinking an item was duplicated via a NULL value.
Reference(s):
. from master branch
commit 25a6ecdbfb
Signed-off-by: Jim Warner <james.warner@comcast.net>
Unlike other cached string fields (such as USER name),
that lxcname field was defined as 'const char *'. This
required a custom 'set' function using a cast in order
to avoid a compiler warning regarding that assignment.
So this commit brings lxcname into line with all those
other cached string fields. And while we're at it, the
reason some string fields have no 'freefunc' will also
be explained with some new comments in the Item_table.
Signed-off-by: Jim Warner <james.warner@comcast.net>
On MacOS the system already has user_from_uid and group_from_gid.
These are renamed pwcache_get_user and pwcache_get_group.
They were also exported but did not appear in the symbol file
or used by any of the procps binaries. They are no longer exported.
References:
https://gitlab.com/procps-ng/procps/issues/34
Signed-off-by: Craig Small <csmall@enc.com.au>
In that reference below a specific systemd problem was
fixed in the commit shown. However lurking deep within
the <pids> interface was yet one final case where NULL
could be returned, involving 'strv' and the following:
. a user requested both a single string vector (always
returned as a normal string) & the vectorized version,
as with PROCPS_PIDS_CMDLINE and PROCPS_PIDS_CMDLINE_V.
. a user simply duplicated some vectorized enum items.
The root of that NULL problem is the fact those single
string vectors shared the same proc_t field with their
true vectorized version. So while multiple occurrences
for most strings could be satisfied with strdup versus
the normal ownership usurpation, those true vectorized
fields could not be quite so easily copied/duplicated.
Thus newlib chose to return a NULL result.strv pointer
under either of the above scenarios (which perhaps was
just a user boo-boo in the first place). In any event,
the NULL was a potential for true string vectors only.
Now, since newlib is the sole caller into the readproc
module, separate fields have been created for what are
just normal strings (never vectorized) and those which
remain the true vectorized versions. And, former flags
which only worked if combined, now act as stand alone.
Thus, both PROCPS_PIDS_CMDLINE & PROCPS_PIDS_CMDLINE_V
can be used simultaneously (as they should have been).
Also with this patch, items which a user duplicates in
the stack (beyond the first such item) will return the
the string "[ duplicate ENUM_ID ]". This practice will
apply to both single strings and true vectorized ones.
In addition to informing users of their error, it will
also mean potential NULLs need now never be a concern.
Reference(s);
http://www.freelists.org/post/procps/systemd-binary-vs-library
commit 0580a7b4c6
Signed-off-by: Jim Warner <james.warner@comcast.net>
A potential conflict arises should should some program
attempt to access systemd data using our library built
without that support (--disable-systemd). And while we
could argue that all callers should check against NULL
string pointers, it's not expected of libprocps users.
So we'll guarantee 'em valid string consisting of "?".
Reference(s):
http://www.freelists.org/post/procps/systemd-binary-vs-library
Signed-off-by: Jim Warner <james.warner@comcast.net>
This is just beginning an effort to minimize/normalize
the sheer variety of results types in use for our API.
In taking these first baby steps, a few anomalies were
found. There'll no doubt be many more yet to discover.
. the _FLT_ (fault) fields were already signed long in
the proc_t (even though their sscanf format used %lu).
. although strtoul will alway return an unsigned long,
all of the _VM_ fields were made signed long just like
other memory fields (& signed sorts more efficiently).
Reference(s):
http://www.freelists.org/post/procps/newlib-drip-drip,4
Signed-off-by: Jim Warner <james.warner@comcast.net>
A while loop used n >=0
A branch inside the loop had a test for n<0
That's never going to happen!
So the branch must go. There is similiar code outside
the branch. Yes, I miss pythons while else here.
Also cleaned up the ***-awful indentation this function had.
No wonder that branch lived for so long there.
References:
Coverity #99119
Signed-off-by: Craig Small <csmall@dropbear.xyz>
readproc still had some of the old API hanging around that
was good while we were trying it out, but its time to say
goodbye:
readproc.h removed from public procps.h header file
enum ns_type - not used - removed
get_ns_name() - not defined - removed
get_ns_id() - not defined - removed
PROCTAB proc_t only used internal to library
readproctab() and 2,3 friends not used - removed
proc_data_t was used by readproctab23() - removed
readtask() - not used - removed
read_cmdline() - copy,renamed and made generic - remove original
freeproc() - not used - removed
get_proc_status - not used - removed, however there should be
a new function created that does this. Given a PID return data
about it instead of scanning the entire procfs. Maybe it already does.
Left as internal-to-library only functions:
readproc(), readeither(), look_up_our_self(), openproc(), closeproc()
Updated libprocps.sym to export only what we use.
Now that the conditional OOMEM_ENABLE has been removed
and, after reviewing current library support, it turns
out we've been using a deprecated /proc/<pid>/oom_adj.
What we should use instead is that more recent tunable
/proc/<pid>/oom_score_adj. This new field will provide
a range of -1000 thru +1000 (former was -17 thru +15).
Reference(s):
. removal of misguided OOMEM_ENABLE
commit 64238730fa
. linux, partial oom_adj revert (Nov, 2012)
commit fa0cbbf145aabbf29c6f28f8a11935c0b0fd86fc
. linux, removal oom_adj (Oct, 2012)
commit 01dc52ebdf472f77cca623ca693ca24cfc0f1bbe
Signed-off-by: Jim Warner <james.warner@comcast.net>
The includes used to define a lot of things a library include
should not. It was also a bit messy what was exposed in the library
and what was not.
get_pid_digits -> procps_pid_length and exported correctly
MALLOC attribute move into relevant .c files
NORETURN attribute moved to relevant .c, not used in library
PURE attribute removed, it wasn't used
KLONG/KLF/STRTOUKL were fixed for long, so now just use long
HIDDEN attribute removed. It was for 3 functions. The PROCPS_EXPORT
seems to do the same (opposite) thing.
likely/unlikely removed from most places, its highly debateable
this does anything useful as CPUs have gotten smarter about branches.
Re-arranged the includes, ALL external programs should just #include
<proc/procps.h> then proc/procps.h includes headers for files that
have exported functions. procps.h and the headers it includes should
not use items that are not exportable (e.g. hidden functions or
macros) they go in procps-private.h
Multiple scanf()s use the GNU-permitted %Lu. This is not supported in
other libraries and isn't to the POSIX specification. The L modifier
is only used for floats in POSIX.
Replacing %Lu with %llu is the same for GNU libc (scanf(3) says as much)
but means other libraries will work fine.
From master commit da715e3
References:
http://pubs.opengroup.org/onlinepubs/009695399/functions/fscanf.html
Beginning with linux-4.5, the following new fields are
being added under that /proc/<pid>/status pseudo file:
. RssAnon - size of resident anonymous memory
. RssFile - size of resident file mappings
. RssShmem - size of resident shared memory
This patch just represents the initial library and top
support, sharing a commit message with 2 more patches.
p.s. locked resident memory support was also added but
isn't directly related to the kernel 4.5 enhancements.
Reference(s):
commit 1f8e41d019
Signed-off-by: Jim Warner <james.warner@comcast.net>
[ but most definitely not all of them by a long shot ]
Reference(s):
proc/diskstat.c:186:17: warning: unused variable 'is_disk' [-Wunused-variable]
int retval, is_disk;
^
proc/namespace.c:110:1: warning: control may reach end of non-void function [-Wreturn-type]
}
^
proc/readproc.c:1131:50: warning: address of array 'ent->d_name' will always evaluate to 'true' [-Wpointer-bo
if(unlikely(unlikely(!ent) || unlikely(!ent->d_name))) return 0;
~~~~~~^~~~~~
proc/readproc.c:1158:50: warning: address of array 'ent->d_name' will always evaluate to 'true' [-Wpointer-bo
if(unlikely(unlikely(!ent) || unlikely(!ent->d_name))) return 0;
~~~~~~^~~~~~
proc/sysinfo.c:45:12: warning: unused variable 'stat_fd' [-Wunused-variable]
static int stat_fd = -1;
^
proc/sysinfo.c:49:12: warning: unused variable 'meminfo_fd' [-Wunused-variable]
static int meminfo_fd = -1;
^
proc/sysinfo.c:51:12: warning: unused variable 'vminfo_fd' [-Wunused-variable]
static int vminfo_fd = -1;
^
proc/sysinfo.c:53:12: warning: unused variable 'vm_min_free_fd' [-Wunused-variable]
static int vm_min_free_fd = -1;
^
proc/uptime.c:157:12: warning: unused variable 'realseconds' [-Wunused-variable]
time_t realseconds;
^
proc/uptime.c:158:16: warning: unused variable 'realtime' [-Wunused-variable]
struct tm *realtime;
^
vmstat.c:574:20: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat]
DSTAT(PROCPS_DISKSTAT_READ_TIME),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vmstat.c:578:20: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat]
DSTAT(PROCPS_DISKSTAT_WRITE_TIME),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
watch.c:230:7: warning: variable 'endptr' is uninitialized when used here [-Wuninitialized]
if (*endptr == '\0') set_ansi_attribute(0); /* [m treated as [0m */
^~~~~~
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit is an outgrowth of the research into a bug
that recently surfaced with the 'w' program. And while
that program was just a victim several inconsistencies
were found in the handling of library flags during the
research. This patch just address such irregularities.
Reference(s):
http://www.freelists.org/post/procps/newlib-at-the-precipice,4
Signed-off-by: Jim Warner <james.warner@comcast.net>
Functions related to namespaces were half-in half-out of the
procps library and didn't fit the standard naming scheme.
While struct { long ns[x]} is a bit clunky, its the only way
to "lock in" x. The alternative is to use ns_* variables.
This work was needed before pgrep could be converted.
The patch below is where most proc_t fixed size arrays
became simple pointers to char. In that commit changes
to the above function were made so that dynamic memory
was freed which included the program name (cmd) field.
That change was prompted by a valgrind reported memory
leak during development that no longer seems to exist.
However, by keeping the look_up_our_self() changes the
ps command without args then fails to report anything.
So this patch just restores the expected old behavior.
Reference(s):
commit 3881a0844a
Signed-off-by: Jim Warner <james.warner@comcast.net>
It was probably always wrong to have a variable length
proc_t structure. This patch takes all remaining oomem
former suse only options and makes them unconditional.
Signed-off-by: Jim Warner <james.warner@comcast.net>
A few minor changes are being made to position the old
readproc logic for a transition to the newlib pid api.
These changes will not impact current users beyond the
the need to recompile such code. Hopefully this should
be very last version change to the deprecated library.
. most char arrays were replaced via char * to dynamic
memory. this was done so that newlib could just assume
ownership of such strings without using a strdup call.
. former user and group name arrays also became char *
but here the reason was because pwcache already cached
those names. so, copying to an array never made sense.
. the concept of QUICK_THREADS used to avoid duplicate
overhead for string data was disabled. it could not be
integrated with the newlib design, at least initially.
. any #define which influenced the size of that proc_t
was disable in the header. it was probably a poor idea
to approach optional features in such a manner anyway.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit adds a lxc container name to every proc_t.
If a process is not running in a container, then a '-'
will be provided, making such a field always sortable.
Unlike other proc_t character pointers, lxc containers
will find many duplicate shared values. So rather than
strdup 'em (with a later free required upon reuse), we
try to keep track of those already seen and share that
address among all tasks running within each container.
We rely on the lines in the task's cgroup subdirectory
which may initially seem somewhat unsophisticated. But
the lxc library itself uses a similar approach when it
is called to list active containers. In that case, the
/proc/net/unix directory is parsed for the '/lxc' eye-
catcher, with potential complications from hashed path
and names that are too long (something we don't face).
[ too bad docker abandoned lxc - our commit won't do ]
[ anything for the users of those kind of containers ]
Reference(s):
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1424253https://bugs.launchpad.net/ubuntu/+source/procps/+bug/1424253
Signed-off-by: Jim Warner <james.warner@comcast.net>
Under some circumstances the ksh shell doesn't fork new processes
when executing scripts and the script is interpreted by the
parent process. That makes the execution faster, but it means
ksh needs to reuse the /proc/PID/cmdline for the new script name
and arguments while the file length needs to stay untouched.
The fork is skipped only when the new cmdline is shorter than
the parent's cmdline and the rest of the file is filled
with '\0'. This is perfectly ok until we try to read the cmdline
of such process. As the read_unvectored() function replaces
all zeros with chosen separator, these trailing zeros are replaced
with spaces in case of the ps tool. Consequently it appends
multiple spaces at the end of the arguments string even when these
zeros do not represent any separators and therefore shouldn't
be replaced.
With this commit the read_unvectored() function skips the
replacement of trailing zeros and separates valid content only.
Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1057600
While 'invisible' thread subdirectories are accessible
under /proc/ with stat/opendir calls, they have always
been treated as non-existent, as is true with readdir.
This patch trades the /proc/#/ns access convention for
the more proper /proc/#/task/#/ns approach when thread
access is desired. In addition some namespace code has
been simplified and made slightly more efficient given
the calloc nature of proc_t acquisition and its reuse.
Reference(s):
commit a01ee3c0b3
Signed-off-by: Jim Warner <james.warner@comcast.net>
One recent patch to dynamic buffer management involved
over-allocating the buffer increase to lessen calls to
xrealloc. That was successful, but the actual increase
amount did not attempt to optimize size or alignments.
With this commit, we'll copy an approach recently used
by the top program and round up buffer sizes to 1 KiB.
More importantly, while buffers are quickly reaching a
KiB optimum multiple, no memcpy will ever be employed!
To illustrate just how effective top's algorithm would
be, just change the initial and subsequent allocations
from the current 1024 bytes to just a single byte then
add an fprintf. Those one byte reallocations while on
the way to optimum buffer size will be a one-time cost
and won't represent any recurring performance penalty.
( gosh, that top program *must be* one fart smeller, )
( or was that a smart feller, i can't remember which )
Reference)s):
commit 6d605f521c
commit a45dace4b8
Signed-off-by: Jim Warner <james.warner@comcast.net>
Each process in Linux has a /proc/<pid>/ns directory which contains
symbolic links to pipes that identify which namespaces that process
belongs to. This patch adds support for ps to display that information
optionally.
Signed-off-by: Aristeu Rozanski <arozansk@redhat.com>
When utility buffers were introduced for file2str read
requests, a subtle change was inadvertently introduced
such that a read of zero no longer returns a -1 value.
This commit ensures that zero bytes read returns a -1.
And although the solution differs from a merge request
submitted by sergey.senozhatsky@gmail.com, a thank you
is offered for revealing this potential abend problem.
References(s):
commit a45dace4b8http://gitorious.org/procps/procps/merge_requests/11
Signed-off-by: Jim Warner <james.warner@comcast.net>
Signed-off-by: Craig Small <csmall@enc.com.au>
When dynamic buffers were recently introduced for read
of the status, stat and statm subdirectories one extra
call to read() was required for end-of-file detection.
This patch avoids most all such extra calls to read().
Additionally, the frequency of memory reallocations is
reduced by overallocating each increase more than 25%.
Reference)s):
commit a45dace4b8
Signed-off-by: Jim Warner <james.warner@comcast.net>
Signed-off-by: Craig Small <csmall@enc.com.au>
readproc.c: In function 'stat2proc' :
readproc.c:516: warning: use of assignment suppression and length modifier together in gnu_scanf format
readproc.c:516: warning: use of assignment suppression and length modifier together in gnu_scanf format
Signed-off-by: Gilles Espinasse <g.esp@free.fr>
A recent Debian bug report, dealing with release 3.2.8
and its even more restrictive buffer sizes (1024) used
in stat, statm and status reads via file2str calls, is
a reminder of what could yet happen to procps-ng. Size
needs are determined by kernel evolution and/or config
options so that bug could resurface even though buffer
size is currently 4 times the old procps-3.2.8 limits.
Those sizes were raised from 1024 to 4096 bytes in the
patch submitted by Eric Dumazet, and referenced below.
This patch makes libprocps immune to future changes in
the amount of stuff that is ultimately found in a proc
'stat', 'statm' or 'status' subdirectory. We now trade
the former static buffer of 4096 bytes for dynamically
allocated buffers whose size can be increased by need.
Even though this change is solely an internal one, and
in no way directly affects the API or the ABI, libtool
suggests that the LIBprocps_REVISION be raised. I hope
Craig remembers to do that just before a next release.
We don't want a repeat of the procps-ng-3.3.4 boo-boo,
but with no API/ABI impact that probably can't happen.
p.s. A big thanks to Jaromir Capik <jcapik@redhat.com>
who reviewed my original version and, of course, found
some of my trademark illogic + unnecessary code. After
his coaxing, he helped make this a much better commit.
Reference(s):
. procps-3.2.8
http://bugs.debian.org/702965
. allow large list of groups
commit 7933435584
Signed-off-by: Jim Warner <james.warner@comcast.net>
Reviewed by: Jaromir Capik <jcapik@redhat.com>
Current linux kernels output no more than 32 groups
in /proc/{pid}/status.
Plan is to increase this limit.
This patch allows ps to not core dump if the buffer used to read status
file was too small.
# ps aux
Signal 11 (SEGV) caught by ps (procps-ng version 3.3.3).
ps:display.c:59: please report this bug
Also increases the size of the buffer from 1024 to 4096, since even with
32 groups we are close to the limit.
cat /proc/12731/status | wc
39 128 961
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
In preparation for top scrollable environment display,
the new flag PROC_EDITENVRCVT was added to mirror the
existing single vector string handling for cgroup and
cmdline.
Signed-off-by: Jim Warner <james.warner@comcast.net>
The control group hierarchies for any particular task
could conceivably grow quite large. However, the
library might impose an arbitrary limit of 1024 bytes
via fill_cgroup_cvt.
Two utility buffers of 128 KiB each were already
available for command line use. This commit simply
trades the smaller 1024 byte stack based buffers for
those much larger existing ones. Thus, truncation
can be avoided with no additional run-time costs.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Some inconsistencies have emerged during development
of support for these relatively new proc_t fields.
For example, a PROC_FILLCGROUP flag (via file2strvec)
could return NULL in cgroup whereas PROC_EDITCGRPCVT
(via fill_cgroup_cvt) *almost* guaranteed a return
address (as is true for PROC_EDITCMDLCVT and cmdline).
But even PROC_EDITCGRPCVT could return NULL if the
kernel version was less than 2.6.24. Then with NULL
ps would display a "-" while top would show "n/a".
And while unlikely, with the PROC_FILLSTATUS flag (via
status2proc) a NULL supgid address was theoretically
possible and both ps and top would then show "n/a".
This commit standardizes the following usage:
. PROC_FILLSTATUS (via status2proc)
guarantees a valid supgid address
representing either a true comma
delimited list or "-"
. PROC_FILLCGROUP plus
PROC_EDITCGRPCVT (via fill_cgroup_cvt)
guarantees a cgroup single vector
representing either a true control
group hierarchy or "-"
And as was true before, the following remains true:
PROC_FILLCOM or
PROC_FILLARG (via file2strvec)
may return a NULL cmdline pointer
. PROC_FILLCGROUP (via file2strvec)
may return a NULL cgroup pointer
. PROC_FILLCOM or
PROC_FILLARG plus
PROC_EDITCMDLCVT (via fill_cmdline_cvt)
guarantees a cmdline single vector
representing either a true command
line or a bracketed program name
. PROC_FILLSTATUS plus
PROC_FILLSUPGRP (via supgrps_from_supgids)
guarantees a valid supgrp address
representing either a true comma
delimited list or "-"
Signed-off-by: Jim Warner <james.warner@comcast.net>
Commit a5881b5a4e, addressing
command lines with a trailing space, produced an undesirable
side effect in the fill_cgroup_cvt() function.
This patch restores correct cgroup behavior while still
producing command lines with no trailing space.
When PROC_FILLARG was used (invoking file2strvec)
command lines contained no trailing space.
When PROC_EDITCMDLCVT was used (invoking read_unvectored)
command lines contained a trailing space.
Now both routes to a cmdline act the same -- no trailing space.
The CodingStyle document suggests programs should allow for
cmdlines of at least 128k. Only the ps program can display
such a cmdline, and only with multiple -w switches.
The library function file2strvec can essentially return
a cmdline of unlimited length. However, the library function
fill_cmdline_cvt used an arbitrary upper limit of 2048 for
buffers automatically allocated on the stack.
This patch raises the fill_cmdline_cvt upper limit to 128k via
dymaically acquired utility buffers ensured by the openproc
function.
It also makes indentation consistent in the openproc function.
There were numerous library memory allocation inconsistencies.
Some were checked for failure and others were not.
All library source modules were modified to utilize the alloc.h
memory rouines which are consistent in dealing with errors.
ps
. added tgid support
. made process/task naming consistent
top
. added tgid support
. standardized pid related width logic
documents
. added tgid to ps & top man documents
library
. fixed OOMEM_ENABLE readtask bug
. corrected header typo for tgid
Library changes
readproc
. added readeither to more efficiently
fill both process and thread proc_t
. added readproctab3, uses readeither
. included task path support in:
fill_cgroup_cvt, fill_cmdline_cvt,
read_unvectored
. QUICK_THREADS #define allows copying
process info vs. repeatedly reading
. simple_nexttid no longer values ppid
. path var made non-static in readtask
. documented 'proc_data_t' in .h file
. tweaked some c comments & formatting
library.map
. added new readeither, readproctab3
Program changes
ps
. exploits readproctab3 where possible
. improved args/comm compliance
top
. exploits readeither
