Commit Graph

2727 Commits

Author SHA1 Message Date
Jim Warner
b2c4fcfddb library: reposition those 'info' structures in headers
For some unknown reason all the 'info' structures were
declared between macros and function prototypes rather
than right after all the other structure declarations.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-12-16 20:50:07 +11:00
Jim Warner
957b74292f ps,top: convert 'PIDS_PROCESSOR' into a signed integer
Not only does that library tweak help to simplify some
top code, but now that ps snprintf fmtstr will finally
be accurate. That is two birdies with a single pebble!

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-12-16 20:50:07 +11:00
Jim Warner
aeb35b18da library: change 'PIDS_PROCESSOR' into a signed integer
This change is really being made on behalf of the ps &
top programs. Besides, over 2 billion CPUs are plenty!

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-12-16 20:50:07 +11:00
Craig Small
bcb837b8c7 First cut at subset=pid proc mount handling
The procfs mount option subset=pid only shows the processes, not things
such as /proc/stat etc.

For certain programs, this should mean they still work, but have reduced
functionality. This is the first cut at some of them.

pgrep - Removed always loading uptime which we never used anyway. The
program now works fine unless we use --older. Add note in man page
stating it will silently fail.

ps - Load boot time and memory total only when required instead of
always. Changed the error messages to something the user actually
cares about "can't get system boot time" vs "create a structure".
Works for most fields except starts and percent memory.

uptime - Give more useful error messages if uptime not available.

vmstat - move header generation after testing for required proc
files, makes the default output more consistent with the rest
of the options.

References:
 procps-ng/procps#227
 https://www.kernel.org/doc/html/latest/filesystems/proc.html#chapter-4-configuring-procfs
 6814ef2d99

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-12-16 20:36:00 +11:00
Craig Small
dae897b54d vmstat: Use KiB instead of pages for paged in/out
While the kernel calls the fields pgpgin and pgpgout, the units
here are not pages, but KiB (or 2x 512 sectors).

The comments come from the referenced merged request, this commit fixes
the "vmstat -s lies" part:

https://elixir.bootlin.com/linux/v5.15-rc7/source/block/blk-core.c#L1057
has submit_bio() which includes the count_vm_events(PGPGIN, count) but what
is count? it is usually what bio_sectors() returns.

bio_sectors() is a macro in
https://elixir.bootlin.com/linux/v5.15-rc7/source/include/linux/bio.h#L49
that defines that as bio->bi_iter.bi_size >> 9. 2^9 is 512 or the sector
size. So our count is incremented by the number of 512-byte sectors.

As @dublio has already pointed out before this result is printed to vmstat,
it is /= 2 to give the number of kibibytes (as the sectors were 512 bytes,
we now made the block size 2*512 or 1024). The code even has
"sectors -> kbytes".

So unless there is something very strange going on, pgpgin and pgpgout in
/proc/vmstat return kibibytes.

What about pages (which is sort of implied in the name) or blocks (as
described on the man page)?

Pages can vary, but they are generally 4 KiB so they're out. That also means
vmstat -s lies :(

Blocks are harder to discount. While these too can vary, they can be 1 KiB;
they could also be something else (e.g dd its 512, filesystems 4096).
However, for memory management inside the kernel, there are sectors and
there are (near userland export) KiB, nothing else. It's probably more
accurate to say sectors are shifted in and out of block devices and the
kernel expresses these transfers to userland as KiB by halving the numbers.

What all this means is that using KiB for bi/bo aka pgpgin/pgpgout is more
accurate than saying blocks or pages.

Signed-off-by: Craig Small <csmall@dropbear.xyz>

References:
 procps-ng/procps!64
2021-11-14 18:50:58 +11:00
Weiping Zhang
992a37e5ed vmstat: use KiB/s for bi/bo instead of blocks/s
/proc/vmstat provide kbytes to pgpgin and pgpgout instead of blocks,
correct unit for bi/bo.

References:
 procps-ng/procps!64

Signed-off-by: Weiping Zhang <zhangweiping@didichuxing.com>
Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-11-14 18:48:12 +11:00
Jim Warner
eafd8e3112 library: refine support for multiple concurrent access
Our new library's now well protected against potential
problems which arise when a multi-threaded application
opens more than one context within the same API at the
same time. However, with a single-threaded application
designed along those same lines, some problems remain.

So, to avoid potential corruption of some data (which
was classified as local 'static __thread') from those
single-threaded designs, we'll move several variables
to the info structure itself and remove the '__thread'
qualifier. Now they're protected against both designs.

[ we'll not be protected against some multi-threaded ]
[ application that shares a single context yet calls ]
[ that interface from separate threads. this is just ]
[ bad application design & no different than sharing ]
[ other modifiable global data between such threads! ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-11-14 16:23:07 +11:00
Jim Warner
126b14470e library: this is why we hate those darn tab characters
Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-11-14 16:23:07 +11:00
Craig Small
10824b0655 lib: Initialise uptime variables
upminutes and uphours could both not get initialised in the
procps_uptime_sprint_short() function.

Error was probably introduced at the referenced commit.

References:
 Coverity 240787 Uninitialized scalar variable
 Coverity 240776 Uninitialized scalar variable
 commit 0496b39876

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-11-02 17:23:37 +11:00
Craig Small
2e0e865ca5 pgrep: Fix format security
do_regcomp() error message didn't use a string literal for the format string.

pgrep.c:538:24: error: format not a string literal and no format arguments
[-Werror=format-security]

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-11-02 17:19:37 +11:00
Craig Small
f3e5290012 pmap: minor Coverity fix for -N option
99126 Explicit null dereferenced
Not 100% sure this is valid (the same branch that sets the variable
is the one that sets N_option) but not too hard to fix.

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-11-02 17:14:29 +11:00
Jim Warner
ad51fef1aa top: tweak some end-of-job logic when separate threads
The separate threads for background updates were added
to top in the commit shown below. At that time cleanup
logic was added to end-of-job processing to cancel any
active threads and destroy any semaphores then in use.

That seemed like simple good stewardship with an added
benefit of avoiding potential valgrind 'possibly lost'
warnings for 320 byte blocks. Those blocks represented
an initial stack allocation for each of three threads.

All of that worked perfectly if such code was executed
under the main thread. In other words, if the keyboard
or a signal directed to any thread was used to trigger
program end. However, it might not always be the case.

Each of those 'refresh' routines supporting a separate
thread interacts with a newlib interface. As a result,
each is required to check the library's return result.
Should some error be detected, 'error_exit' is called.
Now we've got big problems with that eoj cleanup code.

One can't 'pthread_cancel' and 'pthread_join' a thread
from withing that same thread. Thus, when an error was
returned by the library with threads active, top would
hang with no possibility of removal short of a reboot.

So, this commit only executes that cancel/join cleanup
code when we are running under the main thread. Should
program end be triggered by a library error under some
sibling thread, all such cleanup will now be bypassed.
In the latter case, we will rely on documentation that
says any thread calling exit(3) will end every thread.

[ now, the only time we'll see any valgrind warnings ]
[ is with a library error, which is the least likely ]
[ scenario for running valgrind & top to begin with. ]

[ besides, if the valgrind warnings became a problem ]
[ one could easily add a 'warning-suppression' file. ]

Reference(s):
. Sep 2021, top introduced threads
commit 29f0a674a8

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-31 17:12:32 +11:00
Jim Warner
3e43e6273e library: remedy several 'coverity scan' resource leaks
This commit will place the 'file2strvec' function on a
par with the 'file2str' function if ENOMEM was raised.

Plus, just to keep coverity happy, we'll also clean up
after potential failures with the 'openproc' function.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-28 07:06:57 +11:00
Jim Warner
dfcdc0c2c0 library: oh no, failed to right-justify 1 comment line
Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-28 07:06:57 +11:00
Craig Small
4cfb0fb763 pgrep: Match on cgroup v2 paths
You can match or filter on cgroup paths. Currently the match is only
done for version 2 cgroups because these are way simpler as they have
a unified name and always start with "0::".

cgroup v1 can have:
 named groups "1:name=myspecialname:"
 controllers "9:blkio:"
 multiple controllers! "4:cpu,cpuacct:"

So they are very much more complicated from a options parsing and
cgroup matching point of view.

In addition, both my Debian bookworm and bullseye systems use
v2 cgroups.

$ ./pgrep --cgroup /system.slice/cron.service
760

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-10-26 20:56:19 +11:00
Craig Small
14d6ab27d6 NEWS: Add entry for free -h --si fix
References:
 commit 91e5879228
 Issues #133 #223
 merge !140
2021-10-26 18:25:37 +11:00
Todd Lewis
e500ba6d97 fix uid/gid > 2^31
This MR revisits a partial fix from 2018. The problem stems from incorrect
handling of unsigned 32-bit uid_ts and gid_ts as signed when values are
large - i.e. when the high bit is set. In that case, pgrep and pkill fail to
identify processes by uid. (They succeed when finding the same processes by
username.) The primary fix for this is to impliment the "FIXME" comment in
proc/readproc.h, the implementation of which allows the removal of the (int)
casts from the partial fix from 2018.

The other fixed code in this MR consists of tests in strict_atol() that
detects and errors out on overflows.

References:
 Merge !146
2021-10-26 18:13:48 +11:00
Jim Warner
2f293dbf74 library: expand warnings in 'warning-suppression' file
With the addition of more '__thread' attributes in the
previous commit, additional valgrind warnings might be
encountered if developing multi-threaded applications.

So, this patch expands the libproc.supp file which was
originally introduced with the patch referenced below.

Reference(s):
commit be1ddc2756

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-25 21:51:18 +11:00
Jim Warner
d6e6722fd6 library: extend thread safety to more static variables
In the commit referenced below, a '__thread' attribute
was added to numerous static variables to protect them
from concurrent access conflicts with multi-threading.

Unfortunately, that patch did not go quite far enough.

So, this commit adds a few more '__thread' qualifiers.

Reference(s):
commit 23cfb71366

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-25 21:51:18 +11:00
Jim Warner
b652c35f7f doc: added the new valgrind 'warning-suppression' file
Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-18 17:38:43 +11:00
Jim Warner
be1ddc2756 library: created a valgrind 'warning-suppression' file
Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-18 17:38:43 +11:00
Jim Warner
5d09875488 top: only use 'pthread_sigmask' under separate threads
When multi-threading was introduced in the patch shown
below, the former calls to sigprocmask were traded for
a pthread_sigmask call. This was done unconditionally.

As a result, even when those threads weren't enabled a
need to link with libpthread was created. In hindsight
the need should only arise when top is multi-threaded.

This commit will make pthread_sigmask use conditional.

Reference(s):
. 09/2021, separate threads introduced
commit 29f0a674a8

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-18 17:38:43 +11:00
ed neville
0496b39876 uptime: Correctly print pretty/short format
uptime -p would show empty output after 52 weeks of uptime. This commit
is largely the work of Ed but reformatted for newlib branch.

Signed-off-by: Craig Small <csmall@dropbear.xyz>

References:
 procps-ng/procps!141
 procps-ng/procps#217
2021-10-14 19:37:42 +11:00
Craig Small
5f760d5085 vmstat: Add NEWS and man page for previous 2021-10-14 18:52:34 +11:00
Sanskriti Sharma
528158ed77 vmstat: Provides option to omit first report
The first report from vmstat provides statistics since system boot. This is
often thrown out. Thus, this provides a command line option to omit it. The
program still provides <count> reports.

Signed-off-by: Sanskriti Sharma <sansharm@redhat.com>
2021-10-14 18:50:37 +11:00
林博仁(Buo-ren, Lin)
7a6437ce42 misc: Fix typo
Signed-off-by: 林博仁(Buo-ren, Lin) <Buo.Ren.Lin@gmail.com>
2021-10-14 07:57:27 +11:00
Craig Small
db861f2ce6 misc: Add NEWS for ps ignoring SIGURG
References:
 commit b6ccf865f8
2021-10-14 07:52:43 +11:00
Tommi Rantala
b6ccf865f8 ps: ignore SIGURG
Stop registering signal handler for SIGURG, to avoid ps failure if
someone sends such signal. Without the signal handler, SIGURG will
just be ignored.

  Signal 23 (URG) caught by ps (3.3.16).
  ps:ps/display.c:66: please report this bug

https://man7.org/linux/man-pages/man7/signal.7.html
https://www.freebsd.org/cgi/man.cgi?sektion=3&query=signal
2021-10-14 07:50:37 +11:00
Rafael Kitover
3f2d7a0bf6 Remove autogen.sh libtool prog check #222.
Remove the check for the libtool executable as many libtool distribution
packages do not have it and it is not necessary for building, as
libtoolize is also checked for to determine the presence of libtool.

Signed-off-by: Rafael Kitover <rkitover@gmail.com>
2021-10-13 20:38:22 +00:00
Craig Small
d39d9db079 testsuite: Make test program compile on Hurd
procps fails to build from source due to usage of field si_int of struct
siginfo_t in lib/test_process.c which does not exist on GNU/Hurd.

Thanks to Svante Signell for the patch.

References:
 https://bugs.debian.org/987557

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-10-13 07:56:39 +11:00
Jim Warner
67c150ed7c top: restore ending ']' when summary graphs are scaled
When a visual separator was added to 2 abreast summary
items in a recent commit, this bug was introduced. So,
from that earlier patch we'll revert one line of code.

The bug surfaced under an 80/even column terminal only
when that '4' toggle was off. With an an 81/odd column
screen, it existed in both single and 2 abreast modes.

[ this commit also goes the extra distance to ensure ]
[ two abreast mode maximizes available screen width. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-08 18:30:44 +11:00
Jim Warner
a5b51bc417 top: revert part of a patch regarding separate threads
In the patch referenced below, some logic was moved in
that frame_make function in an effort to overlap those
background threads with additional main thread duties.

Unfortunately, the auto-size column feature was broken
in the process. So this patch will revert a portion of
the earlier patch so as to restore the broken feature.

Reference(s):
. Sep 2021, top overlap additional processing
commit f11f43b4f7

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-02 13:00:00 +10:00
Jim Warner
ca30741a30 library: ensure thread safety via function substitutes
Even though we we had to abandon the master branch top
multi-thread effort and even though the newlib version
of a multi-threaded top provides no real benefit, that
whole exercise was not wasted. Rather, it has revealed
some deficiencies in our library which this addresses.

If two or more threads in the same address space tried
to use procps_loadavg or procps_uptime simultaneously,
there's a chance they would experience problems due to
thread-unsafe functions our library called internally.

So, this patch switches them for thread-safe versions.

[ along the way we will also make that procps_uptime ]
[ initialization of his 'up' & 'idle' variables mean ]
[ something by delaying the -ERANGE return a little. ]

Reference(s):
https://www.freelists.org/post/procps/a-few-more-patches,7

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-02 12:55:31 +10:00
Jim Warner
23cfb71366 library: ensure thread safety for all static variables
Even though we we had to abandon the master branch top
multi-thread effort and even though the newlib version
of a multi-threaded top provides no real benefit, that
whole exercise was not wasted. Rather, it has revealed
some deficiencies in our library which this addresses.

If two or more threads in the same address space tried
to access the same api simultaneously, there is a good
chance some function-local static variables will yield
some of those renowned unpredictable results. So, this
patch protects them with the '__thread' storage class.

Reference(s):
https://www.freelists.org/post/procps/a-few-more-patches,7

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-10-02 12:55:31 +10:00
Jim Warner
69978e3650 top: ensure those potential focused tasks stay focused
When that 'F' focus command has been applied to a task
in forest view it should remain as the topmost process
in a particular window. But without this patch that is
not guaranteed. Newly forked/cloned tasks 'above' such
a process result in task(s) appearing which shouldn't.

The effect was as if that up arrow key scrolled beyond
the topmost parent task, which would never be allowed.

[ since scrolling is permitted within a focus range, ]
[ when any task 'above' our focus/topmost task ends, ]
[ we respond as if scrolled with the down arrow key. ]

[ that result is completely appropriate. if the user ]
[ wishes to return to a focused parent, the up arrow ]
[ or home key can be used to accomplish such a goal. ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-09-26 14:47:21 +10:00
Jim Warner
f11f43b4f7 top: overlap some more processing for separate threads
In an effort to improve the performance & decrease the
cost of our separate background update threads we will
relocate some overhead so that it might be overlapped.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-09-26 14:47:21 +10:00
Jim Warner
d89047659c top: modify semaphore destruction for separate threads
This commit just ensures that at program end all those
semaphores are removed in the same order for symmetry.
From thread's view of importance, we do least to most.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-09-26 14:47:21 +10:00
Jim Warner
476327048c top: reverse the #define defaults for separate threads
Rather than having those 3 separate background threads
enabled by default, we'll turn them off until somebody
chooses to activate them. That seems more appropriate.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-09-20 20:05:44 +10:00
Jim Warner
29f0a674a8 top: introduce background updates via separate threads
After the stage had been set in the previous patch, in
this patch we will actually implement those background
updates via 3 separate threads. The design was simple:

. the do-while loops have now been made truly infinite
. 2 semaphores per thread allow needed synchronization
. 1 semaphore will provide for each thread to sem_wait
. 1 semaphore will provide for display o/p to sem_wait
. and all 3 thread's program name was made descriptive

A complication was the potential for a signal directed
to one of our new threads. Rather than having a thread
try to deal with such signals, we pass a mask with all
signals blocked at pthread_create time. Thereafter any
subsequent signals are forwarded to the parent thread.

[ also sigprocmask was exchanged for pthread_sigmask ]
[ since warned about use "in multithreaded process". ]

[ plus we also modified each of those POSIX comments ]
[ about 2004 to agree with current signal-safety(7). ]

Sadly, after all this effort there were no performance
benefits to having separate threads. In fact there was
a measurable performance degradation when running with
ever smaller delay intervals. But even with a delay of
1/10 second the 'real' cost increase is only about 1%.

There is one way whereby any additional costs might be
eliminated (at least seemingly). One could introduce 2
separate sets of contexts for each of those 3 threads.
Then retrieval & display could be overlapped. However,
the resulting display wouldn't represent the real-time
results. Rather it would be stale by 1 delay interval.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-09-20 20:05:44 +10:00
Jim Warner
b4b68f11d6 top: refactor 3 routines ahead of the separate threads
This commit sets the stage for turning three functions
into code that can support a separate thread to update
their respective data while working in the background.

It involved relocating 1 function, renaming 2 routines
and adding parameters plus return addresses to each of
three functions. Those latter changes will be required
when issuing 'pthread_create' calls in the next patch.

The final step was organizing this code into what will
become the infinite do-while loop supporting a thread.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-09-20 20:05:44 +10:00
Jim Warner
1d1c1b84df top: tweak end-of-job processing if invoked via signal
The master branch top is pretty well insulated against
signal-unsafe function calls during end-of-job action.

However, this newlib branch top had exposed himself to
a bunch of such unsafe activity in the form of 'unref'
function calls to the new library. In those guys there
will be several 'free' invocations which are not safe.

So, this commit will simply correct such an oversight.

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-09-20 20:05:44 +10:00
Jim Warner
6049c13b4d top: justify those header #define description comments
This commit is an example of what can happen when this
author is waiting around for the release of our newlib
& otherwise runs out of legitimate changes to be made.

[ on the other hand, such changes surely make us run ]
[ quite a bit more efficiently, if i am not mistaken ]

Signed-off-by: Jim Warner <james.warner@comcast.net>
2021-09-20 20:05:44 +10:00
Craig Small
75ba637182 docs: Add note in sysctl.conf.5 for line length
References:
 procps-ng/procps#77
2021-09-15 21:45:26 +10:00
Craig Small
fdab991bb2 docs: Fix comment for priority
Adjust the pri field for ps.1 because a higher pri field means a
higher priority.

References:
 procps-ng/procps#115
2021-09-15 21:36:40 +10:00
Jan Rybar
08aa15ce4b sysctl: Return non-zero if param setting failed
Real usecase:
'sysctl -w user.max_uts_namespaces=2147483648; echo $?'
returns 0 even though it failed with EINVAL

The close_stream() realised there was an issue and printed an
error but didn't change the return value.

Slightly modified merge request.

References:
 procps-ng/procps!76

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-09-15 21:07:43 +10:00
Craig Small
e886010ef7 misc: Add NEWS for previous
References:
 procps-ng/procps!91
2021-09-15 20:54:30 +10:00
Tom Levy
2737850e66 pgrep: Add a newline after regex error messages
The message from 'regerror' does not include a newline.

Test with `pgrep '*'`.

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-09-15 20:53:12 +10:00
Craig Small
4706f225c9 testsuite: Add sysctl test for directory checks
Adds a test to try to write a sysctl parameter above /proc
and passes if it gives an error message.

References:
 commit f25d462166
2021-09-15 20:16:06 +10:00
Craig Small
2a7c909a1b sysctl: Update previous patch for newlib
This change was cherry picked from old library branch but
missed the change to check the path.

References:
 commit 6389deca5b
2021-09-15 20:09:44 +10:00
Craig Small
6389deca5b sysctl: Support systemd glob patterns
systemd-sysctl handles glob patterns along with overrides and
exceptions. Now the procps sysctl does it too.

The return value for sysctl is consistently either 0 or 1.

Added tests to check sysctl functions.

References:
 procps-ng/procps#191

Signed-off-by: Craig Small <csmall@dropbear.xyz>
2021-09-15 20:07:32 +10:00