The procfs mount option subset=pid only shows the processes, not things
such as /proc/stat etc.
For certain programs, this should mean they still work, but have reduced
functionality. This is the first cut at some of them.
pgrep - Removed always loading uptime which we never used anyway. The
program now works fine unless we use --older. Add note in man page
stating it will silently fail.
ps - Load boot time and memory total only when required instead of
always. Changed the error messages to something the user actually
cares about "can't get system boot time" vs "create a structure".
Works for most fields except starts and percent memory.
uptime - Give more useful error messages if uptime not available.
vmstat - move header generation after testing for required proc
files, makes the default output more consistent with the rest
of the options.
References:
procps-ng/procps#227https://www.kernel.org/doc/html/latest/filesystems/proc.html#chapter-4-configuring-procfs6814ef2d99
Signed-off-by: Craig Small <csmall@dropbear.xyz>
While the kernel calls the fields pgpgin and pgpgout, the units
here are not pages, but KiB (or 2x 512 sectors).
The comments come from the referenced merged request, this commit fixes
the "vmstat -s lies" part:
https://elixir.bootlin.com/linux/v5.15-rc7/source/block/blk-core.c#L1057
has submit_bio() which includes the count_vm_events(PGPGIN, count) but what
is count? it is usually what bio_sectors() returns.
bio_sectors() is a macro in
https://elixir.bootlin.com/linux/v5.15-rc7/source/include/linux/bio.h#L49
that defines that as bio->bi_iter.bi_size >> 9. 2^9 is 512 or the sector
size. So our count is incremented by the number of 512-byte sectors.
As @dublio has already pointed out before this result is printed to vmstat,
it is /= 2 to give the number of kibibytes (as the sectors were 512 bytes,
we now made the block size 2*512 or 1024). The code even has
"sectors -> kbytes".
So unless there is something very strange going on, pgpgin and pgpgout in
/proc/vmstat return kibibytes.
What about pages (which is sort of implied in the name) or blocks (as
described on the man page)?
Pages can vary, but they are generally 4 KiB so they're out. That also means
vmstat -s lies :(
Blocks are harder to discount. While these too can vary, they can be 1 KiB;
they could also be something else (e.g dd its 512, filesystems 4096).
However, for memory management inside the kernel, there are sectors and
there are (near userland export) KiB, nothing else. It's probably more
accurate to say sectors are shifted in and out of block devices and the
kernel expresses these transfers to userland as KiB by halving the numbers.
What all this means is that using KiB for bi/bo aka pgpgin/pgpgout is more
accurate than saying blocks or pages.
Signed-off-by: Craig Small <csmall@dropbear.xyz>
References:
procps-ng/procps!64
The first report from vmstat provides statistics since system boot. This is
often thrown out. Thus, this provides a command line option to omit it. The
program still provides <count> reports.
Signed-off-by: Sanskriti Sharma <sansharm@redhat.com>
Since 2.6.24/33 the kernel knows about guest and guest nice time. That
is the time that is spend in KVM guests. To handle userspace programs
that do not know about this the guest time is also added to user.
Let us provide a guest time column in vmstat that collects both guest
and guest nice into a gu value.
We also subtract that value from the user time as we are now aware of
the guest value.
This commit is different to !113 in several ways:
* newlib already knows about these to values
* vmstat summary already shows these values
* non-wide vmstat squishes the values
So its around the wide vmstat output.
References:
procps-ng/procps!113
Signed-off-by: Craig Small <csmall@dropbear.xyz>
With the 4 header files removed in the previous patch,
this commit just changes all those obsolete references
to that new consolidated 'misc.h' header file instead.
Signed-off-by: Jim Warner <james.warner@comcast.net>
I'm not sure why, but the make check will now fail for vmstat
Running ./vmstat.test/vmstat.exp ...
FAIL: vmstat disk information (-d option)
With the _new function returning the error.
In vmstat all other structures are set to NULL before calling _new
except the diskstat ones. This has been corrected.
There was a time when that procps.h file served a more
traditional role. Prior to the commit referenced below
it held just macros plus manifest constants. But, with
that change, such items were replaced with a series of
includes embracing all the library exported functions.
That approach was known to disguise errors which would
have otherwise yielded a compiler warning. And without
such a warning, there was no way to address the error.
So this patch will trade the all inclusive header file
approach for individual includes only where necessary.
Reference(s):
. April 2016, procps.h header file revamped
commit ccb6ae8de1
. Sept 2018, top abandoned use of procps.h
commit a6dfc2382e
Signed-off-by: Jim Warner <james.warner@comcast.net>
Reverted the translation hint as the fields are used for both
normal and wide modes, so need to fit the smaller of the two.
Added NEWS item
References:
commit 01c1b2345e
2 digits are not enough for number of runnable/blocked processes on modern
systems. Changed to 4 digits with given the -w option.
References:
procps-ng/procps!48
This does not happen with the default string (" -----timestamp-----"),
but this string is translated (to unknown lengths).
Signed-off-by: Craig Small <csmall@enc.com.au>
Otherwise it leads to NULL-pointer dereferences (in case of localtime()
errors) and indeterminate contents of timebuf (in case of strftime()
errors).
Signed-off-by: Craig Small <csmall@enc.com.au>
Otherwise this may read out-of-bounds (there is no guarantee that 5
bytes are actually available at partition/optarg).
Signed-off-by: Craig Small <csmall@enc.com.au>
In the commit referenced below, we lost several format
string qualifiers which produced several new warnings.
Reference(s):
commit ccbd818cb4
Signed-off-by: Jim Warner <james.warner@comcast.net>
The disk partition format translation hint, the actual text and the
printf statements were all diagreeing with each other. This meant
the disk partition output from vmstat was a mess.
Now all the titles and values line up and everything is right-aligned
which looks better than some half-hearted attempt at center-aligned
for titles and right-aligned for values.
Thanks to @goeran for the heads-up in #69
That preceding commit corrected a little mismanagement
regarding the MEMINFO_MEM_CACHED enumerator, which was
used in the following programs: free, top plus vmstat.
This patch simply adapts those programs to use the new
MEMINFO_MEM_CACHED_ALL enumerator, which reflects both
the 'Cached' plus 'SReclaimable' values they expected.
Signed-off-by: Jim Warner <james.warner@comcast.net>
These 2 programs accessed newlib stacks directly which
meant incorrect result type specifications couldn't be
detected using our new result type validation feature.
And, while the usage was correct, to put each on a par
with all of our other programs, they now rely on those
newlib offered VAL macros for accessing stack results.
[ the ps and top programs retain direct stack access ]
[ when assignment to some result struct is necessary ]
[ PIDS_extra is used by top to store the forest view ]
[ level, while ps uses it for cooked cpu percentages ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch is a response to errors found in those type
references now that our library allows for validation.
In two cases, former assignments to a result structure
could no longer employ that VAL macro if validation is
active. Thus, direct reference to some stack was used.
For the record, those instances were to be found here:
. ps - uses PIDS_extra to store the cooked pcpu values
. top - uses PIDS_extra to store the forest view level
Signed-off-by: Jim Warner <james.warner@comcast.net>
Since the VAL macro now requires a 4th parameter, this
commit simply adds the 'info' context structure to it.
In some cases, that context structure needed to become
global, since it was referenced in multiple functions.
Signed-off-by: Jim Warner <james.warner@comcast.net>
. a more appropriate error return code was substituted
. a safer form of comparison is utilized in two places
. a STAT_VAL macro replaced by more proper MEMINFO_VAL
. several of the silly 'xerrx' formats had been missed
. a few whitespace changes have also been incorporated
Signed-off-by: Jim Warner <james.warner@comcast.net>
With this patch we've completed a progression toward a
standard approach to naming conventions which follows:
* Only functions will begin with that 'procps_' prefix
. ........................................... examples
. procps_vmstat_get ()
. procps_diskstats_select ()
- ----------------------------------------------------
* Exposed structures begin with the module/header name
. ........................................... examples
. struct pids_info
. struct stat_reaped
- ----------------------------------------------------
* Item enumerators begin like structs, but capitalized
. ........................................... examples
. VMSTAT_COMPACT_FAIL
. MEMINFO_DELTA_ACTIVE
[ slabinfo varies slightly due to some item variants ]
. SLABINFO_extra
. SLABS_SIZE_ACTIVE
. SLABNODE_OBJS_PER_SLAB
[ could cure with a prefix of SLABINFO, but too long ]
- ----------------------------------------------------
* Other enumerators work exactly like item enumerators
. ........................................... examples
. PIDS_SORT_ASCEND
. STAT_REAP_CPUS_AND_NODES
- ----------------------------------------------------
* Macros and constants begin just like the enumerators
. ........................................... examples
. #define SLABINFO_GET
. #define DISKSTATS_TYPE_DISK
- ----------------------------------------------------
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch is a response to changes to the <diskstats>
interface. And the following represents the summary of
significant unrelated alterations that were also made.
+ corrected that 'milli weighted IO' output, which has
been wrong since that original patch referenced below.
as implemented, it duplicated 'milli spent IO' output.
+ restored original commit intent regarding disks with
a partition switch as represented in references below.
+ moved all item enumerators up near the source start.
+ removed all remaining tabs and inconsistent indents.
+ reformatted the silly style applied to 'xerrx' uses.
Reference(s):
. disallowed 'disks' under 'partition' switch
commit 7df7795b92
. original commit with disk/partition rational
commit e445f7e6c5
Signed-off-by: Jim Warner <james.warner@comcast.net>
Emulating the presumed proper behavior of the original
program, when the -p switch is used we will now report
an error if that provided name matches some disk name.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Back in July of last year, when vmstat was modified to
exploit the 2nd gen <slabinfo> alloc & sort provision,
yours truly introduced this bug (in the commit below).
Reference(s):
commit 8d9612f782
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit just eliminates all the procps_vmstat_read
calls since the interface now handles that requirement
as needed when the 'get' and 'select' guys are called.
Signed-off-by: Jim Warner <james.warner@comcast.net>
When the API changed for the use_format the type changed for the
system variables out of stat, but the printf format was not changed
along with it.
vmstat.c:868:14: warning: format ‘%lld’ expects argument of type ‘long
long int’, but argument 2 has type ‘long unsigned int’ [-Wformat=]
printf(_("%13lld interrupts\n"), SYSv(sstat_INT));
References:
commit 2e6435a39d
Signed-off-by: Craig Small <csmall@dropbear.xyz>
While new_format was using the new stat select feature the
use_format did not. This type of format now uses the select setup
for both the meminfo and stat set of information.
In view of Craig's work with the vmstat <STAT> change,
adaptation to <MEMINFO> changes is offered separately.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch brings the ps program (very small impact) &
that vmstat program (major impact) into agreement with
the new <stat> API. In the case of the latter guy, the
many separate calls to 'get' should be replaced by one
single 'select' call obtaining all data in one stroke.
[ but, i was too tired to undertake that enhancement ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The includes used to define a lot of things a library include
should not. It was also a bit messy what was exposed in the library
and what was not.
get_pid_digits -> procps_pid_length and exported correctly
MALLOC attribute move into relevant .c files
NORETURN attribute moved to relevant .c, not used in library
PURE attribute removed, it wasn't used
KLONG/KLF/STRTOUKL were fixed for long, so now just use long
HIDDEN attribute removed. It was for 3 functions. The PROCPS_EXPORT
seems to do the same (opposite) thing.
likely/unlikely removed from most places, its highly debateable
this does anything useful as CPUs have gotten smarter about branches.
Re-arranged the includes, ALL external programs should just #include
<proc/procps.h> then proc/procps.h includes headers for files that
have exported functions. procps.h and the headers it includes should
not use items that are not exportable (e.g. hidden functions or
macros) they go in procps-private.h
[ but most definitely not all of them by a long shot ]
Reference(s):
proc/diskstat.c:186:17: warning: unused variable 'is_disk' [-Wunused-variable]
int retval, is_disk;
^
proc/namespace.c:110:1: warning: control may reach end of non-void function [-Wreturn-type]
}
^
proc/readproc.c:1131:50: warning: address of array 'ent->d_name' will always evaluate to 'true' [-Wpointer-bo
if(unlikely(unlikely(!ent) || unlikely(!ent->d_name))) return 0;
~~~~~~^~~~~~
proc/readproc.c:1158:50: warning: address of array 'ent->d_name' will always evaluate to 'true' [-Wpointer-bo
if(unlikely(unlikely(!ent) || unlikely(!ent->d_name))) return 0;
~~~~~~^~~~~~
proc/sysinfo.c:45:12: warning: unused variable 'stat_fd' [-Wunused-variable]
static int stat_fd = -1;
^
proc/sysinfo.c:49:12: warning: unused variable 'meminfo_fd' [-Wunused-variable]
static int meminfo_fd = -1;
^
proc/sysinfo.c:51:12: warning: unused variable 'vminfo_fd' [-Wunused-variable]
static int vminfo_fd = -1;
^
proc/sysinfo.c:53:12: warning: unused variable 'vm_min_free_fd' [-Wunused-variable]
static int vm_min_free_fd = -1;
^
proc/uptime.c:157:12: warning: unused variable 'realseconds' [-Wunused-variable]
time_t realseconds;
^
proc/uptime.c:158:16: warning: unused variable 'realtime' [-Wunused-variable]
struct tm *realtime;
^
vmstat.c:574:20: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat]
DSTAT(PROCPS_DISKSTAT_READ_TIME),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vmstat.c:578:20: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat]
DSTAT(PROCPS_DISKSTAT_WRITE_TIME),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
watch.c:230:7: warning: variable 'endptr' is uninitialized when used here [-Wuninitialized]
if (*endptr == '\0') set_ansi_attribute(0); /* [m treated as [0m */
^~~~~~
Signed-off-by: Jim Warner <james.warner@comcast.net>
With the new perspective on potential uses of a 'noop'
enumerator (or whatever we decide to call it) there is
no longer a need to provide for any extra 'user' space
in the stack header structures used by slab & meminfo.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In addition to that text shown below the line which is
common to several commit messages, this patch contains
several minor changes with lessor impact upon the API:
. A call to procps_stat_read_jiffs() has been added to
those jiffs functions carrying the 'fill' nomenclature
to parallel like functions in some of our other files.
. The #include header files are ordered alphabetically
now, with all those <sys/??> types separately grouped.
. Standard copyright boilerplate was added in .c file.
. The header file follows the conventions of indenting
(by 4 spaces) those parameters too lengthy for 1 line.
------------------------------------------------------
. The former 'chains' have now become 'stacks' without
the 'next' pointer in each result struct. The pointers
initially seemed to offer some flexibility with memory
allocations and benefits for the library access logic.
However, user access was always via displacement and a
a statically allocated chain was cumbersome to define.
. An enumerator ending in '_noop' will no longer serve
as a fencepost delimiter. Rather, it has become a much
more important and flexible user oriented tool. Adding
one or more such 'items' in any items list passed into
the library becomes the means of extending the 'stack'
to also include user (not just library) data. Any such
data is guaranteed to never be altered by the library.
. Anticipating PID support, where many different types
must be represented in a result structure, we'll adopt
a common naming standard. And, while not every results
structure currently needs to reflect disparate types a
union will be employed so the same dot qualifier ('.')
can be used consistently when accessing all such data.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In addition to that text shown below the line which is
common to several commit messages, this patch contains
several minor changes with lessor impact upon the API:
. A 'read' was added to function procps_slabnode_count
(but only when necessary, i.e. info->nodes_used == 0).
. The #include header files are ordered alphabetically
now, with all those <sys/??> types separately grouped.
------------------------------------------------------
. The former 'chains' have now become 'stacks' without
the 'next' pointer in each result struct. The pointers
initially seemed to offer some flexibility with memory
allocations and benefits for the library access logic.
However, user access was always via displacement and a
a statically allocated chain was cumbersome to define.
. An enumerator ending in '_noop' will no longer serve
as a fencepost delimiter. Rather, it has become a much
more important and flexible user oriented tool. Adding
one or more such 'items' in any items list passed into
the library becomes the means of extending the 'stack'
to also include user (not just library) data. Any such
data is guaranteed to never be altered by the library.
. Anticipating PID support, where many different types
must be represented in a result structure, we'll adopt
a common naming standard. And, while not every results
structure currently needs to reflect disparate types a
union will be employed so the same dot qualifier ('.')
can be used consistently when accessing all such data.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In addition to exploiting the new interface, a warning
for a meminfo variable 'used uninitialized' was fixed.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This was Craig's original patch, referenced below, but
it was never pushed to newlib. It has now been rebased
on top of some diskstat stuff to serve as a beginning.
The original effort was perfectly serviceable (after a
memory leak was fixed) but the approach would not have
served future PID needs when that proc_t went bye bye.
The slabtop requirements are similar to those of PIDs.
One must accommodate the variable number of slab nodes
(PIDs) while also accepting different data (char * and
unsigned long). Furthermore, some generalized means to
sort all that acquired stuff must somehow be provided.
So I wanted to try a different approach that seemed to
hold potential for satisfying future top and ps needs.
Subsequent commits will make that attempt, building on
Craig's original patch whose commit msg appears below.
------------------------------------------------------
All of the /proc/slabinfo related calls have been changed
here. They follow the same procps_slabinfo_* format.
Made both the slabtop and vmstat programs use the new
API as one was using the old one and one was just sort
of trying to do its own thing.
Sorting of slabnodes is also possible via the library.
Reference(s):
http://www.freelists.org/post/procps/Sorting-slabsprocesses,3http://www.freelists.org/post/procps/library-rework-slabinfo-calls
Signed-off-by: Craig Small <csmall@enc.com.au>
Signed-off-by: Jim Warner <james.warner@comcast.net>
The calls for reading diskstat have been moved out of
sysinfo and into new files diskstat.[ch] These new
library calls follow the standard pattern for the
new libprocps.
vmstat is updated to use the new API and also got
the weighted IO time added.
vmstat -p previously would only show partitions, not
disks. There does not appear to be any good reason to
artifically deny a user to use this command on a disk,
rather than a partition so this restriction was lifted.
I also realised using int for devid means you can send
the library negative numbers, the index uses unsigned int.
Other similiar calls will need to be fixed too.
Signed-off-by: Craig Small <csmall@enc.com.au>
The new library meminfo & vmstat modules use structure
names for their context which exactly mirror the names
of the very /proc/ files whose particulars they yield.
The one exception to this rule was the readstat module
whose struct was named statinfo yet the file was stat.
This commit simply renames that structure (only) so as
to hopefully establish such a naming convention as our
standard going forward. And, it's makes good symmetry.
[ this module's name itself is just perfect as it is ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch mostly just eliminates darn tab characters.
Plus the library function declarations and definitions
have been standardized. Most visibly, the input params
now have all been indented on their own separate line.
The following names were changed to more closely match
meminfo.c or provide a certain symmetry. Unfortunately
that also impacted some other pgms which were updated.
. 'procps_stat_get' evolved into 'procps_stat_get_sys'
. 'procps_stat_info' is now known as 'procps_statinfo'
[and just a little trailing whitespace was eliminated]
Signed-off-by: Jim Warner <james.warner@comcast.net>
