2b6ad61f3c
15 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jim Warner
|
6671a3a8b7 |
library: refactored some header file items and origins
This commit is intended as a refinement of the patches
mentioned below, where origins/sources of newlib items
were added to the header files for user documentation.
However, if those additions are to be truly effective,
along with kernel documentation (where available), the
following prerequisites must also have been satisfied:
. our identifiers closely align with linux field names
. our derived items are documented or self-documenting
Satisfying those prerequisites prompted this patch and
for these changes, kernel sources were emphasized over
available documentation (shame on me, it should always
have been so). And, while some 'new' fields were found
to be conditional, they were included unconditionally.
These changes appear more extensive than they actually
need be since I have attempted to enforce some spacing
conventions. So, I've summarize the significant things
in the sections that follow. For a proper perspective,
use: 'git diff --ignore-space-change' (good as alias).
___________________________________________ <PIDS> api
This api is unique in that there exists many different
file/directory origins subordinate to /proc/<pid>. And
our item identifiers are sometimes coerced so as to be
able to group related or similar enumerators together.
So, users needed more help relating our identifiers to
an actual documented field. Thus, we will now also add
the field names as with 'stat: delayacct_blkio_ticks'.
Each item ending with a '_C' now consistently includes
both the parent's count/time plus waited for children.
That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT
since its original name is an implementation artifact.
___________________________________________ <STAT> api
The only api change was to correct a typo ('dervied').
_________________________________________ <VMSTAT> api
Even ignoring white space, this interface received the
largest number of changes. Mostly, this was because of
deficiencies in the proc(5) documentation. Recall that
this documentation already sorely lacks any substance.
Usually, just kernel releases are noted, not contents.
When compared to kernel source, that proc(5) contained
many non-existent fields and also omitted many others.
________________________________________ <MEMINFO> api
Sadly, with this api many of the changes were simply a
correction of some earlier 'human error' where several
fields where hashed then tracked but never represented
with an item enumerator in this meminfo.h header file.
_______________________________________ <SLABINFO> api
The 'SLABS' (summary) & 'SLABNODE' items were reversed
since the former are derived from the separate caches.
More significantly, those 'SLABNODE' guys were renamed
to 'SLAB' since they concern individual caches and the
concept of 'nodes' is really an implementation detail.
Also, several enumerators were changed to more closely
agree with official slabinfo(5) documentation referred
to in what we're treating as a base document: proc(5).
Lastly, while those 'SLABS' items are solely a product
of our library and not represented in slabinfo(5), the
names attempt to parallel those found as 'SLAB' items.
______________________________________ <DISKSTATS> api
One enumeration identifier was changed so as to better
reflect its relationship to that actual documentation:
'Documentation/iostats.txt', as referenced in proc(5).
Reference(s):
. 12/2018, item origins added (and commit msg history)
commit
|
||
|
Jim Warner
|
96d59cbf46 |
library: add item origin (as comments) to header files
A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit |
||
|
Jim Warner
|
fab37662ef |
library: refactor the XTRA_PROCPS_DEBUG implementation
If we ever were to eliminate the procps.h header file, as discussed in the thread referenced below, then that would impair the current XTRA_PROCPS_DEBUG provisions. The only remaining way to verify result types would be to explicitly include that <proc/xtra-procps-debug.h>. So, this commit will once again enable the ./configure provision for defining the -DXTRA_PROCPS_DEBUG option. Reference(s): https://www.freelists.org/post/procps/newlib-Qualys-patches,6 Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
b0908eec4b |
library: replace the troublesome '__BEGIN_DECLS' macro
When 'newlib' was introduced, in the commit referenced
below, the use of that glibc '__BEGIN_DECLS' macro was
standardized. However, as issue #88 revealed, this may
result in a fatal build error with other environments.
So, this patch just trades that macro for the standard
'#ifdef __cplusplus' conventions (thus avoiding use of
all those '#include <features.h>' directives as well).
Reference(s):
. newlib introduced
commit
|
||
|
Jim Warner
|
82a0dcda0f |
library: strictly cosmetic, absolutely no code changes
This commit just contains some tweaks to comments plus a few adjustments to whitespace for alignment purposes and a normalization of the header inclusion #define's. [ plus a spelling error in one header file was fixed ] Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
bef8c7fb70 |
library: ensure that all those 'GET' macros are robust
When users call the native 'get' functions they have a responsibility to check that the result struct address was indeed returned. But when using those 'GET' macros there was no protection for possible NULL dereference. So this patch will add some protection for a potential failure of an underlying 'get' function. And should it occur then those 'GET' macros will just return a zero. Plus, we'll also mirror that behavior in the debugging header should the XTRA_PROCPS_DEBUG #define be active. And, we might as well add a warning when invalid items are passed to 'GET' macros, just like we do for 'VAL'. [ lastly, we added the missing opening parens/braces ] [ to 2 'GET' macros in that xtra-procps-debug.h file ] [ which went unnoticed until the qa folks caught up. ] Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
6cafe3abec |
library: expand VAL macros to include the context parm
This patch will set the stage for validating the types referenced in the result union. For now, the parameter representing that 'info' structure will remain unused. [ and while we're at it, let us correct a faulty GET ] [ macro in the diskstats header. that puppy missed a ] [ parm which ain't so good if that guy is ever used! ] Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
c4d097c709 |
library: removed all the 'PROCPS_' enumerator prefixes
Many of our item enumerator identifiers are very long, especially in that <VMSTAT> module. Additionally, they all contain the exact same universal 'PROCPS_' prefix. The origins for this are likely found in the desire to avoid name clashes with other potential include files. But with procps-ng newlib, we've probably gone way too far. Did 'PROCPS_PIDS_TICS_SYSTEM' actually offer more protection against clash than 'PIDS_TICS_SYSTEM' does? I don't think so. Besides, no matter how big that name becomes, one can never guarantee they'll never be some clash. And, conversely, extremely short names will not always create conflict. Of course, in either case when some clash occurs, one can always #undef that problem. Thus, this commit will eliminate that 'PROCPS_' prefix making all of those enum identifiers a little shorter. And, we'll still be well above some ridiculously short (criminally short) names found in some common headers: - - - - - - - - - - <term.h> - 'tab', 'TTY', etc - - - - - - - - - - - - - - - - <search.h> - 'ENTER', ENTRY', 'FIND', etc ------------------------------------------------------ Finally, with this as a last of the wholesale changes, we will have established the naming conventions below: . only functions will begin with that 'procps_' prefix . exposed structures begin with the module/header name . item enumerators begin like structs, but capitalized . other enumerators work exactly like item enumerators . macros and constants begin just like the enumerators ------------------------------------------------------ Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
82d5661603 |
library: standardize all the 'context' structure names
This patch attempts to standardize the naming of those most important (declared not defined) context structs. The present practice represents a hodge podge of names only some of which reflect the source /proc file name. And 2 of those file names embed a literal 'info' which is likely the origin of that required parm identifier. Now we'll append a universal '_info' to such structure names, while including the names of those /proc pseudo files where possible. In any case, that context struct will *always* begin with the actual module/header file name. And only the following two sound a little weird! ---------> 'meminfo_info' + 'slabinfo_info' <--------- Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
e41b40e6b0 |
library: <SLABINFO> api, change that 'get' return type
The following commit message is shared with 4 patches. ------------------------------------------------------ Under the newlib interface most of our 'get' functions represent a bit of a compromise in that the actual raw values are coerced into one (probably ok) return type. That approach creates the possibility of truncation at best, and wouldn't serve future needs should something other than numeric data be added to the 'get' results. This commit trades the current compromise for a return value guaranteed to satisfy all future needs, namely a pointer to a particular api's specific results struct. The impact on existing programs is minimal, especially when using a new supplied macro. Otherwise, native 'C' syntax could be used, but may feel somewhat unnatural. [ as an aside, this new approach allows us to delete ] [ all 'getsfunc' table entries & the supporting code ] Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
4a0e974b7f |
library: more tweaks for code and/or comments, 3rd gen
Following is a summary of significant changes (if any) to each of these now upgraded 3rd gen library modules. <meminfo> ............................................ . eliminated duplicate decl of 'struct procps_meminfo' . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . how did i miss relocating all these friggin' #undefs . cleanup 'get' return logic (remove a redundant 'if') <pids> ............................................... . repositioned the procps_pidsinfo structure in header . removed the extra trailing comma from enum pids_item . standardized/normalized results struct union members <slabinfo> ........................................... . corrected comment typo (jeeze, in an 'aligned' para) . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . removed an obsolete #undef from procps_slabinfo_sort . cleanup 'get' return logic (remove a redundant 'if') <stat> ............................................... . how did i miss relocating all these friggin' #undefs . corrected an initialization fencepost used with numa <=== see Craig, here's a bug fix . removed the extra trailing comma from enum stat_item . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . strengthen those parm checks in procps_stat_get func . cleanup 'get' return logic (remove a redundant 'if') <vmstat> ............................................. . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . cleanup 'get' return logic (remove a redundant 'if') [ virtually all of these tweaks reflect the author's ] [ continuing pursuit of an unreasonable goal -- that ] [ of a 'perfect' (plus 'pretty') C language program! ] Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
876ec555c3 |
library: final tweaks to code and/or comments, 3rd gen
With the dust now settling on all those 3rd generation upgrades, this patch tries to provide some consistency among the separate modules involved. Someday we should consider a 4th generation where all redundant code has been removed and isolated in a new shared source file. Following is a summary of significant changes (if any) to each of these now upgraded 3rd gen library modules. <meminfo> ............................................ . strictly formatting/comment changes, code unaffected <pids> ............................................... . replaced a local mkSTR macro with existing STRINGIFY . added fetch narrative explaining duplicate addresses <slabinfo> ........................................... . rearranged some free logic for procps_slabinfo_unref . added fetch narrative explaining duplicate addresses <stat> ............................................... . added #define ENFORCE_LOGICAL, just as in <slabinfo> . replaced a local mkSTR macro with existing STRINGIFY . alphabetized the function declarations in the header <vmstat> ............................................. . made one coverity concession with read_vmstat_failed [ several of these changes may reflect this author's ] [ continuing pursuit of an unreasonable goal -- that ] [ of a 'perfect' (plus 'pretty') C language program! ] Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
6a70466ae5 |
library: overcome effect of copy/paste, <SLABINFO> api
Needless to say, when one copies and then pastes, it's always better when those results are refined somewhat. The qsort callback declarations used the wrong results type and 2 of their 3 input parameters were wrong too. So this patch will fix that & generalize declarations. [ plus add comment guidance about noop & extra enums ] [ & provide for 'noop' sorting, as is done in <PIDS> ] Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
5d5a52a380 |
library: normalize/standardize the i/f, <SLABINFO> api
Before this major redesign, the slabs interface likely was our messiest 2nd generation attempt at opaqueness. Beyond the standard 'new', 'ref' & 'unref', there were a total of 12 exported functions. Now, there are four. The 1st step was to remove several of those functions. These were quick to go since they were not used (yet): . procps_slabnode_count . procps_slabnode_getname . procps_slabnode_getstack Then, the following were internalized so users needn't be burdened with implementation details in the future: . procps_slabinfo_read (renamed: read_slabinfo_failed) . procps_slabnode_stacks_alloc (renamed: stacks_alloc) Still others evolved into the minimal interface we had strived for in the other upgraded 3rd generation APIs: . procps_slabnode_get -----------> procps_slabinfo_get . separate stack_alloc/fill --> procps_slabinfo_select . separate stacks_alloc/fill ---> procps_slabinfo_reap . procps_slabnode_stacks_sort --> procps_slabinfo_sort ------------------------------------------------------ Beyond those reductions, the major modifications were: . This API tries to be as forgiving as possible and as such won't throw errors when a caller request makes no sense. For example, if a 'get' or 'select' requested a SLABNODE item (with no current means to id that node), results will be zero. By the same token, should 'reap' include a global SLABS item (meaning those values will be duplicated in *every* node stack) it'll be allowed. . If the above behavior is undesired, a new #define of ENFORCE_LOGICAL can be used to restrict certain items. . Permission problems will now be caught at 'new' time thanks to a priming 'read' call. That read also serves to make DELTA values potentially useful at 1st access. . Separate slab/slabnode enumerators were consolidated into one, simplifying validation & the results struct. . Several internal parameter checks were relaxed since they were already checked by the caller. Besides if we cannot trust our own code we might as well hang it up. . That sort provision was made more efficient and will offer the ascending choice, in addition to descending. ------------------------------------------------------ Lastly, some additional thoughts regarding the future: . It would not be difficult to expand 'select' to also accept a nodeid, or to clone it as 'select_node'. And, should the same be extended to 'get', a results struct could be returned instead of signed long accommodating the extra data type(s) like a node name (string data). . The 'get' function is not currently affected by that define ENFORCE_LOGICAL. However, at some future point perhaps -EINVAL would be more appropriate than a zero. Signed-off-by: Jim Warner <james.warner@comcast.net> |
||
|
Jim Warner
|
5be4167782 |
library: rename the 'slab' sources as 'slabinfo' files
Where possible, libprocps files convey the name of the actual source pseudo file under the '/proc' directory. This patch brings slab into line with such a standard. Signed-off-by: Jim Warner <james.warner@comcast.net> |