procps/proc/pids.h

248 lines
11 KiB
C
Raw Normal View History

/*
* pids.h - process related declarations for libprocps
*
* Copyright (C) 1998-2005 Albert Cahalan
2021-01-21 12:07:48 +05:30
* Copyright (C) 2015 Craig Small <csmall@dropbear.xyz>
* Copyright (C) 2015-2021 Jim Warner <james.warner@comcast.net>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef PROCPS_PIDS_H
#define PROCPS_PIDS_H
library: removed all the 'PROCPS_' enumerator prefixes Many of our item enumerator identifiers are very long, especially in that <VMSTAT> module. Additionally, they all contain the exact same universal 'PROCPS_' prefix. The origins for this are likely found in the desire to avoid name clashes with other potential include files. But with procps-ng newlib, we've probably gone way too far. Did 'PROCPS_PIDS_TICS_SYSTEM' actually offer more protection against clash than 'PIDS_TICS_SYSTEM' does? I don't think so. Besides, no matter how big that name becomes, one can never guarantee they'll never be some clash. And, conversely, extremely short names will not always create conflict. Of course, in either case when some clash occurs, one can always #undef that problem. Thus, this commit will eliminate that 'PROCPS_' prefix making all of those enum identifiers a little shorter. And, we'll still be well above some ridiculously short (criminally short) names found in some common headers: - - - - - - - - - - <term.h> - 'tab', 'TTY', etc - - - - - - - - - - - - - - - - <search.h> - 'ENTER', ENTRY', 'FIND', etc ------------------------------------------------------ Finally, with this as a last of the wholesale changes, we will have established the naming conventions below: . only functions will begin with that 'procps_' prefix . exposed structures begin with the module/header name . item enumerators begin like structs, but capitalized . other enumerators work exactly like item enumerators . macros and constants begin just like the enumerators ------------------------------------------------------ Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-07-21 10:30:00 +05:30
#ifdef __cplusplus
extern "C" {
#endif
enum pids_item {
library: removed all the 'PROCPS_' enumerator prefixes Many of our item enumerator identifiers are very long, especially in that <VMSTAT> module. Additionally, they all contain the exact same universal 'PROCPS_' prefix. The origins for this are likely found in the desire to avoid name clashes with other potential include files. But with procps-ng newlib, we've probably gone way too far. Did 'PROCPS_PIDS_TICS_SYSTEM' actually offer more protection against clash than 'PIDS_TICS_SYSTEM' does? I don't think so. Besides, no matter how big that name becomes, one can never guarantee they'll never be some clash. And, conversely, extremely short names will not always create conflict. Of course, in either case when some clash occurs, one can always #undef that problem. Thus, this commit will eliminate that 'PROCPS_' prefix making all of those enum identifiers a little shorter. And, we'll still be well above some ridiculously short (criminally short) names found in some common headers: - - - - - - - - - - <term.h> - 'tab', 'TTY', etc - - - - - - - - - - - - - - - - <search.h> - 'ENTER', ENTRY', 'FIND', etc ------------------------------------------------------ Finally, with this as a last of the wholesale changes, we will have established the naming conventions below: . only functions will begin with that 'procps_' prefix . exposed structures begin with the module/header name . item enumerators begin like structs, but capitalized . other enumerators work exactly like item enumerators . macros and constants begin just like the enumerators ------------------------------------------------------ Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-07-21 10:30:00 +05:30
PIDS_noop, // ( never altered )
PIDS_extra, // ( reset to zero )
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
// returns origin, see proc(5)
// ------- -------------------
PIDS_ADDR_CODE_END, // ul_int stat: end_code
PIDS_ADDR_CODE_START, // ul_int stat: start_code
PIDS_ADDR_CURR_EIP, // ul_int stat: eip
PIDS_ADDR_CURR_ESP, // ul_int stat: esp
PIDS_ADDR_STACK_START, // ul_int stat: start_stack
PIDS_CGNAME, // str derived from CGROUP ':name='
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_CGROUP, // str cgroup
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_CGROUP_V, // strv cgroup, as *str[]
PIDS_CMD, // str stat: tcomm or status: Name
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_CMDLINE, // str cmdline
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_CMDLINE_V, // strv cmdline, as *str[]
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_ENVIRON, // str environ
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_ENVIRON_V, // strv environ, as *str[]
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_EXE, // str exe
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_EXIT_SIGNAL, // s_int stat: exit_signal
PIDS_FLAGS, // ul_int stat: flags
PIDS_FLT_MAJ, // ul_int stat: maj_flt
PIDS_FLT_MAJ_C, // ul_int stat: maj_flt + cmaj_flt
PIDS_FLT_MAJ_DELTA, // s_int derived from FLT_MAJ
PIDS_FLT_MIN, // ul_int stat: min_flt
PIDS_FLT_MIN_C, // ul_int stat: min_flt + cmin_flt
PIDS_FLT_MIN_DELTA, // s_int derived from FLT_MIN
PIDS_ID_EGID, // u_int status: Gid
PIDS_ID_EGROUP, // str derived from EGID, see getgrgid(3)
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_ID_EUID, // u_int status: Uid
PIDS_ID_EUSER, // str derived from EUID, see getpwuid(3)
PIDS_ID_FGID, // u_int status: Gid
PIDS_ID_FGROUP, // str derived from FGID, see getgrgid(3)
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_ID_FUID, // u_int status: Uid
PIDS_ID_FUSER, // str derived from FUID, see getpwuid(3)
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_ID_LOGIN, // s_int loginuid
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_ID_PGRP, // s_int stat: pgrp
PIDS_ID_PID, // s_int from /proc/<pid>
PIDS_ID_PPID, // s_int stat: ppid or status: PPid
PIDS_ID_RGID, // u_int status: Gid
PIDS_ID_RGROUP, // str derived from RGID, see getgrgid(3)
PIDS_ID_RUID, // u_int status: Uid
PIDS_ID_RUSER, // str derived from RUID, see getpwuid(3)
PIDS_ID_SESSION, // s_int stat: sid
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_ID_SGID, // u_int status: Gid
PIDS_ID_SGROUP, // str derived from SGID, see getgrgid(3)
PIDS_ID_SUID, // u_int status: Uid
PIDS_ID_SUSER, // str derived from SUID, see getpwuid(3)
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_ID_TGID, // s_int status: Tgid
PIDS_ID_TID, // s_int from /proc/<pid>/task/<tid>
PIDS_ID_TPGID, // s_int stat: tty_pgrp
PIDS_LXCNAME, // str derived from CGROUP 'lxc.payload'
PIDS_MEM_CODE, // ul_int derived from MEM_CODE_PGS, as KiB
PIDS_MEM_CODE_PGS, // ul_int statm: trs
PIDS_MEM_DATA, // ul_int derived from MEM_DATA_PGS, as KiB
PIDS_MEM_DATA_PGS, // ul_int statm: drs
PIDS_MEM_RES, // ul_int derived from MEM_RES_PGS, as KiB
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_MEM_RES_PGS, // ul_int statm: resident
PIDS_MEM_SHR, // ul_int derived from MEM_SHR_PGS, as KiB
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_MEM_SHR_PGS, // ul_int statm: shared
PIDS_MEM_VIRT, // ul_int derived from MEM_VIRT_PGS, as KiB
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_MEM_VIRT_PGS, // ul_int statm: size
PIDS_NICE, // s_int stat: nice
PIDS_NLWP, // s_int stat: num_threads or status: Threads
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_NS_IPC, // ul_int ns/
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_NS_MNT, // ul_int "
PIDS_NS_NET, // ul_int "
PIDS_NS_PID, // ul_int "
PIDS_NS_USER, // ul_int "
PIDS_NS_UTS, // ul_int "
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_OOM_ADJ, // s_int oom_score_adj
PIDS_OOM_SCORE, // s_int oom_score
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_PRIORITY, // s_int stat: priority
PIDS_PRIORITY_RT, // s_int stat: rt_priority
PIDS_PROCESSOR, // u_int stat: task_cpu
PIDS_PROCESSOR_NODE, // s_int derived from PROCESSOR, see numa(3)
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_RSS, // ul_int stat: rss
PIDS_RSS_RLIM, // ul_int stat: rsslim
PIDS_SCHED_CLASS, // s_int stat: policy
PIDS_SD_MACH, // str derived from PID/TID, see sd-login(3)
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_SD_OUID, // str "
PIDS_SD_SEAT, // str "
PIDS_SD_SESS, // str "
PIDS_SD_SLICE, // str "
PIDS_SD_UNIT, // str "
PIDS_SD_UUNIT, // str "
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_SIGBLOCKED, // str status: SigBlk
PIDS_SIGCATCH, // str status: SigCgt
PIDS_SIGIGNORE, // str status: SigIgn
PIDS_SIGNALS, // str status: ShdPnd
PIDS_SIGPENDING, // str status: SigPnd
PIDS_STATE, // s_ch stat: state or status: State
PIDS_SUPGIDS, // str status: Groups
PIDS_SUPGROUPS, // str derived from SUPGIDS, see getgrgid(3)
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_TICS_ALL, // ull_int stat: stime + utime
PIDS_TICS_ALL_C, // ull_int stat: stime + utime + cstime + cutime
PIDS_TICS_ALL_DELTA, // s_int derived from TICS_ALL
PIDS_TICS_BLKIO, // ull_int stat: blkio_ticks
PIDS_TICS_GUEST, // ull_int stat: gtime
PIDS_TICS_GUEST_C, // ull_int stat: gtime + cgtime
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_TICS_SYSTEM, // ull_int stat: stime
PIDS_TICS_SYSTEM_C, // ull_int stat: stime + cstime
PIDS_TICS_USER, // ull_int stat: utime
PIDS_TICS_USER_C, // ull_int stat: utime + cutime
PIDS_TIME_ALL, // ull_int derived from (utime + stime) / hertz
PIDS_TIME_ELAPSED, // ull_int derived from /proc/uptime - (starttime / hertz)
PIDS_TIME_START, // ull_int stat: start_time
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_TTY, // s_int stat: tty_nr
PIDS_TTY_NAME, // str derived from TTY
PIDS_TTY_NUMBER, // str derived from TTY as str
library: refactored some header file items and origins This commit is intended as a refinement of the patches mentioned below, where origins/sources of newlib items were added to the header files for user documentation. However, if those additions are to be truly effective, along with kernel documentation (where available), the following prerequisites must also have been satisfied: . our identifiers closely align with linux field names . our derived items are documented or self-documenting Satisfying those prerequisites prompted this patch and for these changes, kernel sources were emphasized over available documentation (shame on me, it should always have been so). And, while some 'new' fields were found to be conditional, they were included unconditionally. These changes appear more extensive than they actually need be since I have attempted to enforce some spacing conventions. So, I've summarize the significant things in the sections that follow. For a proper perspective, use: 'git diff --ignore-space-change' (good as alias). ___________________________________________ <PIDS> api This api is unique in that there exists many different file/directory origins subordinate to /proc/<pid>. And our item identifiers are sometimes coerced so as to be able to group related or similar enumerators together. So, users needed more help relating our identifiers to an actual documented field. Thus, we will now also add the field names as with 'stat: delayacct_blkio_ticks'. Each item ending with a '_C' now consistently includes both the parent's count/time plus waited for children. That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT since its original name is an implementation artifact. ___________________________________________ <STAT> api The only api change was to correct a typo ('dervied'). _________________________________________ <VMSTAT> api Even ignoring white space, this interface received the largest number of changes. Mostly, this was because of deficiencies in the proc(5) documentation. Recall that this documentation already sorely lacks any substance. Usually, just kernel releases are noted, not contents. When compared to kernel source, that proc(5) contained many non-existent fields and also omitted many others. ________________________________________ <MEMINFO> api Sadly, with this api many of the changes were simply a correction of some earlier 'human error' where several fields where hashed then tracked but never represented with an item enumerator in this meminfo.h header file. _______________________________________ <SLABINFO> api The 'SLABS' (summary) & 'SLABNODE' items were reversed since the former are derived from the separate caches. More significantly, those 'SLABNODE' guys were renamed to 'SLAB' since they concern individual caches and the concept of 'nodes' is really an implementation detail. Also, several enumerators were changed to more closely agree with official slabinfo(5) documentation referred to in what we're treating as a base document: proc(5). Lastly, while those 'SLABS' items are solely a product of our library and not represented in slabinfo(5), the names attempt to parallel those found as 'SLAB' items. ______________________________________ <DISKSTATS> api One enumeration identifier was changed so as to better reflect its relationship to that actual documentation: 'Documentation/iostats.txt', as referenced in proc(5). Reference(s): . 12/2018, item origins added (and commit msg history) commit 96d59cbf46b3ff687bd29fad4708074a0e1cea14 . 01/2019, <stat> origins tweaked commit 201e816b26ddaccc923ec40977c92037cdd0c34e Signed-off-by: Jim Warner <james.warner@comcast.net>
2019-03-12 11:30:00 +05:30
PIDS_VM_DATA, // ul_int status: VmData
PIDS_VM_EXE, // ul_int status: VmExe
PIDS_VM_LIB, // ul_int status: VmLib
PIDS_VM_RSS, // ul_int status: VmRSS
PIDS_VM_RSS_ANON, // ul_int status: RssAnon
PIDS_VM_RSS_FILE, // ul_int status: RssFile
PIDS_VM_RSS_LOCKED, // ul_int status: VmLck
PIDS_VM_RSS_SHARED, // ul_int status: RssShmem
PIDS_VM_SIZE, // ul_int status: VmSize
PIDS_VM_STACK, // ul_int status: VmStk
PIDS_VM_SWAP, // ul_int status: VmSwap
PIDS_VM_USED, // ul_int status: VmRSS + VmSwap
PIDS_VSIZE_PGS, // ul_int stat: vsize
library: add item origin (as comments) to header files A lack of documentation seems to be the major obstacle to releasing this new library. So, in an effort to get the ball rolling again, this patch adds the origins of each item as a comment to six of the new header files. However, before reviewing how such changes may benefit that documentation objective, it seemed appropriate to first reflect on newlib's background & current status. ___________________________________________ BACKGROUND Discussions about and work on a new library began back in July 2012 but quickly died. After a lull of 2 years those discussions were resumed in August 2014 but soon died also (and no code survived the gitorious demise). With those early discussions, the recommended approach was to encapsulate all of the libprocps data offerings in individual functions. When it came to extensibility it was suggested we should rely on symbols versioning. Unfortunately that approach would have made for a huge Application Programming Interface virtually impossible to master or even document. And, runtime call overhead would have been substantial for ps and especially top. So, an alternative design was sought but there were no new suggestions/contributions via freelists or gitlab. Thus, in spite of a lack of library design experience, the procps-ng team (Craig & Jim) set out to develop an alternative API, more concise and with lower overhead. Reference(s): . 07/01/2012, begin library design discussion https://www.freelists.org/post/procps/Old-library-calls . 08/12/2014, revival of library design discussion https://www.freelists.org/post/procps/libprocs-redesign _____________________________________ DESIGN EVOLUTION Our newlib branch first appeared on June 14, 2015. And our current API actually represents the 4th generation during the past 3 years of evolution. First, there was a basic 'new', 'get' and 'unref' approach, using enums to minimize the proliferation of 'get' function calls. Then, in anticipation of other programs like ps, where multiple fields times multiple processes would greatly increase the number of 'get' function calls, a concept of 'chains' was introduced. This became generation #2. Such 'chains' proved unnecessarily complex so 'stacks' replaced them. This was considered the 3rd generation, but too many implementation details were still exposed requiring those users to 'alloc', 'read', 'fill', etc. Finally, a 4th generation emerged representing several refinements to standardize and minimize those exported functions, thus hiding all implementation details from the users. Lastly, handling of 'errno' was normalized. Reference(s): . 06/14/2015, revival of new API discussion https://www.freelists.org/post/procps/The-library-API-again . 06/24/2015, birth of the newlib branch https://www.freelists.org/post/procps/new-library . 06/29/2015, 2nd generation introduced 'chains' https://www.freelists.org/post/procps/new-library,8 . 07/22/2015, 3rd generation introduced 'stacks' https://www.freelists.org/post/procps/newlib-stacks-vs-chains . 06/18/2016, 4th generation refinements begin https://www.freelists.org/post/procps/newlib-generation-35 . 11/10/2017, 4th generation standardized 'errno' https://www.freelists.org/post/procps/some-more-master-newlib-stuff _______________________________________ CURRENT DESIGN Central to this new design is a simple 'result' struct reflecting an item plus its value (thanks to a union). As a user option, these item structures can be grouped into 'stacks', yielding many results with just 1 call. Such a 'stack' can be seen as a variable length record whose content/order is determined solely by the users. Within that 'result' structure, the union has standard C language types so there is never a doubt how a value should be used in a printf statement. Given that linux requires a least a 32-bit platform the only difference in capacity surrounds 'long' integers. And, where such types might be used, the 32-bit maximums are adequate. The items themselves are simply enumerators defined in the respective header files. A user can name any items of interest then the library magically provides result structure(s). The approach was proven to be extensible without breaking the ABI (in commit referenced below). The 6 major APIs each provide for the following calls: . 'new' ---------> always required as the first call . . 'ref' -------------------------> strictly optional . . 'unref' --------> optional, if ill-behaved program . . 'get' --------------------> retrieve a single item . . 'select' ----------------> retrieve multiple items . And the 'get' and 'select' functions provide for delta results representing the difference between successive get/select calls (or a 'new' then 'get/select' call). For the <diskstats>, <pids>, <slabinfo> & <stat> APIs, where results are unpredictable, a 'reap' function can return multiple result structures for multiple stacks. The <pids> API differs from others in that those items of interest must be provided at 'new' or 'reset' time, a function unique to this API. And the <pids> 'select' function requires PIDs or UIDs which are to be fetched which then operates as a subset of 'reap'. Lastly, the 'get' function is an iterator for successive PIDs/TIDs returning items previously identified via 'new/reset'. To provide assistance to users during development, the special header 'proc/xtra-procps-debug.h' is available to check type usage against library expectations. That check is activated by including this header explicitly or via build using: ./configure '-DXTRA_PROCPS_DEBUG'. Reference(s): . 08/05/2016, type validation introduced https://www.freelists.org/post/procps/newlib-types-validation commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f . 08/11/2016, extensibility while preserving ABI example https://www.freelists.org/post/procps/new-meminfo-fields commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de _________________________ INITIAL DOCUMENTATION EFFORT The initial attempt, referenced below, dealt primarily with the <pids> interface. Separate man pages for each exported function were created. Plus there was another document describing the items, among other miscellany. Adopting such an approach encounters several problems: 1. In order to use these man pages, users are required to already know how to use the library. Or alternately one could randomly search each of them while trying to ascertain which function call satisfies their need and what exactly was the proper compliment/order required. 2. While we can explain what all of those <pids> items represent, that certainly isn't true for all the APIs. See the gaps in kernel documentation for <meminfo> and complete lack of documentation with that <vmstat> API. 3. Our documentation effort should take pains to avoid unnecessary implementation details. Here's an example: . "The pointer to info will have memory" . "allocated and a structure created." Alternatively, the following conveys user requirements while not offering any internal implementation detail: . "You must provide the address of a NULL" . "info structure pointer." Reference(s): . 01/04/2017, initial documentation offering https://www.freelists.org/post/procps/Using-reap-and-get commit 2598e9f2ce39c93ebf55f664454d3bea919ed4e0 ___________________ RECOMMENDED DOCUMENTATION APPROACH I recommend that the newlib documentation consist of 3 man pages only. The first would cover the 5 major APIs and their common functions. The second would deal with the <pids> API exclusively, explaining how it differs. Any remaining exported libproc functions which are yet to be included could be represented in a 3rd document. For these new documents the following are are assumed: 1. Since we will not be able to document all items, we shouldn't try to document any items. We should instead rely on proc(5) or Documentation/filesystems/proc.txt. 2. Program development often involves referencing some header file(s). So, make that an absolute requirement. 3. With the addition of item origins, represented with this commit, and considering that 'types' were already present, the header file might be all some users need. 4. And who knows, when a user of our libproc complains about gaps in their documentation, it might prompt the kernel folks to correct those long standing omissions. To summarize, I suggest that we replace that libproc.3 document with a more general one explaining the basics of accessing this new library and the common calls for most of the major interfaces. We can then create a new document (libproc-pids.3?), which explains differences in using the <PIDS> application programming interface. A final document (libproc-misc.3?) covers what's left. Signed-off-by: Jim Warner <james.warner@comcast.net>
2018-12-20 11:30:00 +05:30
PIDS_WCHAN_NAME // str wchan
};
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
enum pids_fetch_type {
library: removed all the 'PROCPS_' enumerator prefixes Many of our item enumerator identifiers are very long, especially in that <VMSTAT> module. Additionally, they all contain the exact same universal 'PROCPS_' prefix. The origins for this are likely found in the desire to avoid name clashes with other potential include files. But with procps-ng newlib, we've probably gone way too far. Did 'PROCPS_PIDS_TICS_SYSTEM' actually offer more protection against clash than 'PIDS_TICS_SYSTEM' does? I don't think so. Besides, no matter how big that name becomes, one can never guarantee they'll never be some clash. And, conversely, extremely short names will not always create conflict. Of course, in either case when some clash occurs, one can always #undef that problem. Thus, this commit will eliminate that 'PROCPS_' prefix making all of those enum identifiers a little shorter. And, we'll still be well above some ridiculously short (criminally short) names found in some common headers: - - - - - - - - - - <term.h> - 'tab', 'TTY', etc - - - - - - - - - - - - - - - - <search.h> - 'ENTER', ENTRY', 'FIND', etc ------------------------------------------------------ Finally, with this as a last of the wholesale changes, we will have established the naming conventions below: . only functions will begin with that 'procps_' prefix . exposed structures begin with the module/header name . item enumerators begin like structs, but capitalized . other enumerators work exactly like item enumerators . macros and constants begin just like the enumerators ------------------------------------------------------ Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-07-21 10:30:00 +05:30
PIDS_FETCH_TASKS_ONLY,
PIDS_FETCH_THREADS_TOO
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
};
enum pids_select_type {
library: removed all the 'PROCPS_' enumerator prefixes Many of our item enumerator identifiers are very long, especially in that <VMSTAT> module. Additionally, they all contain the exact same universal 'PROCPS_' prefix. The origins for this are likely found in the desire to avoid name clashes with other potential include files. But with procps-ng newlib, we've probably gone way too far. Did 'PROCPS_PIDS_TICS_SYSTEM' actually offer more protection against clash than 'PIDS_TICS_SYSTEM' does? I don't think so. Besides, no matter how big that name becomes, one can never guarantee they'll never be some clash. And, conversely, extremely short names will not always create conflict. Of course, in either case when some clash occurs, one can always #undef that problem. Thus, this commit will eliminate that 'PROCPS_' prefix making all of those enum identifiers a little shorter. And, we'll still be well above some ridiculously short (criminally short) names found in some common headers: - - - - - - - - - - <term.h> - 'tab', 'TTY', etc - - - - - - - - - - - - - - - - <search.h> - 'ENTER', ENTRY', 'FIND', etc ------------------------------------------------------ Finally, with this as a last of the wholesale changes, we will have established the naming conventions below: . only functions will begin with that 'procps_' prefix . exposed structures begin with the module/header name . item enumerators begin like structs, but capitalized . other enumerators work exactly like item enumerators . macros and constants begin just like the enumerators ------------------------------------------------------ Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-07-21 10:30:00 +05:30
PIDS_SELECT_PID = 0x1000,
PIDS_SELECT_UID = 0x4000
};
enum pids_sort_order {
library: removed all the 'PROCPS_' enumerator prefixes Many of our item enumerator identifiers are very long, especially in that <VMSTAT> module. Additionally, they all contain the exact same universal 'PROCPS_' prefix. The origins for this are likely found in the desire to avoid name clashes with other potential include files. But with procps-ng newlib, we've probably gone way too far. Did 'PROCPS_PIDS_TICS_SYSTEM' actually offer more protection against clash than 'PIDS_TICS_SYSTEM' does? I don't think so. Besides, no matter how big that name becomes, one can never guarantee they'll never be some clash. And, conversely, extremely short names will not always create conflict. Of course, in either case when some clash occurs, one can always #undef that problem. Thus, this commit will eliminate that 'PROCPS_' prefix making all of those enum identifiers a little shorter. And, we'll still be well above some ridiculously short (criminally short) names found in some common headers: - - - - - - - - - - <term.h> - 'tab', 'TTY', etc - - - - - - - - - - - - - - - - <search.h> - 'ENTER', ENTRY', 'FIND', etc ------------------------------------------------------ Finally, with this as a last of the wholesale changes, we will have established the naming conventions below: . only functions will begin with that 'procps_' prefix . exposed structures begin with the module/header name . item enumerators begin like structs, but capitalized . other enumerators work exactly like item enumerators . macros and constants begin just like the enumerators ------------------------------------------------------ Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-07-21 10:30:00 +05:30
PIDS_SORT_ASCEND = +1,
PIDS_SORT_DESCEND = -1
};
struct pids_result {
enum pids_item item;
union {
library: more tweaks for code and/or comments, 3rd gen Following is a summary of significant changes (if any) to each of these now upgraded 3rd gen library modules. <meminfo> ............................................ . eliminated duplicate decl of 'struct procps_meminfo' . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . how did i miss relocating all these friggin' #undefs . cleanup 'get' return logic (remove a redundant 'if') <pids> ............................................... . repositioned the procps_pidsinfo structure in header . removed the extra trailing comma from enum pids_item . standardized/normalized results struct union members <slabinfo> ........................................... . corrected comment typo (jeeze, in an 'aligned' para) . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . removed an obsolete #undef from procps_slabinfo_sort . cleanup 'get' return logic (remove a redundant 'if') <stat> ............................................... . how did i miss relocating all these friggin' #undefs . corrected an initialization fencepost used with numa <=== see Craig, here's a bug fix . removed the extra trailing comma from enum stat_item . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . strengthen those parm checks in procps_stat_get func . cleanup 'get' return logic (remove a redundant 'if') <vmstat> ............................................. . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . cleanup 'get' return logic (remove a redundant 'if') [ virtually all of these tweaks reflect the author's ] [ continuing pursuit of an unreasonable goal -- that ] [ of a 'perfect' (plus 'pretty') C language program! ] Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-06-14 10:30:00 +05:30
signed char s_ch;
signed int s_int;
unsigned int u_int;
unsigned long ul_int;
unsigned long long ull_int;
char *str;
char **strv;
} result;
};
struct pids_stack {
struct pids_result *head;
};
struct pids_counts {
int total;
int running, sleeping, stopped, zombied, other;
};
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
struct pids_fetch {
struct pids_counts *counts;
2015-08-20 10:30:00 +05:30
struct pids_stack **stacks;
};
#define PIDS_VAL( relative_enum, type, stack, info ) \
stack -> head [ relative_enum ] . result . type
struct pids_info;
library: more tweaks for code and/or comments, 3rd gen Following is a summary of significant changes (if any) to each of these now upgraded 3rd gen library modules. <meminfo> ............................................ . eliminated duplicate decl of 'struct procps_meminfo' . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . how did i miss relocating all these friggin' #undefs . cleanup 'get' return logic (remove a redundant 'if') <pids> ............................................... . repositioned the procps_pidsinfo structure in header . removed the extra trailing comma from enum pids_item . standardized/normalized results struct union members <slabinfo> ........................................... . corrected comment typo (jeeze, in an 'aligned' para) . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . removed an obsolete #undef from procps_slabinfo_sort . cleanup 'get' return logic (remove a redundant 'if') <stat> ............................................... . how did i miss relocating all these friggin' #undefs . corrected an initialization fencepost used with numa <=== see Craig, here's a bug fix . removed the extra trailing comma from enum stat_item . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . strengthen those parm checks in procps_stat_get func . cleanup 'get' return logic (remove a redundant 'if') <vmstat> ............................................. . standardized/normalized results struct union members . added 'std' & 'var' dividers in .c file, like <pids> . cleanup 'get' return logic (remove a redundant 'if') [ virtually all of these tweaks reflect the author's ] [ continuing pursuit of an unreasonable goal -- that ] [ of a 'perfect' (plus 'pretty') C language program! ] Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-06-14 10:30:00 +05:30
int procps_pids_new (struct pids_info **info, enum pids_item *items, int numitems);
int procps_pids_ref (struct pids_info *info);
int procps_pids_unref (struct pids_info **info);
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
struct pids_stack *fatal_proc_unmounted (
struct pids_info *info,
int return_self);
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
struct pids_stack *procps_pids_get (
struct pids_info *info,
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
enum pids_fetch_type which);
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
struct pids_fetch *procps_pids_reap (
struct pids_info *info,
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
enum pids_fetch_type which);
int procps_pids_reset (
struct pids_info *info,
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
enum pids_item *newitems,
int newnumitems);
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
struct pids_fetch *procps_pids_select (
struct pids_info *info,
unsigned *these,
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
int numthese,
enum pids_select_type which);
struct pids_stack **procps_pids_sort (
struct pids_info *info,
2015-08-20 10:30:00 +05:30
struct pids_stack *stacks[],
int numstacked,
library: standardize portions of interface, <PIDS> api This represents a rather major interface redesign. The following highlights most of the changes/enhancements. . The 'read' interface (employed by pgrep & pidof) saw the biggest change. The 'open', 'next' and 'shut' guys all went bye-bye, replaced by a single 'get' function. . The items specified at 'new' time no longer serve as the maximum. In fact, items & numitems are now treated as optional, should callers prefer to wait until later when the 'reset' function would then become mandatory. . Even at 'reset' time, the stacks are not tied to any sort of maximum. They will grow dynamically as needed. . The order of some parameters was changed to parallel that found in our other APIs. Specifically, when items & numitems are needed they're specified in that order. . A user will no longer be prevented from concurrently employing any accessor functions. In other words, that 'get' (old 'read') won't preclude 'reap' and 'select'. . A duplicate enumerator was found dealing with locked resident pages. So, the name VM_LOCK was eliminated in favor of VM_RSS_LOCKED, which is way more descriptive. . The struct address returned to callers following any reap() or select() is now more sharable as pids_fetch. . Some input parameter names were changed to make them more descriptive of the intended purpose/requirements. ------------------------------------------------------ Internally, there were numerous implementation changes made that did not directly impact any potential users. . That #define FPRINT_STACKS was eliminated along with the associated supporting function and its invocation. . Addresses returned following 'reap' or 'select' will now be NULL delimited, so one has the option of stacks access via the total count or this new NULL fencepost. . Input params were simplified and generalized in both oldproc_open() & close() to enable more than 1 PROCTAB to be open simultaneously, which was required for get. . The PROCPS_PIDS_logical_end enum was relocated after the Item_table making the need to keep it synchronized more apparent (if the table expands it's right there). . The 'Public function' section of the source file was subdivided into 1) the three basic required functions; and 2) functions that can sometimes vary between APIs. Signed-off-by: Jim Warner <james.warner@comcast.net>
2016-05-14 10:30:00 +05:30
enum pids_item sortitem,
enum pids_sort_order order);
#ifdef XTRA_PROCPS_DEBUG
# include "xtra-procps-debug.h"
#endif
#ifdef __cplusplus
}
#endif
#endif