-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add block-info module #37
Conversation
Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application. When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated. If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public. |
9b4a8e9
to
d553087
Compare
b303671
to
e0072a4
Compare
Please give an output of this command. And also describe how you test it. |
|
We tested this module by running it on uek5, uek6 and uek7 vmcores and validated the output with crash. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code overall looks pretty good, I have a few comments but nothing too crazy.
My main gripe is the code organization:
scsiHosts
andscsidevs
are bad Python naming style. And both modules are rather small. I think it makes sense to put all that code into a single module,drgn_tools/scsi.py
.- I don't think we need a separate file
drgn_tools/blockinfo.py
. Corelens modules don't need to be in a file with the same name. In this case, it would be better for you to add this module to thedrgn_tools/block.py
file where the rest of the block-layer helpers are.
Thank you!
drgn_tools/scsiHosts.py
Outdated
def print_scsi_hosts(prog: Program) -> None: | ||
""" | ||
Prints scsi host information | ||
""" | ||
shostlist = _list_of_scsi_host(prog) | ||
print("SCSI HOSTS\n==========\nHOST\t\t\tNAME") | ||
for x in range(len(shostlist)): | ||
# print(f"{shostlist[x].hostt.name}\n") | ||
print( | ||
f"{hex(shostlist[x].value_())}\thost{shostlist[x].host_no.value_()}" | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Try using the drgn_tools.table.print_table()
function to help this. It's the standard way to print these things so they get formatted properly.
drgn_tools/scsiHosts.py
Outdated
try: | ||
class_in_private = prog.cache["knode_class_in_device_private"] | ||
except KeyError: | ||
class_in_private = prog.type("struct device_private").has_member( | ||
"knode_class" | ||
) | ||
prog.cache["knode_class_in_device_private"] = class_in_private |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't bother caching this. Looking up a type, and testing whether the type has a member, is not a very expensive task. Plus, this function isn't called repeatedly in a performance-sensitive area, so caching doesn't really make sense.
drgn_tools/scsiHosts.py
Outdated
def _list_of_scsi_host(prog: Program) -> List[Object]: | ||
""" | ||
Iterates through all scsi hosts and returns a | ||
list of scsi devices | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is marked private (starts with an _
), but it looks useful. Could you name it such that it's not private? And also, be sure to document the C type of the returned objects in the docstring. EG:
def _list_of_scsi_host(prog: Program) -> List[Object]: | |
""" | |
Iterates through all scsi hosts and returns a | |
list of scsi devices | |
""" | |
def for_each_scsi_host(prog: Program) -> List[Object]: | |
""" | |
Iterates through all scsi hosts and returns a | |
list of scsi devices | |
:returns: list of ``struct Scsi_Host *`` | |
""" |
drgn_tools/scsidevs.py
Outdated
) | ||
for disk in for_each_disk(prog): | ||
partno = 0 | ||
has_bdev_struct = is_blkdev_hdpart(prog) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is_blkdev_hdpart()
could be expensive (iterates over every disk) and you're calling it inside the loop. Please move this call outside the loop to ensure we don't get an O(N^2) situation.
drgn_tools/scsidevs.py
Outdated
print( | ||
"%-10s %-10s %-20s %-20s %-20s %-20s" | ||
% ("MAJOR", "MINOR", "TIMEOUT", "#BLOCKS", "GENDISK", "NAME") | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, this is a good place for print_table()
drgn_tools/scsidevs.py
Outdated
) | ||
) | ||
else: | ||
while disk.part_tbl.part[partno]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like the part_tbl
has a len
field. Try this instead:
for partno in range(disk.part_tbl.len):
Then you shouldn't need to do the FaultError
check. And you can remove the statements adjusting partno
and the declaration.
drgn_tools/scsidevs.py
Outdated
def print_scsi_devs_info(prog: Program) -> None: | ||
""" | ||
Prints the scsi device information | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, I'm trying to understand what is scsi-specific here. As far as I can tell, this is simply printing block device info?
If that's the case, then it really doesn't belong in a module named with scsi
. Probably the drgn_tools.block
module, and it should be named something like print_block_devs_info()
.
Thanks for the review Stephen, will update PR keeping your comments in mind. |
e0072a4
to
11582ab
Compare
6e0115a
to
11582ab
Compare
drgn_tools/block.py
Outdated
num_inflight_ios = 0 | ||
for disk in for_each_disk(prog): | ||
num_inflight_ios += get_inflight_io_nr(prog, disk) | ||
print(f"{num_inflight_ios} inflight IOs found") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please extend helper "get_inflight_io_nr" with an extra parameter, if it points to a special disk, then return the inflight-io for that disk, otherwise return the inflight-io for all the disk in the system, then you can just print the value anywhere, no need a special print function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually the better way is define a new helper to get the inflight io number for all the disk, and have the new helper invoking get_inflight_io_nr() to get per disk inflight io number.
drgn_tools/block.py
Outdated
print("\n\nInFlight I/Os\n=============") | ||
dump_inflight_io(prog, "all") | ||
print("\n") | ||
print_total_inflight_ios(prog) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This BlockInfo module did multiple things, I think we should separated it, make it only do the things like "dev -d", and move scsi host dump to scsi.py using new corelen module, finally have InflightIOModule module do the inflight dump thing?
Regarding all the partition related helper, i think we can integrate through a new pull request, that pull request should provide a helper which can list partition for a particular disk or all disks, it should include name, offset, length, and also whether the partition is read-only.
drgn_tools/block.py
Outdated
has_bdev_struct = False | ||
|
||
return has_bdev_struct | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sound too heavy to tell whether gendisk using hd_struct or block_device, i guess there will be better way. Can we check the type of part_tbl or part0? Please ask Stephen if you are not sure how to improve it.
A couple notes to add on to Junxiao's review:
Thanks! |
11582ab
to
764bdd0
Compare
As suggested by junxiao, removed the code corresponding to scsi and partitions. This module just prints block device related information. Will create a separate PRs for scsiinfo and partitioninfo. |
sample output:
|
I’m working on scsiinfo, will move the list_of_scsi_host() / for_each_scsi_host() to generic module that could be used by other modules.
Rajan
From: SRIVATHSA DARA ***@***.***>
Date: Monday, December 18, 2023 at 01:53
To: oracle-samples/drgn-tools ***@***.***>
Cc: Subscribed ***@***.***>
Subject: Re: [oracle-samples/drgn-tools] Add block-info module (PR #37)
As suggested by junxiao, removed the code corresponding to scsi and partitions. This module just prints block device related information. Will create a separate PRs for scsiinfo and partitioninfo.
|
drgn_tools/block.py
Outdated
""" | ||
output = [["MAJOR", "GENDISK", "NAME", "REQUEST_QUEUE", "Inflight I/Os"]] | ||
for disk in for_each_disk(prog): | ||
mjr = int(disk.major) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
'major' sounds a better name.
drgn_tools/block.py
Outdated
name = disk.disk_name.string_().decode("utf-8") | ||
rq = hex(disk.queue.value_()) | ||
ios = int(get_inflight_io_nr(prog, disk)) | ||
output.append([str(mjr), gendisk, name, rq, str(ios)]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need converting 'mjr' and 'ios'?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, pre_commit complained that the list has different types of objects.
I have use int conversion for major because it has "(int 32)" prefix, and for IOs as you mentioned it does not need int conversion, i will remove that. Converted both of them to str to make pre_commit happy.
drgn_tools/block.py
Outdated
gendisk = hex(disk.value_()) | ||
name = disk.disk_name.string_().decode("utf-8") | ||
rq = hex(disk.queue.value_()) | ||
ios = int(get_inflight_io_nr(prog, disk)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
'get_inflight_io_nr()' already return int, convert is not required.
Hi @vashanmu , @srivathsa729 already posted a basic scsiinfo pull request, i know it's pulled from your scsiinfo helpers, would you like to merge that one and then you can use new pull request to extend it or you would like leaving the whole scsiinfo for you to work on? https://github.com/oracle-samples/drgn-tools/pull/50/files |
Hi Junxiao,
I thought Gulam was working on iSCSI module that he wanted to use the list of scsihost from my scsiinfo module.
Rajan
From: Junxiao Bi ***@***.***>
Date: Monday, December 18, 2023 at 15:14
To: oracle-samples/drgn-tools ***@***.***>
Cc: Rajan Shanmugavelu ***@***.***>, Mention ***@***.***>
Subject: Re: [oracle-samples/drgn-tools] Add block-info module (PR #37)
I’m working on scsiinfo, will move the list_of_scsi_host() / for_each_scsi_host() to generic module that could be used by other modules. Rajan From: SRIVATHSA DARA @.> Date: Monday, December 18, 2023 at 01:53 To: oracle-samples/drgn-tools @.> Cc: Subscribed @.***> Subject: Re: [oracle-samples/drgn-tools] Add block-info module (PR #37<#37>) As suggested by junxiao, removed the code corresponding to scsi and partitions. This module just prints block device related information. Will create a separate PRs for scsiinfo and partitioninfo.
Hi @vashanmu<https://github.com/vashanmu> , @srivathsa729<https://github.com/srivathsa729> already posted a basic scsiinfo pull request, i know it's pulled from your scsiinfo helpers, would you like to merge that one and then you can use new pull request to extend it or you would like leaving the whole scsiinfo for you to work on? https://github.com/oracle-samples/drgn-tools/pull/50/files
—
Reply to this email directly, view it on GitHub<#37 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIQN7XY6W65PKPAQ3BNIR4TYKDE4JAVCNFSM6AAAAABABANIZKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRRHAZTQOBUG4>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Have an offline discussion with Rajan, we will go ahead to merge scsiinfo, and Rajan will extend it further using new pull request. This way Gulam can move forward his iscsi helpers in parallel. |
this is the error pre_commit is reporting when all the elements of the list are not strings
|
764bdd0
to
ce5f42b
Compare
Removed int conversion for inflight-IOs, but found that some times
|
drgn_tools/block.py
Outdated
|
||
class BlockInfo(CorelensModule): | ||
""" | ||
Corelens Module for scsi-devs-info |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please fix this comment, it's block device info.
Then please fix get_inflight_io_nr() to make it always return int. |
blockinfo prints block device information Authored-by: Gulam Mohamed <[email protected]> Co-authored-by: Srivathsa Dara <[email protected]>
ce5f42b
to
ea608da
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
block-info prints scsihosts, scsi-device information and pending in-flight IOs
Authored-by: Gulam Mohamed [email protected]
Co-authored-by: Srivathsa Dara [email protected]