HPA/DCO detection confusion #542
Comments
|
Forgot to mention that the USB drive is encrypted and has got only 1 "partition" (according to GParted and |
|
Can you post the nwipe log, thanks. |
Thanks for the fast reply. What log is that? I haven't wiped anything yet. |
|
You don't need to wipe anything, just control C it exit nwipe, then ALT F2 to switch to the 2nd virtual terminal, ls will display the / directory contents. There will be a file called nwipexxxxxxx. If you more that and take a photo. If you prefer to anonymize the serial no. Etc at the prompt run nwipe -q, once it's displayed the selection just control c to exit, there will be a 2nd nwipe log with serial numbers replaced with xxxxxx. |
|
There's no "nwipexxxxxxx" files, ls shows only "packages-list.txt", "packages-size.txt" and "version" files. However, exiting nwipe, it outputs text to tty1, same where nwipe was started/exited. Could not get the full text, but luckily the problematic
|
|
Interesting, can you post the output of |
|
|
|
Thanks, looks like you have discovered a bug. Nwipe reporting HS = yes, is incorrect in this case. The reason for the error is that the HPA and DCO are being reported as 512 byte sectors, by both hdparm and nwipe's own HPA/DCO functions, but libata is reporting the number of sectors as 4096 byte sectors for both physical and logical which results in the number of sectors being 8 times less. 976754646 * 8 = 7814037168. I'll submit a patch over the next couple of days to fix this. I think all my drives report physical/logical as 512/512 although I do have a encrypted device it also reports physical/logical sector size as 512 so would not show this bug and correctly reports HS status. Thank you for reporting this. That is much appreciated. |
|
Ok. Thanks. Good to know, now I can start wiping the drive, no need to manipulate the drive with hdparm (which I don't know how to). :) |
|
Can you post a link from Amazon/Ebay where I can get hold of the exact same device you have. I don't think any of the hardware I have reports as 4096/4096 so it would be handy to have for testing. Are you able to build from source? I might need you to test the patched version if you have some spare time. |
It's an old drive, I don't think you could get it anymore, unless someone sells it used.. I think. Can't even remember where I bought it. EDIT: Is it not possible to format a drive and manually select sector sizes?
Well... I really haven't done that much, only once (RHash) and although there were only few commands, I had to google/learn a lot so I could get it working (the github page didn't have that great instructions.. IIRC, it was rather outdated info). I'm running Linux Mint 20.3 Cinnamon (I have that also in a VirtualBox machine, so I could easily play with it without damaging my host system). But the biggest problem is that how am I going to get the test build to run on my old laptop with USB stick? I'm now running nwipe that is included in the SystemRescue ISO, which I'm starting from a Ventoy "formatted" USB stick (can boot basically any ISO file). If you can figure out a (relatively) easy way for me to test, sure I can test it for you, no problem. I really wouldn't like to play with building on my main laptop (might need to install a lot of packages.. and not sure if even compatible packages are available.. that happened to me with some other software I was trying to build (in same OS in VirtualBox)). |
hdparm ?: |
|
I've just tried a WD & a Seagate EXOS drive with that command, both unfortunately fail with the error Not to worry, I'm sure somebody out there has a drive that reports logical/physical as 4096, the important part of that for this discussion is that logical reads 4096. I can still work on the patch in the meantime. |
I hope somebody could chip in, sorry that I couldn't help more. |
|
Hey, I got the repo master compiled (hopefully ok) in a virtualbox machine and now I can test the USB drive with the patch.
I don't understand why the sector count is now different.
nwipe 0.35.6: & output after quitting (including vbox hdd, etc., all info): I don't understand why some of the data is now different, maybe something went wrong with the building of nwipe? Anyhoo, if you have any further questions/instructions, I'm waiting here. :) P.S. I'm currently wiping 2 USB drives on my old laptop and one of them just finished but the other one is still going at ~50%; can I disconnect the already ready USB drive while the other one still has approximately 55 hours to go? |
Yes, it's ok to pull the finished USB. I'll take a closer look at the output you posted and let you know. I can see something that's not quite correct in the line |
|
I have a request. When you get a chance can you plug in the 4096 (4kn) Intenso USB 3.0 4000 GB USB device and start nwipe with the verbose option. You don't need to run a wipe, just start nwipe with the above command and then control C to exit. The log will be displayed in stdout and also written to nwipelog.txt. Can you post nwipelog.txt. Thanks. |
Noticed that the I'll try your request next. |
|
Here's the log. It's from the virtualbox machine (which itself might not work ok, the USB "passthrough" might produce errors). I noticed that the smartctl says I just actually realized that my old laptop, I have same Linux Mint 20.x installed on it (kind of forgot it since I haven't used it in a long while). I can test the drive/nwipe with only that, no virtual environments, no SystemRescue USB boot with Ventoy (which had a small problem; I couldn't see the SystemRescue boot menu and couldn't load it into memory (I wanted to remove my Ventoy boot USB stick after booting)). I'll just take a snapshot with a Timeshift and then compile nwipe and try the USB drive there. BUT, I'll have to wait until the wiping ends for the remaining one USB drive. Takes a few days, so maybe on Monday/Tuesday(?) I'll report back. But in the meantime I can test with the virtualbox machine if you want. |
No problem, I think I've found an issue with the soft block size being initialised once the drive/s for wiping have been selected and the wipe started, where as HPA/DCO determination is conducted prior to this, so I need to make a change to the code over the next few days. I'll let you know when there is a revised version 0.35.7 for downloading & compiling. |
|
Hey, I just noticed (from an old smartctl log I had) that my 2TB USB drive (the last one now being wiped), smartctl reports sectors as |
That's ok, I believe 512 logical and 4096 physical is working ok, as is 512 logical and 512 physical, it's only 4096 logical and 4096 physical that has an issue. |
|
as for exotic disks: I have a SMR currently wiping, some more on the shelf (had been bought by accident not realising that "archive disk" was a code for "shingled magenetic recoding". Started with apparently normal data rates (~150MB/s), but now at 64.43% (round 1 of 1, pass 3 of 3) (very short blanking, long syncing) the data rate lowered to 104MB/s It is a ST8000AS0002. Had hoped to find it 4kn, but the drive reports 512e. |
|
Well you learn something new everyday. I'd never heard of "Shingled magnetic recording" until I just googled it. |
Just making sure that I understood correctly; I was supposed to wait for the new version before doing anything? |
Yes, sorry I forgot to bump the minor version. You can now download the master (0.35.7) and give that a try. |
Hmm, you only updated the version number? I thought you wanted to change some actual code.
|
|
I did, look at the previous commit. |
Ignore that, you are perfectly correct there was one other change I wanted to make. Hold fire with that testing, it will be 0.35.8 ! Apologies. |
|
@ggruber Do you have a 512/4096 drive on your server I could test with, maybe a smaller EXOS ? After a few changes I've made in 0.35.8 I want to check the logical/physical is being reported correctly in the logs and then I'll do a zeros wipe with verification. Would that be possible? |
|
@ggruber like that ST8000AS0002 should be fine. |
|
@Perkolator Master (0.35.8) is ready for download & testing. If you plug that encrypted USB disk in that shows as 4096/4096 and start |
|
@PartialVolume I will be able to take the one ST8000AS0002 with me that's currently being wiped (from @work to my homelab) when the wipe processed is finished. Or: I take some more from the shelf (this was a unintentional puchase) to the homelab. Then I could mount them in the 3,5" wipebox, together with an "normal" 6TB Exos drive (ST6000NM0115), also 512e. |
nwipe: Zeros wipe + verification, no blanking pass: nwipe_log.txt For some reason |
|
@Perkolator Thanks for the detail, much appreciated.
There is a bug in hdparm 9.60, including possibly earlier or later versions but which is fixed in 9.65, that returns an incorrect (negative) value displayed as a very large positive number for some drives that are possibly over a certain size. I wrote a low level function in nwipe that retrieves the DCO and correctly interprets it. My guess is you are running the version of hdparm that is between 9.60 and 9.64. The latest version of ShredOS is running hdparm v9.65 so will show the correct number of sectors with the --dco-identify command. Looking at the nwipe log, it all looks correct, apart from one thing (which isn't an nwipe issue). As you can see hdparm incorrectly reports real max sectors as -1 So in summary it all looks as i would expect. |
|
@Perkolator oh, and of course the logical/physical sizes are reported correctly as 4096/4096 and nwipe reports no hidden sectors which is as expected. It would be nice to create a hidden sector on that device using hdparm which would then show that nwipe positively detects it. I'll leave it up to you whether you want to try that :-)
nwipe should report hidden sectors, in the certificate it will report 8 hidden sectors or 4096 bytes hidden. |
It's actually 9.58 (Linux Mint 20.3 (Ubuntu focal 20.04)), but anyways I'm happy that it's not affecting your testing of this stuff. :)
What certificate? The PDF report? I did as you instructed and I see these (ignored the incorrect hdparm:DCO):
And nwipe (GUI and log output after exit) shows hidden sectors detected. All seems to be ok. Anything else you'd want me to test or do I get to dismantle my test bench? :) |
|
@PartialVolume I have four ST8000AS0002 in the 3,5" wipebox, together with an "normal" 6TB Exos drive (ST6000NM0115), also 512e. Another 4TB SATA, 4TB SAS, 2TB SATA |
|
@Perkolator Looks good, thanks again, your help is much appreciated. |
That bug in hdparm goes back a little further than I thought.
Yes, the PDF report. Sometimes I call it a certificate, other times a report. I don't need it BTW, it's just worth checking that it recorded the fact there were hidden sectors. |
Thanks, I need to make a minor commit to the code and then I'll start a wipe on these some time tomorrow. |
I have an USB 3.0 drive and nwipe shows
HS? YESfor the drive, but when I run thehdparmcommands your previous version of the "readme.md" hadall the sector values are the same.
hdparm -Nreports: max sectors = 7814037168/7814037168, HPA is disabledhdparm --dco-identifyreports: Real max sectors: 7814037168I don't understand deeply what this HPA/DCO thing is, but I'd like to erase drives completely.
Any idea why nwipe says that there are hidden sectors but hdparm doesn't?
How does that happen with hdparm?
The text was updated successfully, but these errors were encountered: