Skip to content

Latest commit

 

History

History
258 lines (188 loc) · 16.4 KB

README.md

File metadata and controls

258 lines (188 loc) · 16.4 KB

nvmetool-win: Sample program of accessing NVMe device using Windows' inbox NVMe driver

Abstract

This software demonstrates that issuing some NVMe commands from userland to NVMe device using Windows inbox NVMe driver (stornvme.sys)[1].

Specification that this software refered to is NVMe 1.3d[2] and NVMe 1.4b[6].

For StorNVMe on Windows 10 version 1903 and later, Supported NVMe Admin and NVM command sets and associated opcodes are shown in [8], and supported features are shown in [9].

Command matrix

Table 1. Command matrix (Admin Command Set)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

Opcode Command name Status Note
00h Delete I/O Submission Queue 1
01h Create I/O Submission Queue 1
02h Get Log Page See Table 4
04h Delete I/O Completion Queue 1
05h Create I/O Completion Queue 1
06h Identify See Table 3
08h Abort 1
09h Set Features See Table 5
0Ah Get Features See Table 5
0Ch Asynchronous Event Request 1
0Dh Namespace Management 2
10h Firmware Commit
11h Firmware Image Download
14h Device Self-test x
15h Namespace Attachment 2
18h Keep Alive Not listed in [8]
19h Directive Send 1
1Ah Directive Receive 1
1Ch Virtualization Management Not supported
1Dh NVMe-MI Send 2
1Eh NVMe-MI Receive 2
7Ch Doorbell Buffer Config Not supported
80h Format NVM x Only for data drive. Need to support Format NVM command and cryptographic erase is supported along Format NVM (bit 2 of the FNA field in Identify Controller data structure)
81h Security Send
82h Security Receive x Getting TCG Level 0 Discovery Data for example in this tool
84h Sanitize 2
86h Get LBA Status Not supported
C0h -- FFh Vendor Specific

Note 1: probably cannot issue this command because only "Internal Driver Use" is written in [8]

Note 2: can issue this command only in WinPE mode according to [8]

Table 2. Command matrix (NVM Command Set)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

Opcode Command name Status Note
00h Flush x
01h Write x
02h Read x
04h Write Uncorrectable 1
05h Compare Can issue only in WinPE mode [8]
08h Write Zeroes 1
09h Dataset Management x Only Deallocate is supported
0Ch Verify 1
0Dh Reservation Register 1
0Eh Reservation Report 1
11h Reservation Acquire 1
15h Reservation Release 1
80h -- FFh Vendor Specific

Note 1: these command are stated as "Not supported" in [8]

Controller or Namespace Structure (CNS) for Identify command

CNS values from 10h to 13h are seemed to be able to issue only in WinPE mode, because these values are mandatory for controllers that support the Namespace Management capability in NVMe spec., but Namespace Management command can only be issued in WinPE mode according to [8].

CNS values 14h and 15h are seemed to be unable to issue, because these values are mandatory for controllers that support Virtualization Enhancements in NVMe spec., but Virtualization Management command is not supported according to [8].

Table 3. Controller or Namespace Structure (CNS)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

CNS Description Status Note
00h Namespace x NSID = 1
01h Controller x CNTID = 0
02h Active Namespace ID list returned data seems to be the data for Identify Namespace
03h Namespace Identification Descriptor list returned data seems to be the data for Identify Namespace
04h -- 0Fh Reserved
10h Allocated Namespace ID list Probably can issue only in WinPE mode [8]
11h Specified allocated namespace Probably can issue only in WinPE mode [8]
12h List of controllers attached to specified namespace Probably can issue only in WinPE mode [8]
13h List of controllers exist in the NVM subsystem Probably can issue only in WinPE mode [8]
14h Primary Controller Capabilities Probably cannot issue [8]
15h Secondary controller list Probably cannot issue [8]
16h -- 1Fh Reserved
20h -- FFh Reserved

Log Identifier (LID) for Get Log Page command

Table 4. Log Identifier (LID)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

LID Description Status Note
01h Error Information x Number of entries to be retrieved from devices depends on the value "ELPE" in Identify Controller data
02h SMART / Health Information x CNTID = 0, NSID = 0xFFFFFFFh
03h Firmware Slot Information x
05h Command Supported and Effects x
06h Device Self-test x
07h Telemetry Host-initiated x Getting header only. see also Note 1.
08h Telemetry Controller-initiated x Getting header only.
09h -- 6Fh Reserved
70h Discovery For NVMe over Fabrics (NVMeoF)
71h -- 7Fh Reserved for NVMeoF
80h Reservation Notification Probably cannot issue [8]
81h Sanitize Status Probably can issue only in WinPE mode [8]
82h -- BFh Reserved
C0h -- FFh Vendor Specific

Note 1: If you want to set Create Telemetry Host-Initiated Data to 1, you need Windows 10 version 2004 (build 10041) or later. And to do this, this tool uses IOCTL_STORAGE_GET_DEVICE_INTERNAL_LOG described in [9]. But I don't know products (SSDs) that can respond this type of request (command)...

Feature Identifier (FID) for Get/Set Features command

According to [8], only Host Controlled Thermal Management is supported via Set Features command with IOCTL_STORAGE_SET_PROPERTY.

Table 5. Feature Identifier (FID)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

FID Description Status (Get/Set) Note
01h Arbitration x /
02h Power Management x /
03h LBA Range Type -- / error code = 1117 (The request could not be performed because of an I/O device error.)
04h Temperature Threashold x /
05h Error Recovery x /
06h Volatile Write Cache x / -- error code = 1117 (The request could not be performed because of an I/O device error.)
07h Number of Queues -- / error code = 1117 (The request could not be performed because of an I/O device error.)
08h Interrupt Coalescing x /
09h Interrupt Vector Configuration /
0Ah Write Atomicity Normal x /
0Bh Asynchronous Event Configuration x /
0Ch Autonomous Power State Transition -- / error code = 1117 (The request could not be performed because of an I/O device error.)
0Dh Host Memory Buffer -- / error code = 1117 (The request could not be performed because of an I/O device error.)
0Eh Timestamp -- / error code = 1117 (The request could not be performed because of an I/O device error.)
0Fh Keep Alive Timer /
10h Host Controlled Thermal Management x / x "SV = 1" is not supported
11h Non-operational Power State Config x /
12h -- 77h Reserved /
78h -- 7Fh Refer to the NVMe-MI spec. /
80h Software Progress Marker x /
81h Host Identifier /
82h Reservation Notification Mask / 1
83h Reservation Persistence / 1
84h -- BFh Reserved /
C0h -- FFh Vendor specific /

Note 1: FID 82h and 83h are seemed to be unable to issue, because commands related to Reservation are not supported according to [8].

Note for some commands

Read command

Read command is issued via SCSI Pass-through[3].

Issue Read command with Starting Logical Block Address (SLBA) = 0 and Number of Logical Blocks (NLB) = 1.

Write command

Write command is issued via SCSI Pass-through[3].

Issue Write command with SLBA = 0, NLB = 1, and data is "DEADBEEF".

Dataset Management command

Dataset Management command is issued via SCSI Pass-through[3] with IOCTL_STORAGE_MANAGE_DATA_SET_ATTRIBUTES [4][5] (only Deallocate bit is set).

LBA region to be deallocated is SLBA = 0 and NLB = 1 (Sector 0).

An NVMe drive that supports Dataset Management command is also needed.

Supporting Dataset Management command can be checked by bit 2 of "Optional NVM Command Support (ONCS)" field in Identify Controller Data Structure.

If the bit is set to 1, the controller support Dataset Management command.

For further information, see "5.15.3 Identify Controller data structure (CNS 01h)" in the NVMe specification[2].

An NVMe drive that return fixed data pattern for reading deallocated logical block is recommended for checking the result of deallocation.

It can be checked by bit 2:0 of "Deallocate Logical Block Features (DLFEAT)" field in Identify Namespace Data Structure.

If it is 001b, the drive returns all bytes cleared to 00h for reading deallocated logical block.

If it is 010b, the drive returns all bytes set to FFh for reading deallocated logical block.

For further information, see "5.15.2 Identify Namespace data structure (CNS 00h)" in the NVMe specification[2].

Caution

Issuing "Write", "Dataset Management (Deallocate)", and "Format NVM" command to devices under management by file systems (e.g. FAT, NTFS) will corrupt your system.

These commands should be issued with highest care.

"Read" command does not have such a side effect... :-)

Environment

Confirmed on the following software environment:

  • Operating system and device driver
    • Windows 10 Professional 64bit (Version 2004, Build 21286.1000)
    • stornvme.sys (version 10.0.21286.1000, WinBuild 160101.0800)
  • Developping environment
    • Microsoft Visual Studio Community 2019 (Version 16.7.6)
      • Microsoft Visual C++ 2019
      • Windows Driver Kit (10.0.19030.1000)

Limitations

Only tested with the NVMe drive directly attached to PC via PCIe.

It is ok to access to M.2 drives and M.2 drives that attached to M.2-PCIe converter Add-In-Card (AIC).

Also, it may be ok with U.2 drives.

But it may not work over protocol translations such as usb-nvme.

To build and run program

  1. Double click NVMeTool.sln to start Visual Studio
  2. Build program with Build -> Build Solution (or Control+Shift+B)
  3. Run program (F5)

Note that, when you run (debug) in Visual Studio, confirm that the command line argument is set to appropriate drive ID for your system.

You can find the settings of command line arguments from Debug menu -> NVMeTool Debug Properties -> Configuration Properties -> Debugging -> Command Arguments.

Future Works

  • Test commands, LIDs, FIDs, and CNSs in the NVMe specification and listed in [8]
  • Test in WinPE mode

License

This software is released under the MIT License and Microsoft Public License (MS-PL), see LICENSE.

Reference

[1] Microsoft, "Working with NVMe drives", https://docs.microsoft.com/en-us/windows/desktop/fileio/working-with-nvme-devices, Retrieved May 14, 2019

[2] NVM Express, "NVM Express[TM] Base Specification", Revision 1.3d, March 20, 2019

[3] Microsoft, "SCSI Pass-Through Interface Tool", https://github.com/microsoft/Windows-driver-samples/tree/master/storage/tools/spti, Retrieved May 14, 2019.

[4] Mark E. Russinovich, et al., "Windows Internals, Part 2", 6th Edition, October 2012, Microsoft Press, ISBN: 978-0735665873

[5] Microsoft, "IOCTL_STORAGE_MANAGE_DATA_SET_ATTRIBUTES IOCTL", https://docs.microsoft.com/ja-jp/windows/desktop/api/winioctl/ni-winioctl-ioctl_storage_manage_data_set_attributes, Retrieved May 14, 2019

[6] NVM Express, "NVM Express[TM] Base Specification", Revision 1.4b, Sept. 21, 2020

[7] Microsoft, "NVMe Features Supported by StorNVMe", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/nvme-features-supported-by-stornvme, Retrieved June 25, 2020

[8] Microsoft, "StorNVMe Command Set Support", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/stornvme-command-set-support, Retrieved June 25, 2020

[9] Microsoft, "NVMe Features Supported by StorNVMe", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/stornvme-feature-support, Retrieved June 29, 2020