Replies: 6 comments 8 replies
-
|
I'm still getting familiarized with the JSON spec. Here's how I'm making sense of the proposal: I conceptualize the LMT hierarchies as System::Link::TorV::Port::Lane::Offset. The testStep corresponds to the Port scope, because a Port has one common test spec, and Ports are tested sequentially. On the other end of the hierarchy, the lowest level of an LMR measurement has the following raw data: step (with +/- direction), status (response payload[7:6]), error count, sample count. That doesn't align with the MeasurementSeriesElement vs. the validator scheme. I see why we have to interpret it into a "bit errors" measurement. Do we plan to keep the raw data in this output? Somehow, I feel it's more appropriate to assign the lane number, instead of the BDF, to the measurementSeriesId. Then each measurementSeriesElement can represent an offset. Overall, the LMT outputs a lot more details that we need to tuck away in the extension fields. The LMR reported capability parameters may go into the MeasurementSeriesStart.Metadata. We should standardize the format for that. |
Beta Was this translation helpful? Give feedback.
-
|
Summarizing the proposal provided by Dan via email:
Summarizing the discussion happened on 10/5 meeting with Dan (Google), Hua (Google), Francesco (Meta), Leland (Meta), Adrian (Meta), Sathish (Meta):
Next steps:
|
Beta Was this translation helpful? Give feedback.
-
|
@sksekar, Adrian and I had a 2.5hr meeting on 2024-02-23. Here are my takeaways from the meeting: The spec we are defining is to facilitate the consumer of the diag output to interpret, to process, and to present the test result. The diag themselves may also play a role in processing and presenting the readings from the hardware. The spec should support a variety of LMT diags and use cases. Therefore, it needs to be expandable and unrestrictive as much as possible. Our approach is to each come up with a spec proposal that suits our existing diags: https://github.com/opencomputeproject/ocp-diag-pci_lmt and https://github.com/google/pcie_lmt. We then resolve any conflict between them and find common ground. We have agreed on a hierarchy of the LMT subjects (LMT is the diag. LMR is the PCI-SIG-specified PCIe feature. LMT conducts LMR.):
At the very basic level, an LMR step margin operation returns three readings from the hardware: status, error_count, sample_count. We should start with mapping those to the output spec. This enables raw measurement collection. Here are a few rule-of-thumb I learned about mapping info to the output spec:
Based on the above takeaways from the meeting, here's my proposal. Let's first review the ideas. The details need to be refined. An LMT subject is mapped to a subcomponent. Here's an example : The name The "location" parser pattern for "PCIELMT-MARGINPOINT-PCI" is There are three measurements per a The value is specified by the PCIe spec: 3:NAK; 2:Margining in progress; 1:Set up for margin in progress. 0:Too many errors There are another set of raw measurements which are the RX LMR parameters read from the lane. These include Those measurements has the RXLANE subcomponent: Also at this RXLANE level, the diag can output processed measurements, such as I can't think of anything need to be specified at the TestStep level. One consideration is that the amount of measurement output can be a lot if we require the raw measurements. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @makestuffwork for summarizing our discussion and thanks @mimir-d for your inputs during the discussion. Here is my proposal for the https://github.com/opencomputeproject/ocp-diag-pci_lmt tool. Mapping System HierarchySystem Hierarchy
My proposal (slightly different from what we discussed) is to map the System level info in dutInfo and Device level info in hardwareInfo within the testRunArtifact as below: Open Questions:
Mapping Margining CapabilitiesFollowing capabilities are read from the device's config-space.
Note: These are identical across all the lanes (within the link) and hence it's sufficient to report per Rx Device level (instead of per Rx Lane level) Current proposal is to map the above information as individual Measurement in testStepArtifact for each Rx Device with optional validators (limits are available in the spec): Sample: Margining ResultsFollowing information are reported by the device to determine the status of margining at a given Margin Point:
Note: These values need to reported for each lane within the link and hence this will be the majority of the output. Current proposal is to report the ErrorCount using the MeasurementSeries in testStepArtifact. The SampleCount and MarginStatus are surfaced using Metadata Sample: Questions:
|
Beta Was this translation helpful? Give feedback.
-
|
in reverse order, I can address some of the questions
in
in
in
So, the answer is:
Those sound to me like subcomponents (and it matches with the spec semantics as the "lane", "receiver", etc are not physical). However, you can choose to increase the granularity of the
That is expected. The diag output as per the spec semantics should be as complete as possible and let the consumer filter whatever they need. Having said that, there is nothing to say that you couldnt filter the output from the diag by providing some input parameter. both @sksekar @makestuffwork your mappings ( |
Beta Was this translation helpful? Give feedback.
-
That is a valid assumption. However, validating the assumptions is the purpose of testing. We can totally apply one parameter readings across all lanes as a test condition. However, when we treat the parameter readings a measurements, it should be attributed to the RxLane from where it's read. We should always leave room for defects.
I remember that we gave up the MeasurementSeries approach during our first meeting, because of its specific use case which is not applicable here. For example, the MarginStatus is a reading from the hardware. We want the spec to make sure that the consumer can get this data from the diag. Therefore, it deserves to be aMeasurement on its own. The Metadata is too loosely specified for that purpose. Let's talk more about this.
I think the common ground here is that they point out where the measurement is made, and we can use the Subcomponent to capture that info. The difference is only in the format. If you look at my "PCIELMT-MARGINPOINT-PCI" Subcomponent format example, we can use various format identifiers and parser strings to accommodate both scheme. |
Beta Was this translation helpful? Give feedback.
-
@sksekar please confirm if this measurement series is time based. Are those measurement element sample points taken at different points in wallclock time? If so, that would be a valid use of series on a related note, looking at the json, i think we can actually use the id to encode the testing scenario, so instead of "measurementSeriesStart": {
"measurementSeriesId": "0",
"name": "0000:00:01.1-timing_margin_right-step10-error_count",it could be "measurementSeriesStart": {
"measurementSeriesId": "0000:00:01.1-timing_margin_right-step10",
"name": "error_count", |
Beta Was this translation helpful? Give feedback.
-
|
I want to give the diag the flexibility of using either measurement series, or a single measurement just for the end BER. However, Measurement does not have a measurementSeriesId. Also, there are three readings per MarginPoint:
So, we need three measurement series or measurements, and we need to use the name to identify each. For the purpose of maximizing flexibility, I want to use the Name field to identify how to interpret the info in the object. Then the Name field should not need parsing. I don't think there's a better place to encode the timing/voltage offset than under the Subcomponent. We can attach the same Subcomponent to multiple measurement series and/or measurements |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @mimir-d. Some comments/follow-up
SGTM. My preference would be use
Looking closer, I think we can surface these information as part of Device/Margining Capabilities measurement which is already surfaced per Device.
Yes, that's correct. In our case, user is allowed to select the goal (eye-scan vs spot-check) using the config file provided as input.
Yes, there is no restriction in doing both (eye-scan and spot-check) as different steps. |
Beta Was this translation helpful? Give feedback.
-
|
As discussed, attached is an output example for our discussion: |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing the output. This helps answering multiple questions that I had in my mind. But, there are few more:
My solution is almost the same with few minor differences:
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the quick review! First, that output example is by no means my ideal. It's mainly to demonstrate some constraints we are facing, and the flexibility we desire. The diag execution has at least two dimensions: One is time, and the other is the system components which can be hierarchical, parallel, or a mixture of both. One constraint is that the TestStep forces the output into a single chronological thread. Therefore, we have to use the Subcomponent to traverse the system within that single thread. This was brought up a couple weeks ago, and I remember @mimir-d was open to support a threading nomenclature in the output spec. Yes, my LMT tests multiple lanes simultaneously and independently for the sake of test time saving. The PCI-SIG's LMR spec has its own "validator", and the checking result is reflected in the PCIELMT-MARGINPOINT-STATUS measurement. My intention is to keep my LMT within the scope of PCI-SIG's standard. I use it as a generic bench testing tool. To use it as a DUT-specific production diag, there is another piece. It translates the PCIe BDF to HW location for repair instructions. It can also add the OCP Validator to qualify the PCIELMT-MARGINPOINT-STATUS, or any other measurement. Those are very production-DUT-specific. I want to standardize the flexibility. Therefore, I use the "subcomponent.name" field to indicate the format of the "location". For example, the name PCIELMT-MARGINPOINT-PCI has three parts. PCIELMT is a self-context identifier. MARGINPOINT identifies the info type. PCI identifies the format as PCI enumeration, rather than a hardware port. If a new diag has info that does not fit into this format, it can define a new "name". This does seem very complicated. It'd be nice if the OCP format spec can help simplify it. Another guideline-constraint is that the diag should report all raw measurements. However, the output consumer may not care about all the details. An interpreted PASS/FAIL may be sufficient. The raw measurement can be dumped to a separate non-OCP-format log file for offline analysis purpose. Then, it makes this LMT output much more consumer-centric and standardizable. |
Beta Was this translation helpful? Give feedback.
-
|
I asked around. There is nothing preventing us from using the TestStep to model each parallel executing test threads. If so, I propose to use TestStep to model at the receiver-port level. For example: "BUS:a1h--RX:6". There is still parallelism at the lane level within a TestStep. However, the receiver-port usually points to a hardware component that's not sub divisible, and subject to a repair. |
Beta Was this translation helpful? Give feedback.
-
|
We saw the ocp-diag-core-viewer demo a few weeks ago. With that, I tuned the pcie_lmt OCP output the way I'd like to see as a user. This pcie_lmt_ocp.json is a sample output. I can also demo it from the ocp-diag-core-viewer in the meeting. The pcie_lmt can stream the OCP artifacts to a file or a named pipe. Our use case has a diag-runner which creates this OCP pipe and listens to it. It converts the PCIe-domain BDF info to the DUT-specific HW-Info.This way, the pcie_lmt can stay generic and only PCI-SIG-aware. The diag-runner is also generic in the sense that it can run various diags as a sub-process. The pcie_lmt runs parallel TestSteps, each maps to an RX-port. Within each TestStep, the lanes are also running in parallel. I'm counting on the sorting and filtering features of the result viewers. Instead of dumping all the raw measurements, I now only output what "matters". The raw measurements are still dumped in a log for reference. As a user, I'd like to see the interpreted results upfront. So the pcie_lmt outputs eye size, eye corner margin, BER, and/or status. Irrelevant and/or implied information, such as 0-error margin points in an eye-scan, are omitted. Still, there are more info to fit in a Measurement artifact. I'm overloading the |
Beta Was this translation helpful? Give feedback.
-
Objective
Add support for emitting test results compliant with OCP Test and Validation Output Specification from the PCIe LMT Diagnostic tool.
Background
A sample LMT test run performs:
Current Output Format
CSV format
JSON format
Proposal
Current proposal is to use:
Sample Execution
pci_lmt -o ocp config_file
To Be Discussed
Beta Was this translation helpful? Give feedback.
All reactions