Refinement of survey responses - troubleshooting

[bertfrees]

We are currently in the process of analysing your responses to the Pipeline usage survey held earlier this year. Some of the feedback requires some more elaboration or discussion. Below are some responses in the topic "troubleshooting":

---

• Q: What do you think are the 3 greatest weaknesses of Pipeline?
  • Two organisations brought up the issue of diagnostic output that is hard to interpret:

    • Error logs are overcomplicated for non IT users.
    • Diagnostic output is hard to interpret, making error triage hard

• Q: Would you encourage other organizations to consider using Pipeline in their production workflow?
  • One organisation referred to Pipeline 2 as a "black box" that can be hard to diagnose:

    • It takes the work of one set of transformations off our plate, albeit at the cost of it being a black box that can be difficult to diagnose when problems occur.
    • There are lots of tools available within the Pipeline, while the black-box nature of it might make it hard for an organization to triage errors it runs into.

---

I would like to know:

• We would like to get some specific examples of diagnostic output that is hard to interpret, or things that make Pipeline a "black box".
• Does the detailed log need to be understandable by regular users? Or should we hide the detailed log from them and only use it to send error reports to the maintainers?

[torchtrust]

I agree with this as this happens here, but I do appreciate that some issues do require an understanding of DAISY 2.02 structure, but it would be good to have a simple PASS or FAIL, which it might already and I can appreciate that some issues are hard to summarise but the reports would be easier to digest if for example they said "one or more timing values are out of tolerance" and "Some Meta Data seem to be missing" etc.
HTH
Paul

[bertfrees]

@torchtrust I guess you're talking a different kind of troubleshooting than some other organisations. You are referring to the validation report output rather than the messages and log files.

The DAISY 2.02 validator output can indeed be a bit technical, as the implementation is based on RelaxNG schema. It would indeed be interesting to look into how we could make the report more human-readable.