Future of the library

[tailhook]

I was used to thinking that this library is not needed anymore because clap and structopt do the job just well. But there are valid points by @d-e-s-o, the most important of it is that clap is very large in size.

On the other hand, I very much like the model structopt provides (i.e. parse in the structure instead of complex borrows). The actual semantics of structopt annotations are sometimes ugly though because it is a thin wrapper around clap (with different actual syntax).

So we have a few options:

1. Continue support of argparse as is (we need to write rust-style docs, as well as fix few current bugs, requiring breaking changes)
2. Make structopt-like wrapper (I mean derive) on top of argparse-based parser
3. Try to optimize clap+structopt if possible

[d-e-s-o]

Thanks for bringing this topic up for discussion.

I agree that structopt provides a nicer model to work with and I would be in favor of moving more towards that. But at the same time: I don't think that is a must. The degree of "pain" (for lack of a word with less negative connotations) of using argparse directly depends on how many things you need it for. For small programs the existing model is fine, I would argue. And even for a program with a good amount of parameters and subcommands it's not the worst (some getopt style solution is probably much worse). It's a compromise as so many things (which is also what I was getting at when saying that "it fills a niche") :-)

In nitrocli, a project we currently use the crate in, we've gone through some effort to generate things such as enums used by argparse and descriptions of parameters. Right now that is only happening through the usage of declarative macros and very limited. I plan to look into this a bit more and perhaps the results of that investigation could be of benefit here (although I should mention that this is not a priority of mine right now and I have limited time available for the project to begin with; I also don't necessarily want to go down the procedural macro route unless I see a big need).

That being said, I don't think points 1. and 2. are necessarily mutually exclusive. As you said 2. could easily be an addon, if, say, it was procedural macro based. As for point 3., it may be worthwhile getting in contact with the authors of clap and structopt to understand whether that is a concern of theirs, if they have done anything to do about that, and to see if the root causes are understood. Perhaps we should also first check if structopt incurs the same binary bloat, which I assumed but did not verify (if the compiler is unable to eliminate much code for clap for a simple project it appears unlikely that it will do so in the context of another crate built on top; but we should be sure).
As for clap I am fairly confident that the author is interested in getting the size down as there are multiple articles on the subject (this one that I linked in the pull request already, for example). However, given what I have read so far I am cautiously pessimistic about the outcome. I shall inform myself some more on what has been done and where things are, though.

My suggestion would be as follows:

1. Check whether structopt is as bloated as clap (I would probably be able to report something on that front unless you want to look into that @tailhook)
2. Read up on clap's optimization attempts and see if
   2.1) The root cause of the bloat is fully understood
   2.2) We can do anything about that
   I plan on reading what the author of clap has written up in full. Of course, it doesn't hurt if multiple people have that background.
3. At the same time (mainly for the reason that I don't think moving closer to the structopt model is a must) I do think that Make Print action print to the proper io::Write object #50 should be merged and a new version released, as it is a bug fix. Okay, that one is driven by my need for the fix :)

[d-e-s-o]

I have updated numbers now that include structopt. Note that total numbers are smaller than those mentioned here #50 because I updated to Rust 1.32 which changes the allocator used (removing roughly 324 KiB from binaries according to measurements I've done in the past).

(unit is bytes)

With default settings:
structopt: 862432
clap: 866528
getopts: 358624

When optimizing for size (opt-level = "z", lto = true, codegen-units = 1, incremental = false):
structopt: 575712 (d-e-s-o/dictcc-cli@6cad765)
clap: 579808 (d-e-s-o/dictcc-cli@0a48fe9)
getopts: 280800 (d-e-s-o/dictcc-cli@06c7e73)

We can see that while structopt does not add anything over clap (in fact, it reduces the overall size slightly), it still plays in the same ball park. For small programs, I would argue, the increases that clap and structopt incur are quite significant.

[tailhook]

It's still not clear how much argparse adds. In my expericence, it's ~100Kb, which isn't too small either :(

Other than that, I haven't had time to investigate it myself, yet...

[d-e-s-o]

It's still not clear how much argparse adds. In my expericence, it's ~100Kb, which isn't too small either :(

Yeah, I first wanted to address the structopt situtation. Anyway, now tested with argparse I get the following:

argparse: 301280 (d-e-s-o/dictcc-cli@e64a968)

So that's a bit more than getopts but still significantly less than clap or structopt. I'd also not make this only about binary size. The number of dependencies (including transitive ones) is another factor influencing build time and maintenance effort (keeping up & auditing), among probably others. And while clap/structopt are not the worst offenders in this space (having less dependencies than other crates), they still have infinitely more than argparse :P

[d-e-s-o]

I read through the two articles that the author of clap has published. I doubt we can do much on this front (for the following I equate clap and structopt as the size of the former directly influences the size of the latter by virtue of structopt just being a wrapper; we have seen very similar binary size characteristics).
The size truly seems to come down to code (.text) and the author already has made effort to get this down. It seems unlikely that someone less familiar with the code base can change that much for the better, given that there was already work on this front that picked the low hanging fruits.

[cronokirby]

I like this crate more than structopt, mainly because it's quite a bit more lightweight, and straight to the point, since it avoids dabbling in macros.

[ijackson]

I am using this library primarily because clap (and therefore structopt) does not understand (and so cannot be used to implement) conventional Unix command line parsing. For example, it is usual for a Unix program to understand later options as overriding earlier ones, but clap treats them as conflicting. You can't even do something as simple as have a boolean --thing --no-thing pair where --no-thing can be used to override an earlier --thing. There are other anomalies and clap in particular lacks the flexibility that argparse provices.

I would love to have something more declarative, structopt-style. I don't care about the code size very much :-).

To go back to this library, my view is:

• The documentation needs a lot of work.
• Some of the liftetime requirements are pretty annoying. For example, add_option et al insists on taking &'parser str for help text etc. This makes it more awkward than necessary to have programatically-generated strings. IMO these parameters should all be Into<String> and argparse should make a copy. The extra heap allocations are trivial in cost compared to program execution so we should provide an easier interface and not try to hyperoptimise by taking references to avoid copies.
• Different options affecting the same setting (Ref) should be able to have different metavars.
• There are some aspects of the API a that are un-idiomatic (at least by modern standards). For example, StoreOption ought probably to be called StoreSome.

Some of these changes would be API breaks.

[tailhook]

@ijackson I agree with most of what you have said. I think this was my first project in Rust and dates back to Rust ~0.7, so certainly needs updating. And I'm also not happy with some of the clap's limitations. However, I do not have enough time to work on this now.

[ijackson]

@ijackson I agree with most of what you have said. I think this was my first project in Rust and dates back to Rust ~0.7, so certainly needs updating. And I'm also not happy with some of the clap's limitations,

Well I think you have done well for your first project then! Thank you :-).

However, I do not have enough time to work on this now.

I may find time to do so. If I do, should I submit MRs, or pick a new name and fork it, or what? I guess the first step would be an MR with a branch containing proposed changes.

[sunshowers]

You can't even do something as simple as have a boolean --thing --no-thing pair where --no-thing can be used to override an earlier --thing.

BTW this can be done with overrides_with in clap.