Move from firejail to docker for building docs

[nobodywasishere]

Docker is what docs.rs uses, and it would give a lot more flexibility towards the environment docs are generated in, allowing for using older versions of the docs generator, adding custom libs, better security, and easier setup.

docs.rs uses rustwide, a secure execution environment built on Docker. A similar shard should be developed for Crystal for safe execution.

[nobodywasishere]

It would be beneficial as well to be able to run all the crystaldoc programs via docker with one command - if that's feasible.

[straight-shoota]

Alternative: Make everything uper simple and build docs directly in the host environment.
Ideally doc generation should require no input except from the source code itself. So it should be completely safe to do without a sandbox and should not need any custom libs etc.
That would of course require a compiler option to disable arbitrary code execution and macro run.

Installing multiple Crystal versions in parallel should be no issue.

I believe this could be a very feasible alternative to building more and more complexity for the relatively simple task of extracting comments from the source code and turning it into HTML.

[nobodywasishere]

That would of course require a compiler option to disable arbitrary code execution and macro run.

I'm not sure how this could be accomplished without disabling macro file access altogether, which may limit some documentation generation. Not that I'm opposed to a compiler flag to disable some of these things, I just don't think it's possible to completely eliminate risk while keeping Crystal's macro system intact.

for the relatively simple task of extracting comments from the source code and turning it into HTML.

I don't think this is a simple task though. It requires parsing all of the shard and stdlib code and doing a top-level semantic. If it weren't for Crystal's macro system it could potentially be a lot simpler.

I do like the macro system, but the way it's currently implemented puts a large hamper on what can be done here, and for tooling support, plus requires doing parsing all the way up to the final stages of semantic analysis. I think it may be interesting to explore a more structured macro system for 2.0, but that's neither here nor there.

[straight-shoota]

I'm not sure how this could be accomplished without disabling macro file access altogether, which may limit some documentation generation.

Accessing files should generally be fine, as long as the executing user doesn't have access to any sensitive information. Which it shouldn't anyway.
Perhaps it could be tightened down a bit more by restricting file access to the current source tree (either directly in the compiler or via access restrictions in the build system).

I just don't think it's possible to completely eliminate risk while keeping Crystal's macro system intact.

The vast majority of macros are self-contained within the source code. There are only a few features which break out from that, and they are used relatively rarely. And I'd argue that most of the time when they're used, they are more of a crook than an ideal solution. We should offer better alternatives for these use cases.
Anyways, macros that rely on external information are unlikely to influence the API. So in most cases it should be quite possible to skip that when generating docs. This may require some macro conditions to handle things well in doc mode, but that shouldn't be hard and overall lead to a much simpler process for generating docs, with reduced complexity of the entire system.

[nobodywasishere]

Perhaps it could be tightened down a bit more by restricting file access to the current source tree (either directly in the compiler or via access restrictions in the build system).

If this could be added that would be much appreciated. That's one of the features of firejail and why I reached for it (along with restricting network access and restricting how much disk space / memory docs gen could take up).

And I'd argue that most of the time when they're used, they are more of a crook than an ideal solution. We should offer better alternatives for these use cases.

I agree.