Understanding the source code

[fullofcaffeine]

Hey Skial,

I was curious about how the web app for Haxe.io was implemented and started digging the code. At first I thought uFront was used, but doesn't look like it.

But what called my attention was the Main.hx file. Why is its body almost all commented out?

[skial]

Hi @fullofcaffeine,

Unfortunately your timing to look into how haxe.io works is bad, as last weekend I had an issue which required me to rebuild all the plugins, but I unfortunately failed to keep the dependencies in check. Basically, you are extremely unlikely to build haxe.io with everything in its current state.

If your interested in how it used to work and its current state, more in-depth detail below.

How it used to work.

The main program which handles all the file discovery is Tuli which consists of LibRunner.hx and Tuli.hx. LibRunner.hx is the entry point from the haxelib command haxelib run tuli.

• LibRunner.hx depends on the cmd library which consists of Lod, a command line arguments parser, Liy, which at runtime using reflection attempts to map the commands to methods and properties and finally Ede, which does the same thing as Liy but as a build macro.
• Tuli.hx depends on the Tappi library which finds and loads other neko files as plugins.

Tuli reads a config.json file located in the root directory, which lists the plugins needed to build the final static output. Tuli has a bunch of plugins included which are built as individual neko files. This repo also includes some site specific plugins.

I won't go into detail on how each plugin works, but the basis is that each plugins entry point, usual static function main() {..} returns itself which is then stored by Tappi. When Tuli gets the map of loaded plugins, it creates an instance of each class and passes to the constructor an instance of itself. In the plugins constructor it can register callbacks with Tuli.

Tuli allows plugins to register to run on every file found by Tuli, to run on a specific file extension, to run on any data loaded and to run when Tuli is about to finish.

If any file content was read, it will be cached by Tuli, also if any content was generated by a plugin it also is cached by Tuli, all of which was then saved to the output directory in the config.json file.

Current refactor

Because of the issues I've had last weekend, I'm rebuilding each plugin as a separate, individual command line tool so no dependency on Tappi and no dependency on Tuli, the Haxe community really doesn't need another build tool.

So far the markdown and codehighlighter plugins now run as standalone command line tools. I'm currently working on fisel which will be an enhanced version of the ImportHTML plugin but as a command line tool. I plan on integrating this into Tem in the distant future 😉.

If you have any other questions, I'll happy to answer them.

[fullofcaffeine]

Interesting, thanks for taking the time to explain. I now see it's a preprocessor/generator.

I will close this one and wait for the updates. Maube this could be extracted into a generator web fw for Haxe, like Jekyll/Middleman is for Ruby?

[skial]

Yep, thats what Tuli was initial created for but its similar to metalsmith as it only handles loading plugins and files and then passing files around to those plugins.

My problem came from the ImportHTML plugin, which was caused by my fork of Detox by @jasononeil, which uses the HTML parser in Mo on sys platforms. Detox by default uses the Xml parser from the std library which is just too strict.

So Tuli works well enough to load plugins, pass files around and file creation, as long as each plugin follows this format shown between lines 17 and 24 and compiled to a neko file.

I'll try and remember to post to this thread when things get more stable/usable 😄.