Use a CI configuration file instead of a build script

[NiklasRosenstein]

It should be possible to build the same repository with different options and environment variables from the same push event. One build would need to be split into multiple builds and we need a new entity "Push Event" (or alike) that contains all the builds triggered for that event.

It should be configured in the .flux.ci CSON configuration file. (See also #23)

matrix:
  <build_name>:
    <override_key>: <override_value>

[gsantner]

I would go to not reinvent the wheel. I think gitlab CI has already a pretty nice format here, which we don't have to support fully, but to have the basics compatible. This is basically:

jobname:
  script:
    - mycmd1
    - mycmd2
  artfiacts:
    - dist/

this could e.g. look like this

build:coolapp:
  script:
    - gradle assembleDebug
  artfiacts:
    - app/flavor*/*.apk
build:somethingelse:
  script:
    - echo "something"

This would be what I first go for - step by step. For now to just parse this, to execute, build and get the results. And that one after another. Next up step could be e.g. specifying stage which allows to order execution steps (as in matrix).

This basically too depends for me on #41 as I want to try to setup CI/CD for projects I don't have push access to but want to provide nightly builds for.

[NiklasRosenstein]

I'd be totally in for that kind of CI spec format. I would very much like to avoid YAML though.

[gsantner]

YAML is what you suggested at your very top comment too :D - and most CIs use it currently. I think it's best to also go this way. Easier to go over to Flux-CI then when coming from any other CI.

[NiklasRosenstein]

No I was suggesting CSON ;-)

It should be configured in the .flux.ci CSON configuration file. (See also #23)

It is very similar to YAML (uses indentation, can have keys without quotes). But you can not specify a string value without quotes, which I think is a very good thing.

It's probably quite simple to just support both flavors, so YAML by default is fine to make it easier for people used to other CI systems.

[gsantner]

If the only big difference for our goals are things like quoted vs unquoted or different array specify syntax - I think it should not matter - as long as it gives the expected model object with properties.

I really thought your example is YAML, so I don't think it gonna be hard beside throwing file to parser - from config file view.

May I ask for #23 to get closed/merged into this issue? It is basically about the same but non-technical

[NiklasRosenstein]

I would definitely prefer CSON, but as I said before, it should be really easy to just support both formats as the actual structure you get from parsing them should be the same.

May I ask for #23 to get closed/merged into this issue? It is basically about the same but non-technical

A'ight, you're too fast my friend. I was actually about to do that. :)

[gsantner]

So, the CI parser if mostly done. It has to the biggest extend comptibility with GitLab CI, Travis CI and is more convienient use. It does so by doing many refactors etc so it is just one hierarchy that is all same for all jobs.

For example the two script lines at bottom example will get converted to a script: block, so it works like every other job. For example we just have todo for loop over script block then, and don't have to think about if it's existing, and how it is formed. I made sure it's always existing, and a list of strings. Same I have done for everything else (according to how it is expected).

job:name:
  stage: test
  script:
    - echo hello
    - echo world
job:name:
  - echo hello
  - echo world

The whole thing I have developed works with a standard Python Dictionary. At it's base it supports to load from YAML, or dictionary. This means if someone wants to load from web (e.g. JSON), or some CSON this will totally possible - as long as python can make a dictionary out of it, and the blocks are correctly ordered. And this is the case for most exchangeable data formats I think.

It also support injection of variables. This means when we get variables like BUILD_FEATURE_X=1 from web-api, you can easily pass this in. The same goes for other variables, CI infos etc. (Like Build number).

What it too allows already is to export a hierachy (which we can then use for example for html - job matrix table generation).`

What I don't know is, how to get it in best now, I too want to make sure the parsing and converting stays seperate. We get a fully fledged dictionary out of it, which we just have to use.
At this point I'm asking for some help - can we go something like I show it, describe it at best and you guys (@NiklasRosenstein @tvrzna ) do more of the integration part? Summed up, this was work of many nights across 3 weeks :D, would really love to get it in use now :D.