Reorg Notes

We recently reorganized the original fds-smv GitHub repo. We did this for two reasons: (1) the repo had grown to over 2 GB in size and (2) we wanted the flexibility to allow CFAST and FDS to utilize the same experimental data repository.

One of the most difficult issues in reorganizing is maintaining the project history, including issues. Also, to reduce the size of the repository requires rewriting its history with BFG. Migrating issues from one repo to another is quite complicated. In fact, as of now, GitHub does not have an official way to do it, though they are working on it here.

Because of these complications, we decided to take the following approach: We first cloned the fds-smv repo to fds-smv_deprecated and froze development as of 9-2-2016. This repo can still be accessed via the Fire Models organization. We then created new repos to hold Smokeview (smv), Experimental_Data (exp), FDS_Output_Files (out), and FPE Empirical Correlations (cor). Files from the old repo were transitioned to the new repos while, as best we could, maintaining history of the files. Note that, because of the new directory structure, files may look as though they have a short commit history on the GitHub site. But this history is only for the current directory structure. If you checkout an old commit (on a clean repo)---if you are doing a bisect, for example---then you will get back the old directory structure. Last, we cleaned fds-smv and renamed it fds. In this way, fds maintained its issue history and also its wiki repo (though the wikis could have been migrated more easily).

The downside of this strategy is that your forked fds-smv repos now track the new, cleaned fds repo. You DO NOT want to merge the new repo into your old fork. Here is what you need to do to get going again.

##Reorg Checklist

###1. Backup your fork

If you have any outstanding commits relative to when we froze the fds-smv repo, make sure you save these. If you were not up-to-date with fds-smv as of 9-2-2016, you will need to add the new fds-smv_deprecated repo to track as a remote. Then you can merge any changes to your local fds-smv repo. Below I assume the local repo is named fds-smv_fork and that you are currently on the development branch.

$ git remote add deprecated [email protected]:firemodels/fds-smv_deprecated.git
$ git remote update
$ git merge deprecated/development
...

###2. Delete your forked repo

Now go to your GitHub account and delete the repo that currently says forked from firemodels/fds. It may take some time (10 min or so) for this change to register on GitHub. If you get an error in step 3 (next) you may need to wait a little longer.

###3. Fork the new repos

You will need to make new forks of any repo you plan to make pull requests on. If you are working with fds, you will also need exp and out. To fork the repos, go to the specific repo on the GitHub site and click the "fork" button (upper right). GitHub will ask you where you want to fork the repo to (if you are a member of other organizations, you may have several options). It will also tell you that you have forked this repo before (you did, when it was called fds-smv). Just select your GitHub space and continue.

###4. Clone your forked repos

Next, you need to clone each repo locally. We recommend you create a directory that identifies your repos as being forked. The repo names need to stay the same, as many scripts rely on relative paths between the new repos.

$ mkdir FireModels_fork
$ cd FireModels_fork
$ git clone [email protected]:<username>/fds.git
...
$ git clone [email protected]:<username>/smv.git
...
# exp uses macfp-db as a submodule
$ git clone --recursive [email protected]:<username>/exp.git
...
$ git clone [email protected]:<username>/out.git
...

You can now check the disk usage to see that fds has been drastically reduced in size (KB):

$ du -sk *
187472	exp
204876	fds
109976	out
84768	smv

###5. Set up firemodels for remote tracking

You now need to go into each repo and add the central firemodels repo for remote tracking so you can pull down changes made by others. For each repo (take fds as an example) do the following:

$ cd FireModels_fork/fds
$ git remote add firemodels [email protected]:firemodels/fds.git
$ git remote -v
firemodels	[email protected]:firemodels/fds.git (fetch)
firemodels	[email protected]:firemodels/fds.git (push)
origin	[email protected]:<username>/fds.git (fetch)
origin	[email protected]:<username>/fds.git (push)

##Workflow for New FDS Repos

If you are working on FDS, you will now be juggling three separate repositories: fds, exp, and out. As mentioned above, the names of these repositories cannot be changed, because many relative paths are hardwired in the process scripts. We suggest you create a folder (name it whatever you want) that indicates that these are your forked repos. Below I will assume you have created a folder called "FireModels_fork" to hold your forked repos.

###Merging changes from firemodels

The process for staying up-to-date with the central repo is more or less the same as before, you just have a couple more repos to deal with. If you like, Glenn has created scripts that will do some of the repetitive work for you. Below I will go a bit slower.

Unless you are committing new validation results or adding new experimental data, you can do most of your work from within the fds repo. To update fds from firemodels, as usual, do

$ cd FireModels_fork/fds
$ git branch -a
* master  # NOTE: NEW DEFAULT BRANCH IS "master"!
  remotes/firemodels/master
  remotes/firemodels/nist-pages
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
$ git remote update
$ git merge firemodels/master
...

Notice that the new default branch name is master.

If you are running validation scripts from within fds/Utilities/Matlab, remember that these scripts access both the exp and out repos, so you will want to make sure these are also up-to-date with firemodels.

$ cd FireModels_fork/exp
$ git remote update
$ git merge firemodels/master
...

$ cd FireModels_fork/out
$ git remote update
$ git merge firemodels/master
...

We do not expect exp to change that often, but you should check periodically. The out repo may change more often; as we get "Validation_Bot" up and running we may be pushing changes to output files on a regular basis.

###Validation

The post-processing scripts for validation will now copy your "Current_Results" to out/CASENAME/FDS_Output_Files/. Remember that this is now a separate repository. To commit these results, cd into FireModels_fork/out; do a remote update and merge to make sure you capture any updates to out from the central repo; then add, commit, and push your changes to your fork; then send a pull request to Fire Models. Suppose you are in fds/Validation/CASENAME/. After running your validation case, the results are in the local Current_Results directory. Run the Process_Output.sh script to copy the results to the out repo, then commit the results.

# make sure out is up-to-date with firemodels, see above
$ cd FireModels_fork/fds/Validation/CASENAME
$ ./Process_Output.sh
$ cd FireModels_fork/out/CASENAME
$ git status -uno  # you should see modified files
$ git add -u       # stage the files for commit
$ git commit -m "FDS Validation: updates to CASENAME"
$ git push origin master  # note default branch names are now "master"

Now go to GitHub and send a pull request to firemodels.