| Author: | Tiago Alves Macambira |
|---|---|
| Licence: | Creative Commons By-SA |
Table of Contents
This is yet another guide describing how to setup private
HTTP-accessible Git repositories on Dreamhost using Git's
git-http-backend (a.k.a git's Smart HTTP protocol). While
similar guides can easily be found by the thousands in the Web (I've
listed some of them in the References section), I've found that some
guides have outdated information or that the setup described in
them could be improved. Thus, this guide tries to update, improve and
consolidate the information dispersed in such sources.
Some might ask "Why on Earth would someone opt to create its own private Git hosting solution while better offerings are available from sites such as, let's say, GitHub?" As the guy from RailsTips pointed out in one of his articles, sometimes you don't need or don't want to "share" a project with anyone but yourself and paying for a GitHub-like service might just not make sense. If that's your case, than this guide is for you.
While aimed at a Dreamhost-hosted accounts and the environment such accounts have as of 2010-10-17, I believe the process described here can be used in other hosting providers as well.
It is important to highlight that one of the objectives of this guide is to describe a process that:
- should be easy to perform by just renaming or editing this guide's companion files and
- once complete, can be easily be reused to generate other "collection of repositories", with different URLs and passwords.
No WebDav or SSH support is needed nor used for serving Git repositories.
Once again, the focus is on HTTP access using
git-http-backend. As for SSH, guides describing how to setup a similar environment for SSH-accessible repositories can be found easily on the web.Repositories will be password protected and available for both reading and writing.
As we will explain later, in the Setup git-http-backend for your repositories section, we will have to password-protect our repositories in order to be able to
git pushto them through HTTP.We will stick to a DRY (Don't Repeat Yourself) philosophy.
Thus, we want configuration options to be repeated in as few places as possible. We will use environment variables in Apache configuration files to do this. If this makes you uncomfortable, well, you can always manually spread configuration options all over the place. :-) Your call.
The web server being used is Apache.
Well, that is what Dreamhost allows me to use so that is going to be the focus of this guide. While it should not be that difficult to port the settings here to something suitable for another web server, describing how to do it is out of the scope of this guide.
We are able to run CGI scripts.
DreamHosts puts some restrictions on how a CGI script can be executed and the environment where it runs. We will abide to those restrictions.
ScriptAliasis not allowed by the web server.The instructions given in git-http-backend manpage will not work as they use
ScriptAlias. The idea is to use common CGI scripts andmod_rewriteinstead, roughly following the ideas presented in http://wiki.dreamhost.com/Git#Smart_HTTP .SuExec is used to run CGI scripts.
Notice that, as stated in DreamHost page on CGI,
SuExec"wipes out all environment variables that don't start with HTTP_". All our env. vars. will have this prefix.Your private git repositories will be accessible in a subpath of your domain.
The idea is that your private git repos will be available in an address such as http://www.example.tld/corporate-git/. Adapting the instructions below so you can serve them from the root of a domain of its own, say http://corporate-git.example.tld should be fairly simple.
This document and its companion files are initially hosted on http://github.com/tmacam/private-git-on-dreamhost.
The file README.rst is generated from README-real.rst. So, if
you plan on doing any updates or fixes, README-real.rst is the
file you ought to edit. Just run make afterwards in order to get
README.rst updated as well. This is done because I wanted to use
GitHub's automatic rendering of README files but I didn't want to
just paste the contents of the companion files in this README.rst
and risk getting the Guide and files out of sync. Unfortunately,
GitHub does not allow the use of RestructuredText's include
directive, so I had to fake it -- and here is the reason why we have
README-real.rst.
This guide is distributed under the Creative Common BY-SA license while companion files are distributed under a MIT License.
All the commands and instructions given bellow should be performed on the machine in Dreamhost where your account is installed. So ssh to it and let's start.
Well, this should probably be a non-issue since git comes pre-installed on most Dreamhost machines. To verify it:
$ git --version git version 1.7.1.1
As you see, the box that serves my domain in DreamHost has git version 1.7.1.1 installed. Anything greater than 1.6.6 shall do.
If you don't have git installed in you box, have an old version or if for some other reason your need to compile git, follow Craig's instructions in Hosting Git Repositories on Dreamhost.
It should reside somewhere not accessible from the web or directly
served by the web server. We will tell Apache and git-http-backend
how to properly and securely serve those repositories later. For now,
we want them protected from third parties.
Say we decided to store them in ~/private_repos/. We will refer to
this directly by GIT_REPOS_ROOT in the rest of this guide. Create
this directory and protect it against file system access from others:
export GIT_REPOS_ROOT="~/private_repos/"
mkdir ${GIT_REPOS_ROOT}
chmod 711 ${GIT_REPOS_ROOT}
We will use the script newgit.sh, presented bellow, to create new
repositories [1] [2] . Remember to modify
the value of the GIT_REPOS_ROOT variable in it to match our setup:
#!/bin/bash
# this script is based on code from the following blog post
# http://arvinderkang.com/2010/08/25/hosting-git-repositories-on-dreamhost/
# and http://gist.github.com/73622
set -e
# Please, configure a default GIT_REPOS_ROOT to match your config
#GIT_REPOS_ROOT="~/private_repos/"
DEFAULT_DESCRIPTION='no description :('
# describe how the script works
usage()
{
echo "Usage: $0 [ -h ] [ -r directory] [ -d description ] [ -n projectname ]"
echo ""
echo "If no projectname is given, the name of the parent folder will be used as project name."
echo ""
echo " -r directory : (root) directory holding your git repositories"
echo " -d description : description for gitweb"
echo " -h : print this screen"
echo " -n name : name of the project (should end in .git)"
echo ""
}
DESCRIPTION=${DEFAULT_DESCRIPTION}
# evaluate the options passed on the command line
while getopts r:d:n:h option
do
case "${option}"
in
r) GIT_REPOS_ROOT=${OPTARG};;
d) DESCRIPTION=${OPTARG};;
n) REPONAME=${OPTARG};;
h) usage
exit 1;;
esac
done
# check if repositories directory is given and is accessible
if [ -z $GIT_REPOS_ROOT ]; then
usage
exit 1
fi
if ! [ -d $GIT_REPOS_ROOT ]; then
echo "ERROR: '${GIT_REPOS_ROOT}' is not a directory"
echo ""
usage
exit 1
fi
# check if name of repository is given. if not, use folder name
if [ -z $REPONAME ]; then
REPONAME=$(basename $PWD)
fi
# Add .git at and if needed
if ! ( echo $REPONAME | grep -q '\.git$'); then
REPONAME="${REPONAME}.git"
fi
#
# Ready to go
#
REP_DIR="${GIT_REPOS_ROOT}/${REPONAME}"
mkdir ${REP_DIR}
pushd ${REP_DIR}
git --bare init
git --bare update-server-info
cp hooks/post-update.sample hooks/post-update
chmod a+x hooks/post-update
echo $DESCRIPTION > description
# This mark the repository as exportable.
# For more info refer to git-http-backend manpage
touch git-daemon-export-ok
popd
exit 0
Move or copy this file to an appropriate path (say, your home directory would be fine) and turn it into an executable:
chmod u+x ~/newgit.sh
| [1] | This script is based in http://gist.github.com/73622 |
| [2] | Other guides prefer to use something similar wrapped as a Bash function but I'd rather have it as a script |
Now, let's configure Apache to securely serve those repositories.
As we stated in Assumptions and requirements, we want to serve our files from
http://www.example.tld/corporate-git/. So, go to the directory
holding your domain files (~/www.example.tld, in our example),
create a corporate-git directory in it if it doesn't exist yet and create
a .htaccess file in it:
cd ~/www.example.tld mkdir corporate-git cd corporate-git export GIT_WEB_DIR=`pwd` # we will use it in later steps touch .htaccess chmod 644 .htaccess
Now, edit this .htaccess contents to match the text presented
bellow or just copy the contents of the file model-htaccess into
it and adapt it to match your config:
Options +Indexes
# GIT BEGIN ###########################################################
SetEnv HTTP_GIT_PROJECT_ROOT /home/user/private_repos/
SetEnv HTTP_GITWEB_CONFIG /home/user/private_repos/gitweb_config.perl
RewriteEngine On
DirectoryIndex gitweb_wrapper.cgi
# The following two rules can be used instead of DirectoryIndex
#RewriteRule ^$ gitweb_wrapper.cgi/ [L,E=SCRIPT_URL:/$1]
#RewriteRule ^([?].*)$ gitweb_wrapper.cgi/ [L,E=SCRIPT_URL:/$1]
# Everything else that is not a file is forwarded to git-http-backend
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^?].+)$ git-http-backend-private.cgi/$1
# GIT END ############################################################
# AUTHENTICATION BEGIN ###############################################
AuthType Digest
AuthName "Private Git Repository Access"
# UNCOMMENT THE LINE BELLOW FOR BETTER PERFORMANCE
# AuthDigestDomain /corporate-git/
AuthUserFile /home/user/private_repos/.htpasswd
Require valid-user
# AUTHENTICATION END ################################################
For now we will focus on the area between the # GIT BEGIN and #
GIT END blocks. Modify HTTP_GIT_PROJECT_ROOT to match you setup:
it should point to the full path where you store your private
repositories. Just expand the value of GIT_REPOS_ROOT to get this
information:
$ (cd ${GIT_REPOS_ROOT}; pwd)
/home/user/private_repos/
So, in our example, HTTP_GIT_PROJECT_ROOT value should be set to
/home/user/private_repos/, as presented in the example above.
Not we will create a CGI script that will invoke
git-http-backend. In your .htaccess this script is referred as
git-http-backend-private.cgi. Create it in the same directory
where your .htaccess is by coping the one that comes with this guide
to that directory or by creating an empty file with the following
contents:
#!/bin/sh
export GIT_HTTP_EXPORT_ALL=1
export GIT_PROJECT_ROOT=${HTTP_GIT_PROJECT_ROOT:?HTTP_GIT_PROJECT_ROOT env. variable not set. Aborting.}
/usr/lib/git-core/git-http-backend
Turn it into an executable file:
chmod 755 git-http-backend-private.cgi
Attention!
You may need to update the path to git-http-backend executable
if git was installed in a non-default location.
And that's it. No need to setup anything: all the settings this
scripts are passed to it through environment variables set by Apache
and defined in the .htaccess file.
From this point on you should be able to create repositories from the
command line and
access them through HTTP, but they will be
read-only. As stated in git-http-backend manpage, "by default,
only the ``upload-pack`` service is enabled, which serves git ``fetch-pack``
and git ls-remote clients, which are invoked from ``git fetch``, ``git pull``,
and ``git clone``". For write access, i.e., to be able to perform a
git push, the receive-pack service is needed, and it is only
enabled when the client is authenticated.
We are almost set. Let's configure password protection for this whole
thing. We will focus on the latter part of your .htaccess, the one
between # AUTHENTICATION BEGIN and # AUTHENTICATION END that we
reproduce bellow:
# AUTHENTICATION BEGIN ######################## AuthType Digest AuthName "Private Git Repository Access" AuthUserFile /home/user/private_repos/.htpasswd Require valid-user # AUTHENTICATION END #########################
You will have to create the password file pointed by AuthUserFile
and use the htdigest tool to add a user to this file
touch /home/user/private_repos/.htpasswd htdigest /home/user/private_repos/.htpasswd "Private Git Repository Access" username
You will be prompted for a password. And that's it.
Notice:
- we are using Digest Authentication. It is supposed to be more secure than plain authentication.
- The password file should be keep in a place not directly accessible
from the web. Ideally it should not even be placed in the directory
to be served by
git-http-backendbut I'm lazy and I hope this will be enough. :) - If you update the value of the
AuthNamesetting you must also change the 2nd. parameter passed tohtdigest, i.e., the Realm, as they must match! Odd, I know. But that's the way it is.
If you followed this guide up to this point than you are able to use your repositories with git with no major issues. But you will not be able to browse them with a web browser, retrieve the list of repositories you have, see diffs, commit messages nor nothing like that. To make things better, let's install GitWeb, another CGI interface that will provide a web interface that allows to do all those things I just said you couldn't.
Note
Most of the content in this section comes from Kang's Hosting Git repositories on Dreamhost.
GitWeb comes in the same source package as git itself. Unfortunately, Dreamhost doesn't install it by default so we will have to install it manually ourselves. Do your remember what is your git version? No? Find it all:
git --version
Go to git homepage and download the corresponding source
package. In my example, in which my git version is 1.7.1.1, I would
need to grab the git-1.7.1.1.tar.gz source package:
cd ~ # Yep, we will download it in our home directory wget http://www.kernel.org/pub/software/scm/git/git-1.7.1.1.tar.gz
Unpack it, build GitWeb:
tar zxvf git-1.7.1.1.tar.gz cd git-1.7.1.1 make prefix=/usr/bin gitweb/gitweb.cgi rm gitweb/gitweb.perl # we won't need it
We will install it into ~/gitweb/:
export GITWEB_INSTALL_DIR="~/gitweb"
cp -r gitweb ${GITWEB_INSTALL_DIR}
We are almost there.
Now, copy all the GitWeb's media files into the directory
where your .htaccess is:
cp ${GITWEB_INSTALL_DIR}/*.{css,png,js} ${GIT_WEB_DIR}
# in this example, GIT_WEB_DIR points
# to ~/www.example.tld/corporate-git
Get back to where your .htaccess file is
(i.e. GIT_WEB_DIR). We will create a wrapper CGI for
GitWeb. Just copy gitweb_wrapper.cgi or create an empty file with
the contents bellow:
#!/bin/bash
export GITWEB_CONFIG=${HTTP_GITWEB_CONFIG:?HTTP_GITWEB_CONFIG env. variable not set. Aborting.}
export GIT_PROJECT_ROOT=${HTTP_GIT_PROJECT_ROOT:?HTTP_GIT_PROJECT_ROOT env. variable not set. Aborting.}
# Replace "/home/user/gitweb/" for the correct path to where
# gitweb.cgi was installed.
/home/user/gitweb/gitweb.cgi
Turn it into an executable file:
chmod 755 gitweb_wrapper.cgi
Attention!
Remember to to update this file by replacing /home/user/gitweb
to match gitweb's install location, i.e., to match the contents
of ${GIT_WEB_DIR}.
Once again, we are using settings stored in .htaccess file and
passing them to a script using environment variables set by Apache. In
this case, we are informing the wrapper script where our repositories
are with HTTP_GIT_PROJECT_ROOT, and informing it where GitWeb
configuration file is with HTTP_GITWEB_CONFIG. The wrapper script,
in turn, will forward these informations to both GitWeb and to its
config file.
Now, let's create GitWeb configuration file. Just
copy gitweb_config.perl provided with this guide to
${GIT_REPOS_ROOT}/gitweb_config.perl or create an empty file in
that path location with the following contents:
# where is the git binary?
$GIT = "/usr/bin/git";
# where are our git project repositories?
$projectroot = $ENV{'GIT_PROJECT_ROOT'};
# what do we call our projects in the gitweb UI?
$home_link_str = "My Git Projects";
# where are the files we need for gitweb to display?
@stylesheets = ("gitweb.css");
$logo = "git-logo.png";
$favicon = "/favicon.png";
# what do we call this site?
$site_name = "My Personal Git Repositories";
You can customize it a little bit, if you want, but the most important
setting, $projectroot, is set to match the value of
HTTP_GIT_PROJECT_ROOT, a env. var. set by Apache.
Notice that this file, gitweb_config.perl is stored in the same
directory where your repositories are, in ${GIT_REPOS_ROOT}. If,
for some reason, you prefer to store it elsewhere, you will have to
update this information in the .htaccess file.
So, something is not working as expected?
Comment out the authentication code. This will ease your "debugging" process.
Remember to uncomment it later.
A nice way to check if there is something really wrong with your setup
is to use the info.cgi, whose code is presented bellow. This
script is only a minor modification to the one presented in Dreamhost
wiki page on CGI and allows your to do verify if you are able to
execute CGI scrips and what settings Apache is passing to the other
CGI scripts we use here.
#!/bin/sh # disable filename globbing set -f echo "Content-type: text/plain; charset=iso-8859-1" echo echo CGI/1.0 test script report: echo echo argc is $#. argv is "$*". echo echo SERVER_SOFTWARE = $SERVER_SOFTWARE echo SERVER_NAME = $SERVER_NAME echo GATEWAY_INTERFACE = $GATEWAY_INTERFACE echo SERVER_PROTOCOL = $SERVER_PROTOCOL echo SERVER_PORT = $SERVER_PORT echo REQUEST_METHOD = $REQUEST_METHOD echo HTTP_ACCEPT = "$HTTP_ACCEPT" echo PATH_INFO = "$PATH_INFO" echo PATH_TRANSLATED = "$PATH_TRANSLATED" echo SCRIPT_NAME = "$SCRIPT_NAME" echo QUERY_STRING = "$QUERY_STRING" echo REMOTE_HOST = $REMOTE_HOST echo REMOTE_ADDR = $REMOTE_ADDR echo REMOTE_USER = $REMOTE_USER echo AUTH_TYPE = $AUTH_TYPE echo CONTENT_TYPE = $CONTENT_TYPE echo CONTENT_LENGTH = $CONTENT_LENGTH echo "" echo HTTP_GIT_PROJECT_ROOT = $HTTP_GIT_PROJECT_ROOT echo HTTP_GITWEB_CONFIG = $HTTP_GITWEB_CONFIG exit 0
Copy it to GIT_WEB_DIR, turn it into an executable script
(chmod 755 ...) and point your browser to it ( That would be
http://www.example.tld/corporate-git/ in our example).
We are listing this as a last step but that's probably the fist place where you should have looked for clues: your server logs.
For example:
[Mon Oct 25 18:30:28 2010] [error] [client 150.164.3.192] Service not enabled: 'receive-pack'
This message says that 'receive-pack' was not enable -- probably because you are trying to push to a repository and authentication was disabled. As we explained in Setup git-http-backend for your repositories, you must use authentication to be able to write (push) to repositories using git-http-backend.
This one should be pretty obvious:
Digest: user username: password mismatch: /corporate-git/test.git/info/refs
And so on...
So everything is ready to use. How do you actually create and use these new repositories?
In order to create a new repository, say toyproject.git, all you
have to do is ssh into your Dreamhost account and:
~/newgit.sh -r ${GIT_REPOS_ROOT} -d "My first private repository" -n toyproject
That's it: your created and empty repository in you repository collection. You can clone it if you want.
So, you got a new pristine and empty repository. Let's clone it, shall we?:
$ git clone http://[email protected]/corporate-git/toyproject.git Initialized empty Git repository in /private/tmp/teste/.git/ Password: warning: You appear to have cloned an empty repository.
Important
Have you noticed that we have a username@ in the URL? This
tells git that it must athenticate to the server before trying to
access the git repository.
In this example, we are acessing the
repository with the crentials of the user username, the one we
setup in Password-protect your repository. Modify it to match
the user you created in that step.
But what if you already have a local repository and all you want is push it and its history to the server?
What you usually do is creating a local repository, adding file to it and committing this repository history to the new, empty and pristine repository in your web server:
mkdir toyproject cd toyproject git init touch README git add README git commit -m 'first commit' git remote add origin http://[email protected]/corporate-git/toyproject.git git push origin master
If you have an existing Git Repo, that's the procedure:
cd existing_toyproject_git_repo git remote add origin http://[email protected]/corporate-git/toyproject.git git push origin master
The above workflow follows what is presented in http://help.github.com/creating-a-repo/.
If you need more than one collection of private repositories (say, one for you and one to share privately with a group of coworkers), all you need to do is:
- Create a directory for each of these collections.
- Create copies of
newgit.sh, one for each collection, and setup the value of GIT_REPOS_ROOT in each of them.- Adapt each .htaccess accordingly.
- GitWeb: copy its files too.. Or just sym-link it from a pristine copy.
- Focus on reusability.
- Write the Final remarks section properly.
http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#rewritecond -- serve directly w/ apache if...
Adding project .description directly in the scripts
- arvinderkang.com - Hosting Git repositories on Dreamhost
- craigjolicoeur.com - Hosting Git Repositories on Dreamhost
- Git'n Your Shared Host On
- http://faves.eapen.in/guide-to-hosting-git-repositories-on-dreamhos
- http://gist.github.com/73622
- http://wiki.dreamhost.com/Git#Smart_HTTP
- http://arvinderkang.com/2010/08/25/hosting-git-repositories-on-dreamhost/
- git-http-backend manpage
- Pro Git - Smart HTTP Transport
- http://www.jedi.be/blog/2009/05/06/8-ways-to-share-your-git-repository/
- http://help.github.com/creating-a-repo/