template: title
.mega-octicon.octicon-circuit-board[]
Patrick McKenna
template: content
Git contains tons of useful data and offers powerful commands...
--
...but the interface for those commands is often cumbersome to use
--
... and accessing Git data can be tricky
template: content
First, we'll give an introduction to writing shell scripts for Git
--
We'll develop 2 examples:
-
List the new commits on a branch after a
git pull -
Run a test against a range of commits
template: content
Next, we'll show how to interact directly with Git objects, using modern languages (Python, Ruby, Go, ...), thanks to libgit2
--
We'll develop an example using Rugged, the Ruby binding to libgit2:
- Git-backed web server
template: content
We'll do this hands-on, going slowly enough for everyone to write these scripts
--
Our assumptions:
-
You're comfortable with command line Git, and understand the basics of its internals
-
You familiar with the basics of shell scripting
-
You have a local Git repo to mess about with
template: section
.mega-octicon.octicon-circuit-board[]
template: content
We often ask the same questions of Git
--
For example: what commits were added to master by my last pull?
--
Let's answer this, but for an arbitrary branch (not just master)
git-show-new <branch>template: content
First, we need accept a <branch> arg
If none is received, error out
--
#!/bin/bash
if [ $# -ge 1 ]; then
branch="$1"
else
echo "git-show-new requires a branch name!"
exit 1
fi
# ...template: content
Note:
-
We explicitly invoke Bash
-
We ignore anything past the first arg
template: content
Next, we need to generate the list of new commits
--
We use git rev-list
template: content
git rev-list fundamentally important plumbing command
--
It generates and traverses commit graphs
Reminder: commits form a directed acyclic graph (DAG)
template: content
How do we use it?
--
git rev-list A ^B
Lists commits reachable from A but not B
--
A, B anything that resolves to a commit (i.e. they're commit-ish)
template: content
git rev-list A ^B
In our case (assuming we care about master):
-
Ais the commit wheremastercurrently points -
Bis the commitmasterpointed to before our latestpull
template: content
How do we specify the commit master used to point to?
--
Using Git's @{...} syntax
master@{1}template: content
Hence
git rev-list master ^master@{1}--
Equivalently, using alternate syntax (and generalizing to use our var branch)
git rev-list "$branch"@{1}.."$branch"template: content
Should we output just a list of SHA1s?
--
Let's count them instead
git rev-list "$branch"@{1}.."$branch" | wc -ltemplate: content
We'll use git log, with same revision range syntax, to output commit info
git --no-pager log "$branch"@{1}.."$branch" --onelinetemplate: content
Putting it all together
#!/bin/bash
if [ $# -ge 1 ]; then
branch="$1"
else
echo "git-show-new requires a branch name!"
exit 1
fi
printf "\n%s%s\n\n" $(git rev-list $branch@{1}..$branch | wc -l) \
" commits were added by your last update to $branch:"
git --no-pager log "$branch"@{1}.."$branch" --onelinetemplate: content
Notes:
--
- Requires an arg
--
- Doesn't accept multiple args
--
- Could also make this a Git alias
git config --global alias.show-new \
"!f() { # contents of script go here } ; f"template: section-with-subtitle
.mega-octicon.octicon-circuit-board[]
test a range of commits
template: content
Run a test on a range of commits, stopping (by default) if test fails
(based on @mhagger's script)
--
git-test-range [-k|--keep-going] RANGE -- COMMANDtemplate: content
Where do we start?
--
Need to verify:
-
Script run from inside a valid repo
-
Working directory is clean
template: content
--
Git includes git-sh-setup "scriplet"
Meant for inclusion in other scripts
-
Performs some useful checks
-
Offers helper functions
--
Let's use it!
template: content
Make sure we run git-test-range inside valid repo?
--
Just source the script
. "$(git --exec-path)/git-sh-setup"template: content
What if we wanted to allow git-test-range to be run from anywhere?
--
Set var NONGIT_OK before sourcing git-sh-setup
# NONGIT_OK=true
# the source the script
. "$(git --exec-path)/git-sh-setup"???
true,falseactually commands- could also use non-empty string
template: content
Check for clean working directory
--
# after sourcing script
require_clean_work_tree <command>--
Pass <command> to add info to error message
template: content
Now define a function to actually test a commit
--
test_rev() {
local rev="$1" # keyword local a Bash thing
local command="$2"
git checkout -q "$rev" && # suppress feedback messages
eval "$command" # don't run command unless checkout successful
# ...
}template: content
Now define a function to actually test a commit
test_rev() {
local rev="$1"
local command="$2"
git checkout -q "$rev" &&
eval "$command"
local retcode=$? # shell functions can't return values, so we
if [ $retcode -ne 0 ] # use return (exit) codes instead
then
printf "\n%s\n" "$command FAILED ON:"
git --no-pager log -1 --decorate $rev
return $retcode # make test_rev's return code same as
fi # command's
}template: content
--
Need this so we can checkout back to commit we were on when we run git-test-range
--
head=$(git symbolic-ref HEAD 2>/dev/null || git rev-parse HEAD)--
First command will error out if we run the script from a detached HEAD state
template: content
Define what we'll loop through
--
We already know about:
-
git rev-list -
specifying commit ranges, e.g.
feature..master
--
What's left? Dealing with test results...
template: content
fail_count=0
for rev in $(git rev-list --reverse $range); do
test_rev $rev "$command"
retcode=$?
if [ $retcode -eq 0 ]; then # all good, test next commit!
continue
fi
# ...template: content
fail_count=0
for rev in $(git rev-list --reverse $range); do
test_rev $rev "$command"
retcode=$?
if [ $retcode -eq 0 ]; then
continue
fi
if [ $keep_going ]; then # if a test fails, only continue if
fail_count=$((fail_count + 1)) # user chose that option
continue
else
git checkout -fq ${head#refs/heads/} # get back to where we started
exit $retcode # otherwise HEAD detached
fi
done
git checkout -fq ${head#refs/heads/} # get back to where we startedtemplate: content
We've skipped a few things...
--
- Dealing with input args
--
- Printing out final results
--
Complete version here: https://git.io/vVgTY
template: section
.mega-octicon.octicon-circuit-board[]
template: content
Problems arise when you want to:
--
- Start doing complicated things
--
- Use a more modern, fully-featured language (often for above reason)
--
- Scale — shelling out to Git can get expensive
template: section
.mega-octicon.octicon-circuit-board[]
template: content
--
-
Portable, pure C implementation of Git core methods
- Git is mostly C, but some shell, Perl
--
-
Bindings to it in most major languages
- Python, Ruby, Node.js, Go, ...
--
-
Well-established, actively maintained OSS project with wide industry support
- GitHub, Microsoft, Atlassian, Canonical, others use in production
template: section-with-subtitle
.mega-octicon.octicon-circuit-board[]
Ruby binding to libgit2
template: content
--
-
Ruby benefits
-
Widely used
-
Very readable syntax (even for newcomers)
-
Great for scripting
-
Huge package ecosystem
-
--
- (Also: time limits)
template: content
Rugged is distributed as a self-contained gem, so:
gem install ruggedIt's easiest to let Rugged use its bundled version of libgit2; to do that, you'll need:
-
CMake
-
pkg-config
???
consider taking break here, so people can install as needed
template: content
Main object class (rarely created directly)
Rugged::Object--
Primary interface to local Git repos
Rugged::Repositorytemplate: content
Git has 4 fundamental object types
--
Rugged has corresponding class for each:
-
Rugged::Blob -
Rugged::Tree -
Rugged::Commit -
Rugged::Tag
template: content
--
We can live in Ruby, use all of our favorite gems...
--
... and integrate Git natively!
template: content
Ruby is widely used for building web apps
--
Let's build an app that serves a website directly from a Git branch
(Based on @carlosmn's git-httpd)
template: content
--
Serve content directly from the Git object store (.git directory)
--
No need to checkout files onto disk
--
Example use case: locally deploy one branch that needs a web server (e.g. gh-pages), while you work on another
template: content
Let's make use of two Ruby gems
gem install sinatra
gem install mimetemplate: content
Starting our script
--
#!/usr/bin/env ruby
require 'sinatra'
require 'rugged'
require 'mime/types'template: content
Specify the repo, the branch(ref), and create the repo object
--
repo_path = ENV['HOME'] + '/PATH/TO/REPO'
ref_name = 'refs/remotes/REMOTE/BRANCH'
repo = Rugged::Repository.new(repo_path)template: content
Use Sinatra to start defining how our server responds to GET requests
--
get '*' do |path| # Sinatra method
commit = repo.ref(ref_name).target
path.slice!(0) # strip leading slash
path = 'index.html' if path.empty?
# ...template: content
Use the supplied path to retrieve the associated Git tree entry
--
# ...
entry = commit.tree.path path # entry is a Ruby hash (i.e. map/dictionary)
puts path
blob = repo.lookup entry[:oid]
content = blob.content # return contents of blob as string
halt 404, "404 Not Found" unless content
# ...template: content
The whole thing (nearly)
--
get '*' do |path|
commit = repo.ref(ref_name).target
path.slice!(0)
path = 'index.html' if path.empty?
entry = commit.tree.path path
puts path
blob = repo.lookup entry[:oid]
content = blob.content
halt 404, "404 Not Found" unless content
content_type MIME::Types.type_for(path).first.content_type
content
endtemplate: content
Full example here: https://git.io/vVgJp
template: title
.mega-octicon.octicon-circuit-board[]
Patrick McKenna
template: title
.mega-octicon.octicon-circuit-board[]
template: content
Git comes from the UNIX tradition
-
Small programs designed to do one thing well
-
Known, stable interfaces
git rm,git mv, ...
Sound like microservices...
template: content
Git comes from the UNIX tradition
-
Small programs designed to do one thing well
-
Known, stable interfaces
git rm,git mv, ...
template: content
Troll alert: let's not take that comparison too seriously
But, we can think writing a Git script as (borrowing Fowler's words again):