diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..8dac8cb --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +pkg +.idea +.ruby-version diff --git a/ChangeLog.markdown b/ChangeLog.markdown new file mode 100644 index 0000000..f8d7239 --- /dev/null +++ b/ChangeLog.markdown @@ -0,0 +1,199 @@ +# 2.4.0 - 2016-10-30 + + This release introduces the ability to supply a password for SVN repositories that can't authenticate by other means. + It also adds the ability to specify the '--branches' and '--tags' arguments multiple times to better support those with + more complicated SVN repository layouts. + + * Added support for the '--password' option for authentication (thanks edpbx). + * Added the ability to specify the '--branches' and '--tags' arguments multiple times (thanks pdf). + * Fixed a problem with processing of the '--exclude' argument (improper quoting internally) (thanks pdf). + +# 2.3.2 - 2014-06-08 + + This is a bugfix release. It fixes issues running with Windows using MRI ruby and fixes a problem with Ruby 1.8.7. + + * Removed open4 dependency. svn2git no longer has any runtime dependencies and things work well on Windows again. + * Fixed an issue with Ruby 1.8.7, which doesn't implicitly require the 'thread' library meaning classes that library weren't in scope. + + +# 2.3.1 - 2014-05-14 + + This is a critical bugfix release if you're running git >= 1.8.3.2. In the days of svn2git 1.x we supported syncing + local branches with upstream by tracking the branch as we set them up. This allowed one to checkout the branch and + issue a "git pull" to fetch the changes. git-svn ceased allowing this in 1.8.3.2, which broke svn2git with that + version of git and all subsequent versions. The rationale seemed to be in order to prevent pushing changes from + git-svn back up and breaking the remote link, but this was never something svn2git supported anyway. + + Acknowledging the new reality of upstream, the old behavior is retained but deprecated for users of git < 1.8.3.2. + We'll be removing the establishment of remote tracking SVN branches in the 2.5.0 release. If you wish to sync back + with upstream, run `svn2git --rebase`. If you're on git >= 1.8.3.2 your only option for resynchronizing is to + use `svn2git --rebase`. + + Many thanks to ktdreyer for modernizing the test suite and Daniel Ruf (DanielRuf) for pushing on the git compatibility + issue. + + * Fixed creating local branches for remote SVN branches in git >= 1.8.3.2. + * Fixed verbose logging of sub-process STDERR stream. + * Added MIT license metadata to gemspec. + * Switched to minitest to get tests working on Ruby 1.9+ with minitest 5+ installed. + + +# 2.3.0 - 2014-05-14 + + This release passes STDIN through to the underlying git-svn process, allowing users to interact with the + git-svn prompt. Principally, it will allow users to choose what to do when prompted about unverified + SSL certificates. + + * Pass STDIN through to the underlying git-svn process so users can respond to prompts. + +# 2.2.5 - 2014-03-09 + + This is a bugfix release. It improves handling of quotes in SVN commit messages. + + + * Fixed an with single quote escaping (thanks aucl). + * Escape double quotes (e.g., if they appear in a commit message) before passing to the shell (thanks aucl). + +# 2.2.4 - 2014-03-08 + + There was a permissions problem with some of the files packed in 2.2.3. This was caught immediately after the gem + was pushed, so it was yanked as it simply wouldn't work for anyone. 2.2.4 contains everything 2.2.3 did, but with + proper packaging. + +# 2.2.3 - 2014-03-08 + + This is a bugfix release. First change done by FeeJai. + + * Fixed an issue with password protected svn-repositories. The prompt to enter the password is now displayed. + * Fixed an issue with server certificates. If the certificate is untrusted, the prompt to confirm or deny the certificate is now shown. + * Fixed an issue with using the `--local` flag for `git config` in git versions < 1.7.4. + + +# 2.2.2 - 2012-10-07 + + This is an overdue bugfix release. No new features were added, but several long-standing bugs fixed by the community + have been merged. Many thanks to Edson de Lima (edsonlima), Rudger (Rud5G), Ben Wolfe (bwolfe), CyberTech, PowerKiKi, and Philipp Riemer (ruderphilipp) for the pull requests. + + * Fixed an issue working with repositories that contained a space in the name (thanks edsonlima). + * Fixed an issue working with tags that contain a hyphen (thanks Rud5G). + * Fixed an issue with fixing tags during a rebase (thanks PowerKiKi). + * Double-quote git-svn commands working with tags to avoid issues with special strings (thanks CyberTech). + * Improved the documentation example of fetching the author list for an SVN repository (thanks bwolfe). + * Set the git committer date for tags in a more cross-platform manner (thanks CyberTech). + * Improved documentation formatting (thanks ruderphilipp). + +# 2.2.1 - 2012-02-25 + + This is a critical bugfix release if your repository has tags. Thanks to David Zülke (dzuelke) for the patches making up this release. + + * Added the ability to specify an end revision for migration (thanks dzuelke). + * Fixed an issue with initial conversion if the repo had tags (thanks dzuelke). + +# 2.2.0 - 2012-01-25 + + Thanks to Craig Hobbs (craigahobbs) and Simon Chiang (thinkerbot) for the patches making up this release. + It rounds out our tag support by handling tags with special characters and preserving original tag author info. + + * Fixed an issue with not quoting tag names (thanks craigahobbs and thinkerbot) + * Fixed an issue whereby the person running the svn2git conversion became the author of every tag (i.e., we lost the + original tag committer info) (thanks thinkerbot) + +# 2.1.2 - 2011-12-28 + + * Fixed a regression in improperly quoting branch names (thanks ziangsong). + +# 2.1.1 - 2011-12-27 + + * Fixed SVN branch detection (thanks thinkerbot). + * Stop processing when a git subprocess fails (thanks thinkerbot). + * Fixed an issue with SVN branches containing shell special characters (thanks sleicht). + +# 2.1.0 - 2011-04-03 + + Thanks to Francois Rey (fmjrey), Sven Axelsson (svenax), and Julian Taylor (juliantaylor) for submitting all the patches + that comprise this release. svn2git now works with a much wider array SVN repositories because of their efforts. + + * Added --no-minimize-url option for migrating specific subprojects from an SVN repo containing several projects (thanks fmjrey). + * Added --username option for migrating password-protected repositories (thanks svenax). + * Added --revision option for specifying the revision to start importing from (thanks svenax). + * Fixed compatibility with older versions of git (thanks juliantaylor). + +# 2.0.0 - 2010-05-29 + + This release adds the oft requested incremental SVN update support. If you run svn2git with the `--rebase` option on an existing + repository that you've converted with svn2git, it will fetch new branches & tags from SVN and update existing ones. There are + two important things to note: + + * This will not work on already converted repositories because the tracking information isn't set up correctly. You could do that + yourself, but it's probably a lot easier to do the conversion over. + * svn2git now maintains remote tracking information. If this is a problem for you because you don't want any links to the SVN server + you can either stick with a 1.x release of svn2git or simply clone the repo created with svn2git, which will lose the tracking information. + + A great deal of thanks to Nathaniel McCallum (npmccallum) for coming up with an elegant solution and then providing the patch for this release. + +# 1.3.3 - 2010-03-31 + + Thanks to Jeff Ramnani (jramnani) for finding a problem with with the --excludes tag and providing a patch. + + * Fix error when using '--exclude' option. + +# 1.3.2 - 2010-03-12 + + Thanks to Rajit Singh (rajit) for finding a problem with quoting in tag comments that were causing issues with svn2git's internal + quoting and providing a patch. + + * Deal cleanly with any single quotes found in tag comments so that the 'git tag' commands run correctly. + +# 1.3.1 - 2009-06-09 + + Thanks to KUBO Atsuhiro (iteman) for finding a problem with the tagging process and providing a patch. + + * Fixed a problem with creating actual git tags when the SVN tags path was named anything other than 'tags.' + +# 1.3.0 - 2009-06-09 + + Many thanks to Malte S. Stretz (mss) for the patches making up most of this release. + + * Fixed a problem where tags didn't get the original date and time. + * New switch --exclude which can be used to specify a PCRE pattern to exclude paths from the import. + * New switches --no{trunk,branches,tags} to skip import of those. + * Improved docs. + +# 1.2.4 - 2009-05-04 + + * No changes. I ran the jeweler command twice inadvertently. Tearing down the release would be more harmful than helpful. + +# 1.2.3 - 2009-05-04 + + * Yanked out the code referencing the gem by name. This shouldn't be necessary at all. + +# 1.2.2 - 2009-05-04 + + * Updated the reference gem in the binary to use this one and not the one on RubyForge. + +# 1.2.1 - 2009-04-19 + + * Fixed a problem with the svn2git binary not loading command-line args properly. + +# 1.2.0 - 2009-04-17 + + * Reworked command-line options so they work similarly to every other app in the world. + * Better error messaging when no URL provided. + * Improved docs. + +# 1.1.1 - 2009-04-15 + + * Started using Jeweler for gem management. + * Fixed issue with not loading up RubyGems appropriately. + +# 1.1.0 - 2009-01-02 + + * First release since nirvdrum fork. + + * Fixed issues with handling of tags and branches. + * Added better logging of output from git-svn. + * Wrap external command processing to capture failures. + +# 1.0.0 - 2008-07-19 + + * Forked version from jcoglan. diff --git a/MIT-LICENSE b/MIT-LICENSE index 714d101..94f6821 100644 --- a/MIT-LICENSE +++ b/MIT-LICENSE @@ -1,4 +1,4 @@ -Copyright (c) 2008 James Coglan +Copyright (c) 2008 James Coglan, Kevin Menard Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/README b/README deleted file mode 100644 index 4278cd9..0000000 --- a/README +++ /dev/null @@ -1,89 +0,0 @@ -== svn2git - -+svn2git+ is a tiny utility for migrating projects from Subversion to Git while keeping -the trunk, branches and tags where they should be. It uses git-svn to clone an svn repository -and does some clean-up to make sure branches and tags are imported in a meaningful way, and -that the code checked into master ends up being what's currently in your svn trunk rather -than whichever svn branch your last commit was in. - -=== Examples - -Say I have this code in svn: - - trunk - ... - branches - 1.x - 2.x - tags - 1.0.0 - 1.0.1 - 1.0.2 - 1.1.0 - 2.0.0 - -git-svn will go through the commit history to build a new git repo. It will import all branches -and tags as remote svn branches, whereas what you really want is git-native local branches and -git tag objects. So after importing this project I'll get: - - $ git branch - * master - $ git branch -a - * master - 1.x - 2.x - tags/1.0.0 - tags/1.0.1 - tags/1.0.2 - tags/1.1.0 - tags/2.0.0 - trunk - $ git tag -l - [ empty ] - -After svn2git is done with your project, you'll get this instead: - - $ git branch - * master - 1.x - 2.x - $ git tag -l - 1.0.0 - 1.0.1 - 1.0.2 - 1.1.0 - 2.0.0 - -Finally, it makes sure the HEAD of master is the same as the current trunk of the svn repo. - -=== Installation - -Make sure you have git installed, then install the gem: - - $ sudo apt-get install git-core git-svn - $ sudo gem install svn2git - -=== Usage - -To create a git repo from an existing svn repo: - - $ svn2git http://svn.yoursite.com/path/to/repo - -This will create a git repository in the current directory with the git version of the svn -repository. If you're not using the standard trunk/branches/tags layout, you can pass arguments -to tell git-svn what to look for: - - $ svn2git http://svn.yoursite.com/path/to/repo trunk=the_trunk tags=taggings - -=== Authors - -To convert all your svn authors to git format, create a file somewhere on your system with -the list of conversions to make, one per line, for example: - - jcoglan = James Coglan - stnick = Santa Claus - -Then pass an +authors+ option to +svn2git+ pointing to your file: - - svn2git http://repos.com/myproject authors=~/authors.txt - diff --git a/README.markdown b/README.markdown new file mode 100644 index 0000000..7b1165c --- /dev/null +++ b/README.markdown @@ -0,0 +1,256 @@ +svn2git +======= + +_svn2git_ is a tiny utility for migrating projects from Subversion to Git +while keeping the trunk, branches and tags where they should be. It uses +git-svn to clone an svn repository and does some clean-up to make sure +branches and tags are imported in a meaningful way, and that the code checked +into master ends up being what's currently in your svn trunk rather than +whichever svn branch your last commit was in. + +Examples +-------- + +Say I have this code in svn: + + trunk + ... + branches + 1.x + 2.x + tags + 1.0.0 + 1.0.1 + 1.0.2 + 1.1.0 + 2.0.0 + +git-svn will go through the commit history to build a new git repo. It will +import all branches and tags as remote svn branches, whereas what you really +want is git-native local branches and git tag objects. So after importing this +project I'll get: + + $ git branch + * master + $ git branch -a + * master + 1.x + 2.x + tags/1.0.0 + tags/1.0.1 + tags/1.0.2 + tags/1.1.0 + tags/2.0.0 + trunk + $ git tag -l + [ empty ] + +After svn2git is done with your project, you'll get this instead: + + $ git branch + * master + 1.x + 2.x + $ git tag -l + 1.0.0 + 1.0.1 + 1.0.2 + 1.1.0 + 2.0.0 + +Finally, it makes sure the HEAD of master is the same as the current trunk of +the svn repo. + +Installation +------------ + +Make sure you have git, git-svn, and ruby installed. svn2git is a ruby wrapper around git's native SVN support through git-svn. It is possible to have git installed without git-svn installed, so please do verify that you can run `$ git svn` successfully. For a Debian-based system, the installation of the prerequisites would look like: + + $ sudo apt-get install git-core git-svn ruby + +Once you have the necessary software on your system, you can install svn2git through rubygems, which will add the `svn2git` command to your PATH. + + $ sudo gem install svn2git + + +Usage +----- + +### Initial Conversion ### + +There are several ways you can create a git repo from an existing +svn repo. The differentiating factor is the svn repo layout. Below is an +enumerated listing of the varying supported layouts and the proper way to +create a git repo from a svn repo in the specified layout. + +1. The svn repo is in the standard layout of (trunk, branches, tags) at the +root level of the repo. + + $ svn2git http://svn.example.com/path/to/repo + +2. The svn repo is NOT in standard layout and has only a trunk and tags at the +root level of the repo. + + $ svn2git http://svn.example.com/path/to/repo --trunk dev --tags rel --nobranches + +3. The svn repo is NOT in standard layout and has only a trunk at the root +level of the repo. + + $ svn2git http://svn.example.com/path/to/repo --trunk trunk --nobranches --notags + +4. The svn repo is NOT in standard layout and has no trunk, branches, or tags +at the root level of the repo. Instead the root level of the repo is +equivalent to the trunk and there are no tags or branches. + + $ svn2git http://svn.example.com/path/to/repo --rootistrunk + +5. The svn repo is in the standard layout but you want to exclude the massive +doc directory and the backup files you once accidently added. + + $ svn2git http://svn.example.com/path/to/repo --exclude doc --exclude '.*~$' + +6. The svn repo actually tracks several projects and you only want to migrate +one of them. + + $ svn2git http://svn.example.com/path/to/repo/nested_project --no-minimize-url + +7. The svn repo is password protected. + + $ svn2git http://svn.example.com/path/to/repo --username <> + +If this doesn't cooperate and you need to specify a password on the command-line: + + $ svn2git http://svn.example.com/path/to/repo --username <> --password <> + +8. You need to migrate starting at a specific svn revision number. + + $ svn2git http://svn.example.com/path/to/repo --revision <> + +9. You need to migrate starting at a specific svn revision number, ending at a specific revision number. + + $ svn2git http://svn.example.com/path/to/repo --revision <>:<> + +10. Include metadata (git-svn-id) in git logs. + + $ svn2git http://svn.example.com/path/to/repo --metadata + +The above will create a git repository in the current directory with the git +version of the svn repository. Hence, you need to make a directory that you +want your new git repo to exist in, change into it and then run one of the +above commands. Note that in the above cases the trunk, branches, tags options +are simply folder names relative to the provided repo path. For example if you +specified trunk=foo branches=bar and tags=foobar it would be referencing +http://svn.example.com/path/to/repo/foo as your trunk, and so on. However, in +case 4 it references the root of the repo as trunk. + +### Repository Updates ### + +As of svn2git 2.0 there is a new feature to pull in the latest changes from SVN into your +git repository created with svn2git. This is a one way sync, but allows you to use svn2git +as a mirroring tool for your SVN repositories. + +The command to call is: + + $ cd && svn2git --rebase + +Authors +------- + +To convert all your svn authors to git format, create a file somewhere on your +system with the list of conversions to make, one per line, for example: + + jcoglan = James Coglan + stnick = Santa Claus + +Then pass an _authors_ option to svn2git pointing to your file: + + $ svn2git http://svn.example.com/path/to/repo --authors ~/authors.txt + +Alternatively, you can place the authors file into `~/.svn2git/authors` and +svn2git will load it out of there. This allows you to build up one authors +file for all your projects and have it loaded for each repository that you +migrate. + +If you need a jump start on figuring out what users made changes in your +svn repositories the following command sequence might help. It grabs all +the logs from the svn repository, pulls out all the names from the commits, +sorts them, and then reduces the list to only unique names. So, in the end +it outputs a list of usernames of the people that made commits to the svn +repository which name on its own line. This would allow you to easily +redirect the output of this command sequence to `~/.svn2git/authors` and have +a very good starting point for your mapping. + + $ svn log --quiet | grep -E "r[0-9]+ \| .+ \|" | cut -d'|' -f2 | sed 's/ //g' | sort | uniq + +Or, for a remote URL: + + $ svn log --quiet http://path/to/root/of/project | grep -E "r[0-9]+ \| .+ \|" | cut -d'|' -f2 | sed 's/ //g' | sort | uniq + +Debugging +--------- + +If you're having problems with converting your repository and you're not sure why, +try turning on verbose logging. This will print out more information from the +underlying git-svn process. + +You can turn on verbose logging with the `-v` or `--verbose` flags, like so: + + $ svn2git http://svn.yoursite.com/path/to/repo --verbose + +Options Reference +----------------- + + $ svn2git --help + Usage: svn2git SVN_URL [options] + + Specific options: + --rebase Instead of cloning a new project, rebase an existing one against SVN + --username NAME Username for transports that needs it (http(s), svn) + --password PASS Password for transports that needs it (http(s), svn) + --trunk TRUNK_PATH Subpath to trunk from repository URL (default: trunk) + --branches BRANCHES_PATH Subpath to branches from repository URL (default: branches); can be used multiple times + --tags TAGS_PATH Subpath to tags from repository URL (default: tags); can be used multiple times + --rootistrunk Use this if the root level of the repo is equivalent to the trunk and there are no tags or branches + --notrunk Do not import anything from trunk + --nobranches Do not try to import any branches + --notags Do not try to import any tags + --no-minimize-url Accept URLs as-is without attempting to connect to a higher level directory + --revision START_REV[:END_REV] + Start importing from SVN revision START_REV; optionally end at END_REV + -m, --metadata Include metadata in git logs (git-svn-id) + --authors AUTHORS_FILE Path to file containing svn-to-git authors mapping (default: ~/.svn2git/authors) + --exclude REGEX Specify a Perl regular expression to filter paths when fetching; can be used multiple times + -v, --verbose Be verbose in logging -- useful for debugging issues + + -h, --help Show this message + +FAQ +--- + +1. Why don't the tags show up in the master branch? + + The tags won't show up in the master branch because the tags are actually + tied to the commits that were created in svn when the user made the tag. + Those commits are the first (head) commit of branch in svn that is + associated with that tag. If you want to see all the branches and tags + and their relationships in gitk you can run the following: gitk --all + + For further details please refer to FAQ #2. + +2. Why don't you reference the parent of the tag commits instead? + + In svn you are forced to create what are known in git as annotated tags. + It just so happens that svn annotated tags allow you to commit change + sets along with the tagging action. This means that the svn annotated tag + is a bit more complex then just an annotated tag it is a commit which is + treated as an annotated tag. Hence, for there to be a true 1-to-1 mapping + between git and svn we have to transfer over the svn commit which acts as + an annotated tag and then tag that commit in git using an annotated tag. + + If we were to reference the parent of this svn tagged commit there could + potentially be situations where a developer would checkout a tag in git + and the resulting code base would be different than if they checked out + that very same tag in the original svn repo. This is only due to the fact + that the svn tags allow changesets in them, making them not just annotated + tags. + diff --git a/Rakefile b/Rakefile index bae6ea9..80abf54 100644 --- a/Rakefile +++ b/Rakefile @@ -1,30 +1,31 @@ require 'rake' -require 'rake/gempackagetask' +require 'rake/testtask' +require 'rubygems/package_task' -spec = Gem::Specification.new do |spec| - spec.name = "svn2git" - spec.version = "1.0.0" - spec.platform = Gem::Platform::RUBY - spec.summary = "A tool for migrating svn projects to git" +begin + require 'jeweler' + Jeweler::Tasks.new do |spec| + spec.name = "svn2git" + spec.summary = "A tool for migrating svn projects to git" + spec.authors = ["James Coglan", "Kevin Menard"] + spec.homepage = "https://github.com/nirvdrum/svn2git" + spec.email = "nirvdrum@gmail.com" + spec.license = 'MIT' + spec.add_development_dependency 'minitest' + end + Jeweler::GemcutterTasks.new - spec.require_path = "lib" - spec.files = FileList["lib/**/*"].to_a - spec.autorequire = "lib/svn2git.rb" - spec.bindir = "bin" - spec.executables = ["svn2git"] - spec.default_executable = "svn2git" - - spec.author = "James Coglan" - spec.email = "james@jcoglan.com" - spec.homepage = "http://github.com/jcoglan/svn2git/" - - spec.test_files = FileList["test/**/*"].to_a - spec.has_rdoc = true - spec.extra_rdoc_files = ["README"] - spec.rdoc_options << "--main" << "README" << '--line-numbers' << '--inline-source' +rescue LoadError + puts "Jeweler not available. Install it with: gem install jeweler" end - -Rake::GemPackageTask.new(spec) do |pkg| - pkg.need_tar = true + +desc 'Test the rubber plugin.' +Rake::TestTask.new(:test) do |t| + t.libs << 'lib' + t.libs << 'test' + t.pattern = 'test/**/*_test.rb' + t.verbose = true end +desc 'Default: run unit tests.' +task :default => :test \ No newline at end of file diff --git a/VERSION.yml b/VERSION.yml new file mode 100644 index 0000000..900900f --- /dev/null +++ b/VERSION.yml @@ -0,0 +1,5 @@ +--- +:major: 2 +:minor: 4 +:patch: 0 +:build: diff --git a/bin/svn2git b/bin/svn2git index 2d2055c..63bbf00 100644 --- a/bin/svn2git +++ b/bin/svn2git @@ -20,15 +20,8 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. +require 'rubygems' require 'svn2git' -url = ARGV.shift -options = ARGV.inject({}) do |memo, arg| - parts = arg.split('=') - memo[parts.first.to_sym] = parts.last - memo -end - -migration = Svn2Git::Migration.new(url, options) +migration = Svn2Git::Migration.new(ARGV) migration.run! - diff --git a/lib/svn2git/migration.rb b/lib/svn2git/migration.rb old mode 100644 new mode 100755 index 795a43b..4de8a9e --- a/lib/svn2git/migration.rb +++ b/lib/svn2git/migration.rb @@ -1,73 +1,491 @@ +require 'optparse' +require 'pp' +require 'timeout' +require 'thread' + module Svn2Git + DEFAULT_AUTHORS_FILE = "~/.svn2git/authors" + class Migration - + attr_reader :dir - - def initialize(url, options = {}) - @url = url - @dir = @url.scan(/[^\/]+/).last - - @options = options - @options[:trunk] ||= 'trunk' - @options[:branches] ||= 'branches' - @options[:tags] ||= 'tags' - - @authors = options[:authors] - end - + + def initialize(args) + @options = parse(args) + if @options[:rebase] + show_help_message('Too many arguments') if args.size > 0 + verify_working_tree_is_clean + elsif @options[:rebasebranch] + show_help_message('Too many arguments') if args.size > 0 + verify_working_tree_is_clean + else + show_help_message('Missing SVN_URL parameter') if args.empty? + show_help_message('Too many arguments') if args.size > 1 + @url = args.first.gsub(' ', "\\ ") + end + end + def run! - clone! - fix_tags + if @options[:rebase] + get_branches + elsif @options[:rebasebranch] + get_rebasebranch + else + clone! + end fix_branches + fix_tags fix_trunk + optimize_repos + end + + def parse(args) + # Set up reasonable defaults for options. + options = {} + options[:verbose] = false + options[:metadata] = false + options[:nominimizeurl] = false + options[:rootistrunk] = false + options[:trunk] = 'trunk' + options[:branches] = [] + options[:tags] = [] + options[:exclude] = [] + options[:revision] = nil + options[:username] = nil + options[:password] = nil + options[:rebasebranch] = false + + if File.exists?(File.expand_path(DEFAULT_AUTHORS_FILE)) + options[:authors] = DEFAULT_AUTHORS_FILE + end + + + # Parse the command-line arguments. + @opts = OptionParser.new do |opts| + opts.banner = 'Usage: svn2git SVN_URL [options]' + + opts.separator '' + opts.separator 'Specific options:' + + opts.on('--rebase', 'Instead of cloning a new project, rebase an existing one against SVN') do + options[:rebase] = true + end + + opts.on('--username NAME', 'Username for transports that needs it (http(s), svn)') do |username| + options[:username] = username + end + + opts.on('--password PASSWORD', 'Password for transports that need it (http(s), svn)') do |password| + options[:password] = password + end + + opts.on('--trunk TRUNK_PATH', 'Subpath to trunk from repository URL (default: trunk)') do |trunk| + options[:trunk] = trunk + end + + opts.on('--branches BRANCHES_PATH', 'Subpath to branches from repository URL (default: branches); can be used multiple times') do |branches| + options[:branches] << branches + end + + opts.on('--tags TAGS_PATH', 'Subpath to tags from repository URL (default: tags); can be used multiple times') do |tags| + options[:tags] << tags + end + + opts.on('--rootistrunk', 'Use this if the root level of the repo is equivalent to the trunk and there are no tags or branches') do + options[:rootistrunk] = true + options[:trunk] = nil + options[:branches] = nil + options[:tags] = nil + end + + opts.on('--notrunk', 'Do not import anything from trunk') do + options[:trunk] = nil + end + + opts.on('--nobranches', 'Do not try to import any branches') do + options[:branches] = nil + end + + opts.on('--notags', 'Do not try to import any tags') do + options[:tags] = nil + end + + opts.on('--no-minimize-url', 'Accept URLs as-is without attempting to connect to a higher level directory') do + options[:nominimizeurl] = true + end + + opts.on('--revision START_REV[:END_REV]', 'Start importing from SVN revision START_REV; optionally end at END_REV') do |revision| + options[:revision] = revision + end + + opts.on('-m', '--metadata', 'Include metadata in git logs (git-svn-id)') do + options[:metadata] = true + end + + opts.on('--authors AUTHORS_FILE', "Path to file containing svn-to-git authors mapping (default: #{DEFAULT_AUTHORS_FILE})") do |authors| + options[:authors] = authors + end + + opts.on('--exclude REGEX', 'Specify a Perl regular expression to filter paths when fetching; can be used multiple times') do |regex| + options[:exclude] << regex + end + + opts.on('-v', '--verbose', 'Be verbose in logging -- useful for debugging issues') do + options[:verbose] = true + end + + opts.on('--rebasebranch REBASEBRANCH', 'Rebase specified branch.') do |rebasebranch| + options[:rebasebranch] = rebasebranch + end + + opts.separator "" + + # No argument, shows at tail. This will print an options summary. + # Try it and see! + opts.on_tail('-h', '--help', 'Show this message') do + puts opts + exit + end + end + + @opts.parse! args + options + end + + def self.escape_quotes(str) + str.gsub(/'|"/) { |c| "\\#{c}" } + end + + def escape_quotes(str) + Svn2Git::Migration.escape_quotes(str) + end + + def self.checkout_svn_branch(branch) + "git checkout -b \"#{branch}\" \"remotes/svn/#{branch}\"" end - + private - + def clone! trunk = @options[:trunk] branches = @options[:branches] tags = @options[:tags] - `git svn init --no-metadata --trunk=#{trunk} --branches=#{branches} --tags=#{tags} #{@url}` - `git config svn.authorsfile #{@authors}` if @authors - `git svn fetch` + metadata = @options[:metadata] + nominimizeurl = @options[:nominimizeurl] + rootistrunk = @options[:rootistrunk] + authors = @options[:authors] + exclude = @options[:exclude] + revision = @options[:revision] + username = @options[:username] + password = @options[:password] + + if rootistrunk + # Non-standard repository layout. The repository root is effectively 'trunk.' + cmd = "git svn init --prefix=svn/ " + cmd += "--username='#{username}' " unless username.nil? + cmd += "--password='#{password}' " unless password.nil? + cmd += "--no-metadata " unless metadata + if nominimizeurl + cmd += "--no-minimize-url " + end + cmd += "--trunk='#{@url}'" + run_command(cmd, true, true) + + else + cmd = "git svn init --prefix=svn/ " + + # Add each component to the command that was passed as an argument. + cmd += "--username='#{username}' " unless username.nil? + cmd += "--password='#{password}' " unless password.nil? + cmd += "--no-metadata " unless metadata + if nominimizeurl + cmd += "--no-minimize-url " + end + cmd += "--trunk='#{trunk}' " unless trunk.nil? + unless tags.nil? + # Fill default tags here so that they can be filtered later + tags = ['tags'] if tags.empty? + # Process default or user-supplied tags + tags.each do |tag| + cmd += "--tags='#{tag}' " + end + end + unless branches.nil? + # Fill default branches here so that they can be filtered later + branches = ['branches'] if branches.empty? + # Process default or user-supplied branches + branches.each do |branch| + cmd += "--branches='#{branch}' " + end + end + + cmd += @url + + run_command(cmd, true, true) + end + + run_command("#{git_config_command} svn.authorsfile #{authors}") unless authors.nil? + + cmd = "git svn fetch " + unless revision.nil? + range = revision.split(":") + range[1] = "HEAD" unless range[1] + cmd += "-r #{range[0]}:#{range[1]} " + end + unless exclude.empty? + # Add exclude paths to the command line; some versions of git support + # this for fetch only, later also for init. + regex = [] + unless rootistrunk + regex << "#{trunk}[/]" unless trunk.nil? + tags.each{|tag| regex << "#{tag}[/][^/]+[/]"} unless tags.nil? or tags.empty? + branches.each{|branch| regex << "#{branch}[/][^/]+[/]"} unless branches.nil? or branches.empty? + end + regex = '^(?:' + regex.join('|') + ')(?:' + exclude.join('|') + ')' + cmd += "--ignore-paths='#{regex}' " + end + run_command(cmd, true, true) + get_branches end - + def get_branches - @branches = `git branch -a`.split(/\n/) - @local = `git branch`.split(/\n/) - @remote = @branches.find_all { |b| not @local.include?(b) } - @tags = @remote.find_all { |b| b.strip =~ %r{^#{@options[:tags]}\/} } + # Get the list of local and remote branches, taking care to ignore console color codes and ignoring the + # '*' character used to indicate the currently selected branch. + @local = run_command("git branch -l --no-color").split(/\n/).collect{ |b| b.gsub(/\*/,'').strip } + @remote = run_command("git branch -r --no-color").split(/\n/).collect{ |b| b.gsub(/\*/,'').strip } + + # Tags are remote branches that start with "tags/". + @tags = @remote.find_all { |b| b.strip =~ %r{^svn\/tags\/} } + end - + + def get_rebasebranch + get_branches + @local = @local.find_all{|l| l == @options[:rebasebranch]} + @remote = @remote.find_all{|r| r.include? @options[:rebasebranch]} + + if @local.count > 1 + pp "To many matching branches found (#{@local})." + exit 1 + elsif @local.count == 0 + pp "No local branch named \"#{@options[:rebasebranch]}\" found." + exit 1 + end + + if @remote.count > 2 # 1 if remote is not pushed, 2 if its pushed to remote + pp "To many matching remotes found (#{@remotes})" + exit 1 + elsif @remote.count == 0 + pp "No remote branch named \"#{@options[:rebasebranch]}\" found." + exit 1 + end + pp "Local branches \"#{@local}\" found" + pp "Remote branches \"#{@remote}\" found" + + @tags = [] # We only rebase the specified branch + + end + def fix_tags + current = {} + current['user.name'] = run_command("#{git_config_command} --get user.name", false) + current['user.email'] = run_command("#{git_config_command} --get user.email", false) + @tags.each do |tag| - id = tag.strip.gsub(%r{^#{@options[:tags]}\/}, '') - `git checkout #{tag}` - `git tag -a -m "Tagging release #{id}" #{id}` + tag = tag.strip + id = tag.gsub(%r{^svn\/tags\/}, '').strip + subject = run_command("git log -1 --pretty=format:'%s' \"#{escape_quotes(tag)}\"").chomp("'").reverse.chomp("'").reverse + date = run_command("git log -1 --pretty=format:'%ci' \"#{escape_quotes(tag)}\"").chomp("'").reverse.chomp("'").reverse + author = run_command("git log -1 --pretty=format:'%an' \"#{escape_quotes(tag)}\"").chomp("'").reverse.chomp("'").reverse + email = run_command("git log -1 --pretty=format:'%ae' \"#{escape_quotes(tag)}\"").chomp("'").reverse.chomp("'").reverse + run_command("#{git_config_command} user.name \"#{escape_quotes(author)}\"") + run_command("#{git_config_command} user.email \"#{escape_quotes(email)}\"") + + original_git_committer_date = ENV['GIT_COMMITTER_DATE'] + ENV['GIT_COMMITTER_DATE'] = escape_quotes(date) + run_command("git tag -a -m \"#{escape_quotes(subject)}\" \"#{escape_quotes(id)}\" \"#{escape_quotes(tag)}\"") + ENV['GIT_COMMITTER_DATE'] = original_git_committer_date + + run_command("git branch -d -r \"#{escape_quotes(tag)}\"") + end + + ensure + # We only change the git config values if there are @tags available. So it stands to reason we should revert them only in that case. + unless @tags.empty? + current.each_pair do |name, value| + # If a line was read, then there was a config value so restore it. + # Otherwise unset the value because originally there was none. + if value.strip != '' + run_command("#{git_config_command} #{name} \"#{value.strip}\"") + else + run_command("#{git_config_command} --unset #{name}") + end + end end end - + def fix_branches - svn_branches = @remote.find_all { |b| not @tags.include?(b) } + svn_branches = @remote - @tags + svn_branches.delete_if { |b| b.strip !~ %r{^svn\/} } + + if @options[:rebase] + run_command("git svn fetch", true, true) + end + svn_branches.each do |branch| - branch = branch.strip - next if branch == @options[:trunk] - `git checkout #{branch}` - `git checkout -b #{branch}` + branch = branch.gsub(/^svn\//,'').strip + if @options[:rebase] && (@local.include?(branch) || branch == 'trunk') + lbranch = branch + lbranch = 'master' if branch == 'trunk' + run_command("git checkout -f \"#{lbranch}\"") + run_command("git rebase \"remotes/svn/#{branch}\"") + next + end + + next if branch == 'trunk' || @local.include?(branch) + + if @cannot_setup_tracking_information + run_command(Svn2Git::Migration.checkout_svn_branch(branch)) + else + status = run_command("git branch --track \"#{branch}\" \"remotes/svn/#{branch}\"", false) + + # As of git 1.8.3.2, tracking information cannot be set up for remote SVN branches: + # http://git.661346.n2.nabble.com/git-svn-Use-prefix-by-default-td7594288.html#a7597159 + # + # Older versions of git can do it and it should be safe as long as remotes aren't pushed. + # Our --rebase option obviates the need for read-only tracked remotes, however. So, we'll + # deprecate the old option, informing those relying on the old behavior that they should + # use the newer --rebase otion. + if status =~ /Cannot setup tracking information/m + @cannot_setup_tracking_information = true + run_command(Svn2Git::Migration.checkout_svn_branch(branch)) + else + unless @legacy_svn_branch_tracking_message_displayed + warn '*' * 68 + warn "svn2git warning: Tracking remote SVN branches is deprecated." + warn "In a future release local branches will be created without tracking." + warn "If you must resync your branches, run: svn2git --rebase" + warn '*' * 68 + end + + @legacy_svn_branch_tracking_message_displayed = true + + run_command("git checkout \"#{branch}\"") + end + end end end - + def fix_trunk - trunk = @remote.find { |b| b.strip == @options[:trunk] } - if trunk - `git branch -D master` - `git checkout trunk` - `git checkout -f -b master` + trunk = @remote.find { |b| b.strip == 'trunk' } + if trunk && ! @options[:rebase] + run_command("git checkout svn/trunk") + run_command("git branch -D master") + run_command("git checkout -f -b master") + else + run_command("git checkout -f master") + end + end + + def optimize_repos + run_command("git gc") + end + + def run_command(cmd, exit_on_error=true, printout_output=false) + log "Running command: #{cmd}\n" + + ret = '' + @stdin_queue ||= Queue.new + + # We need to fetch input from the user to pass through to the underlying sub-process. We'll constantly listen + # for input and place any received values on a queue for consumption by a pass-through thread that will forward + # the contents to the underlying sub-process's stdin pipe. + @stdin_thread ||= Thread.new do + loop { @stdin_queue << $stdin.gets.chomp } + end + + # Open4 forks, which JRuby doesn't support. But JRuby added a popen4-compatible method on the IO class, + # so we can use that instead. + IO.popen("2>&1 #{cmd}") do |output| + threads = [] + + threads << Thread.new(output) do |output| + # git-svn seems to do all of its prompting for user input via STDERR. When it prompts for input, it will + # not terminate the line with a newline character, so we can't split the input up by newline. It will, + # however, use a space to separate the user input from the prompt. So we split on word boundaries here + # while draining STDERR. + output.each(' ') do |word| + ret << word + + if printout_output + $stdout.print word + else + log word + end + end + end + + # Simple pass-through thread to take anything the user types via STDIN and passes it through to the + # sub-process's stdin pipe. + Thread.new do + loop do + user_reply = @stdin_queue.pop + + # nil is our cue to stop looping (pun intended). + break if user_reply.nil? + + stdin.puts user_reply + stdin.close + end + end + + threads.each(&:join) + + # Push nil to the stdin_queue to gracefully exit the STDIN pass-through thread. + @stdin_queue << nil + end + + if exit_on_error && $?.exitstatus != 0 + $stderr.puts "command failed:\n#{cmd}" + exit -1 + end + + ret + end + + def log(msg) + print msg if @options[:verbose] + end + + def show_help_message(msg) + puts "Error starting script: #{msg}\n\n" + puts @opts.help + exit + end + + def verify_working_tree_is_clean + status = run_command('git status --porcelain --untracked-files=no') + unless status.strip == '' + puts 'You have local pending changes. The working tree must be clean in order to continue.' + exit -1 end end - + + def git_config_command + if @git_config_command.nil? + status = run_command('git config --local --get user.name', false) + + @git_config_command = if status =~ /unknown option/m + 'git config' + else + 'git config --local' + end + end + + @git_config_command + end + end end diff --git a/svn2git.gemspec b/svn2git.gemspec new file mode 100644 index 0000000..c5a786f --- /dev/null +++ b/svn2git.gemspec @@ -0,0 +1,52 @@ +# Generated by jeweler +# DO NOT EDIT THIS FILE DIRECTLY +# Instead, edit Jeweler::Tasks in Rakefile, and run 'rake gemspec' +# -*- encoding: utf-8 -*- +# stub: svn2git 2.4.0 ruby lib + +Gem::Specification.new do |s| + s.name = "svn2git" + s.version = "2.4.0" + + s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version= + s.require_paths = ["lib"] + s.authors = ["James Coglan", "Kevin Menard"] + s.date = "2016-10-30" + s.email = "nirvdrum@gmail.com" + s.executables = ["svn2git"] + s.extra_rdoc_files = [ + "ChangeLog.markdown", + "README.markdown" + ] + s.files = [ + "ChangeLog.markdown", + "MIT-LICENSE", + "README.markdown", + "Rakefile", + "VERSION.yml", + "bin/svn2git", + "lib/svn2git.rb", + "lib/svn2git/migration.rb", + "svn2git.gemspec", + "test/commands_test.rb", + "test/escape_quotes_test.rb", + "test/test_helper.rb" + ] + s.homepage = "https://github.com/nirvdrum/svn2git" + s.licenses = ["MIT"] + s.rubygems_version = "2.5.1" + s.summary = "A tool for migrating svn projects to git" + + if s.respond_to? :specification_version then + s.specification_version = 4 + + if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then + s.add_development_dependency(%q, [">= 0"]) + else + s.add_dependency(%q, [">= 0"]) + end + else + s.add_dependency(%q, [">= 0"]) + end +end + diff --git a/test/commands_test.rb b/test/commands_test.rb new file mode 100644 index 0000000..c28b4f5 --- /dev/null +++ b/test/commands_test.rb @@ -0,0 +1,9 @@ +require File.expand_path(File.join(__FILE__, '..', 'test_helper')) + +class CommandsTest < Minitest::Test + def test_checkout_svn_branch + actual = Svn2Git::Migration.checkout_svn_branch('blah') + + assert_equal 'git checkout -b "blah" "remotes/svn/blah"', actual + end +end \ No newline at end of file diff --git a/test/escape_quotes_test.rb b/test/escape_quotes_test.rb new file mode 100644 index 0000000..3f5150c --- /dev/null +++ b/test/escape_quotes_test.rb @@ -0,0 +1,22 @@ +require File.expand_path(File.join(__FILE__, '..', 'test_helper')) + +class EscapeQuotesTest < Minitest::Test + def test_identity + expected = 'A string without any need to escape.' + actual = Svn2Git::Migration.escape_quotes(expected) + + assert_equal expected, actual + end + + def test_escape_single_quotes + actual = Svn2Git::Migration.escape_quotes("Here's a message with 'single quotes.'") + + assert_equal "Here\\'s a message with \\'single quotes.\\'", actual + end + + def test_escape_double_quotes + actual = Svn2Git::Migration.escape_quotes('Here is a message with "double quotes."') + + assert_equal 'Here is a message with \\"double quotes.\\"', actual + end +end \ No newline at end of file diff --git a/test/test_helper.rb b/test/test_helper.rb new file mode 100644 index 0000000..8327638 --- /dev/null +++ b/test/test_helper.rb @@ -0,0 +1,12 @@ +$:.unshift "#{File.dirname(__FILE__)}/../lib" + +require 'rubygems' +require 'svn2git' +require 'minitest/autorun' + +if Minitest.const_defined?('Test') + # We're on Minitest 5+. Nothing to do here. +else + # Minitest 4 doesn't have Minitest::Test yet. + Minitest::Test = MiniTest::Unit::TestCase +end \ No newline at end of file