I’ve been working in a single git repository with many projects (subfolders) and needed to divide it by moving some folders (including their history) to a new repository, thereby removing these files (with history) from the original repository. There are a number of posts on this, however for my case none worked exactly as indicated. I rather had to combine instructions from various sources, that’s why I decided to write a summarizing blog post here.
Short Instructions
Basically, you have to perform these steps:
- Make a backup
- Clone the source repository
- Execute git subtree commands on all folders to be moved
- Create a new repository
- Pull the isolated folders into the new repository and move them to subdirectories
- Restore the original repository
- Delete all moved folders (including history) from the original repository
Detailed Instructions
First of all: Make a backup. Seriously, these operations will rewrite your git history and can be destructive, so make sure everything is stored elsewhere.
The first step is to create a new working copy of the existing repository.
1 | git clone git://my-server.tld/my-repo.git |
Imagine the following git repository with the following folders on top level:
A B C D
Let’s say we want to move A and B to another, new git repository and leave C and D in the original repository. Therefore we invoke git subtree split – a magic command that will create a branch containing only the history of a given subdirectory. The syntax is:
1 | git subtree split -P <directory> -b <branch> |
In our example, we isolate folder A on branch onlyA and folder B on branch onlyB:
1 2 | git subtree split -P A -b onlyA git subtree split -P B -b onlyB |
The source repository is now prepared. The next step is to create the new target repository:
1 2 3 4 | cd .. mkdir newrepo cd newrepo git init |
Now we pull a branch from the original repository:
1 | git pull /path/to/original/repository onlyA |
The path to the original repository must be an absolute path, at least relative paths did not work for me. When listing the directory content you will notice that everything was imported to the root of your repository. If you want the merged content to be in a subfolder again, you need to create it and move the files using git mv commands manually. Dont’t forget to commit your moved files. Don’t forget to move hidden files also! These can be discovered using ls -la.
1 2 3 4 5 | mkdir A git mv file1 A git mv file2 A ... git commit -m "Merged A and moved to subfolder" |
Note: Apparently it is not possible to move the files in the original repository and then use the split command. When doing this, my history got lost. For me it only worked the other way round (first split, then move the files in the target repository). Feel free to comment if you can explain this behaviour.
Repeat this process for all other folders to be moved:
1 2 3 4 5 6 | git pull /path/to/original/repository onlyB mkdir B git mv file1 B git mv file2 B ... git commit -m "Merged B and moved to subfolder" |
After that we restore the original repository completely (either using git clone or from a backup). Then we remove all moved folders (including history) from that repository. This is achieved using git filter-branch commands:
1 2 | git filter-branch -f --tree-filter 'rm -rf A' HEAD git filter-branch -f --tree-filter 'rm -rf B' HEAD |
That’s it! We have now moved a part of the original repository (including history) to a new repository and removed that part (including history) from the original repository. Effectively this divided the original repository into two independent repositories.
Resources
Here are some links that might be helpful for details and performance optimized operations:
Thank you Dave!
This is a great breakdown! I also found that other posts talking about this seemed to have conflicting instructions and with the different versions of git and subtree command, I was glad to find and follow your instructions! I couldn’t get the old folders removed by using the : “git filter-branch -f –tree-filter ‘rm -rf A’ HEAD” , but that was the least of my worries.
Thanks again!
-fabio
Thanks for that really useful article. Everybody developing with git may have this problem at some day. But I’m sure nobody really knows how to do this correctly and without pain.
I’m able to run `git pull /path/to/original/repository onlyA`, but when I try `git pull /path/to/original/repository onlyB`, I get the error “fatal: refusing to merge unrelated histories”. This is with git 2.12.1. Any idea why?
I also received this message, but solved it by instead using the command “git pull /path/to/original/repository onlyB –allow-unrelated-histories”. Hope this helps someone!
Thanks for these instructions. When doing multiple iterations of:
git pull, mkdir, git mv
for several directories, it was necessary to commit each one before proceeding to the next. However after src files were split into a submodule and supermodule of a new repo, busy coworkers were making new commits in the old repo and I only got a one line suggestion in my StackOverflow posting for how to catch up on these commits.
Since I did a git pull from one local repo to a new local repo, I do not want to subtree push to some remote, so I’m not sure subtree push helps.
https://stackoverflow.com/questions/48471530/git-subtree-split-into-new-repo-meanwhile-commits-to-old-repo?noredirect=1#comment83945959_48471530
I have one Question when you say hidden files , I do see only .git folder ? other than that there is no hidden folder, Let say you have 2 folders to move
should I move complete .git folder inside Folder A ?
then I when I pull folder B , I move its .git project under folder B?
$ ls -la
total 12
drwxr-xr-x 1 xyz 2147484161 0 Jun 4 13:20 ./
drwxr-xr-x 1 xyz 2147484161 0 Jun 4 01:16 ../
drwxr-xr-x 1 xyz 2147484161 0 Jun 4 13:20 .git/
drwxr-xr-x 1 xyz 2147484161 0 Jun 4 13:20 3.0/
drwxr-xr-x 1 xyz 2147484161 0 Jun 4 13:20 3.2/
No, the .git folder should be ignored in the process. In your case, only the folders 3.0 and 3.2 are relevant.