Merging Two Git Repositories with Combined Histories

This procedure merges two Git repositories while preserving the complete commit history of both repositories.

Goal

We have:

• Repo A — the final/main repository
• Repo B — the repository being imported
• Repo B should appear inside Repo A under the priorwork/ directory
• Both repositories use the main branch

The final layout should look like:

repo-A/
├── existing files from Repo A
└── priorwork/
    ├── files from Repo B
    └── ...

Both repositories' Git histories will be preserved.

---

1. Start with a fresh clone of Repo B

Do not run git filter-repo inside Repo A.

Clone Repo B into a separate directory:

git clone <REPO-B-URL> repo-B
cd repo-B

Make sure you are on Repo B's main branch:

git checkout main

---

2. Rewrite Repo B so its files live under priorwork/

Run this command inside the fresh Repo B clone:

git filter-repo --to-subdirectory-filter priorwork

Because this is a fresh clone, you should not need --force.

Verify the result:

git ls-tree -r HEAD --name-only | head -20

You should see paths like:

priorwork/.gitignore
priorwork/README.md
priorwork/file1...
priorwork/src/...

This confirms that Repo B's files have been moved under priorwork/ in its history.

---

3. Go to Repo A

Switch to the final repository:

cd ../repo-A

Make sure you are on the branch that should receive Repo B:

git checkout main

Check that your working tree is clean:

git status

Ideally, Git should report:

nothing to commit, working tree clean

---

4. Add the rewritten Repo B as a remote

From inside Repo A:

git remote add priorwork ../repo-B

Verify:

git remote -v

You should see something similar to:

origin     <REPO-A-URL>
priorwork  ../repo-B

Here, priorwork is simply the name of the Git remote. It is not a directory in the working tree.

---

5. Fetch Repo B's rewritten history

Run:

git fetch priorwork

This downloads Repo B's Git history into Repo A's .git directory.

It does not create a priorwork/ directory in the working tree yet.

Verify that Repo B's files are stored under the desired path:

git ls-tree -r priorwork/main --name-only | head -20

You should see:

priorwork/.gitignore
priorwork/README.md
priorwork/file1...
priorwork/src/...

This confirms that Repo B was correctly rewritten before the merge.

---

6. Merge the two histories

Now merge Repo B into Repo A:

git merge priorwork/main --allow-unrelated-histories

The --allow-unrelated-histories option is required because Repo A and Repo B originally had independent Git histories.

If there are no conflicts, the merge should complete successfully.

The working tree should now look approximately like:

repo-A/
├── .git/
├── <Repo A files>
├── ...
└── priorwork/
    ├── .gitignore
    ├── README.md
    ├── file1...
    └── src/
        ├── ...
        └── ...

---

7. Verify that both histories were preserved

View the complete commit graph:

git log --graph --oneline --all --decorate

You should see the two independent histories joining at the merge commit.

You can also inspect Repo B's history:

git log --oneline priorwork/main

And inspect all history:

git log --oneline --all

---

8. Push the combined repository

Once everything has been verified:

git push origin main

The remote Repo A repository will now contain:

• Repo A's original files
• Repo B's files under priorwork/
• Repo A's complete Git history
• Repo B's complete Git history
• A merge commit connecting the two histories

---

Important Concept

The overall process is:

             Rewrite Repo B
Repo B ───────────────────────────►
                                    Repo B'
                                    priorwork/
                                        │
                                        │
Repo A ───────────────────────────────►│
                 merge                │
                                        ▼
                             Combined Repo A
                             ├── Repo A files
                             └── priorwork/
                                 └── Repo B files

The most important rule is:

Run git filter-repo on a fresh clone of Repo B, not on Repo A.

---

Quick Command Summary

Repo B

git clone <REPO-B-URL> repo-B
cd repo-B
git checkout main
git filter-repo --to-subdirectory-filter priorwork
git ls-tree -r HEAD --name-only | head -20

Repo A

cd ../repo-A
git checkout main
git status

git remote add priorwork ../repo-B
git fetch priorwork

git ls-tree -r priorwork/main --name-only | head -20

git merge priorwork/main --allow-unrelated-histories

Verify and push

git log --graph --oneline --all --decorate
git push origin main

---

Expected Result

repo-A/
├── files originally belonging to Repo A
└── priorwork/
    └── files originally belonging to Repo B

Both repositories' commit histories are preserved.

Signing Git Commits: Inappropriate ioctl for device

When I attempted to sign a Git commit, I encountered the following error:

$ git commit -S -m "important change"
error: gpg failed to sign the data:
[GNUPG:] KEY_CONSIDERED AAAABBBBCCCCDDDD00000 2
[GNUPG:] BEGIN_SIGNING H8
[GNUPG:] PINENTRY_LAUNCHED 583247 curses 1.2.1 - xterm localhost:10.0 - 1000/2001 0
gpg: signing failed: Inappropriate ioctl for device
[GNUPG:] FAILURE sign 83918950
gpg: signing failed: Inappropriate ioctl for device

fatal: failed to write commit object

After an investigation, I learned that the problem could be somethign with gpg itself. To gauge whether it is the problem with gpg, we could sign a message with gpg, e.g.,

$ echo "test" | gpg --clearsign
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

test
gpg: signing failed: Inappropriate ioctl for device
gpg: [stdin]: clear-sign failed: Inappropriate ioctl for device

Clearly, we observe the same error message. We can conclude it is the issue with gpg. It turns out that one reason of this is that tty is not set properly. To fix this, we set tty for gpg using

export GPG_TTY=$(tty)

This can be added to shell script's profiles

Completing Removing a Git Submodule

This is a note about removing  a Git submodule completely.

1. Revise the .gitmodules by removing the reference to the submodule Git repo;
2. Revise the .git/config file by removing the reference to the submodule Git repo;
3. Remove the submodule from Git index, e.g.,

       git rm --cached path_to_submodule
   

4. Delelete the meta data from .git directory, by removing .git/module/path_to_submodule
5. Stage the changes, and commit the changes, e.g.,

       git add -u
       git commit -m "removed the submodule"
       

Reference

The above is a summary from these references

• https://gist.github.com/myusuf3/7f645819ded92bda6677
• https://www.atlassian.com/git/articles/core-concept-workflows-and-tips
• https://stackoverflow.com/questions/1260748/how-do-i-remove-a-submodule

Purging a directory or a list of files from a Git repository

Sometimes we want to remove a directory or a file at a given path completely from a Git repository's history. BFG Repo-Cleaner is a recommended tool for this. Typical examples examples given are to remove files matching a name or a pattern anywhere in the repository. Sometimes this is not what we wanted, for instance, we want to remove a Readme.md in a particular directory, or remove a doc directory from a repository that has multiple files or directories of those names instead.

Below we show an example where we remove a file or a directory at a particular path. This relies on the following option of BFG Repo-Cleaner,

-bi, --strip-blobs-with-ids <blob-ids-file>  
           strip blobs with the specified Git object ids

where it is noteworthy that the argument given to this option is a file. The file contains Git object hashes of the files we wish to purge. 

In addition, to purge a directory is to purge all the files in the directory; once all of the files are gone, the directory is gone; however, BFG Repo-Cleaner would yield an error if we pass the Git object hash of a directory to it. 

The question becomes how we figure out the Git object hashes of the files we wish to purge. For this, we can use git rev-list command, e.g.,

git rev-list --all --objects | \
      grep "src/Readme.md" | \
      cut -d' ' -f1 > file_hashes.txt

Following this,

bfg -bi file_hashes.txt 
git push --force

How do we get the "diff" or the patch for the very first commit in Git?

This is a summary of several Stack Overflow discusses about how to get the "diff" or the patch for the very first commit in Git.

1. To get the diff from the very first commit to the empty tree, we can use the following command.

   git diff \
       4b825dc642cb6eb9a060e54bf8d69288fbee4904 \
       $(git log --reverse | head -1 | cut -d' ' -f2)
     

   where 4b825dc642cb6eb9a060e54bf8d69288fbee4904 is the SHA of the empty tree and $(git log --reverse | head -1 | cut -d' ' -f2) retrieves the SHA of the very first commit. Alternatively, we can rewrite the above in a way that perhaps is easier to read.
2. To get the diff from the very first commit to the empty tree, we can use the following command.

   git diff \
       $(printf '' | git hash-object -t tree --stdin) \
       $(git log --reverse | head -1 | cut -d' ' -f2)
     

3. To get the patch from the very first commit to the empty tree, we can use the following command.

   git format-patch \
       --root --stdout \
       $(git log --reverse | head -1 | cut -d' ' -f2)
       

Reference 

1. How to get Git diff of the first commit?  
2. How to use git format-patch on initial commit

"git status" shows modifications, but "git diff" shows nothing

I encountered a strange problem about a git repository today, that is, when I do a git status, git reports there are modifications. However, when I do a git diff, git reports no difference, like the following,

$ git status .
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add ..." to update what will be committed)
  (use "git checkout -- ..." to discard changes in working directory)

        modified:   .gitignore

$ git diff
$

After a little bit digging, apparently a few others encountered the same problem as a Stack Overflow discussion indicates.

The cause of my case are two,

1. Some files have different line endings from the version in the repository as a result that I was working on a Windows Subsystem for Linux and had the git's configuration property core.autocrlf set as input. I revealed the Windows line ending using vim, as in the following,
   

   
   $ vim -b .gitignore
   # Compiled class file^M
   *.class^M
   ^M
   # Log file^M
   *.log^M
   ^M
   # BlueJ files^M
   *.ctxt^M
   

   
   where the ^M characters are the Windows line endings, i.e., CR LF. I got rid of the line endings using sed, as in,
   

   
    sed -i -e s/^M// .gitignore
   

   
   Note that we enter the line ending by pressing CRTL-V CRTRL-M
   
2. The second issue is that some files have different permissions from those in the repository. Since I was using a Windows Subsystem for Linux on a Windows host, it is difficult for me to set the permissions identical to those in the repository. However, we can let git to ignore permission difference, i.e.,
   

   
   git config --local core.filemode false
   

   
   where I only did this for the repository. If we wish to do this for globally for all repositories, just issue the command without the --local option.

Making New Git Repository from Directories in Existing Git Repository

I want to make a new Git repository from a few directories in an existing Git repository. I found the solution provided in this Stack Overflow answer was exactly what I wanted.

• Do it with multiple and easy understandable steps.

Pushing Git Repositories to Multiple Remotes

According to this post, we configure as follows,

git remote add all URL_1
git remote set-url --add --push all URL_1
git remote set-url --add --push all URL_2
...
git remote set-url --add --push all URL_N

To push to the N remotes, do,

git push all