To view the objects in the Git database, we first need a repository to be examined. For this recipe, we will clone an example repository located here:

$ git clone https://github.com/dvaske/data-model.git
$ cd data-model

Now you are ready to look at the objects in the database, we will start by looking first at the commit object, then the trees, the files, and finally the branches and tags.

Let's take a closer look at the object's Git stores in the repository.

$ git cat-file -p HEAD
tree 34fa038544bcd9aed660c08320214bafff94150b
parent a90d1906337a6d75f1dc32da647931f932500d83
author Aske Olsson <aske.olsson@switch-gears.dk> 1386933960 +0100
committer Aske Olsson <aske.olsson@switch-gears.dk> 1386941455 +0100

This is the subject line of the commit message

It should be followed by a blank line then the body, which is this text. Here you can have multiple paragraphs etc. and explain your commit. It's like an email with subject and body, so get people's attention in the subject

$ git cat-file -p 34fa038544bcd9aed660c08320214bafff94150b 
100644 blob f21dc2804e888fee6014d7e5b1ceee533b222c15    README.md
040000 tree abc267d04fb803760b75be7e665d3d69eeed32f8    a_sub_directory
100644 blob b50f80ac4d0a36780f9c0636f43472962154a11a    another-file.txt
100644 blob 92f046f17079aa82c924a9acf28d623fcb6ca727    cat-me.txt
100644 blob bb2fe940924c65b4a1cefcbdbe88c74d39eb23cd    hello_world.c

$ git cat-file -p 92f046f17079aa82c924a9acf28d623fcb6ca727

This is the content of the file: "cat-me.txt."

Not really that exciting, huh?

$ git cat-file master
usage: git cat-file (-t|-s|-e|-p|<type>|--textconv) <object>
   or: git cat-file (--batch|--batch-check) < <list_of_objects>

<type> can be one of: blob, tree, commit, tag.
...
$ git cat-file -p master
tree 34fa038544bcd9aed660c08320214bafff94150b
parent a90d1906337a6d75f1dc32da647931f932500d83
...

$ cat .git/refs/heads/master
34acc370b4d6ae53f051255680feaefaf7f7850d 

$ git log -1
commit 34acc370b4d6ae53f051255680feaefaf7f7850d
Author: Aske Olsson <aske.olsson@switch-gears.dk>
Date:   Fri Dec 13 12:26:00 2013 +0100

    This is the subject line of the commit message
...

$ cat .git/HEAD
ref: refs/heads/master

$ git tag
v0.1
v1.0

$ git cat-file -p v1.0
object 34acc370b4d6ae53f051255680feaefaf7f7850d
type commit
tag v1.0
tagger Aske Olsson <aske.olsson@switch-gears.dk> 1386941492 +0100

We got the hello world C program merged, let's call that a release 1.0

The Git command git cat-file -p will pretty print the object given as an input. Normally, it is not used in everyday Git commands, but it is quite useful to investigate how it ties together the objects. We can also verify the output of git cat-file, by rehashing it with the Git command git hash-object; for example, if we want to verify the commit object at HEAD (34acc370b4d6ae53f051255680feaefaf7f7850d), we can run the following command:

$ git cat-file -p HEAD | git hash-object -t commit --stdin
34acc370b4d6ae53f051255680feaefaf7f7850d

If you see the same commit hash as HEAD pointing towards you, you can verify whether it is correct with git log -1.

There are many ways to see the objects in the Git database. The git ls-tree command can easily show the contents of trees and subtrees and git show can show the Git objects, but in a different way.

We'll use the same data-model repository as seen in the last recipe:

$ git clone https://github.com/dvaske/data-model.git
$ cd data-model

First, we'll make a small change to the file and check git status:

$ echo "Another line" >> another-file.txt
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

  modified:   another-file.txt

no changes added to commit (use "git add" and/or "git commit -a")

This, of course, just tells us that we have modified another-file.txt and we need to use git add to stage it. Let's add the another-file.txt file and run git status again:

$ git add another-file.txt
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

  modified:   another-file.txt

The file is now ready to be committed, just as you have probably seen before. But what happened during the add command? The add command, generally speaking, moves files from the working directory to the staging area, but more than this actually happens, though you don't see it. When a file is moved to the staging area, the SHA-1 hash of the file is created and the blob object is written to Git's database. This happens for all the files added and every time a file is added, but if nothing changes for a file, this means it is already stored in the database. At first, this might seem that the database is growing quickly, but this is not the case. Garbage collection kicks in at times, compressing and cleaning up the database and keeping only the objects that are required.

We can edit the file again and run git status:

$ echo 'Whoops almost forgot this' >> another-file.txt
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

  modified:   another-file.txt

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

  modified:   another-file.txt

Now, the file shows up both in the Changes to be committed and Changes not staged for commit sections. This looks a bit weird at first, but there is of course an explanation. When we added the file the first time, the content of it was hashed and stored in Git's database. The changes from the second change of the file have not yet been hashed and written to the database; it only exists in the working directory. Therefore, the file shows up in both the Changes to be committed and Changes not staged for commit sections; the first change is ready to be committed, the second is not. Let's also add the second change:

$ git add another-file.txt
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

  modified:   another-file.txt

Now, all the changes we have made to the file are ready to be committed and we can record a commit:

$ git commit -m 'Another change to another file'
[master 55e29e4] Another change to another file
 1 file changed, 2 insertions(+)

As we learned previously, the add command creates the blob object, the tree, and commit objects; however, they are created when we run the commit command. We can view these objects with the cat-file command, as we saw in the previous recipe:

$ git cat-file -p HEAD
tree 162201200b5223d48ea8267940c8090b23cbfb60
parent 34acc370b4d6ae53f051255680feaefaf7f7850d
author Aske Olsson <aske@schantz.com> 1401744547 +0200
committer Aske Olsson <aske@schantz.com> 1401744547 +0200

Another change to another file

The root-tree object from the commit is:

$ git cat-file -p HEAD^{tree}
100644 blob f21dc2804e888fee6014d7e5b1ceee533b222c15  README.md
040000 tree abc267d04fb803760b75be7e665d3d69eeed32f8  a_sub_directory
100644 blob 35d31106c5d6fdb38c6b1a6fb43a90b183011a4b  another-file.txt
100644 blob 92f046f17079aa82c924a9acf28d623fcb6ca727  cat-me.txt
100644 blob bb2fe940924c65b4a1cefcbdbe88c74d39eb23cd  hello_world.c

From the previous recipe, we know the SHA-1 of the root tree was 34fa038544bcd9aed660c08320214bafff94150b and of the another-file.txt file was b50f80ac4d0a36780f9c0636f43472962154a11a, and as expected, they changed in our latest commit when we updated the another-file.txt file. We added the same file, another-file.txt, twice before we created the commit, recording the changes to the history of the repository. We also learned that the add command creates a blob object when called. So in the Git database, there must be an object similar to the content of another-file.txt the first time we added the file to the staging area. We can use the git fsck command to check for dangling objects, that is, objects that are not referred by other objects or references:

$ git fsck --dangling
Checking object directories: 100% (256/256), done.
dangling blob ad46f2da274ed6c79a16577571a604d3281cd6d9

Let's check the contents of the blob using the following command:

$ git cat-file -p ad46f2da274ed6c79a16577571a604d3281cd6d9
This is just another file
Another line

The blob is, as expected, similar to the content of another-file.txt when we added it to the staging area the first time.

The following diagram describes the tree stages and the commands used to move between the stages:

We will use the example repository from the last section and ensure that the master branch is pointing to 34acc37:

$ git checkout master && git reset --hard 34acc37

In the previous command, we only use the first seven characters (34acc37) of the commit ID; this is fine as long as the abbreviated ID used is unique in the repository.

The simplest way to see the history is to use the git log command; this will display the history in reverse chronological order. The output is paged through less and can be further limited, for example, by providing only the number of commits in history to be displayed:

$ git log -3

This will display the following result:

commit 34acc370b4d6ae53f051255680feaefaf7f7850d
Author: Aske Olsson <aske.olsson@switch-gears.dk>
Date:   Fri Dec 13 12:26:00 2013 +0100

    This is the subject line of the commit message.


    It should be followed by a blank line then the body, which is this text. Here 
    you can have multiple paragraphs etc. and explain your commit. It's like an 
    email with subject and body, so get people's attention in the subject

commit a90d1906337a6d75f1dc32da647931f932500d83
Author: Aske Olsson <aske.olsson@switch-gears.dk>
Date:   Fri Dec 13 12:17:42 2013 +0100

    Instructions for compiling hello_world.c

commit 485884efd6ac68cc7b58c643036acd3cd208d5c8
Merge: 44f1e05 0806a8b
Author: Aske Olsson <aske.olsson@switch-gears.dk>
Date:   Fri Dec 13 12:14:49 2013 +0100

    Merge branch 'feature/1'

    Adds a hello world C program.

By default, git log prints the commit, author's name and e-mail ID, timestamp, and the commit message. However, the information isn't very graphical, especially if you want to see branches and merges.

To display this information and limit some of the other data, you can use the following options with git log:

$ git log --decorate --graph --oneline --all

The previous command will show one commit per line (--oneline) identified by its abbreviated commit ID and the commit message subject. A graph will be drawn between the commits depicting their dependency (--graph). The --decorate option shows the branch names after the abbreviated commit ID, and the --all option shows all the branches, instead of just the current one(s).

$ git log --decorate --graph --oneline --all
* 34acc37 (HEAD, tag: v1.0, origin/master, origin/HEAD, master) This is the sub...
* a90d190 Instructions for compiling hello_world.c
*   485884e Merge branch 'feature/1'
...

This output, however, gives neither the timestamp nor author information, due to the way the --oneline option formats the output.

Fortunately, the log command gives us the possibility to create our own output format. So, we can make a history view similar to the previous. The colors are made with the %C<color-name>text-be-colored%Creset syntax: including the author and timestamp information, and some colors to display it nicely:

$ git log --all --graph --pretty=format:\
'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset'

This is a bit cumbersome to write, but luckily it can be made as an alias so you only have to write it once:

git config ----global alias.graph "log --all --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset'"

Git traverses the DAG by following the parent IDs (hashes) from the given commit(s). The options passed to git log can format the output in different ways; this can serve several purposes, for example, to give a nice graphical view of the history, branches, and tags, as seen previously, or to extract specific information from the history of a repository to use, for example, in a script.

Clone the JGit repository using the following command lines:

$ git clone https://git.eclipse.org/r/jgit/jgit
$ cd jgit

If you want the exact same output as in this example, reset your master branch to the following commit, b14a93971837610156e815ae2eee3baaa5b7a44b:

$ git checkout master && git reset --hard b14a939

You are now ready to look through the commit log for commit messages that describe the bugs fixed. First, let's limit the log to only look through the history since the last tag (release). To find the last tag, we can use git describe:

$ git describe 
v3.1.0.201310021548-r-96-gb14a939 

The preceding output tells us three things:

Now, the log can be parsed from HEAD to v3.1.0.201310021548-r. But just running git log 3.1.0.201310021548-r..HEAD will give us all the 96 commits, and we just want the commits with commit messages that contain "Bug: xxxxxx" for our release note. The xxxxxx is an identifier for the bug, for example, a number. We can use the --grep option with git log for this purpose: git log --grep "Bug: ". This will give us all the commits with "Bug: " in the commit message; all we need now is just to format it to something we can use for our release note.

Let's say we want the release note format to look like the following template:

Commit-id: Commit subject
Fixes-bug: xxx

Our command line so far is as follows:

$ git log --grep "Bug: " v3.1.0.201310021548-r..HEAD

This gives us all the bug fix commits, but we can format this to a format that is easily parsed with the --pretty option. First, we will print the abbreviated commit ID %h, followed by a separator of our choice |, then the commit subject %s, (first line of the commit message), followed by a new line %n, and the body, %b:

--pretty="%h|%s%n%b"

The output of course needs to be parsed, but that's easy with regular Linux tools such as grep and sed:

First, we just want the lines that contain "|" or "Bug: ":

grep -E "\||Bug: "

Then, we replace these with sed:

sed -e 's/|/: /' -e 's/Bug:/Fixes-bug:/'

The entire command put together gives:

\$ git log --grep "Bug: " v3.1.0.201310021548-r..HEAD --pretty="%h|%s%n%b" 
| grep -E "\||Bug: " | sed -e 's/|/: /' -e 's/Bug:/Fixes-bug:/'

The previous set of commands gives the following output:

f86a488: Implement rebase.autostash 
Fixes-bug: 422951 
7026658: CLI status should support --porcelain 
Fixes-bug: 419968 
e0502eb: More helpful InvalidPathException messages (include reason) 
Fixes-bug: 413915 
f4dae20: Fix IgnoreRule#isMatch returning wrong result due to missing reset 
Fixes-bug: 423039 	
7dc8a4f: Fix exception on conflicts with recursive merge 
Fixes-bug: 419641 
99608f0: Fix broken symbolic links on Cygwin. 
Fixes-bug: 419494 
...

Now, we can extract the bug information from the bug tracker and put the preceding code in the release note as well, if necessary.

First, we limit the git log command to only show the range of commits we are interested in, then we further limit the output by filtering the "Bug: " string in the commit message. We pretty print the string so we can easily format it to a style we need for the release note and finally find and replace with grep and sed to completely match the style of the release note.

If we just wanted to extract the bug IDs from the commit messages and didn't care about the commit IDs, we could have just used grep after the git log command, still limiting the log to the last tag:

$ git log  v3.1.0.201310021548-r..HEAD | grep "Bug: "

If we just want the commit IDs and their subjects but not the actual bug IDs, we can use the --oneline feature of git log combined with the --grep option:

$ git log --grep "Bug: " --oneline  v3.1.0.201310021548-r..HEAD

The same repository and HEAD position (HEAD pointing to b14a939) as seen in the previous recipe will be used. The release is also the same, which is v3.1.0.201310021548-r.

The following command lists all the files changed since the last release (v3.1.0.201310021548-r):

$ git diff --name-only v3.1.0.201310021548-r..HEAD
org.eclipse.jgit.packaging/org.eclipse.jgit.target/jgit-4.3.target 
org.eclipse.jgit.packaging/org.eclipse.jgit.target/jgit-4.4.target 
org.eclipse.jgit.pgm.test/tst/org/eclipse/jgit/pgm/DescribeTest.java 
org.eclipse.jgit.pgm.test/tst/org/eclipse/jgit/pgm/FetchTest.java 
org.eclipse.jgit.pgm/src/org/eclipse/jgit/pgm/Describe.java 
...

The git diff command operates on the same revision range as git log did in the previous recipe. By specifying --name-only, Git will only give the paths of the files as output changed by the commits in the range specified.

The output of the command can be further filtered; if we only want to show which files have been deleted in the repository since the last commit, we can use the --diff-filter switch with git diff:

$ git diff --name-only --diff-filter=D  v3.1.0.201310021548-r..HEAD 
org.eclipse.jgit.junit/src/org/eclipse/jgit/junit/SampleDataRepositoryTestCase.java 
org.eclipse.jgit.packaging/org.eclipse.jgit.target/org.eclipse.jgit.target.target 
org.eclipse.jgit.test/tst/org/eclipse/jgit/internal/storage/file/GCTest.java

There are also switches for the files that have been added (A), copied (C), deleted (D), modified (M), renamed (R), and so on.

Make sure you have Gitk installed:

$ which gitk
/usr/local/bin/gitk

If nothing shows up, Gitk in not installed on your system, or at least is not available on your $PATH.

Change the directory to the data-model repository from the objects and DAG examples. Make sure the master branch is checked out and pointing to 34acc37:

$ git checkout master && git reset --hard 34acc37

In the repository, run gitk --all & to bring up the Gitk interface. You can also specify the commit range or branches you want similar to git log or provide --all to see everything:

$ gitk --all &

Gitk parses the information for every commit and the objects attached to it to provide an easy graphical information screen that shows a graph of the history, author, and timestamp for each commit. In the bottom half, the commit message and the patches for each file changed and the list of files changed by the selected commit are displayed.

Though very lightweight and fast, Gitk is a very powerful tool. There are many different context menus regarding clicking on a commit, a branch, or a tag in the history view. You can create and delete branches, revert and cherry-pick commits, diff selected commits, and much more.

From the interface, you can perform a find and search. Find looks through the history and search looks through the information displayed in the lower half of Gitk for the currently highlighted commit.

Again, we will use the JGit repository, trying to find commits related to the keyword "Performance". In this recipe, we will look through the entire history, so we don't need the master branch to point to a specific commit.

As we tried earlier, we can use the --grep option to find specific strings in commit messages. In this recipe, we look at the entire history and search every commit that has "Performance" in its commit message:

$ git log --grep "Performance" --oneline --all 
9613b04 Merge "Performance fixes in DateRevQueue" 
84afea9 Performance fixes in DateRevQueue 
7cad0ad DHT: Remove per-process ChunkCache 
d9b224a Delete DiffPerformanceTest 
e7a3e59 Reuse DiffPerformanceTest support code to validate algorithms 
fb1c7b1 Wait for JIT optimization before measuring diff performance 

In this example, we specifically ask Git to consider all of the commits in the history, by supplying the --all switch. Git runs through the DAG and checks whether the "Performance" string is included in the commit message. For an easy overview of the results, the --oneline switch is also used to limit the output to just the subject of the commit message. Hopefully then the commit(s) we needed to find can be identified from this much shorter list of commits.

Note that the search is case sensitive; had we searched for "performance" (all in lower case), the list of commits would have been very different:

$ git log --grep "performance" --oneline --all
5ef6d69 Use the new FS.exists method in commonly occuring places
2be6927 Always allocate the PackOutputStream copyBuffer
437be8d Simplify UploadPack by parsing wants separately from haves
e6883df Enable writing bitmaps during GC by default.
374406a Merge "Fix RefUpdate performance for existing Refs"
f1dea3e Fix RefUpdate performance for existing Refs
84afea9 Performance fixes in DateRevQueue
8a9074f Implement core.checkstat = minimal
130ad4e Delete storage.dht package
d4fed9c Refactored method to find branches from which a commit is reachable
...

We also could have used the find feature in Gitk to find the same commits. Open Gitk with the --all switch, type Performance in the Find field and hit Enter. This will highlight the commits in the history view and you can navigate to the previous/next result by pressing Shift + up arrow, Shift + down arrow, or the buttons next to the Find field. You will still, however, be able to see the entire history in the view with the matching commits highlighted:

Again, we will use the JGit repository with the master branch pointing to b14a939:

$ git checkout master && git reset --hard b14a939

We would like to find all the commits that have changes made to lines that contain the method "isOutdated". Again, we will just display the commits on one line each then we can check them individually later:

$ git log -G"isOutdated" --oneline 
f32b861 JGit 3.0: move internal classes into an internal subpackage 
c9e4a78 Add isOutdated method to DirCache 
797ebba Add support for getting the system wide configuration 
ad5238d Move FileRepository to storage.file.FileRepository 
4c14b76 Make lib.Repository abstract and lib.FileRepository its implementation 
c9c57d3 Rename Repository 'config' as 'repoConfig' 
5c780b3 Fix unit tests using MockSystemReader with user configuration 
cc905e7 Make Repository.getConfig aware of changed config

Eight commits have patches that involve the string "isOutdated".

Git traverses the history, the DAG, looking at each commit for the string "isOutdated" in the patch between the parent commit and the current commit. This method is quite convenient to find out when a given string was introduced or deleted and to get the full context and commit at that point in time.

The -G option used with git log will look for differences in the patches that contain added or deleted lines that match the given string. However, these lines could also have been added or removed due to some other refactoring/renaming of a variable or method. There is another option that can be used with git log, -S, which will look through the difference in the patch text similar to the -G option, but only match commits where there is a change in the number of occurrences of the specified string, that is, a line added or removed, but not added and removed.

Let's see the output of the -S option:

$ git log -S"isOutdated" --oneline 
f32b861 JGit 3.0: move internal classes into an internal subpackage
c9e4a78 Add isOutdated method to DirCache
797ebba Add support for getting the system wide configuration
ad5238d Move FileRepository to storage.file.FileRepository
4c14b76 Make lib.Repository abstract and lib.FileRepository its implementation
5c780b3 Fix unit tests using MockSystemReader with user configuation
cc905e7 Make Repository.getConfig aware of changed config

The search matches seven commits, whereas the search with the –G option matches eight commits. The difference is the commit with the ID c9c57d3 is only found with the –G option in the first list. A closer look at this commit shows that the isOutdated string is only touched due to renaming of another object, and this is why it is filtered away from the list of matching commits in the last list when using the –S option. We can see the content of the commit with the git show command, and use grep -C4 to limit the output to just the four lines before and after the search string:

$ git show c9c57d3 | grep -C4 "isOutdated"
@@ -417,14 +417,14 @@ public FileBasedConfig getConfig() {
           throw new RuntimeException(e);
         }
       }
-    if (config.isOutdated()) {
+    if (repoConfig.isOutdated()) {
         try {
-              loadConfig();
+              loadRepoConfig();
         } catch (IOException e) {