Showing posts with label git. Show all posts
Showing posts with label git. Show all posts

Friday, November 9, 2012

Merging most of one git branch into another, without history rewrite

This little post explains how to partially commit a feature branch into a stable branch in git, without rewriting the history of any of the branches.

Having worked on a feature branch for a few weeks, I am left with 50+ commits on top of the common ancestor point. Yet I cannot merge all those changes back to stable, as some are still feature branch specific. And the feature branch is public (pushed to a remote repo) and used by our CI server. So I don't really want to rewrite its history.


S1                                (stable)
   \                            
    C1-F1-C2-C3-F2    (feature)

Basically I want to merge into stable C1, C2, C3 but keep F1, F2 in my branch.



So to be able to merge most of my changes into the stable branch, I am going to create an intermediate branch containing the changes I can have.

In effect I will have 3 branches

  • the stable branch
  • the feature branch
  • the stable_feature_common branch: where I keep work in sync between the 2 others.
Here's how I did it.

1. Find the common ancestor between my normal branch and my feature branch

$ git merge-base stable feature
0e43ddd8857fdcff0f50258cde194b6dd2ef26ff

2. find how many changes we had

$ git log --pretty=oneline 0e43ddd8857fdcff0f50258cde194b6dd2ef26ff..HEAD | wc -l
57

3. create a new stable_feature_common branch to contain my changes

$ git checkout -b stable_feature_common 0e43ddd8857fdcff0f50258cde194b6dd2ef26ff

4. take note of the current HEAD of my feature branch

 f19869fd6ef69186bb2e8172833b09521e7aacb5

5. create a rebased copy of my feature branch into my common branch.

$ git rebase --onto stable_feature_common stable_feature_common feature

When using git rebase --onto 1 2 3, git rebase all changes between 2 and 3 onto 1. As consequence of this operation, 3 becomes the new 1. In my case, the resulting work is referenced as feature. We will have to rename it.

Note the new HEAD of the feature branch (for the future feature_common branch)

697176ed7370099a9e559b6c82f3ac12caa36d47

6. force the feature branch reference back to where it was

$ git checkout stable_feature
$ git reset --hard f19869f

7. force the feature_common branch reference

$ git branch -f stable_metro_common 697176ed7370099a9e559b6c82f3ac12caa36d47

Now feature_stable_common and feature are identical code wise (but with different commit ids and istory caused by the rebase). I am going to edit feature_stable_common since the branch point commit and remove/edit the things I need only in that branch.

$ git checkout stable_feature_common
$ git branch -i 0e43ddd8857fdcff0f50258cde194b6dd2ef26ff

[lots of editing/commiting]

8. merge feature_stable_common into feature and stable (to easily track future changes)

$ git checkout stable_feature
$ git merge stable_feature_common

$ git checkout stable
$ git merge stable_feature_common

I now have

S1                     S2           (stable)
  \                      /
    C1´-C2´-C3´                (stable_feature_common)
   \                        \
    C1-F1-C2-C3-F2-M    (feature)

C1´, C2´, C3´ are rebased and potentially edited versions of C1, C2 and C3
S2 is my merged common into stable
M is my merged common into feature

This should allow me to track the future commits and merge them back properly.

Some links I've found entertaining:


Wednesday, January 5, 2011

Publishing your signed OSS project to the Sonatype OSS repository

One thing I wanted to do was to publish the yui-runner releases to ibiblio. It's now pretty easy thanks to the Sonatype OSS repository, which will help you stage and release your repositories.

See the official Sonatype OSS Maven Repository guide.

Because of a few choices (git, scala), I ran into a few troubles or quirks that are worth noting though. Here's the outline of the procedure I picked up:

  • clean up your POM as per the documentation
  • no need yet to add sections for javadoc, pgp, release etc. They are by default included once the parent is inherited from. If you wish to test javadoc or pgp, you should then enable the sonatype-oss-release profile to your maven execution when needed.
  • if you do add the javadoc attach-sources goal, you might ran into problems that causes the reactor to not find the project's own internal dependencies. This is because I think that particular goal forks maven. See https://github.com/lacostej/yui-runner/commit/7f15baa5a73c0da773a283800521a12d8c20a905
  • you might want to configure the git push off by default (http://jira.codehaus.org/browse/SCM-444). This I think fits well with the staging principle of the Sonatype OSS repo. Use something like:

    <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-release-plugin</artifactId>
    <version>2.1</version>
    <configuration>
    <pushChanges>false</pushChanges>
    </configuration>
    </plugin>

  • during release, you might want to add the use of the gpg agent if you don't like to enter your GPG passphrase for every module

    <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-gpg-plugin</artifactId>
    <version>1.1</version>
    <configuration>
    <useAgent>true</useAgent>
    </configuration>
    </plugin>

  • the Sonatype OSS repository expects releases to have a javadoc artifact. As the maven-scala-plugin doesn't yet behave like the maven-javadoc-plugin, and doesn't produce nor attach a javadoc artifact to your build, I cheated a bit. This is of course temporary.

  • Deploying will require the use of GPG signatures. This is what i did on Ubuntu 10.10 to get it working as I wanted (with a gpg-agent):

    1. create a signing key:

      jerome@lenovo-w510:~ $ gpg --gen-key
      jerome@lenovo-w510:~ $ gpg --list-keys
      jerome@lenovo-w510:~ $ gpg --keyserver pgp.mit.edu --send-key 576831db3ae26aaf

    2. install the agent:

      root@lenovo-w510:~ # apt-get install gnupg-agent

    3. configure the gpg-agent. Under ~/.gnupg/gpg-agent.conf, write

      pinentry-program /usr/bin/pinentry-gtk-2
      default-cache-ttl 86400
      max-cache-ttl 86400


    4. start the agent and copy the output in the terminal (this can be moved to your DE session startup instead later on)

      jerome@lenovo-w510:~ $ gpg-agent --daemon
      GPG_AGENT_INFO=/tmp/gpg-t61OEn/S.gpg-agent:28820:1; export GPG_AGENT_INFO;
      jerome@lenovo-w510:~ $ GPG_AGENT_INFO=/tmp/gpg-t61OEn/S.gpg-agent:28820:1; export GPG_AGENT_INFO;


  • Now the release takes the form of:

    1. clean your directory (git status, git ignore, etc)
    2. mvn release:clean (optional)
    3. mvn release:prepare
    4. git push --tags && git push (if you disabled push)
    5. mvn release:perform
    6. follow the Sonatype doc for testing your staging repo and releasing it. If you run into problems, don't forget to drop your repo before releasing again.

Thursday, December 9, 2010

Git svn tree corruption

I use git over svn for all my subversion projects. This morning I hit the following problem:


$ git svn rebase
error: garbage at end of loose object 'd0df538270dcaf65a232fb4127b7c59212420129'
fatal: object d0df538270dcaf65a232fb4127b7c59212420129 is corrupted
diff-index HEAD --: command returned error: 128


Mmm no backup accessible right now, and no way I am pulling remotely a git svn for this 7 years old tree.

First let's find information on the commit.

$ mv .git/objects/d0/df538270dcaf65a232fb4127b7c59212420129 ../d0df538270dcaf65a232fb4127b7c59212420129
$ git fsck --full
broken link from tree 066ce753000ca9e1cfbd07a709cac4c7c3e5da47
to tree d0df538270dcaf65a232fb4127b7c59212420129
dangling tree 866d5ac672d6ef4009319e176f195fd01697165c
missing tree d0df538270dcaf65a232fb4127b7c59212420129


What does the previous tree contains ?

$ git ls-tree 066ce753000ca9e1cfbd07a709cac4c7c3e5da47
100755 blob 888ed870384e87dad0642199b4d7d938246946a8 xxx1
100755 blob 346bd13c9ba0d15046af891ccc88aa225d3c7ee3 xxx2
100755 blob 2eef3f1f2985ff1d9cdd4a23c8ebca54a9af1010 xxx3
100644 blob 6a501e34ee0fb197fa6770b2a8cc40cdf276607a xxx4
100755 blob fe61ef5d88201a7502057a2ff5fdd45db44f5075 xxx5
040000 tree 38354b54d3dca537d9c7c9d9ec1668c0c7287bc9 ddd1
040000 tree 50be3bfee5b37fc97b379ae04fe961a212a70a32 ddd2
040000 tree 6b108f1b534bb9465c2e31efe51d06fb05499376 ddd3
040000 tree 532a6da6ece3151b43a1cd0fd174fdadb071e402 ddd4
040000 tree 45363718750ed5142fad1d7be311581748057297 ddd5
040000 tree d0df538270dcaf65a232fb4127b7c59212420129 ddd6
040000 tree b77d3e9240185ed148727e01d4d03ad726411724 ddd7
040000 tree 8b69621e177132f3dc822c427501f1921f39bbe1 ddd8


Didn't help much.

In that SVN tree, my history is pretty linear. I try to find the commit before or after the one that fails:

$ git log --raw | grep svn+ssh | tail -1 | cut -d '@' -f 2
fatal: unable to read destination tree (d0df538270dcaf65a232fb4127b7c59212420129)
4321 20108eba-4dfa-0310-bbc5-8f3fd2a08cff


Log fails to find the full history, and I now know which commits maps to the broken tree: 4320: the one before the last succesffuly logged commit (4321).

Let's verify:

$ git svn find-rev r4320
4e0264e69a5289c5ac5c461085c900009c90c5b9
$ git show 4e0264e69a5289c5ac5c461085c900009c90c5b9
fatal: unable to read destination tree (d0df538270dcaf65a232fb4127b7c59212420129)


Now let's just fetch some versions around that commit and replace our broken tree file.


$ svn clone svn+ssh://blablabla.com/opt/svn-repository/xxx/trunk/xxx -r 4318:4322 xxx2.git
$ mv xxx2.git/.git/objects/d0/df538270dcaf65a232fb4127b7c59212420129 .git/objects/d0/


Check the commit ?

$ git show 4e0264e69a5289c5ac5c461085c900009c90c5b9


git svn rebase now works !

Wednesday, February 3, 2010

A glimpse of the workflow around the Linux stable tree

Found this small fun to watch. Notice how using scripts and aliases can make things quite fast.

Reminds me when I was using quilt to manage extra patches on top of cruisecontrol (about 6 years ago...).

Wednesday, September 2, 2009

Hudson, maven and git on eapps.com VPS

VPS hosting from eapps.com is a reasonable compromise between cost and hosting service quality. I use it to host a hudson CI server.

This entry explains how I set it up up to a point where I can build using maven a java project hosted on a ssh protected git repository (on github). I will not go into the hudson specific stuff, the hudson wiki is usually complete & well worded. But I will spend more time on the OS specific stuff such as daemon configuration, ssh specifics, etc...