Author Topic: Gemini kernel repository with grafted history  (Read 1586 times)

ali1234

  • Newbie
  • *
  • Posts: 19
    • View Profile
Gemini kernel repository with grafted history
« on: June 07, 2018, 03:18:00 pm »
    [/li]
The Gemini kernel release from Planet has no git history. I spent the past week writing some software to correct this problem.

The result is a repository where unmodified files are linked back to their original upstream history. This covers about 90% of the files in the repository.

The process is mostly automated and was done like this:
  • Export the Planet kernel source to a tarball, extract it, and remove the nested directory.
  • Clone upstream repositories: linux-stable, linux-stable-rc, linux-next, linux-next-history, android-common, android-mediatek.
  • Run 'gitxref <path/to/upstream/repo> <path/to/tarball/directory>' It does the following:
    • For every file in the tarball, find a list of upstream commits containing that exact file.
         
    • For all the found commits, count how many tarball files they exactly match.
         
    • Print out the commit with the most matching files.
         
    • For every remaining commit, count how many matching files they have, excluding the files that match the already printed commit.
         
    • Repeat until there are no matched files left.
  • Check out the upstream commit that has the most files in common with the tarball.
  • Copy the tarball over it.
  • Make a temporary commit of the resulting work dir, including new files: 'git add *; git commit -a'
  • Get the sha1 of the tree of the commit you just made: 'git cat-file -p HEAD'
  • Manually create a merge commit listing every matching upstream commit as a parent, and referring to the tree of the temporary commit: 'git commit-tree -p <parent sha1> -p <parent sha1> ... <tree sha1>'
The result is this: https://github.com/ali1234/linux-gemini/tre...d075f04d4f7065a

This should be considered a proof of concept. There may be errors in the merge. There may also be more upstream repositories which would be candidates for merging in order to produce a smaller diff. There may even be a better approach to the whole thing. Suggestions are welcome, here or in github issues.

Beiriannydd

  • Newbie
  • *
  • Posts: 36
    • View Profile
Gemini kernel repository with grafted history
« Reply #1 on: June 08, 2018, 10:01:15 am »
Quote from: ali1234
    [/li]
The Gemini kernel release from Planet has no git history. I spent the past week writing some software to correct this problem.

The result is a repository where unmodified files are linked back to their original upstream history. This covers about 90% of the files in the repository.

The process is mostly automated and was done like this:
  • Export the Planet kernel source to a tarball, extract it, and remove the nested directory.
  • Clone upstream repositories: linux-stable, linux-stable-rc, linux-next, linux-next-history, android-common, android-mediatek.
  • Run 'gitxref <path/to/upstream/repo> <path/to/tarball/directory>' It does the following:
    • For every file in the tarball, find a list of upstream commits containing that exact file.
         
    • For all the found commits, count how many tarball files they exactly match.
         
    • Print out the commit with the most matching files.
         
    • For every remaining commit, count how many matching files they have, excluding the files that match the already printed commit.
         
    • Repeat until there are no matched files left.
  • Check out the upstream commit that has the most files in common with the tarball.
  • Copy the tarball over it.
  • Make a temporary commit of the resulting work dir, including new files: 'git add *; git commit -a'
  • Get the sha1 of the tree of the commit you just made: 'git cat-file -p HEAD'
  • Manually create a merge commit listing every matching upstream commit as a parent, and referring to the tree of the temporary commit: 'git commit-tree -p <parent sha1> -p <parent sha1> ... <tree sha1>'
The result is this: https://github.com/ali1234/linux-gemini/tre...d075f04d4f7065a

This should be considered a proof of concept. There may be errors in the merge. There may also be more upstream repositories which would be candidates for merging in order to produce a smaller diff. There may even be a better approach to the whole thing. Suggestions are welcome, here or in github issues.[/quote]

Thanks! Your git-fu is strong.  You might be able to pick up more of the files if they exist in the source if you walk the git log for changes to that file looking for a minimal diff.