Managing GIT projects within your GIT project

Ever had that genius moment you created a module which relieves you of a forever recurring problem, turning up in all of your projects? Well, it's a good thing you made a reusable module to cram into all of your projects. And then there is that moment you make a fix in this module, while it's running in 20+ websites.

How to maintain your own custom modules within all of your projects?

It's not always an option to contribute your module to the community due to the fact that it's too specific, or the company just doesn't allow it. There are several other options out there, but this blog post is about a solution using our beloved GIT tool.

The basic idea is that your main GIT project (in my example a Drupal 7 installation, but applicable to any project) is containing GIT submodules. Just repositories within repositories and the main repository is aware of the subrepositories. Let's assume you know GIT a bit (adding and committing stuff, initializing GIT and cloning from a remote repo).

Setup

GIT projects

You have your main project set up and pushed to a remote repository, for example on github: git://github.com/user/my-main-project.git
You have locally made a reusable module, move it out of your main project and create a remote repository for it too. Let's say, also on github: git://github.com/user/my-custom-module.git

Add the GIT submodule

$ cd /to/your/drupal/project
$ git submodule add git://github.com/user/my-custom-module.git sites/all/modules/custom/my_custom_module

By doing this, a new file .gitmodules is created which holds all the information about every GIT submodule in your project. By default it contains the local path, the remote url and remote branch for tracking updates. The submodules are also added to the general .git/config file of your main project so GIT is aware of the submodules. Our custom module is back in the main project.
 

$ git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

  new file:   .gitmodules
  new file:   sites/all/modules/custom/my-custom-module

When we check the git status, we see the new files to be committed.
 

$ git add .
$ git commit -m "Added submodule my_custom_module."
$ git push origin master 

Now we add the new files to the GIT index and commit them like any other changes. Finally we push it to the remote repo so it can be deployed to other environments.
Basically, your submodule is ready for use now.

Update the GIT submodule

If we want to update the GIT submodule, we handle the module as a project on its own. When making changes to the files of the GIT submodule, the GIT status of your main project will just indicate that the content of the module has been modified:

$ git status
On branch develop
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)

	modified:   sites/all/modules/custom/my_custom_module (modified content)

To actually see the changes, navigate to the module's root through the command line interface. Check the GIT status and you'll see the changed files. Commit them like you normally would. You probably want to push them to the remote repo too, as other projects might need the update too. That's the whole idea, obviously.

As a final step to take the changes into account in our main project's GIT repository, we have to commit the updated submodule. Remember the 'modified content'. Go back to your project's root and just commit the the submodule like you would commit a normal file. At this point we're telling our main project it has to point to the currently active commit hash of the submodule. By doing so, other developers or other environments know what version of the submodule to check out (see next step).

Deploy to environments

Say you've done all the above, and you're ready to checkout the project on some other environments (a web server or developer's machine). At this point we have 2 remote repositories containing our submodule and our main project which is aware of the submodule (and pointing to a certain commit hash). We also have everything working on our local machine, now let's check the project out on another machine.
 

$ git clone git://github.com/user/my-main-project.git

Clone the main project to the new machine. At this point, you'll see that out submodule's directory is just empty, because of the very reason it is a git repo itself. Let's fill it up.
 

$ git submodule init
$ git submodule update

Got to your project's root and perform the above commands. The first command is to make your main git repo aware of any new submodules. The second command will pull the files into the submodules (remember the directories were empty at first). It checks out the code of the commit hash it is pointing to in the submodule. This is it. The project is fully checked out and all submodules are present. Say someone else made an update to a submodule (or 10 submodules), you just perform the 'git submodule update' command and you're done.