A while back, Steve Losh posted some excellent Django advice, in particular the part on working with third-party apps. I thought it sounded interesting but was concerned that it would be a hassle to maintain.
Once I finally tried it, it was quite liberating. Never again will I have to work around some issue with a package. Now I just fix it and move on.
It did take a bit of work, though, to figure out an approach that I was happy with. I usually use a pip requirements file to keep track of project dependencies and I wasn’t quite sure how to make it all work together.
One option is to use pip’s
editable flag like so:
What I didn’t like about that was having to constantly update my
requirements.txt to change the revision. And, by default, pip installs
editable packages into
$VIRTUAL_ENV/src while I’d rather they be in a
subdirectory of my project for easy access.
My first attempt at dealing with those issues was to fork every single package I was using, add a tag or branch for the revision my project was using and change the requirements.txt file to look like so:
Then I would manually specify the source directory when installing, like so:
The problems with this approach were that I was now maintaining forks of all these apps, it was just as tedious to be moving tags around as updating the revision hash, and now the requirements file wasn’t self-contained.
What I really wanted was to add all the apps as subrepositories to my project’s Mercurial repository. That way the revisions would be tracked automatically and no need for tags.
The hard part was getting pip to install these packages as editable without
making a copy somewhere. It took some trial and error to get right… though
I did finally figure it out (hint: don’t use
So now, when I want to add an app to my project, I first clone it to the
lib subdirectory. Then I add the above entry to my requirements
file and the following entry in
Finally, I install the package into my virtualenv with:
You might also want to add the following to your project’s
to avoid automatically committing the subrepos:
Don’t forget that you can also include apps that are in git or svn.