In a previous post I wrote about how nice it'd be if Mercurial automatically could choose rebase or merge depending on whether there are conflicts that need manual resolution.
So, "yesternight", after finishing the checkfiles extension I wrote about recently I couldn't go back to sleep (despite it being 2:30 AM). So, I got back up and hacked together the rebaseif extension that adds a command and an option to pull that does exactly that. It turns out it wasn't that hard, although the process could probably be optimized slightly as it currently attempts a rebase, aborts on conflicts and then does a merge. (The better way would be to detect which operation would succeed and perform that.)
Now, some support for that in TortoiseHg and we're all set. :)
Showing posts with label mercurial. Show all posts
Showing posts with label mercurial. Show all posts
Saturday, November 27, 2010
Mercurial extension writing
I just spent the better part of the evening and early night to hack together a Mercurial extension to check for and fix tabs and trailing whitespace in files to be committed.
Diving into a (relatively) poorly documented Python API is not easy. My previous attempts with Django (extremely good docs) and BuildBot (perfectly acceptable docs) went much better in terms of tackling the early learning curve.
As an example, I have not yet managed to figure out how to get the added diff lines of the working directory, or the files included in an upcoming commit (this are slightly different). The MercurialAPI wiki page only gives a brief overview, after that you're on your own and have to use the source. This is pretty fine, except that dynamic languages need more written docs if the source is to be understood better.
Anyway, the extension works pretty nicely as I decided to check the _entire_ file instead of just the changed lines. Shouldn't make a big difference performance wise, and it will allow a code base to go from mixes-tabs to spaces-only much faster. I can always revisit that part later if the need arises.
The next part in the plan is to write a small script that auto-injects this into each developers hgrc file on checkout. (I have a script that one must run, more or less, to deploy/unzip third-party libraries and set up environment variable, so it can just as well edit the hgrc file too.) I'm not worried about the security implications currently, as we're four developers on a corporate network, so everyone should play nicely at the moment, and secondly, I can always safeguard the paths to the extension (or put in in a subrepo) to which no-one but a select few, including myself, have commit/push rights (in case of a rogue developer).
The next little script I write will probably be something like that, to allow deployment of local extensions. I'll also add some extra meta-data to a commit and a central-repo hook to check for that, so that one can't push without the meta-data (ensuring that everyone starts using the hook ASAP) and also run the same checks there (in case someone decides to override the hook).
Hacking should not happen out of pure malice, but I've had developers disabling the global warnings-in-log-at-exit popup just because they where annoyed by them. (The best way would've been to fix the warnings inside the app, or at least have a chat with me on which log messages are warnings and if it's worthwhile to add a feature where said developer may disable the popup for a few hours.) Since the popup was disabled, many others in the team missed warnings or errors in the log and subsequently the CI build failed with warnings after commit. Sadly quite contraproductive and against the spirit of early-warnings.
Summing it up, I really enjoy hacking together small things in Python, but for larger projects it seems to become pretty unbearable and you really want to know the types of the variables. I really wish C# and WPF was available on more than one platform, so that we could go with that instead of C++/Python on Qt. It does seem to be a nicer road to walk.
(Yes, I've thought about Java, and I've worked with it for a few years. And no, I don't really like it, but I suppose SWT is at leats a bearable GUI-API (i.e. better than Swing), but lately with the Oracle takeover I'm very wary of betting on Java going anywhere but down the drain for non-enterprise projects.)
Obligatory links:
Diving into a (relatively) poorly documented Python API is not easy. My previous attempts with Django (extremely good docs) and BuildBot (perfectly acceptable docs) went much better in terms of tackling the early learning curve.
As an example, I have not yet managed to figure out how to get the added diff lines of the working directory, or the files included in an upcoming commit (this are slightly different). The MercurialAPI wiki page only gives a brief overview, after that you're on your own and have to use the source. This is pretty fine, except that dynamic languages need more written docs if the source is to be understood better.
Anyway, the extension works pretty nicely as I decided to check the _entire_ file instead of just the changed lines. Shouldn't make a big difference performance wise, and it will allow a code base to go from mixes-tabs to spaces-only much faster. I can always revisit that part later if the need arises.
The next part in the plan is to write a small script that auto-injects this into each developers hgrc file on checkout. (I have a script that one must run, more or less, to deploy/unzip third-party libraries and set up environment variable, so it can just as well edit the hgrc file too.) I'm not worried about the security implications currently, as we're four developers on a corporate network, so everyone should play nicely at the moment, and secondly, I can always safeguard the paths to the extension (or put in in a subrepo) to which no-one but a select few, including myself, have commit/push rights (in case of a rogue developer).
The next little script I write will probably be something like that, to allow deployment of local extensions. I'll also add some extra meta-data to a commit and a central-repo hook to check for that, so that one can't push without the meta-data (ensuring that everyone starts using the hook ASAP) and also run the same checks there (in case someone decides to override the hook).
Hacking should not happen out of pure malice, but I've had developers disabling the global warnings-in-log-at-exit popup just because they where annoyed by them. (The best way would've been to fix the warnings inside the app, or at least have a chat with me on which log messages are warnings and if it's worthwhile to add a feature where said developer may disable the popup for a few hours.) Since the popup was disabled, many others in the team missed warnings or errors in the log and subsequently the CI build failed with warnings after commit. Sadly quite contraproductive and against the spirit of early-warnings.
Summing it up, I really enjoy hacking together small things in Python, but for larger projects it seems to become pretty unbearable and you really want to know the types of the variables. I really wish C# and WPF was available on more than one platform, so that we could go with that instead of C++/Python on Qt. It does seem to be a nicer road to walk.
(Yes, I've thought about Java, and I've worked with it for a few years. And no, I don't really like it, but I suppose SWT is at leats a bearable GUI-API (i.e. better than Swing), but lately with the Oracle takeover I'm very wary of betting on Java going anywhere but down the drain for non-enterprise projects.)
Obligatory links:
- checkfiles.py on BitBucket.org.
- CheckFilesExtension on the Mercurial Wiki.
- ml-hgext project on Ohloh.net
- Ozric Tentacles on Spotify, great background coding music (instrumental progressive/trance crossover)
Thursday, November 4, 2010
Mercurial rebase/merge musings
I asked about how-to-check-for-potential-merge-rebase-conflicts-in-mercurial on StackOverflow yesterday and got a few interesting responses.
I think I'm going to write a small plugin that rebases if there are no file conflicts, and merges otherwise. This is what I want myself and everyone else to do, as it helps with clearing up bad merges afterwards, and avoid "'rebase broke the build and my life sucks" situtations.
Update 2010-11-27: I've written an extension for this, see this post for details and links.
Update 2010-11-27: I've written an extension for this, see this post for details and links.
Thursday, March 12, 2009
My BuildBotIcon app now hosted on BitBucket
I've uploaded my BuildBotIcon app to BitBucket.org!
It's a small java app that polls a buildmaster and changes the color of an icon in the java tray depending on the status. It also alerts when statuses change so you notice it a bit more.
I wrote it 1,5 years ago and published the jar-file on BuildBot's mail-list, but I decided to put it up as a proper project tonight, just for fun.
Bitbucket and Github are both greats hosting sites for open source projects. Very easy to use, very fast (compared to SourceForge) and they use DVCS:es both. I like Mercurial more, so I opted for Bitbucket.
On a related note, my work with patches for BuildBot on Github is apparently going fairly well, as I was just promoted to contributor with commit rights to the current 'official' BuildBot repository. (Dustin Mitchell's master branch).
On a related related note, Github's network feature is cool, if you like graphs. See mine here.
Oh, I also hang out a bit at StackOverflow.com.
It's a small java app that polls a buildmaster and changes the color of an icon in the java tray depending on the status. It also alerts when statuses change so you notice it a bit more.
I wrote it 1,5 years ago and published the jar-file on BuildBot's mail-list, but I decided to put it up as a proper project tonight, just for fun.
Bitbucket and Github are both greats hosting sites for open source projects. Very easy to use, very fast (compared to SourceForge) and they use DVCS:es both. I like Mercurial more, so I opted for Bitbucket.
On a related note, my work with patches for BuildBot on Github is apparently going fairly well, as I was just promoted to contributor with commit rights to the current 'official' BuildBot repository. (Dustin Mitchell's master branch).
On a related related note, Github's network feature is cool, if you like graphs. See mine here.
Oh, I also hang out a bit at StackOverflow.com.
Sunday, December 14, 2008
Mercurial and python 2.6! :)
This is a follow up on my previous post 'Distributed Version Control Systems on Windows' where I just want to note that I've made some progress with Mercurial by using Python 2.6 instead of 2.5.2.
The good thing is that Python 2.6 uses MSVS 2008, which I do have available. So with that, I can use setuptools to compile python libs. Including Mercurial!
Compiling and install Mercurial into my Python 2.6 64-bit installation was dead easy, by the books all the way! Yay!
That will probably help with running buildbot/mercurial/trac on a pure windows-based server (as in no cygwin). But I'm considering using a linux box for this anyway, since it's smoother and more easy to add misc minor stuff to. (Not disregarding setting up ssh-accounts for mercurial users, which is what I've done currently.)
I'm currently using it to try to assist the Mercurial situation on BuildBot, which is lacking a bit, especially when it comes to in-repo branches. There are patches on the buildbot trac, but they haven't been applied and the crucial one won't be added until it gets a proper test-case.
So, I'm working on that. There's some issue with line-endings preventing try-patches to apply cleanly in the tests. (It works in real life, as I did use the try-functionality at my last job. Never got it ready for every developer though...)
The good thing is that Python 2.6 uses MSVS 2008, which I do have available. So with that, I can use setuptools to compile python libs. Including Mercurial!
Compiling and install Mercurial into my Python 2.6 64-bit installation was dead easy, by the books all the way! Yay!
That will probably help with running buildbot/mercurial/trac on a pure windows-based server (as in no cygwin). But I'm considering using a linux box for this anyway, since it's smoother and more easy to add misc minor stuff to. (Not disregarding setting up ssh-accounts for mercurial users, which is what I've done currently.)
I'm currently using it to try to assist the Mercurial situation on BuildBot, which is lacking a bit, especially when it comes to in-repo branches. There are patches on the buildbot trac, but they haven't been applied and the crucial one won't be added until it gets a proper test-case.
So, I'm working on that. There's some issue with line-endings preventing try-patches to apply cleanly in the tests. (It works in real life, as I did use the try-functionality at my last job. Never got it ready for every developer though...)
Friday, October 17, 2008
Distributed Version Control Systems on Windows...
It's never easy...
So, having used Subversion and Trac successfully for roughly three years, I now have the oppurtunity to research and choose the technology base at the startup where I'm currently at.
DVCS:es seems to be all the rage, with Git and Mercurial being the two topmost contenders (I know about bazaar and a few others, but I didn't bother looking too much.)
Now, my prerequisites was that I wanted to get this to run on a Windows server (for various reasons). Open Source on the bleeding edge and Windows is never easy, but I've learnt a few things and gotten halfway there.
I'm using:
* Apache 2.2
* ModPython
* Python 2.5
* Trac 0.11
* Windows XP (server) and Vista (client)
Git
Git has a standalone installer for windows build on MSys (Mingw) which works well, both from the command-line and using the Tcl GUI.
By using Cygwin on the server, one can use ssh to access the common repo and push/pull. (Cygwin can import domain passwords, so you don't need to bother with dual user/password lists)
Pulling from http using webdav/apache is possible. With mod_auth_sspi you can again use windows domain usernames/password. (It might be possible to use LDAP too, haven't tried yet)
Git in daemon mode with by cygwin and inetd works too. (fast download path)
Trac and Git on windows does _not_ work, yet. This is because GitPlugin uses an old version of PyGit (not the PyGit on PyPi), for detail see this ticket. I might get around to do some hacking there.
I'm working on getting gitweb.cgi to run. I just get "403 Forbidden - No project found." all the time.
Mercurial
There are clients both for command-line (py2exe standalone builds, no python required) as well as windows shell integration (although it still has some rough edges, being 0.5 at the time of writing).
Trying to use Mercurial with mod_python is tricky, since the command-line dist has everything in library.zip (including dll's), which regular python can't import. So you need to unpack it somewhere and add that dir to sys.path. Also, Mercurial expects it's Templates to be in the same dir, so you need to copy that as well (otherwise you'll get hit an assert when using hgweb.cgi). See here for details.
hgweb.cgi works! See here for some tips.
Using MercurialPlugin in Trac from mod_python does not work. There is an issue with demandimport, or expat (under unix), or something. Which simply makes apache or python crash.
Installing Mercurial into c:\Python25 using easy_install might help (to avoid importing library.zip from the stand-alione cmd-line binaries), but that requires Visual C++ 7.1, which I don't have. I haven't found anyone who has precompiled for that use yet. (Copying the files to lib/site-packages did not work.)
I did get trac to run in tracd mode by modifying tracd-script.py and adding 'sys.path.append(r'C:\Program Files\Mercurial\lib')' and running it with 'python tracd-script.py --port ...' !! I have .py files associated with python so can just run py-files as .bat/.cmd, but this did not work here! Very strange.
So, tracd is slower than mod_python (or mod_wsgi, which I haven't tried, but I expect it to have the same wierd issues as mod_python), so I guess I'm stuck with using tracd.
Tracd can be run as a service with the help of the Windows Resource Kit, but you don't get apache authentication (LDAP/SSPI). :(
Anyway, that seems to be the best bet.
Final words
I'm thinking of using apache under cygwin instead. That might help somewhat. Or just usel ighttpd/fastcgi on cygwin and windows-apache's mod_proxy. But.. the layers.. the jumping.. the hoops... arggh..
OTOH, I might go for subversion and wait until the DVCS-on-Win32 state improves (perhaps with my help), or until I get a Unix based server to toy with...
To be continued, as they say...
So, having used Subversion and Trac successfully for roughly three years, I now have the oppurtunity to research and choose the technology base at the startup where I'm currently at.
DVCS:es seems to be all the rage, with Git and Mercurial being the two topmost contenders (I know about bazaar and a few others, but I didn't bother looking too much.)
Now, my prerequisites was that I wanted to get this to run on a Windows server (for various reasons). Open Source on the bleeding edge and Windows is never easy, but I've learnt a few things and gotten halfway there.
I'm using:
* Apache 2.2
* ModPython
* Python 2.5
* Trac 0.11
* Windows XP (server) and Vista (client)
Git
Git has a standalone installer for windows build on MSys (Mingw) which works well, both from the command-line and using the Tcl GUI.
By using Cygwin on the server, one can use ssh to access the common repo and push/pull. (Cygwin can import domain passwords, so you don't need to bother with dual user/password lists)
Pulling from http using webdav/apache is possible. With mod_auth_sspi you can again use windows domain usernames/password. (It might be possible to use LDAP too, haven't tried yet)
Git in daemon mode with by cygwin and inetd works too. (fast download path)
Trac and Git on windows does _not_ work, yet. This is because GitPlugin uses an old version of PyGit (not the PyGit on PyPi), for detail see this ticket. I might get around to do some hacking there.
I'm working on getting gitweb.cgi to run. I just get "403 Forbidden - No project found." all the time.
Mercurial
There are clients both for command-line (py2exe standalone builds, no python required) as well as windows shell integration (although it still has some rough edges, being 0.5 at the time of writing).
Trying to use Mercurial with mod_python is tricky, since the command-line dist has everything in library.zip (including dll's), which regular python can't import. So you need to unpack it somewhere and add that dir to sys.path. Also, Mercurial expects it's Templates to be in the same dir, so you need to copy that as well (otherwise you'll get hit an assert when using hgweb.cgi). See here for details.
hgweb.cgi works! See here for some tips.
Using MercurialPlugin in Trac from mod_python does not work. There is an issue with demandimport, or expat (under unix), or something. Which simply makes apache or python crash.
Installing Mercurial into c:\Python25 using easy_install might help (to avoid importing library.zip from the stand-alione cmd-line binaries), but that requires Visual C++ 7.1, which I don't have. I haven't found anyone who has precompiled for that use yet. (Copying the files to lib/site-packages did not work.)
I did get trac to run in tracd mode by modifying tracd-script.py and adding 'sys.path.append(r'C:\Program Files\Mercurial\lib')' and running it with 'python tracd-script.py --port ...' !! I have .py files associated with python so can just run py-files as .bat/.cmd, but this did not work here! Very strange.
So, tracd is slower than mod_python (or mod_wsgi, which I haven't tried, but I expect it to have the same wierd issues as mod_python), so I guess I'm stuck with using tracd.
Tracd can be run as a service with the help of the Windows Resource Kit, but you don't get apache authentication (LDAP/SSPI). :(
Anyway, that seems to be the best bet.
Final words
I'm thinking of using apache under cygwin instead. That might help somewhat. Or just usel ighttpd/fastcgi on cygwin and windows-apache's mod_proxy. But.. the layers.. the jumping.. the hoops... arggh..
OTOH, I might go for subversion and wait until the DVCS-on-Win32 state improves (perhaps with my help), or until I get a Unix based server to toy with...
To be continued, as they say...
Subscribe to:
Posts
(
Atom
)