Integrating SVN, trac and jenkins with eclipse CDT

I have previously posted on how to install trac, SVN and jenkins on the Raspberry Pi, and also on how to install (see the bottom of this post for related posts). When using Eclipse there are helpful plugins that tightly integrate all of these tools. This post covers the installation and configuration of these tools.
NOTE: I am installing in the Mars release of Eclipse CDT.

Subversion – Install Subclipse

There are two competing SVN integrations for Eclipse, subclipse and subversive. Both appear to be good, subclipse is supported by tigris (the organisation behind subversion), subversive is supported as an official Eclipse project. I have personally used subclipse for a long time and see no compelling reason to change.
If you use any other SVN client to work on the same checkout as Eclipse then they must use the same working copy format. See the subclipse download and install page for details. Because I also use tortoise SVN and am currently using version 1.8.x,  I need to use subclipse 1.10.x. The update site for this version is http://subclipse.tigris.org/update_1.10.x
In Eclipse go to Help->Install New Software…, in the Install dialog press the Add… button paste in the url for the update site that you require and enter a name.
Click OK and then select Subclipse and SVNKit as shown
click through accepting the license agreement and wait for the install to complete. I got a warning about unsigned content during the install and chose to accept the installation.
The built in help for subclipse is good, in Eclipse click Help->Help Contents and then find the topic Subclipse – Subversion Eclipse Plugin, the getting started section walks you through how to add a new project or how to access an existing one.

Trac and Jenkins install Mylyn Connectors

In Eclipse go to Help->Install New Software…, in the Install dialog in the Work with: drop down pick –All Available Sites– and then under Collaboration check Mylyn Builds Connector: Hudson/Jenkins and Mylyn Tasks Connector : Trac.
click through accepting the license agreement and wait for the install to complete.

Configuring the trac connector

Switch to the SVN Repository Exploring perspective in Eclipse, in the bottom left of the display you should see the Task Repositories view. In this view right click and Add Task Repository, or click on the  icon, select the Trac repository type and click Next.
NOTE: you may be prompted to setup a Secure Storage Password.
In the Add Task Repository… dialog, for the Server enter the URL that you use to browse to the trac repository, in the Label field put a short description, complete the User ID and Password that you use to log in to trac. Validate Settings and then Finish.
When you click Next you will be prompted to Add a new query, queries give you the ability to filter lists of trac tickets to those appropriate to you, a component, a release or whatever. The example below shows all of my open tickets.
To then see the results of the queries click Window->Show View->Other, then under Mylyn select Task List and click OK.
In the Task List it is possible to expand the queries to see lists of tickets, and the tickets can be opened and worked on. See below for an example.

Configuring the Jenkins Connector

Open the Mylyn Builds View (Click Window->Show View->Other…, and find Builds under Mylyn)
In the Builds view either click on Create a build server or on the icon. Select Hudson (supports Jenkins).
Fill in the Server:, Label:, User: and Password:, fields assuming you are using password authentication. For me the server URL is http://xxx.xxx.xxx.xxx:8080 where xxx.xxx.xxx.xxx is the IP address of my server. Check any build plans you are interested in , validate your settings and click finish.
From this builds view all build plans selected are visible, builds can be manually started, the build history is available and details for individual builds can be accessed (Such as warnings, test results and changes made).

Related Posts

Using Google Test with CDT in eclipse (Covers the installation of Eclipse CDT and MinGW)

Integrating trac and Jenkins

In this post I cover how I have integrated Trac and Jenkins using HudsonTracPlugin and Jenkins Trac Plugin. I am running both Trac and Jenkins on a Raspberry Pi 2.

Trac Plugin

The Trac Plugin is a plugin for Jenkins that creates links from Jenkins projects to Trac instances. 
As an example, if you committed a change with the following log message: 

The Trac plugin decorates Jenkins “Changes” HTML to create links to your Trac wiki, tickets, changesets, and source browser.
Reverted changeset:123 to close ticket #456.
your Jenkins job changes page would make changeset:123 and #456 hyperlinks to the appropriate Trac changeset and ticket pages, respectively.
In the Jenkins change details, there would be a link to the Trac pages for the:

 

  • changeset committed,
  • files changed, and
  • diff of the changes for each changed file.

Installation involved browsing to Jenkins->Manage Jenkins->Manage Plugins->Available, selecting Available Plugins and then checking the Edgewall Trac Plugin.
The plugin is then configured as part of a build configuration


As simple as that, references to changesets and ticket numbers are now shown as hyperlinks and there is a link from the build page back to trac.

HudsonTracPlugin

The HudsonTracPlugin is a Trac plugin that includes Jenkins build results into the Trac timeline and (optionally) provides a navbar menu to link to the Jenkins instance.

As a newbie I found the documentation for installing this plugin to be somewhat limited. It was actually remarkably easy.

The first step is to install the plugin which is written in python using easy_install, so in a terminal session to the RPi

sudo easy_install https://trac-hacks.org/svn/hudsontracplugin/0.10/

This gave the following output
Downloading https://trac-hacks.org/svn/hudsontracplugin/0.10/
Doing subversion checkout from https://trac-hacks.org/svn/hudsontracplugin/0.10/ to /tmp/easy_install-xAXEIt/0.10
Processing 0.10
Running setup.py -q bdist_egg –dist-dir /tmp/easy_install-xAXEIt/0.10/egg-dist-tmp-vkGSex
unrecognized .svn/entries format; skipping .
unrecognized .svn/entries format in 
zip_safe flag not set; analyzing archive contents…
Adding HudsonTrac 0.5-r0 to easy-install.pth file

 

Installed /usr/local/lib/python2.7/dist-packages/HudsonTrac-0.5_r0-py2.7.egg
Processing dependencies for HudsonTrac==0.5-r0

Finished processing dependencies for HudsonTrac==0.5-r0

I then chose to reboot my RPi, I suspect I could have just restarted apache.

Using a browser I then went to the Trac Admin->Plugins page, HudsonTrac 0.5-r0 was now listed, I enabled it and Applied Changes as shown.




Finally I logged back in to the RPi and edited trac.ini

sudo vi /var/trac/projects/prod/conf/trac.ini

I added the following to the end of the file

[hudson]
alternate_success_icon = true
api_token =
display_build_descriptions = false
display_building = true
display_culprit = false
display_in_new_tab = false
display_modules = false
job_url = http://192.168.0.34:8080/
list_changesets = true
main_page = http://192.168.0.34:8080/
password =
timeline_opt_label = Builds
username  =

 
I restarted Apache and it worked 🙂 Builds are shown in the timeline, they act as links to the builds on Jenkins. There is also a builds tab added to the trac navbar.

trac performance

There are suggestions for improving performance at http://trac.edgewall.org/wiki/TracPerformance. With my existing configuration I had it typical page loads are taking 4 to 5 seconds, after making the changes below typical page loads are around a second, some faster some slower (loading the first page takes longer).

Related Posts

Switch to using mod_wsgi rather than mod_python

Remove the apache configuration for trac
sudo vi /etc/apache2/sites-enabled/000-default
Remove the lines that are struck through below
<Location /trac>
SetHandler mod_python
PythonInterpreter main_interpreter
PythonHandler trac.web.modpython_frontend
PythonOption TracEnvParentDir /var/trac/projects
PythonOption TracUriRoot /trac
PythonOption PYTHON_EGG_CACHE /tmp
AuthType Digest
AuthName "Subversion Repo"
AuthDigestDomain /trac
AuthDigestProvider file
AuthUserFile /etc/apache2/dav_svn.htdigest
Require valid-user
</Location>
Uninstall mod python and install mod_wsgi
sudo apt-get remove --purge libapache2-mod-python
sudo apt-get install libapache2-mod-wsgi
Now configure WSGI (I found good help for this at https://code.google.com/p/modwsgi/wiki/IntegrationWithTrac.
sudo mkdir /var/trac/cgi-bin
sudo vi /var/trac/cgi-bin/trac.wsgi
I then put this content in to trac.wsgi
#!/usr/bin/python

# -*- coding: utf-8 -*-

#

import sys

sys.stdout = sys.stderr

import os

os.environ[‘TRAC_ENV_PARENT_DIR’] = ‘/var/trac/projects’

os.environ[‘PYTHON_EGG_CACHE’] = ‘/tmp’



import trac.web.main



application = trac.web.main.dispatch_request
Ensure that the script is accessible to apache and executable
sudo chown www-data:www-data -R /var/trac/cgi-bin

sudo chmod +x /var/trac/cgi-bin/trac.wsgi 
Now to configure Apache
sudo vi /etc/apache2/sites-enabled/000-default
Add the following after the last </Directory> element
        WSGIScriptAlias /trac /var/trac/cgi-bin/trac.wsgi


        <Directory /var/trac/cgi-bin>

                WSGIApplicationGroup %{GLOBAL}

                Order deny,allow

                Allow from all

        </Directory>
Restart Apache
sudo /etc/init.d/apache2 restart
I then tested trac and it was still functioning as before 🙂

Mapping Static Resources

By default static resources like style sheets and images are served by trac itself, it is more efficient to have Apache serve these directly rather than get trac to serve them.
First extract the static resources and delete the ones we do not want (I don’t want cgi-bin as I’ve already configured it, and I don’t want htdocs/site because I will directly reference the files in the trac install).
sudo trac-admin /var/trac/projects/play deploy /var/trac/static

sudo rm -rf /var/trac/static/cgi-bin/

sudo rm -rf /var/trac/static/htdocs/site/

sudo chown www-data:www-data -R /var/trac/static
Now configure Apache to directly serve the static resources
sudo vi /etc/apache2/sites-enabled/000-default
Add the following after the last </Directory> element
Alias /trac/play/chrome/common /var/trac/static/htdocs/common

Alias /trac/play/chrome/site /var/trac/projects/play/htdocs/

<Directory “/var/trac/static/htdocs/common“> Order allow,deny
Allow from all</Directory><Directory “/var/trac/projects/play/htdocs/“> Order allow,deny Allow from all</Directory>

 

svn pre-commit hook to ensure a valid trac ticket is referenced

In my previous post I configured the svn post-commit hook so that tickets referenced in the svn commit message had a link to the changes and the commit message added to the ticket comments. This post describes how to configure the svn to prevent commits that do not reference a valid ticket.

The trac community have done all the hard work, so I downloaded trac-pre-commit-hook from http://trac.edgewall.org/browser/trunk/contrib/trac-pre-commit-hook, and placed it in a new directory /var/trac/hooks.

I set the ownership of the script to www-data and made sure it was executable

sudo chown www-data:www-data -R /var/trac/hooks
sudo chmod +x /var/trac/hooks/trac-pre-commit-hook

All that remains to be done is to call this from the svn pre-commit hook. I created a file/var/svn/repos/play/hooks/pre-commit with this content

REPOS=”$1″
TXN=”$2″
TRAC_ENV=”/var/trac/projects/play”
LOG=`/usr/bin/svnlook log -t “$TXN” “$REPOS”`
/usr/bin/python /var/trac/hooks/trac-pre-commit-hook “$TRAC_ENV” “$LOG” || exit 1

If a commit fails to reference an open ticket I now get a message like this

svn commit -m “Some message referencing a closed ticket refs #1” test.txt 
Sending        test.txt
Transmitting file data .svn: E165001: Commit failed (details follow):
svn: E165001: Commit blocked by pre-commit hook (exit code 1) with output:

At least one open ticket must be mentioned in the log message.

Configuring trac

This post covers how I have configured trac, and trac svn integration, it follows on from Installing Trac and SVN on the Raspberry Pi. It is intended just as a personal reminder, but if it is helpful…

Adding SVN repository to trac

Detailed instructions can be found in TracRepositoryAdmin in the trac wiki.
You are then told to do perform the steps below
So first perform a manual sync
sudo -u www-data trac-admin /var/trac/projects/play repository resync "play"
Edit trac.ini to configure the updater.
sudo -u www-data vi /var/trac/projects/play/conf/trac.ini
In the [trac] section set repository_sync_per_request blank

repository_sync_per_request =

In /var/svn/repos/play/hooks create a file called post-commit with this content

#!/bin/sh
export PYTHON_EGG_CACHE="/tmp"
/usr/bin/trac-admin /var/trac/projects/play changeset added "$1" "$2"

In the same place create post-revprop-change with this content

#!/bin/sh
export PYTHON_EGG_CACHE="/tmp"
/usr/bin/trac-admin /var/trac/projects/play changeset modified "$1" "$2"
 
Ensure that both of these scripts are owned by www-data and that both are executable
sudo chown -R www-data:www-data post-commit post-revprop-change

sudo chmod +x post-commit post-revprop-change

 CommitTicketUpdater

Enable the CommitTicketUpdater plugin by going to the admin tab in trac and then selecting Plugins from the Administration Panel, open the Trac 0.12.5 plugin and tick CommitTicketUpdater as shown.

Edit trac.ini to configure the updater.

sudo -u www-data vi /var/trac/projects/play/conf/trac.ini

In the [ticket] section add the following

commit_ticket_update_envelope =
commit_ticket_update_commands.close =
commit_ticket_update_commands.refs = <ALL>
commit_ticket_update_check_perms = true
commit_ticket_update_notify = true

Testing Commit Hook

Having completed all of the above configuration I rebooted my RPi, I don’t know that this is required but it seemed sensible.
I then created a ticket #1 in the play project, To test I checked out the trunk for the play repository and committed a new file with the commit message “Test commit hook See #1”, the commit happened and ticket #1 was updated with the following comment

Logo

I like each trac project to be immediately identifiable so that it is obvious which trac I am working on. I just set the project logo. The logo looks best with an image of a little under 100 pixels high. I saved the files as play.png in /var/trac/projects/play/htdocs.
I then edited trac.ini (as www-data)

sudo -u www-data vi conf/trac.ini

I set the content of [header_logo]

[header_logo]
alt = play repository to test ideas
height = 80
src = site/play.png
width = 300

Related posts

Installing SVN and trac on Raspberry Pi

This post is step by step instructions for installing subversion (SVN) 1.8.10 and trac on the Raspberry Pi. I am writing it as a reminder to myself, but if you find it useful…

Beforehand

It’s a good idea to get up to date


sudo apt-get update
sudo apt-get upgrade

Install Subversion

First of all I checked the version included in the raspbian distribution, and I wanted a later version. I looked at using the WANdisco binaries, but found that they seemed to be missing the binaries required for the raspberry pi. So these instructions use the debian backports.
I followed these instructions. First I added wheezy-backports to me sources.list by creating /etc/apt/sources.list.d/wheezy-backports.list 

deb http://http.debian.net/debian wheezy-backports main

I tried just running sudo apt-get update, however I got the following error

W: GPG error: http://http.debian.net wheezy-backports Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 8B48AD6246925553 NO_PUBKEY 7638D0442B90D010

After a little digging I found that I could add the public keys like this

gpg --keyserver pgpkeys.mit.edu --recv-key 8B48AD6246925553
gpg -a --export 8B48AD6246925553 | sudo apt-key add -
gpg --keyserver pgpkeys.mit.edu --recv-key 7638D0442B90D010
gpg -a --export 7638D0442B90D010 | sudo apt-key add -
sudo apt-get update
sudo apt-get -t wheezy-backports install subversion

Now it is possible to update with no errors

sudo apt-get update

Finally we can install subversion

sudo apt-get -t wheezy-backports install subversion
sudo apt-get -t wheezy-backports install python-subversion

(Note: originally I missed python-subversion, this left a version mismatch and caused errors)
Now check the version installed

svn --version
svn, version 1.8.10 (r1615264)

compiled Apr 24 2015, 04:16:08 on arm-unknown-linux-gnueabihf


Copyright (C) 2014 The Apache Software Foundation.

This software consists of contributions made by many people;

see the NOTICE file for more information.

Subversion is open source software, see http://subversion.apache.org/


The following repository access (RA) modules are available:


* ra_svn : Module for accessing a repository using the svn network protocol.

– with Cyrus SASL authentication

– handles ‘svn’ scheme

* ra_local : Module for accessing a repository on local disk.

– handles ‘file’ scheme

* ra_serf : Module for accessing a repository via WebDAV protocol using serf.

– using serf 1.3.7

– handles ‘http’ scheme

Install Apache

I have decided that I am less likely to get conflicts if the versions of apace and subversion come from the same distribution. So I install from wheezy backports.

sudo apt-get -t wheezy-backports install apache2
sudo apt-get -t wheezy-backports install libapache2-svn

I got a single warning during install

Starting web server: apache2apache2: Could not reliably determine the server’s fully qualified domain name, using 127.0.1.1 for ServerName

I have chosen to ignore this for now. To test the installation of apache I browsed to http://localhost/

Create Repositories

I have decided that the structure I want is to have two repositories, a play repository and a prod repository. This example just shows the creation of the play repository.
sudo mkdir /var/svn
sudo mkdir /var/svn/repos
sudo svnadmin create /var/svn/repos/play

Configure Security

I have decided to use digest access authorisation. NOTE: I am not recommending this method of securing access, but it suits my needs.
Start by editing dav_svn.conf (I use vi you could equally use nano if you prefer)
sudo vi /etc/apache2/mods-available/dav_svn.conf
The default file is all comments, and I choose to leave it intact and just append the following to the end of the file.
<Location /svn>
DAV svn
SVNParentPath /var/svn/repos
AuthType Digest
AuthName "Subversion Repo"
AuthUserFile /etc/apache2/dav_svn.htdigest
Require valid-user
</Location>
Next I need to enable digest authentication and restart apache
sudo a2enmod auth_digest
sudo service apache2 restart
Next I want to create a user in the apache/subversion digest file
sudo htdigest -c /etc/apache2/dav_svn.htdigest "Subversion Repo" david
It is important to note two things here. Firstly, if you add more than one user omit the -c on all subsequent calls as this creates the digest file and so would delete all existing users. Secondly the “Subversion Repo” argument specifies the realm and must match the AuthName in dav_svn.conf.
Now is a good time to test that it is working: browse to http://192.168.0.34/svn/play (the ip address of your RPi), you should be prompted for a username and password, enter the ones you specified when creating the digest and you should be in.
One last step is to tie down the native file access permissions, and to grant the apache user address to write to the repository.
sudo chown -R www-data:www-data /var/svn/repos

Install trac

With trac I have decided to install the version bundled with raspbian (0.12.5). I based the decision on the availability of other trac plugins, also for compatibility with subversion 1.8 I need trac >=0.12.3.
sudo apt-get install trac
sudo apt-get install lib-apache2-mod-python

Configure trac

I want trac to share the same digest authentication as subversion. I also want a separate trac database for my play and prod repositories. The steps below only show the setup for the play repository.
sudo mkdir /var/trac
sudo mkdir /var/trac/projects
sudo chown -R www-data:www-data /var/trac/projects

Edit the apache configuration

sudo vi /etc/apache2/sites-enabled/000-default

Add the following to the end of the file, just before the </VirtualHost> tag. Note that the authentication matches the subversion settings.

<Location /trac>
SetHandler mod_python
PythonInterpreter main_interpreter
PythonHandler trac.web.modpython_frontend
PythonOption TracEnvParentDir /var/trac/projects
PythonOption TracUriRoot /trac
PythonOption PYTHON_EGG_CACHE /tmp
AuthType Digest
AuthName "Subversion Repo"
AuthDigestDomain /trac
AuthDigestProvider file
AuthUserFile /etc/apache2/dav_svn.htdigest
Require valid-user
</Location>

Now it is time to create the play trac project environment

sudo -u www-data trac-admin /var/trac/projects/play initenv

You get asked a few questions, I responded play for the name of the project and accepted all other defaults.
Restart apache

sudo /etc/init.d/apache2 restart

Time to test, browse to http://192.168.0.34/trac/ (use the IP address of your RPi) you should see a list of your trac projects.

Click on the play project name and you should see trac

NOTE: According to TracAuthenticationIntroduction Note that in the current version of Trac, clicking on the logout link above the navigation bar does not logs user out because the browser still remembers the HTTP authentication and keeps sending it.

To be able to administer trac add a user as the trac admin. I use my normal trac login.

sudo -u www-data trac-admin /var/trac/projects/play permission add david TRAC_ADMIN

For further configuration to make the integration tighter and to improve performance see