A solution to Jenkins jobs conflicting over shared resources

I have been looking for a way to manage the execution of Jenkins jobs that require exclusive access to a resource other than the Jenkins slave it is being built on. Let's consider two scenarios that can cause problems.

Shared Physical Resource

Working with embedded systems I often want to run tests on an embedded device as part of a continuous integration process. I may have multiple Jenkins slaves that are capable of running the jobs, with a single specific embedded device required. If I have multiple jobs how do I ensure that only one runs at a time?

Software License and Tool Versions

For years I have used vxWorks and the Windriver Workbench IDE. Lets suppose that I have three Jenkins Slave Nodes, each with a different version of the tools installed. The tools are licensed by FlexLM on a license server, only one of these slaves can run the tools at the same time. With no restrictions if two jobs are run concurrently that require different versions of the tool then one will fail to build with a license violation. How do I ensure that the jobs are sequenced rather than running concurrently?

The solution

Basic Usage

The Jenkins Lockable Resource Plugin elegantly solves these problems.
The plugin is installed in the normal way from Manage Jenkins->Manage Plugins->Available, tick the box next to the plugin and Install (I restarted as well). Once installed go to Manage Jenkins->Configure System, scroll down to the Lockable Resources Manager and Add Lockable Resource.
In this example I have created a resource to represent a Raspberry Pi running Codesys. I have chosen a unique name (I may have more than one in the future) and a label for the type of node.
Next configure the jobs that require the resource, in the Configuration page for the jobs check the This build requires lockable resources check box.
 
 
 
In this particular build I have selected a specific resource. When the job runs you can see the resource being locked around the job.

 

If the resource is already locked when the job becomes ready to run then it is held pending in the Build Queue, having NOT blocked the Slave node.
 

Manually Reserving a Resource

One other nice feature of the plugin is that it is possible to manually reserve the resource, so if in my example I wanted to do some maintenance on the raspberry pi I can go to Manage Jenkins->Configure System, and put some text in the Reserved by field to take the lock thus holding off builds until I have finished.
 

Advanced

Suppose that I have two embedded devices that I could run my tests on CODESYS_RPI_1 and CODESYS_RPI_2, both configured identically. However my build job needs to know which has been locked so that it can communicate appropriately. To achieve this I have changed the configuration for my job to depend on the resource Label rather than the resource name, and introduced a variable to hold the name of the resource that was locked, like this
Now looking at the Parameters for the build we can see which resource was locked.
This parameter can be used like any other in the build.
 
Some tests may require multiple resources, let's say that we need two identical devices, setting the Number of resources to request to 2 and building shows the variable now has the names of both resources.
 
 

Conclusion

The lockable Resources Plugin provides a simple, elegant solution to locking/reserving resources for a job. As shown above it is easy to have a lock represent a physical resource, the plugin can equally be used in the licensing scenario, define a single resource to represent the license, each job requires the resource and is locked to running on the appropriate slave. Definitely one to add to my list of favourite Jenkins Plugins.
 
 
 

 

5 Comments

  1. Duong Nguyen22nd February 2017

    Hi,

    I am finding a solution to queue mobile devices from different (master) Jenkins.
    Example:
    – MAC connected to 3 mobile devices: Android_A, Android_B, Android_C
    – 10 Jenkins (no connection between them) connected to this MAC
    – Devices (Android_A and Android_C) are busy and used by Jenkins 1 and Jenkins 5
    – Started new job on Jenkins 3
    – This job should recognize that device Android_B is free and use this device
    – So we have following picture:
    — Jenkins 1 use Android_A
    — Jenkins 5 use Android_C
    — Jenkins 3 use Android_B
    After that started another job on Jenkins 9
    It should put this job to queue until one of devices will free up.

    Would really appreciate it if you could provide me a solution to implement it.

    Duong Nguyen

    Reply
    1. David23rd February 2017

      Hi Duong,
      Sorry, but I haven’t encountered your situation. I have never tried to share resources between different Jenkins Masters. I don’t think you will find this easy, but it may be possible. Have a look at the Gearman Plugin – I’ve never tried it but it may do what you want.

      Please post back to say how you solved this problem.

      David

      Reply
  2. Luke7th June 2017

    Hey David,

    It sounds like you have solved my exact problem – but the code snippets are not being displayed on this page!
    Could you put them back in or something?
    Cheers

    Reply
    1. David7th June 2017

      Hi Luke,
      I’m sorry about the missing images, they must have got lost when I changed the hosting of my blog a year ago. I’ll reload them as soon as possible.
      regards David

      Reply
      1. Luke8th June 2017

        Thanks mate, much appreciated!

        Reply

Leave a Reply to David Cancel reply

Your email address will not be published. Required fields are marked *

Scroll to top