Category: xmlstarlet

I ran into this problem, when I tried to extract values from JasperReport’s JRXML using xmlstarlet. JRXML files introduce a default namespace (which does not seem to serve a lot), and XPath processors need to take that into account.

When I searched for help in xmlstarlet’s documentation and on the web, I came across this very helpful article:

• http://www.howtobuildsoftware.com/index.php/how-do/bp0A/xml-xmlstarlet-querying-value-of-xml-file-using-xmlstarlet-with-namspace

Once your XML has a default namespace, you have to use that default namespace with every node (that has no explicit namespace part) in your XPath expressions.

As the article above explains, for “xml sel” each such node …

• … either needs to make use of “_” as name of the default namespace:

# TBD: use a true JasperReports example!

$ xml sel -t -v '/_:publications/_:magazine' 2-023_node-sets.xml

• or you assign an explicit name (let’s say “a”) to your default namespace on your xmlstarlet command line, and then you have to make use of that “explicit default namespace”:

# TBD: use a true JasperReports example!

$ xml sel -N a:'http://jasperreports.sourceforge.net/jasperreports' -t -v '/a:publications/a:magazine' 2-023_node-sets.xml

CAVEAT: The XPath expressions created by “xml el” do not deal with a default namespace, so you have to adapt them yourself in one of the ways explained above.

A one-liner (e.g. in Perl) can help:

$ xml el FILE.xml | fgrep /subreportExpression |
perl -n -a -F'///' -e 'print "_:",join("/_:",@F)'

Over time Jenkins jobs can grow into something “a little confusing”, in other words: like cancer.

The Jenkins developers were thoughtful enough to provide an API to all the data structures, that Jenkins and its jobs operate on, so we are able to export an entire Jenkins job as XML. You certainly do not want to edit a Shell script encapsulated within this XML, or a Windows batch script. You are certainly not the first one to need an export facility for this, and certainly a couple of approaches got developed over time. I am trying to collect them here for you and myself. I actually only found Ken Dreyer’s tool in the beginning – but only after I started developing something myself. NIH applies maybe …

• https://github.com/ktdreyer/jenkins-job-wrecker – Ken Dreyer’s jenkins-job-wrecker – converts Jenkins job XML to JJB YAML

Translate Jenkins XML jobs to YAML. The YAML can then be fed into Jenkins Job Builder.

Have a lot of Jenkins jobs that were crafted by hand over the years? This tool allows you to convert your Jenkins jobs to JJB quickly and accurately.

• https://github.com/openstack-infra/jenkins-job-builder – from YAML back to XML

Initially I was / I am only interested in “project/builders” build steps:

• project/builders/hudson.tasks.BatchFile/command
• project/builders/hudson.tasks.Shell/command
• project/builders/EnvInjectBuilder/info/propertiesContent
• project/builders/EnvInjectBuilder/info/propertiesFilePath

I am extracting the bits and pieces with XPath expressions using XMLStartlet within Shell scripts. Every build step goes into its own file, with names derived from the step’s ordinal number within the Jenkins job, have a look at this example:

• 00–___.properties
• 01–___.propertiesFilePath
• 02–___.sh
• 03–___.bat

These “raw names” push you to think about more reasonable names, that will remind you of their meaning and content from then on.

I called my script “jenkins_config2files.sh“.

For finding the XPath expressions it uses “xmlstarlet el config.xml”. The XPath expressions are not indexed though, i.e. you need to add the XPath indexes yourself.

I created a small Perl script, that creates indexed XPath expressions from the output of “xmlstarlet el”. That works for ordinary “project” and also for “matrix-project” Jenkins XML. I will add more, as I will come across them.

I am going to upload my scripts after completing some tidying to my github account within a couple of days. (Maybe you want to remind me!)