{"id":6605,"date":"2016-07-09T13:49:51","date_gmt":"2016-07-09T11:49:51","guid":{"rendered":"http:\/\/www.b.shuttle.de\/hayek\/hayek\/jochen\/wp\/blog-en\/?p=6605"},"modified":"2021-09-15T17:14:41","modified_gmt":"2021-09-15T15:14:41","slug":"extract-your-build-steps-from-a-jenkins-ci-job","status":"publish","type":"post","link":"https:\/\/wp.jochen.hayek.name\/blog-en\/2016\/07\/09\/extract-your-build-steps-from-a-jenkins-ci-job\/","title":{"rendered":"extract your build steps from a Jenkins CI job to ordinary and &#8220;easy to edit&#8221; flat files"},"content":{"rendered":"<p>\t\t\t\tOver time Jenkins jobs can grow into something &#8220;a little confusing&#8221;, in other words: like cancer.<\/p>\n<p>The Jenkins developers were thoughtful enough to provide an API to all the data structures, that Jenkins and its jobs operate on, so we are able to export an entire Jenkins job as XML. You certainly do not want to edit a Shell script encapsulated within this XML, or a Windows batch script. You are certainly not the first one to need an export facility for this, and certainly a couple of approaches got developed over time. I am trying to collect them here for you and myself. I actually only found Ken Dreyer&#8217;s tool in the beginning \u2013 but only after I started developing something myself. <a href=\"https:\/\/en.wikipedia.org\/wiki\/Not_invented_here\">NIH<\/a>\u00a0applies maybe \u2026<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/ktdreyer\/jenkins-job-wrecker\">https:\/\/github.com\/ktdreyer\/jenkins-job-wrecker<\/a>\u00a0\u2013\u00a0Ken Dreyer&#8217;s\u00a0jenkins-job-wrecker \u2013\u00a0converts Jenkins job XML to JJB YAML<\/li>\n<\/ul>\n<blockquote><p>Translate Jenkins XML jobs to YAML. The YAML can then be fed into Jenkins Job Builder.<\/p>\n<p>Have a lot of Jenkins jobs that were crafted by hand over the years? This tool allows you to convert your Jenkins jobs to JJB quickly and accurately.<\/p><\/blockquote>\n<ul>\n<li><a href=\"https:\/\/github.com\/openstack-infra\/jenkins-job-builder\">https:\/\/github.com\/openstack-infra\/jenkins-job-builder<\/a>\u00a0\u2013 from YAML back to XML<\/li>\n<\/ul>\n<p>Initially I was \/ I am only interested in &#8220;project\/builders&#8221; build steps:<\/p>\n<ul>\n<li>project\/builders\/hudson.tasks.BatchFile\/command<\/li>\n<li>project\/builders\/hudson.tasks.Shell\/command<\/li>\n<li>project\/builders\/EnvInjectBuilder\/info\/propertiesContent<\/li>\n<li>project\/builders\/EnvInjectBuilder\/info\/propertiesFilePath<\/li>\n<\/ul>\n<p>I am extracting the bits and pieces with XPath expressions using XMLStartlet within Shell scripts. Every build step goes into its own file, with names derived from the step&#8217;s ordinal number within the Jenkins job, have a look at this example:<\/p>\n<ul>\n<li>00&#8211;___.properties<\/li>\n<li>01&#8211;___.propertiesFilePath<\/li>\n<li>02&#8211;___.sh<\/li>\n<li>03&#8211;___.bat<\/li>\n<\/ul>\n<p>These &#8220;raw names&#8221; push you to think about more reasonable names, that will remind you of their meaning and content from then on.<\/p>\n<p>I called my script &#8220;<em>jenkins_config2files.sh<\/em>&#8220;.<\/p>\n<p>For finding the XPath expressions it uses &#8220;xmlstarlet el <em>config<\/em>.xml&#8221;. The XPath expressions are not indexed though, i.e. you need to add the XPath indexes yourself.<\/p>\n<p>I created a small Perl script, that creates indexed XPath expressions from the output of &#8220;xmlstarlet el&#8221;. That works for ordinary &#8220;<em>project<\/em>&#8221; and also for &#8220;<em>matrix-project<\/em>&#8221; Jenkins XML. I will add more, as I will come across them.<\/p>\n<p>I am going to upload my scripts after completing some tidying\u00a0to my github account within a couple of days. (Maybe you want to remind me!)\t\t\t\t<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Over time Jenkins jobs can grow into something &#8220;a little confusing&#8221;, in other words: like cancer. The Jenkins developers were thoughtful enough to provide an API to all the data structures, that Jenkins and its jobs operate on, so we are able to export an entire Jenkins job as XML. You certainly do not want [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"_share_on_mastodon":"0"},"categories":[851,732],"tags":[],"class_list":["post-6605","post","type-post","status-publish","format-standard","hentry","category-xmlstarlet","category-xpath"],"share_on_mastodon":{"url":"","error":""},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/paO0kP-1Ix","jetpack_likes_enabled":true,"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/posts\/6605","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/comments?post=6605"}],"version-history":[{"count":1,"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/posts\/6605\/revisions"}],"predecessor-version":[{"id":11879,"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/posts\/6605\/revisions\/11879"}],"wp:attachment":[{"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/media?parent=6605"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/categories?post=6605"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wp.jochen.hayek.name\/blog-en\/wp-json\/wp\/v2\/tags?post=6605"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}