wp.jochen.hayek.name/blog-en

Category: xmlstarlet

  • listing a Jenkins CI folder recursively

    —

    by

    johayek
    in , ,

    https://github.com/JochenHayek/misc – pls find my script here! First I wrote this as a Shell script … calling XMLStarlet resp. SaxonHE (for XPath) and curl (for retrieving the Jenkins details as XML files). $ …/jenkins_find_jobs.sh https://integration.wikimedia.org/ci folder … freeStyleProject … matrixProject … The listing is quite helpful for documentational purposes. I was really proud on my little achievement…

  • an XQuery recipe: generating lots of documents in a single XQuery run …

    —

    by

    johayek
    in , ,

    http://www.gnu.org/software/qexo/XQ-Gen-XML.html – search there for “Generate all the HTML output files“! … by putting them in a single large XML object – then use a post-processor to split this into separate files. (Alright this isn’t really a true “single XQuery run” approach. But it is close enough.) With Saxon-HE there is no way to write to separate text…

  • Xidel – yet another HTML/XML/JSON data extraction tool

    —

    by

    johayek
    in , , , ,

    Xidel is a command line tool to download html/xml pages and extract data from them using CSS 3 selectors, XPath 3 expressions or pattern-matching templates. http://www.videlibri.de/xidel.html https://en.wikipedia.org/wiki/XQuery – I am “watching” the changes on this article, and somebody just added Xidel, that’s how I came across Xidel Cygwin’s and Fink’s repository do not have Xidel, but Xidel’s…

  • converting a Jenkins CI job’s config.xml to several flat files (.properties, .sh, .bat, …)

    —

    by

    johayek
    in ,

    Over time Jenkins jobs can grow into something “a little confusing”, in other words: like cancer. The Jenkins developers were thoughtful enough to provide an API to all the data structures, that Jenkins and its jobs operate on, so we are able to export an entire Jenkins job as XML. You certainly do not want…

  • once you are getting familiar with XPath and XMLStarlet, you are using it for rather “ordinary tasks”

    —

    by

    johayek
    in , ,

    http://xmlstar.sourceforge.net https://www.cygwin.com – “Get that Linux feeling – on Windows“ https://cygwin.com/cgi-bin2/package-grep.cgi?grep=xmlstarlet http://www.finkproject.org – “The Fink project wants to bring the full world of Unix Open Source software to Darwin and Mac OS X. …“ http://pdb.finkproject.org/pdb/package.php/xmlstarlet Areas, where you will want to make use of XPath expressions and xmlstarlet in order to extract details: HTML web pages –…

  • using XPath on non-XML HTML – how to tidy dirty HTML?

    —

    by

    johayek
    in ,

    Scraping HTML using XPath is far nicer than through low-level text processing. But how to proceed, if your XPath tool cannot deal with the HTML, because it is not XHTML conform resp. properly formatted XML? My XPath tool is XMLStarlet: And it can also help reformatting HTML, so that XPath expressions can get applied. I…

  • Q: how to get updates from web pages w/o RSS feed? A: XPath + cron or Jenkins job

    —

    by

    johayek
    in ,

    sadly enough even now in 2016 a lot of web pages are not XHTML conform, but getting them fairly conform is not that expensive: use “xmlstarlet fo –html –recover“ get the (cron or) Jenkins job to save the current page content in the job’s workspace let the Jenkins job compare the current to the last…

  • Jenkins: how to authenticate as a scripted client?

    —

    by

    johayek
    in ,

    https://wiki.jenkins-ci.org/display/JENKINS/Authenticating+scripted+clients To make scripted clients (such as wget) invoke operations that require authorization (such as scheduling a build), use HTTP BASIC authentication to specify the user name and the API token. This is often more convenient than emulating the form-based authentication. The article quote above mentions “buildToken“, but I don’t need it at all. The…

  • Python and XPath

    —

    by

    johayek
    in ,

    I had done some XPath using XMLStarlet in a shell script, and I quite like it. Feeling “safe enough” with XPath I managed to deal with the Python XPath pitfall(s), and I quite like my 1st respective XPath Python script. And of course my Python script has a nicer CLI interface than my shell script.…

  • XMLStarlet – a command-line utility to deal with XML documents

    —

    by

    johayek
    in ,

    https://en.wikipedia.org/wiki/XMLStarlet https://de.wikipedia.org/wiki/XMLStarlet http://xmlstar.sourceforge.net http://xmlstar.sourceforge.net/docs.php http://xmlstar.sourceforge.net/doc/UG/xmlstarlet-ug.html – User’s Guide http://xmlstar.sourceforge.net/doc/UG/xmlstarlet-ug.html#idm47077139502176 – the User’s Guide section on “Other XmlStarlet Resources” (with a few broken links) http://xmlstar.sourceforge.net/doc/xmlstarlet.txt – yet another document called “User’s Guide” but with more rather educative examples http://www.ibm.com/developerworks/library/x-starlet http://www.heise.de/ct/inhalt/15/14/172 — behind a paywall; my PDF copy lives on my archive at: Computers/Data_Formats/Markup_Languages/XML/Addressing_and_Querying/XPath/ http://www.freesoftwaremagazine.com/articles/xml_starlet CAVEAT: “xmlstarlet sel –template –value-of XPATH” lists all…