Last updated: 2020-01-13 21:43:41 +0000

Upstream URL: git clone


View repository

View issue tracker

Contents of follows


This software is released into the Public Domain – Chris Warburton, 2014-09-28


Tested on GNU/Linux, might work on other POSIX systems.

You’ll need some way to run Haskell. Check your package manager or go to to get a compiler or a <code>runhaskell</code> interpreter.

You’ll also need Pandoc available as a library, which you can get from your package manager or with <code>cabal install pandoc</code>, and will probably want the <code>pandoc</code> command available too.

To use PanPipe, invoke it as a Pandoc “filter”, like this:

<code>pandoc --filter ./panpipe input_file > output_file</code>

You can also run <code>panpipe</code> as a standalone command, but note that its stdio should be in PanDoc’s JSON format, rather than e.g. raw Markdown. You can convert <em>to</em> PanDoc JSON using a command like <code>pandoc -t json</code>, and convert <em>from</em> PanDoc JSON using <code>pandoc -f json</code>. The <code>--filter</code> argument automates this plumbing.


PanPipe is a simple Haskell script using PanDoc. It allows code blocks in PanDoc-compatible documents, eg. Markdown, to be sent to external programs for processing.

Any code blocks or lines with a “pipe” attribute will have the contents of that attribute executed as a shell command. The body of the block/line will be piped to that command’s stdin, and the stdout will replace the body of that block/line. A non-zero exit code will cause PanPipe to exit with that code; stderr will be sent to PanPipe’s stderr.

For example, we can execute shell scripts by piping them to “sh”:

<pre><code>```{pipe="sh"} echo "Hello world" ```</code></pre>

This will cause “sh” to be called, with ‘echo “Hello world”’ as its stdin. It will execute the echo command, to produce ‘Hello world’ as its stdout. This will become the new contents of the code block, so in the resulting document this code block will be replaced by:

<pre><code>``` Hello world ```</code></pre>

Usage Notes


The “pipe” attribute is removed, but other attributes, classes and IDs remain:

<pre><code>```{#foo .bar baz="quux" pipe="sh"} echo 'Hello' ```</code></pre>

Will become:

<pre><code>```{#foo .bar baz="quux"} Hello ```</code></pre>

Execution Order

PanPipe uses two passes: in the first, all code <em>blocks</em> are executed, in the order they appear in the document. Hence later blocks can rely on the effects of earlier ones. For example:

<pre><code>```{pipe="sh"} echo "123" > /tmp/blah echo "hello" ``` ```{pipe="sh"} cat /tmp/blah ```</code></pre>

Will become:

<pre><code>``` hello ``` ``` 123 ```</code></pre>

The second pass executes <em>inline</em> code, in the order they appear in the document.


Commands will inherit the environment from the shell which calls panpipe, except they will all be executed in a temporary directory. This makes it easier to share data between code, without leaving cruft behind:

<pre><code>```{pipe="sh"} echo "hello world" > file1 echo "done" ``` ```{pipe="sh"} cat file1 ```</code></pre>

Will become:

<pre><code>``` done ``` ``` hello world ```</code></pre>

The temporary directory will contain a symlink called “root” which points to wherever Pandoc was called from. This allows resources to be shared across invocations (although it’s not recommended to <em>modify</em> anything in root).

Imperative Blocks

If you want to execute a block for some effect, but ignore its output, you can hide the result using a class or attribute:

<pre><code>```{.hidden pipe="python -"} import random with open('entropy', 'w') as f: f.write(str(random.randint(0, 100))) ```</code></pre>

When rendered to HTML will produce:

<div class="sourceCode" id="cb10"><pre class="sourceCode html"><code class="sourceCode html"><span id="cb10-1"><span class="kw"><pre</span> <span class="er">class</span><span class="ot">=</span><span class="st">"hidden"</span><span class="kw">><code></code></pre></span></span></code></pre></div>

Program Listings

A common use-case is to include a program listing in a document <em>and</em> show the results of executing it. You can do this by passing the source code to the Unix “tee” command, then using a subsequent shell script to run it:

<pre><code>```{.python pipe="tee"} print "Foo bar baz" ``` ```{pipe="sh"} python ```</code></pre>

Will become:

<pre><code>```{.python} print "Foo bar baz" ``` ``` Foo bar baz ```</code></pre>

Changing Block Order

Blocks will always be executed in document-order, so you must arrange dependent blocks appropriately. However, we can display blocks in any order by saving them to files and dumping them later.

For example, to show the output of a program <em>before</em> its source code listing, we can define the program first, using “tee” to save it to a file and a HTML class to hide the listing in the resulting document:

<pre><code>```{.hidden pipe="tee"} print "Hello world" ```</code></pre>

Next we can include a block which executes the file we created:

<pre><code>```{pipe="sh"} python ```</code></pre>

Finally we can include a listing by having a block dump the contents of the file (using “.python” for syntax highlighting):

<pre><code>```{.python pipe="sh"} cat ```</code></pre>

Inline Snippets

PanPipe also works on inline code snippets; for example, my root filesystem is currently at <code>`df -h | grep "/$" | grep -o "[0-9]*%"`{pipe="sh"}</code> capacity.


PanPipe keeps the results of script execution inside code blocks/lines, where they can’t interfere with the formatting. If you want to splice some of these results back into the document, you can use the PanHandle script which was written to complement PanPipe.

To prevent ambiguity, PanHandle requires data to be in PanDoc’s JSON format. We can convert things to that format using the <code>-t json</code> option to the <code>pandoc</code> command. For example, to generate a Markdown list and insert it into the document, we can do this:

<pre><code>```{.unwrap pipe="python - | pandoc -t json"} for n in range(5): print " - Element " + str(n) ```</code></pre>

If we run this document through PanPipe, the Python code will output the following to its stdout:

<pre><code> - Element 0 - Element 1 - Element 2 - Element 3 - Element 4</code></pre>

This will be transformed by <code>pandoc -t json</code> into the following:


Hence PanPipe will end up giving out a document equivalent to the following:

<pre><code>``` {.unwrap} [{"unMeta":{}},[{"t":"BulletList","c":[[{"t":"Plain","c":[{"t":"Str","c":"Element"},{"t":"Space","c":[]},{"t":"Str","c":"0"}]}],[{"t":"Plain","c":[{"t":"Str","c":"Element"},{"t":"Space","c":[]},{"t":"Str","c":"1"}]}],[{"t":"Plain","c":[{"t":"Str","c":"Element"},{"t":"Space","c":[]},{"t":"Str","c":"2"}]}],[{"t":"Plain","c":[{"t":"Str","c":"Element"},{"t":"Space","c":[]},{"t":"Str","c":"3"}]}],[{"t":"Plain","c":[{"t":"Str","c":"Element"},{"t":"Space","c":[]},{"t":"Str","c":"4"}]}]]}]] ```</code></pre>

Running <em>that</em> code through PanHandle will splice the contents into the document to give (a JSON equivalent of):

<pre><code> - Element 0 - Element 1 - Element 2 - Element 3</code></pre>

Note that this is no longer in a codeblock, so it will render like this:

<ul> <li>Element 0</li> <li>Element 1</li> <li>Element 2</li> <li>Element 3</li> </ul>

Hence we can use PanPipe to obtain or generate data, and PanHandle to splice it into the document in a sensible way. For example, it’s easy to include one Markdown document inside another:

<pre><code>```{.unwrap pipe="sh"} cat /some/ | pandoc -t json ``` ```{.unwrap pipe="sh"} wget -O- | pandoc -t json ```</code></pre>