Joe Corneli Joe Corneli - 1 year ago 66
Bash Question

"inferior" process in clojure - redirecting asyncronous output for further processing

I'd like to maintain a long-running interactive shell process inside Clojure -- what would be called an "inferior" process in Emacs. I assume the basic idea is plenty familiar because Clojure itself can run inside of Emacs via CIDER. I'd be happy to know if there are any working examples of this set-up using Clojure as the top-level process.

Edit: I found this "shell" Gist, which is a very nice looking wrapper for Conch which gives me some food for thought.

With my own first attempt with Conch: I wasn't able to feed strings into

and get output back the same way that I can with, e.g.,
. But with a bit of experimentation I figured out the basics and got around the first roadblocks.

(require '[me.raynes.conch.low-level :as sh])
(def sh-python (sh/proc "python" "-i")) ; "-i" needed to get interactive mode
(future (sh/stream-to-out sh-python :out))
#future[{:status :pending, :val nil} 0x516f3fff]
(sh/feed-from-string sh-python "1+1\n") ; just returns nil on the CIDER repl
2 ; <- but we see "2" with the lein repl

So I know the data I'm interested in is available, although it is a bit strange that it doesn't print in CIDER (sub-question... why isn't the 2 printed?). Anyway, for my use case, I don't need it printed; rather I just want to get it back as a string.

First attempt:

(def pyout (future (sh/stream-to-string sh-python :out)))
;=> #'flowrweb.core/py-out
(sh/feed-from-string sh-python "1+2\n")
;=> 3
;=> nil
^C ; <- the process hangs

It seems that
isn't quite doing what I need.

How about doing it with

(def something (with-out-str (sh/feed-from-string sh-python "1+3\n")))
;=> 4
;=> #'user/something
;=> "" ;<- where is the "4"?

Nope, that didn't work either.

tl;dr: How can I redirect the output from a Conch subprocess for future processing?

Answer Source

This is enough to get started with.

(require '[me.raynes.conch.low-level :as sh])
;=> nil
(def my-stringwriter (
;=> #'user/my-stringwriter
(def sh-python (sh/proc "python" "-i"))
;=> #'user/sh-python
(future (sh/stream-to sh-python :out my-stringwriter)) ; NOT redirecting *out* 
;=> #future[{:status :pending, :val nil} 0x4358e46d]
(sh/feed-from-string sh-python "1+1\n")
;=> nil
(.toString my-stringwriter)
;=> "2\n"
(sh/feed-from-string sh-python "1+2\n")
;=> nil
(.toString my-stringwriter)
;=> "2\n3\n"

Keeping in mind that "The agent system supports sharing changing state between threads in an asynchronous and independent manner" ( docs), I think that a sensible way to encapsulate this is:

(require '[clojure.string :as str])

(def a-stringwriter (agent (
(future (sh/stream-to sh-python :out @a-stringwriter))

(defn feed-python [user-input]
  (future (sh/feed-from-string sh-python (str user-input "\n"))
          (Thread/sleep 1000)
          (str/split (.toString @a-stringwriter) #"\n")))

Then you can write e.g. @(feed-python "10+20") to send and receive results from python. This command will show the history of previous interactions as well as the latest one. For most use cases only the latest addition is relevant.

(defn gljcon [user-input]
  (last @(feed-python user-input)))