Possible reader issues when running the Clojerl REPL in a release?

[oubiwann]

As part of exploring the issue in #85, I ran into another problem.

When I start a release-based Clojerl REPL with the following (where PROJ_BIN points to an app's release script):

CODE_LOADING_MODE=interactive $(PROJ_BIN) start
$(PROJ_BIN) eval "'clojure.main':main([<<\"-r\">>])"
$(PROJ_BIN) attach;

And then attempt to execute a call in Clojerl, I get a series of Clojerl prompts written back to the terminal:

clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=> clje.user=>

If I start from just the console and then switch from Erlang into Clojerl:

CODE_LOADING_MODE=interactive $(PROJ_BIN) console

> 'clojure.main':main([<<"-r">>]).

clje.user=> (ports.echo/echo "hey, it's a thing")

then I see a little more:

Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> ports.echo/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> orts.echo/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> rts.echo/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> ts.echo/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> s.echo/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> .echo/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> echo/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> cho/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> ho/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> o/echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> /echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> echo "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> cho "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> ho "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> o "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=>  "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> "hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> hey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> ey, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> y, it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> , it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=>  it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> it's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
clje.user=> t's a thing")
Couldn't unread to erlang.io.PushbackReader: :badargclje.user=> 
's a thing")
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg
Couldn't unread to erlang.io.PushbackReader: :badarg

clje.user=>  a thing")
clje.user=> a thing")
clje.user=>  thing")
clje.user=> thing")
clje.user=> hing")
clje.user=> ing")
clje.user=> ng")
clje.user=> g")
clje.user=> ")
clje.user=> )
clje.user=> 
clje.user=> 

[jfacorro]

The Couldn't unread to erlang.io.PushbackReader: :badarg errors can be avoided by setting the I/O options with the following values before calling clojure.main/main:

ok = io:setopts([{binary, true}, {encoding, unicode}]).

This results in a (sort of) working Clojerl REPL:

Erlang/OTP 21 Klarna-g527298a524 [erts-10.3.5.6] [source-527298a524] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe]

Eshell V10.3.5.6  (abort with ^G)
(try_clojerl@C02X74D6JGH5)1> ok = io:setopts([{binary, true}, {encoding, unicode}]).
ok
(try_clojerl@C02X74D6JGH5)2> 'clojure.main':main([<<"-r">>]).
Clojure 0.6.0
clje.user=> 1

1
clje.user=> 2

2
clje.user=> 3

3
clje.user=> 4

4
clje.user=> :hello
hello
ello
llo
lo
o

:hello
clje.user=>

I'm saying "sort of working" since I still have to investigate why the re-printing of the characters from the input is happening.

[jfacorro]

The same I/O options need to be set before starting the Clojerl REPL through the other method mentioned in #85, and this actually works much better than calling clojure.core/main from the Erlang Shell.

➜  try_clojerl git:(master) _build/prod/rel/try_clojerl/bin/try_clojerl start
➜  try_clojerl git:(master) _build/prod/rel/try_clojerl/bin/try_clojerl eval "io:setopts([{binary, true}, {encoding, unicode}]), 'clojure.main':main([<<\"-r\">>])"
Clojure 0.6.0
clje.user=> 1
1
clje.user=> 2
2
clje.user=> 3
3
clje.user=> 4
4
clje.user=> 5
5
clje.user=> :hello
:hello
clje.user=>

To avoid this situation for users in the future (i.e. REPL failing to start because of I/O options) I think it might be a good idea to either:

• force them in clojure.main/main when starting the REPL...
• ... or add an assert to make sure the required I/O options are set.

[oubiwann]

I think I like your first option, because I imagine this really being needed/wanted any time a REPL is started.

I guess my only questions would be "How big of a change is this?" and "Is this going to impact anything/anyone that currently starts the REPL directly?"

[jfacorro]

"How big of a change is this?"

It's a very small change.

"Is this going to impact anything/anyone that currently starts the REPL directly?"

I don't think so. Those options are currently being set here when the clojerl application is started so someone using the REPL is probably counting on those options being there already.

[jfacorro]

Evaluating io:getopts() with eval in a running node returns [{binary,false},{encoding,latin1}], which are the default values.

$ _build/default/rel/try_clojerl/bin/try_clojerl eval "io:getopts()"
[{binary,false},{encoding,latin1}]

After some investigation I realized this is because the erlang:group_leader() is different from the one where the I/O options are set when the clojerl application starts, and it is the erlang:group_leader() process that keeps the I/O configuration.

$ _build/default/rel/try_clojerl/bin/try_clojerl foreground
Exec: /Users/juan.facorro/.erlang-installs/21.3.8.8+kred6/erts-10.3.5.6/bin/erlexec -noshell -noinput +Bd -boot /Users/juan.facorro/dev/clojerl/try_clojerl/_build/default/rel/try_clojerl/releases/0.1.0/try_clojerl -mode embedded -boot_var ERTS_LIB_DIR /Users/juan.facorro/.erlang-installs/21.3.8.8+kred6/lib -config /Users/juan.facorro/dev/clojerl/try_clojerl/_build/default/rel/try_clojerl/releases/0.1.0/sys.config -args_file /Users/juan.facorro/dev/clojerl/try_clojerl/_build/default/rel/try_clojerl/releases/0.1.0/vm.args -- foreground
Root: /Users/juan.facorro/dev/clojerl/try_clojerl/_build/default/rel/try_clojerl
/Users/juan.facorro/dev/clojerl/try_clojerl/_build/default/rel/try_clojerl
{<0.768.0>,[{binary,true},{encoding,unicode}]}

$ _build/default/rel/try_clojerl/bin/try_clojerl eval "erlang:group_leader()"
<0.68.0>

I was wondering how does Elixir handle this and they call io:setops/2 when the elixir applications is started, but they also set the options in the IEx application when starting the Elixir shell here:

  defp set_expand_fun do
    gl = Process.group_leader()

    expand_fun =
      if node(gl) != node() do
        IEx.Autocomplete.remsh(node())
      else
        &IEx.Autocomplete.expand/1
      end

    # expand_fun is not supported by a shell variant
    # on Windows, so we do two IO calls, not caring
    # about the result of the expand_fun one.
    _ = :io.setopts(gl, expand_fun: expand_fun)
    :io.setopts(gl, binary: true, encoding: :unicode)
  end

[jfacorro]

Fixed by clojerl/clojerl#760

[jfacorro]

I'm saying "sort of working" since I still have to investigate why the re-printing of the characters from the input is happening.

This was fixed by (clojerl/clojerl#763). Starting a REPL from an Erlang shell should work without issues or weird output now.