r3:do(compile). kills embedded yaws service (posted per request in irc)

[iambumblehead]

using the "rebar3 auto" or calling r3:do(compile). kills the embedded yaws service my app uses.

Environment

$ rebar3 report
Rebar3 report
 version 3.3.6+build.3784.ref643a4b21
 generated at 2017-05-14T20:26:58+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: 
Entered as:
  
-----------------
Operating System: x86_64-unknown-linux-gnu
ERTS: Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:0] [kernel-poll:false]
Root Directory: /usr/local/lib/erlang
Library directory: /usr/local/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.3.0
certifi: 0.4.0
cf: 0.2.2
common_test: 1.11
compiler: 6.0
crypto: 3.6
cth_readable: 1.2.5
dialyzer: 2.8
edoc: 0.7.17
erlware_commons: 1.0.0
eunit: 2.2.10
eunit_formatters: 0.3.1
getopt: 0.8.2
inets: 6.0
kernel: 4.0
providers: 1.6.0
public_key: 1.0
relx: 3.22.4
sasl: 2.5
snmp: 5.2
ssl_verify_fun: 1.1.1
stdlib: 2.5
syntax_tools: 1.7
tools: 2.8

-----------------
Escript path: /usr/local/bin/rebar3
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs release relup report shell state tar tree unlock update upgrade upgrade upgrade version xref 
...

Current behaviour

> r3:do(compile).
===> This feature is experimental and may be modified or removed at any time.
Verifying dependencies...
Compiling gani

=ERROR REPORT==== 14-May-2017::13:17:53 ===
Module yaws_dynopts must be purged before loading


=ERROR REPORT==== 14-May-2017::13:17:53 ===
yaws: FATAL gserv died killed

=ERROR REPORT==== 14-May-2017::13:17:53 ===
** Generic server yaws_server terminating 
** Last message in was {'EXIT',<0.320.0>,killed}
** When Server state == {state,
                            {gconf,
                                "/home/chris/software/auth.ganimas.com/_build/default/deps/yaws",
                                false,612,"./log",
                                ["./ebin"],
                                [],[],30000,nolimit,400,1000000,8000,nolimit,
                                [],10240,[],10000000,30,
                                ["/home/chris/software/auth.ganimas.com/_build/default/deps/yaws/examples/include"],
                                "/usr/bin/php-cgi","Yaws 2.0.4",
                                "auth.ganimas",false,[],yaws_session_server,8,
                                undefined,
                                [inet],
                                120000,3600000,disable},
                            [{<0.320.0>,
                              [{sconf,8080,7,[],undefined,undefined,"./www",
                                   [],
                                   {127,0,0,1},
                                   "localhost:8080",[],undefined,380969,
                                   undefined,
                                   [{"./www",[]}],
                                   10240,
                                   [{"/",gani_req}],
                                   [],yaws_outmod,yaws_outmod,yaws_outmod,
                                   yaws,yaws_log,[],undefined,
                                   [yaws,php,cgi,fcgi],
                                   [],
                                   ["index.yaws","index.html","index.php"],
                                   [],
                                   [{listen_opts,[{backlog,1024}]}],
                                   [],undefined,undefined,
                                   {cgi,"/usr/bin/php-cgi"},
                                   undefined,undefined,undefined,undefined}]},
                             {<0.322.0>,
                              [{sconf,4545,7,[],undefined,undefined,"./www",
                                   [],
                                   {127,0,0,1},
                                   "localhost:4545",[],undefined,385066,
                                   {ssl,
                                       "./config/certkeysSrc/server-key.pem",
                                       "./config/certkeysSrc/server-cert.pem",
                                       0,undefined,0,undefined,undefined,
                                       undefined,undefined,undefined,false,
                                       undefined,undefined,undefined,false},
                                   [{"./www",[]}],
                                   10240,
                                   [{"/",gani_req}],
                                   [],yaws_outmod,yaws_outmod,yaws_outmod,
                                   yaws,yaws_log,[],undefined,
                                   [yaws,php,cgi,fcgi],
                                   [],
                                   ["index.yaws","index.html","index.php"],
                                   [],
                                   [{listen_opts,[{backlog,1024}]}],
                                   [],undefined,undefined,
                                   {cgi,"/usr/bin/php-cgi"},
                                   undefined,undefined,undefined,undefined}]}],
                            true}
** Reason for termination == 
** {restartme,[{yaws_server,handle_info,2,
                            [{file,"/home/chris/software/auth.ganimas.com/_build/default/deps/yaws/src/yaws_server.erl"},
                             {line,425}]},
               {gen_server,try_dispatch,4,
                           [{file,"gen_server.erl"},{line,615}]},
               {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,681}]},
               {proc_lib,init_p_do_apply,3,
                         [{file,"proc_lib.erl"},{line,239}]}]}
ok

[vans163]

What is your full Erlang release version?

erl -eval '{ok, Version} = file:read_file(filename:join([code:root_dir(), "releases", erlang:system_info(otp_release), "OTP_VERSION"])), io:fwrite(Version), halt().' -noshell

Any idea if using make:all([load]) or manually doing c/1, l/1 on the file you changed also crashes yaws? This would narrow down if its a rebar3 issue or if yaws is not properly coded to support hotload.

[ferd]

Yeah my bet there is on Yaws not coping with sequential hot-code loads and one of the modules getting killed. You can see process <0.320.0> getting killed (I assume this is gserv). The parent (I assume this is yaws_serv) is holding a reference to <0.320.0> in its state before terminating with reason restartme.

You can see this taking place here in the code: https://github.com/klacke/yaws/blob/b32f7d49eff120f7860453a488acbd358beb5763/src/yaws_server.erl#L421-L430

I don't know that too much can be done if this server dies.

[iambumblehead]

I'll take a look this evening and let you know what I find.

Thank you for your interest and help :)

[iambumblehead]

@vans163 18.0

2> make:all([load]).
up_to_date
3> c("/home/chris/software/auth.ganimas.com/src/gani_req.erl").
{ok,gani_req}
4> l(gani_req).
{module,gani_req}
5> 

is there a "rebar3" way of directing the yaws compilation? I'm not sure how rebar3 is managing the yaws compilation, but @vinoski suggests compiling yaws without HiPE erlyaws/yaws#307

[tsloughter]

Oh, I bet it is hipe.

[iambumblehead]

How can I be certain if HiPE is disabled?

I recompiled otp with "--disable-hipe", deleted the _build directory and retried but the same issue is there.

[ferd]

The shell app reload blacklist feature should cover this now. See https://www.rebar3.org/docs/configuration#section-shell