Sending emails hangs when using RestRserve instead of plumber

[MislavSag]

Hi,

I have first developed plumber API, but wasn't happy with stability and speed.

I have rewritten API using RestRserve. Localy everything is working as expected, but when I create docker image, the hangs at email ntification (sending an email through mailR). I am not sure if this is in anyway connected to this package, but I don't know what to try, so I am posting here if you can find the reason.

Here is the app.R file:

library(RestRserve)
library(mailR)
library(ibrestr)
library(lgr)
library(lgrExtra)
library(DBI)
library(RPostgres)


# connect to database
conn = function() {
  dbConnect(
    RPostgres::Postgres(),
    dbname = "defaultdb",
    host = "xxx",
    port = 25060L,
    user = "doadmin",
    password = "xxx"
  )
}

# set up logger help function
set_logger = function(table_name, conn) {
  lg = get_logger("db_logger")
  lg$add_appender(name = "db",
                  lgrExtra::AppenderDbi$new(
                    conn = conn,
                    table = table_name
                  ))
  return(lg)
}


# Create a new RestRserve app
app = Application$new()

# Echo endpoint
app$add_get(
  path = "/echo",
  FUN = function(.req, .res) {
    msg = as.character(.req$parameters_query[["msg"]])
    .res$set_body(msg)
    .res$set_content_type("text/plain")
  }
)

# Sum 2 numbers endpoint
app$add_post(
  path = "/sum",
  FUN = function(.req, .res) {
    a = as.numeric(.req$body$a)
    b = as.numeric(.req$body$b)
    .res$set_content_type("application/json")
    .res$set_body(a + b)
  }
)

# Ping IB
app$add_post(
  path = "/ping",
  FUN = function(.req, .res) {
    host = as.character(.req$body$host)
    port = as.integer(.req$body$port)
    strategy_name = as.character(.req$body$strategy_name)
    account_id = as.character(.req$body$account_id)
    email_config = .req$body$email_config

    # Create an instance of the IB class with the provided parameters
    ib_instance = IB$new(
      host = host,
      port = port,
      strategy_name = strategy_name,
      account_id = account_id,
      email_config = email_config,
      logger = NULL
    )

    # Call the liquidate method
    result = tryCatch({
      ib_instance$get()
    }, error = function(e) {
      list(error = as.character(e))
    })

    .res$set_content_type("application/json")
    .res$set_body(list(result))
  }
)

# Set Holdings endpoint
app$add_post(
  path = "/set_holdings",
  FUN = function(.req, .res) {
    account_id = as.character(.req$body$account_id)
    symbol = as.character(.req$body$symbol)
    sectype = as.character(.req$body$sectype)
    side = as.character(.req$body$side)
    tif = as.character(.req$body$tif)
    weight = as.numeric(.req$body$weight)
    host = as.character(.req$body$host)
    port = as.integer(.req$body$port)
    strategy_name = as.character(.req$body$strategy_name)
    email_config = .req$body$email_config
    table_name = .req$body$table_name

    # set logger
    if(is.null(table_name)) {
      logger = NULL
    } else {
      connection = conn()
      logger = set_logger(table_name, connection)
    }

    # Create an instance of the IB class with the provided parameters
    ib_instance = IB$new(
      host = host,
      port = port,
      strategy_name = strategy_name,
      account_id = account_id,
      email_config = email_config,
      logger = logger
    )

    # Call the liquidate method
    result <- tryCatch({
      ib_instance$set_holdings(account_id=NULL, symbol=symbol, sectype=sectype,
                               side=side, tif=tif, weight=weight)
    }, error = function(e) {
      list(error = as.character(e))
    })

    # remove appender to the loger
    if(!is.null(table_name)) {
      logger$set_appenders(NULL)
      dbDisconnect(connection)
    }

    .res$set_content_type("application/json")
    .res$set_body(list(result))
  }
)

and here is the Dockerfile:

FROM rocker/r-ver:4.3.1

# always use RSPM
ENV RENV_CONFIG_REPOS_OVERRIDE https://packagemanager.rstudio.com/cran/latest

RUN apt-get update -qq && apt-get install -y --no-install-recommends \
   libcurl4-openssl-dev \
   libicu-dev \
   libsodium-dev \
   libssl-dev \
   make \
   zlib1g-dev \
   default-jdk \
   r-cran-rjava \
   libpq-dev \
   && apt-get clean

COPY renv.lock renv.lock
RUN Rscript -e "install.packages('renv')"
RUN Rscript -e "renv::restore()"
COPY app.R /api/app.R

# Set the working directory to where app.R is located
WORKDIR /api

# Expose port 8080 for the app
EXPOSE 8080

# Run the app
CMD ["Rscript", "app.R"]

Here are the logs when I execute set_holdings:

PS C:\Users\Mislav\Documents\GitHub\ibapp> docker run -p 8080:8080 --name api mislavsag/ibapp:v1

Attaching package: ‘lgr’

The following object is masked from ‘package:RestRserve’:

    Logger

{"timestamp":"2024-02-23 09:26:10.920843","level":"INFO","name":"Application","pid":1,"msg":"","context":{"http_port":8080,"endpoints":{"POST":["/sum","/ping","/set_holdings","/liquidate","/insert"],"HEAD":"/echo","GET":"/echo"}}}
-- running Rserve in this R session (pid=1), 2 server(s) --
(This session will block until Rserve is shut down)
INFO  [09:27:05.964] Checks accounts
INFO  [09:27:06.599] Checks
INFO  [09:27:07.024] Check gateway
INFO  [09:27:08.487] Find conid by symbol for symbol AAPL
INFO  [09:27:09.414] ConId is 120549942
WARN  [09:27:09.848] Get position for AAPL
INFO  [09:27:10.480] Position for AAPL is 0
INFO  [09:27:10.907] Available cash
INFO  [09:27:11.334] Checks accounts
INFO  [09:27:12.935] Cash 1011263.500000
INFO  [09:27:13.363] Create body for AAPL
INFO  [09:27:14.711] Quantity for AAPL is 274
INFO  [09:27:15.164] Checks accounts
INFO  [09:27:15.784] Checks accounts
[[1]]
[[1]]$order_id
[1] "1911009641"

[[1]]$order_status
[1] "Submitted"

[[1]]$encrypt_message
[1] "1"


INFO  [09:27:17.041] Places order for AAPL
INFO  [09:27:17.512] list(list(order_id = "1911009641", order_status = "Submitted", encrypt_message = "1"))
INFO  [09:27:17.512] Order placed successfully
INFO  [09:27:17.961] Check status for AAPL
INFO  [09:27:18.897] Notification - send email for AAPL

I would like to add that I could send an email while I was using plumbe, with same docker iamge. That is why I thinkg there could be some strange reason it doesn't work because of RestRserve.

[s-u]

@MislavSag I don't see the actual code, but my suspicion based on the Docker file is that you may be using some very convoluted way to send emails involving Java which is not possible if you initialize the JVM too soon since you cannot fork a JVM. I would recommend some more sane way to send email such as sendmail? If you really want to use a JVM, make sure you start it only after the fork, i.e. you must call .jinit() in the handler (but initializing an entire JVM just to do something that it trivially done with a single command seems a bit of an overkill...).

[dselivanov]

Also be careful with Postgresql connections. Forks should not use single connection concurrently. So you might need to create connection within function which handles particular endpoint.

…

On Sat, 24 Feb 2024, 06:53 Simon Urbanek, ***@***.***> wrote: @MislavSag <https://github.com/MislavSag> I don't see the actual code, but my suspicion based on the Docker file is that you may be using some very convoluted way to send emails involving Java which is not possible if you initialize the JVM too soon since you cannot fork a JVM. I would recommend some more sane way to send email such as sendmail? If you really want to use a JVM, make sure you start it only *after* the fork, i.e. you must call .jinit() in the handler (but initializing an entire JVM just to do something that it trivially done with a single command seems a bit of an overkill...). — Reply to this email directly, view it on GitHub <#206 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHC5XLI5BA25FT3HJSSGRDYVEMYLAVCNFSM6AAAAABDWM5INCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSGEYDIMBZGI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[MislavSag]

I have changed email dependency package. Now I use blastula instead of mailR. Blastula has only pandoc as system requirement. It works now. So, you are right, it seems there was a problem with init JAVa on cores. But as I see now, sendemailR have minimal dependency, so I will try with it too. Blastula have huge dependency.

@dselivanov , I am openeing and closing connection inside API function. It seems tome this is working as expected.