RetryHelpers trait to implement non-blocking retries

[phelps-sg]

• Implemented a trait io/cequence/openaiscala/RetryHelpers.scala to allow API calls to be made with non-blocking retries as a possible solution to (fixes Blocking IO in OpenAIRetryServiceAdapter #15, and partial implementation of OpenAIRetryServiceAdapter retries non-retryable errors #16).
• Implemented unit tests for RetryHelpers (Missing unit tests #24).
• New implementation of io.cequence.openaiscala.service.OpenAIServiceWrapper in openai-client package which uses RetryHelpers.
• Still TODO: separate exception classes for retryable errors such as rate-limit-error (OpenAIRetryServiceAdapter retries non-retryable errors #16).

Example usage below.

This design has advantages on the client side we can easily apply retry for specific methods, or use different retry settings for different methods (without necessarily having to create separate wrappers).

import akka.actor.{ActorSystem, Scheduler}
import io.cequence.openaiscala.RetryHelpers
import io.cequence.openaiscala.RetryHelpers.RetrySettings
import io.cequence.openaiscala.domain.{ChatRole, MessageSpec}
import io.cequence.openaiscala.service.{OpenAIService, OpenAIServiceFactory}

import javax.inject.Inject
import scala.concurrent.{ExecutionContext, Future}

class MyCompletionService @Inject() (
    val actorSystem: ActorSystem,
    implicit val ec: ExecutionContext,
    implicit val scheduler: Scheduler
)(val apiKey: String)
    extends RetryHelpers {
  val service: OpenAIService = OpenAIServiceFactory(apiKey)
  implicit val retrySettings: RetrySettings = RetrySettings()

  def ask(prompt: String): Future[String] =
    for {
      completion <- service
        .createChatCompletion(
          List(MessageSpec(ChatRole.User, prompt))
        )
        .retryOnFailure
    } yield completion.choices.head.message.content
}

[phelps-sg]

In addition to above, I have now:

• implemented unit tests in RetryHelpersSpec
• Implemented object Retryable in OpenAIScalaClientException to allow pattern matching against retryable exceptions, e.g.: case Retryable(_) =>

[phelps-sg]

I have created a new wrapper OpenAIRetryServiceAdapter which uses RetryHelpers, so that both the wrapper approach and implicit class approach can be used according to preference.

[peterbanda]

Good work! I like that retries are usable both, within the wrapper class, as well as with an implicit "monkey patching". Just three changes:

1. As in the original retry adapter we need to have a way to log an error message (by default a simple println suffices).
2. Related to the point 1 attempts should be increasing from 1 to max rather than decreasing so we can log something like Attempt i: The call xyz failed... (in any case the max number of attempts must be passed in a recursive call).
3. (optional) Exponential delay is nice to have but if a constant delay is required then it might not be totally intuitive that delayBase must be set to one. Maybe two case classes for exponential and linear delay settings would be more user-friendly...

Regarding new exceptions I will add those once this PR is merged.

Thanx

[phelps-sg]

attempts should be increasing from 1 to max rather than decreasing so we can log something like Attempt i: The call xyz failed...

1. Would you be happy with debug logs something like this:

WSRequestHelper: Calling end point v1/models...
RetryHelpers: Attempting max 5 attempts...
RetryHelpers: Attempt failed, 4 attempts remaining, retrying in 2 seconds...
RetryHelpers: Attempt failed, 3 attempts remaining, retrying in 10 seconds...
RetryHelpers: Succeeded.
WSRequestHelper: v1/models succeeded with result [some JSON]

or:

WSRequestHelper: Calling end point v1/models...
RetryHelpers: Attempting max 3 attempts...
RetryHelpers: Attempt failed, 2 attempts remaining, retrying in 2 seconds...
RetryHelpers: Attempt failed, 1 attempts remaining, retrying in 10 seconds...
RetryHelpers: Attempt failed, aborting.
WSRequestHelper: v1/models failed with time out

If not please be aware that rewriting the code to have an increasing counter will take further effort and the code may be more complicated (additional parameters in method calls). Also, could you write tests in RetryHelpersSpec that indicate the exact behavior you require?

As in the original retry adapter we need to have a way to log an error message (by default a simple println suffices).

2. Will await resolution of discussion in Configure standard logging framework #21 before implementing this.

if a constant delay is required then it might not be totally intuitive that delayBase must be set to one

3. I have implemented RetrySettings.constantInterval with corresponding tests:

 "RetrySettings" should {
    "allow easy configuration of a constant interval" in {
      val interval = 10.seconds
      val result = RetrySettings(interval)
      result.delayBase shouldBe 0
      result.delayOffset shouldBe interval
    }
  }
  
  "RetryHelpers" should {
    "compute the correct delay when using constant interval" in {
      val interval = 10.seconds
      val settings = RetrySettings(interval)
      delay(1)(settings) shouldBe interval
      delay(5)(settings) shouldBe interval
    }

    "compute the correct delay when using strictly positive base" in {
      val settings = RetrySettings(
        maxRetries = 5,
        delayOffset = 2.seconds,
        delayBase = 2
      )
      delay(1)(settings) shouldBe 4.seconds
      delay(2)(settings) shouldBe 6.seconds
      delay(3)(settings) shouldBe 10.seconds
    }
}

[peterbanda]

Hey @phelps-sg ,
Thanks for the feedback and implementing the constant interval delay! I think I can nail down the exact details regarding logging and such myself. Could you just resolve the conflicts? I am ready to merge this PR.
Btw. I am going to wait with a code reformatting until this is merged.

Cheers!

[phelps-sg]

Note that I just realised readme needs to be updated with new retry behaviour, which I have just done. Will resolve conflicts shortly.

[phelps-sg]

@peterbanda the checks above are failing because of formatting. You can merge and override checks, or alternatively apply formatting in this branch. I will hold fire unless hear otherwise. Let me know if you need any help resolving issues with the CI.

[phelps-sg]

See #33 for resolving the pipeline issues. Let me know if you want to merge it into this branch.