asyncReplayMessages: Failed to deserialize PersistentRepr with kryo serialization

[eduardofcgo]

Hello,
When using kyro, even though everything works well, when I restart the app, I get the following exception:

[error] - actors.DocumentActor - Persistence failure when replaying events for persistenceId [document-AVftV8GPHepb9fmOjr58-persistent-0]. Last known sequence number [0]
java.lang.RuntimeException: asyncReplayMessages: Failed to deserialize PersistentRepr
    at com.hootsuite.akka.persistence.redis.journal.RedisJournal$$anonfun$asyncReplayMessages$1$$anonfun$apply$2.apply(RedisJournal.scala:109) ~[akka-persistence-redis_2.11-0.3.0.jar:0.3.0]
    at com.hootsuite.akka.persistence.redis.journal.RedisJournal$$anonfun$asyncReplayMessages$1$$anonfun$apply$2.apply(RedisJournal.scala:106) ~[akka-persistence-redis_2.11-0.3.0.jar:0.3.0]
    at scala.collection.Iterator$class.foreach(Iterator.scala:893) ~[scala-library-2.11.8.jar:na]
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) ~[scala-library-2.11.8.jar:na]
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) ~[scala-library-2.11.8.jar:na]
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54) ~[scala-library-2.11.8.jar:na]
    at com.hootsuite.akka.persistence.redis.journal.RedisJournal$$anonfun$asyncReplayMessages$1.apply(RedisJournal.scala:106) ~[akka-persistence-redis_2.11-0.3.0.jar:0.3.0]
    at com.hootsuite.akka.persistence.redis.journal.RedisJournal$$anonfun$asyncReplayMessages$1.apply(RedisJournal.scala:104) ~[akka-persistence-redis_2.11-0.3.0.jar:0.3.0]
    at scala.util.Success$$anonfun$map$1.apply(Try.scala:237) ~[scala-library-2.11.8.jar:na]
    at scala.util.Try$.apply(Try.scala:192) ~[scala-library-2.11.8.jar:na]
    at scala.util.Success.map(Try.scala:237) ~[scala-library-2.11.8.jar:na]
    at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237) ~[scala-library-2.11.8.jar:na]
    at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237) ~[scala-library-2.11.8.jar:na]
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) ~[scala-library-2.11.8.jar:na]
    at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121) ~[scala-library-2.11.8.jar:na]
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library-2.11.8.jar:na]
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library-2.11.8.jar:na]
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library-2.11.8.jar:na]
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library-2.11.8.jar:na]

Am I making a mistake in using kryo with this library or is this something that could be fixed?
Thanks a lot for your time.

[ssong-van]

Have you tried snapshot? Does it have the same deserialization issue? Reason I ask is to rule out the first journal record being invalid, i.e., different schema.

Do you mind posting reference.conf in regards to Akka Serialization? And any further info, if any, for me to reproduce the issue.

[scotdalton]

@ssong-van, @eduardofcbg I'm just starting to use Akka persistence and am looking into using this library, so late to the issue, but I may have some insight into it. Depending on the "ID strategy" used the Akka Kryo serializer may not be able to find the class that a particular ID was previously registered to. I was able to resolve the issue by using the idstrategy=automatic and/or use-manifests=true Kryo config settings. This lets Kryo know which IDs map to which classes when recovering.

@ssong-van, one suggestion would be to return the original exception (maybe wrapped in your own) when serializing/deserializing. It would greatly help with debugging. I can submit an issue for that (and a PR next week) if it would be helpful.

[ssong-van]

@scotdalton Thank you for the insight. Returning the original exception makes sense. In fact I can't believe we missed that! Please submit a PR if you are still interested.

[scotdalton]

@ssong-van, definitely still interested. I'll submit a PR in the next day or so.