Support for Spark 3

[adityashekhar146]

Currently, the spark-sftp package doesn't work when trying to use in Spark 3.0. Are there any plans to extend the support for Spark 3?

[shaikmanu797]

@adityashekhar146 take a look at https://github.com/arcizon/spark-filetransfer repository (supports Spark 3) which I had developed if it helps to solve your issue.

[satishetc4288]

@shaikmanu797 your spark-filetransfer has issues. I tried it with spark 3 and i am seeing file content got changed after sending it into SFTP server

[satishetc4288]

@adityashekhar146 Currently, the spark-sftp package doesn't work when trying to use in Spark 3.0. Are there any plans to extend the support for Spark 3? We are waiting for spark 3 Support .

[hadoopBrogrammers]

We are also looking for same sftp connector for spark3.x, so is there any plan to have this soon.

[kumgaurav]

Why there is no commit or improvement on this repo. I also see scala 2.12 pull request is open from long time too.

[maleksan]

@adityashekhar146 take a look at https://github.com/arcizon/spark-filetransfer repository (supports Spark 3) which I had developed if it helps to solve your issue.

This connector library perfectly works in my case. (Spark 3.1.2, Scala 2.12), @shaikmanu797 thanks for great work.

[rajancherma]

@adityashekhar146 are there any plans for provide support for scala 2.13