Compatibility with Mongo 3.0?

[tcash21]

Hello,

Trying to connect, authenticate and query but I need to switch away from the admin database once authenticated.

Additionally, even querying against the database is resulting in Read operation failures:
Error in .jcall(rmongo.object@javaMongo, "S", "dbGetQuery", collection, :
com.mongodb.MongoException$Network: Read operation to server dscimemd01/10.9.68.7:27017 failed on database huron

Is this an issue with Mongo 3.0 and the driver included in this package?

Thanks,
Tanya

[tcash21]

I actually just found this closed issue: #31

I'll try installing directly from source since it doesn't look like it's on CRAN yet.

[tcash21]

I'm still unable to authenticate even when using the 0.1.0 version of RMongo.

mongo <- mongoDbConnect(host=host , db=auth.db)
authenticated <- dbAuthenticate(mongo, username, password)
> authenticated
[1] FALSE

When I try to query without authenticating I get this:

Error in .jcall(rmongo.object@javaMongo, "S", "dbGetQuery", collection,  :
com.mongodb.MongoException: not authorized for query on db.collectionName

[tc]

@tcash21 are you sure the user/pass is correct with the permissions? the error message sounds like the permissions aren't correct.

[tcash21]

Yes. We actually wrote a new version with the latest driver and it's working. Will let you know once we're able to release it. Thank you though!

[davesgonechina]

Getting the same error with RMongo 0.1.0 and Mongo 3.0.2:

Error in .jcall(rmongo.object@javaMongo, "[S", "dbShowCollections") :
com.mongodb.MongoException: not authorized for query on admin.system.namespaces

Authentication succeeds if I use local mongo client or pymongo to connect to remote mongodb, but fails in R as well as clients like UMongo. Not sure what I've done wrong.

[tc]

@tcash21 do you have any suggestions for @davesgonechina ?

[svensteudter]

Hi,

I adapted your code such that is able to connect to replica sets with the new authentication method of MongoDB 3.0. Unfortunately I have no experience with the used programming language Scala (and almost no exp. in R). ALso I do not have the time to get the code to a higher quality.

I have not tested, if my changes break anything (which could be the case especially for old authentication methods in my code).

I will publish the patch here and write a few remarks in my next post. So if you, @tc , or anybody else has the time, please feel free to adopt the code snippet and commit it to Github.

Also one remark, although it is not urgent, RMongo uses the Monog Java driver in version 2.13 the 3.0 driver is available for at least half a year now, RMongo is using maven to build this library, it would be cool t update also the driver to 3.0.

[svensteudter]

Index: NAMESPACE
===================================================================
--- NAMESPACE   (revision 17)
+++ NAMESPACE   (revision 18)
@@ -1,5 +1,5 @@
 import('methods')
 import('rJava')
-export('mongoDbConnect', 'mongoDbReplicaSetConnect')
+export('mongoDbConnect', 'mongoDbReplicaSetConnect','mongoDbReplicaSetConnectWithCredentials')
 exportClass('RMongo')
 exportMethods('dbInsertDocument', 'dbAuthenticate', 'dbSetWriteConcern', 'dbShowCollections', 'dbGetQuery', 'dbGetQueryForKeys', 'dbDisconnect', 'dbRemoveQuery', 'dbGetDistinct', 'dbAggregate')
Index: R/rmongo.R
===================================================================
--- R/rmongo.R  (revision 16)
+++ R/rmongo.R  (revision 17)
@@ -16,6 +16,11 @@
   rmongo
 }

+mongoDbReplicaSetConnectWithCredentials <- function(dbName, hosts="127.0.0.1:27017", username, pwd){
+rmongo <- new("RMongo", javaMongo = .jnew("rmongo/RMongo", dbName, hosts, TRUE, username, pwd))
+  rmongo
+}
+
 setGeneric("dbAuthenticate", function(rmongo.object, username, password) standardGeneric("dbAuthenticate"))
 setMethod("dbAuthenticate", signature(rmongo.object="RMongo", username="character", password="character"),
    function(rmongo.object, username, password){
Index: src/r-mongo-scala/src/main/scala/rmongo/RMongo.scala
===================================================================
--- src/r-mongo-scala/src/main/scala/rmongo/RMongo.scala    (revision 16)
+++ src/r-mongo-scala/src/main/scala/rmongo/RMongo.scala    (revision 17)
@@ -10,24 +10,29 @@
 import java.util.logging.Logger
 import java.util.logging.Level

-class RMongo(dbName: String, hosts: String, replica: Boolean) {
+class RMongo(dbName: String, hosts: String, replica: Boolean, username: String, pwd: String) {
   val mongoLogger = Logger.getLogger("com.mongodb")
   mongoLogger.setLevel(Level.SEVERE)
   val servers = hosts.split(",").map(_.trim.split(":")).map { a =>
     if (a.size < 2) new ServerAddress(a(0), 27017) else new ServerAddress(a(0), a(1).toInt)
   }.toList
   val this.replica = replica
-  val m = if (!this.replica) new MongoClient(servers(0)) else new MongoClient(servers)
+  val credential = mCreateCredential(username,dbName,pwd)
+  val m = if (!this.replica) new MongoClient(servers(0)) else new MongoClient(servers,List(credential))
   val db = m.getDB(dbName)
   var writeConcern = WriteConcern.NORMAL

-  def this(dbName: String, host: String, port: Int) = this (dbName, host + ":" + port, false)
+  def this(dbName: String, host: String, port: Int) = this (dbName, host + ":" + port, false,"","")

-  def this(dbName: String) = this (dbName, "127.0.0.1:27017", false)
+  def this(dbName: String) = this (dbName, "127.0.0.1:27017", false,"","")

   def dbAuthenticate(username:String, password:String):Boolean = {
     db.authenticate(username, password.toCharArray)
   }
+  
+  def mCreateCredential(userName:String, database:String,password:String):MongoCredential = {
+   MongoCredential.createCredential(username,database,password.toCharArray)
+  }

   def dbSetWriteConcern(w: Int, wtimeout: Int, fsync: Boolean, j: Boolean) {
     writeConcern = new WriteConcern(w, wtimeout, fsync, j)

[svensteudter]

OK,

no idea how to upload patches in a discussion on GIthib. Seems that I would need write rights for uploading a patch file. In my previos post, see above, I posted the content of my patch. The patch should work at least for replica sets with MongoDB 3.0 SCRAM-SHA1 authentictaion.

[tc]

Hi, you can submit this as a pull request. There's a help page on github
about it. It'll make it a lot easier to review
On Thu, Oct 15, 2015 at 3:01 AM svensteudter [email protected]
wrote:

OK,

no idea how to upload patches in a discussion on GIthib. Seems that I
would need write rights for uplaoding a patch file. In my previos post, see
above, I posted the content of my patch. The patch should work at least for
replica sets with MongoDB 3.0 SCRAM-SHA1 authentictaion.

—
Reply to this email directly or view it on GitHub
#34 (comment).

[svensteudter]

Hi, not sure if I did the Pull request stuff right (first time), did you receive it?
Btw: added java older with description how to retrieve the source code of the MongoDB Java driver 2.13.0 which should resolve the issue #33.

[tc]

Got the pull request, thanks! will review.

[midunrajendranpandera]

Hi svensteudter,

I am using patch to connect to a replica sets. How would you initialise a host variable when using
mongoDbReplicaSetConnectWithCredentials or
mongoDbReplicaSetConnect

Am stuck here. Can you help.

[midunrajendranpandera]

Hi svensteudter,
Just to update, I was able to authenticate with the change in your script, but dbGetQueryForKeys and dbGetQuery is not working as expected. These two return after merging all the resultant columns in dataframe.
So, I am using your method for authentication and RMongo package for retrieving data .

[svensteudter]

Hi midunrajendranpandera,
could you please provide me with some more detail on your problem? For clarification, the problem I had, was that MongoDB changed the default authentication method used (from MONGDB-CR to SCRAM-SHA1), the way the RMongo package made authentication did not work for MonogDB 3.0.

What I did was changing some code, to adapt it to the new way how a connection and authentication to MongoDB should be done. I ONLY updated this with the focus on connecting to a replica set. That should work. The rest remained unchanged.

If you ar eable to connect and authentciate to a replica seton MongoDB 3.0 with my patch, than my patch is doing what it is intended for, else not.

The way to use it in R should be
mongoDbReplicaSetConnectWithCredentials(dbname,hosts="server1:port1,server2:port2...", username, password)

If there is another bug not related to connecting and authenticating to MongoDB wirh RMongo please open a new issue.

[svensteudter]

@tc Btw. I have almost no experience in R and absolutely no expereince scala. All I did was adapting your code quick'n dirty for a colleague of mine. What looks a litle bit strange to me is the how the code is built. You include the Java driver, add a Scala intermediate layer which has the responsibility to call the Java methods. From those both, Java Mongo driver and the Scala intermediate layer you build a Java library.

Would it not be the easiest solution, to simply use the Java Mongo driver directly?

I.e., only the code in rmongo.R would be needed, which directly operates on the Java driver. Am I missing here something? (This would still need some work for calling the corresponding Java methods in R)

[davesgonechina]

Here's an alternative method for connecting to MongoDB 3.0, which is to roll back MongoDB's user authentication method from SCRAM-SHA-1 to MONGODB-CR. I can confirm RMongo and rmongodb both can successfully authenticate and query with these changes.

Log in to MongoDB with administrative privileges. If the following returns the value '5', it is set to SCRAM-SHA-1 and you need to change it to '3'

db.system.version.findOne({"_id" : "authSchema"})

First you might need to add the system role to your user account:

db.grantRolesToUser ( "root", [ { role: "__system", db: "admin" } ] )

Then change the authentication scheme:

use admin
var schema = db.system.version.findOne({"_id" : "authSchema"})
schema.currentVersion = 3
db.system.version.save(schema)
exit

Once this is done, restart mongo. Make sure to set auth=true either in the command line or mongod.conf file. Now when you create a new user, it will use MONGODB-CR which compatible with older drivers. Previously existing users, however, will continue to use SCRAM-SHA-1 unless you drop and recreate their accounts.

You can check each users authentication method using:

db.system.users.find()

[svensteudter]

Hi davesgonechina,

this is a well known solution. Good point to add it here in the discussion. Some could use this to reset the authentication protocol to the previous used method and circumvent the problems.

For a productive system, where availablity is a must this is not an option, because you need to shut down the system and (if I understood the solution correct), during changing the authentication method, active users can not access the db until the users are also migrated to this authentication method.

[midunrajendranpandera]

Thanks for your response svensteudter.
Yes our infrastructure team doesn't want to downgrade the security and it will stay with SCRAM-SHA1 algorithm. sventerdter - your function to authenticate did work & Thanks again for that.
But the other methods to get data from Mongo merges all data columns. It's weird because you have adapted the other functions from RMongo package.
So, I removed other functions from your patch and using RMongo functions for data retrieval.

[tc]

@svensteudter, you are right, we can use just the Java mongo driver and
avoid the scala. I coded in scala simply because i was learning it at the
time and java didn't have closures so it would have made it more difficult
to code. I'll happily accept a PR that rewrites the package just with Java

On Wed, Oct 21, 2015 at 1:31 AM, svensteudter [email protected]
wrote:

@tc https://github.com/tc Btw. I have almost no experience in R and
absolutely no expereince scala. All I did was adapting your code quick'n
dirty for a colleague of mine. What looks a litle bit strange to me is the
how the code is built. You include the Java driver, add a Scala
intermediate layer which has the responsibility to call the Java methods.
From those both, Java Mongo driver and the Scala intermediate layer you
build a Java library.

Would it not be the easiest solution, to simply use the Java Mongo driver
directly?

I.e., only the code in rmongo.R would be needed, which directly operates
on the Java driver. Am I missing here something? (This would still need
some work for calling the corresponding Java methods in R)

—
Reply to this email directly or view it on GitHub
#34 (comment).

Tommy Chheng

[primaryobjects]

It looks like RMongo is still incompatible with Mongo 3.0. When I try to execute a dbGetQuery() for a database hosted on mongolab.com, I see the following error:

Error in .jcall(rmongo.object@javaMongo, "S", "dbGetQuery", collection,  : 
  com.mongodb.MongoException: not authorized for query on database.collectionName

Any hope for an update on this?

Btw, in the readme file, the command line for installing the src should use a capitalized "INSTALL" instead of lowercase:

Install:
R CMD INSTALL RMongo*.tar.gz

[tc]

Sorry but i'm a little busy at the moment. Feel free to submit a PR and i
will review/merge any fixes.

On Sat, Nov 7, 2015 at 9:22 AM, Kory Becker [email protected]
wrote:

It looks like RMongo is still incompatible with Mongo 3.0. When I try to
execute a dbGetQuery() for a database hosted on mongolab.com, I see the
following error:

Error in .jcall(rmongo.object@javaMongo, "S", "dbGetQuery", collection, :
com.mongodb.MongoException: not authorized for query on database.collectionName

Any hope for an update on this?

Btw, in the readme file, the command line for installing the src should
use a capitalized "INSTALL" instead of lowercase:

Install:
R CMD INSTALL RMongo*.tar.gz

—
Reply to this email directly or view it on GitHub
#34 (comment).

Tommy Chheng

[rodrigomurta]

This is still open yes? I can't connect using authentication with Mongo 3.x?

[svensteudter]

No,

you should be able with the modifications added to login to MongoDB BUT NOT with the standard login method, but using "mongoDbReplicaSetConnectWithCredentials " at least connecting to RS. As far as I remember this was also working for single dbs.

[pedromorfeu]

Hi there,
Is this issue still open?

[tc]

Have you tried @svensteudter 's solution? mongoDbReplicaSetConnectWithCredentials

[tcash21]

Hi all, finally checking in on this again I wasn't getting alerts. @davesgonechina and @tc unfortunately I don't have permission to release the full code, but we set up auth via java like this:

I have not tried the mongoDbReplicaSetConnectWithCredentials but if that's an easy solution I'd go with that.

	/**
	 * Authentication enabled Constructor
	 *
	 * @param host MongoDB Host
	 * @param port MongoDB Port
	 * @param dbName Name of DB to query
	 * @param username Username
	 * @param password Password
	 */
	public RJMongo(String host, String port, String dbName, String username, String password)
	{
		this(host,port,dbName,username,password,dbName);
	}

	/**
	 * This is the extended authentication enabled constructor with the ability to
	 * auth against a different database than you want to query.  Does not use SSL by default
	 *
	 * @param host MongoDB Host
	 * @param port MongoDB Port
	 * @param dbName Name of DB to query
	 * @param username Username
	 * @param password Password
	 * @param authDB name of DB to authenticate against
	 */
	public RJMongo(String host, String port, String dbName, String username, String password, String authDB)
	{
		this(host,port,dbName,username,password,authDB,"false");
	}```