couchbase_password_issue #1
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I found that updating the dependencies to be compatible with CDH4 made the connector work correctly with no code changes. Note that it won't work with CDH3 with these dependencies. Also note that compiling with CDH3 dependencies yeilds software that won't load the proper classes under CDH4 owing to changes in CDH, not anything we've done in the connector. Note: the copyright was removed in one file here because the new sqoop command does not allow # as a comment for some reason. Change-Id: Ib19c1959d1817a202c53362f2cc2b8bd3ac9f9a1
Change-Id: Ibbe1507942a5052b661dd48f39e5598af39302a0
Change-Id: Ifeffc0b9a591a6df523a35ba4b93f9f35bbdbdf6
Change-Id: I30ddbe8dd0eb2c58d1a004f39ebf44d2235d455e
Change-Id: I917c226d7e9a3c436a0aa668669d9ee25c633172
Change-Id: Ic5582e616b87375bfdbc47f60264536feab45afe
Change-Id: I3e5895e942413b7c655813f13ca71eac3e493878
Change-Id: Idd471304635cbffa6b0e262c49b107d6dbe061f6
Change-Id: I4dfea5be19bc5f4ca4ae798ccd2952fc328c4af1
At around 1.4 of the Couchbase Java Client, we updated to and began using the API of the new httpcore/httpcore-nio. Unfortunately, it appears that Hadoop/CDH5 is still on the previous version. Initially I tried to remove this dependency entirely since we don't technically use it in the hadoop connector, but we do use it to set up unit tests and I didn't want to throw those out. The solution instead was to create a modified branch of this dependency that repackages httpcore into our own namespace. Not ideal, but I checked with both the sqoop project and Jeff Bean at Cloudera to see if there was a better way to handle it. Jeff hasn't gotten back to me and the sqoop-dev list recommended this approach. See http://mail-archives.apache.org/mod_mbox/sqoop-dev/201409.mbox/%[email protected]%3E and http://mail-archives.apache.org/mod_mbox/sqoop-dev/201409.mbox/%[email protected]%3E Change-Id: I0a34ee7e76cf40d185564fbe90b427d19fa8e465
Change-Id: I2bd320660d4377e2a867bc1be403138c963d1ed2
According to JetBrains, all except a couple of files should be shared. See https://intellij-support.jetbrains.com/entries/23393067 Change-Id: Ib32c38b9d601856c537c3cfd7a46ed70c510818e
Starting in some more recent version of sqoop the password is handled differently. Because the default bucket uses a null password, this silently snuck by. Change-Id: I06d55400bdd9fb863729d1d2d4f5d6554e193ed8
Change-Id: I853246d7d7cca444deeac8feffb08ae1fc1527e9
Changes in sqoop mean the password needs to be fetched in a slightly different way than handled before. Now the accessor is used and this allows the following to work. sqoop import --username named -P --verbose --connect http://192.168.1.115:8091/pools --table DUMP sqoop import --username named --password letmein --verbose --connect http://192.168.1.115:8091/pools --table DUMP sqoop import --username default --verbose --connect http://192.168.1.115:8091/pools --table DUMP sqoop import --username named --password-file passwordfile --verbose --connect http://192.168.1.115:8091/pools --table DUMP Note that the first includes the -P param, which will prompt for the password at the CLI. The second includes the password inline (which is a minor security hole). The third format shows how to use a default bucket with no password. For the last, the sqoop documentation isn't very clear and it appears that the "passwordfile" file must be in HDFS *and* it must be the password without any newlines at the end of the file. It can be put there with `hdfs dfs -put <filename>` passwordfile. Change-Id: Iba024bfdc04f0d4bcb582b7d88637420d6e2adde
Change-Id: If3fbca71a0b61823f98ea895270b3b243b4e9e9f
Change-Id: I6e7ef88724b15f2807d11bb5d1e90d34becc0d82
The earlier setup was not working correctly when trying to execute individual unit tests so I cleaned up project files and reimported the project. At least for me, with these settings, I can resolve dependencies with IvyIDEA and can execute individual unit tests. Change-Id: I2cf39cd5dc5fd28dc7b10bcdfe950b77e950f63e
Change-Id: I32db491db5be2faeb44a71c69b2e4d020ba352c0
Change-Id: I1530d4af7eab604277e0815e7e27f9bce12bc4da Reviewed-on: http://review.couchbase.org/41743 Reviewed-by: Brett Lawson <[email protected]> Tested-by: Matt Ingenthron <[email protected]>
Change-Id: I8377af51a90d0e5094605db7476cf6364bac0dc4 Reviewed-on: http://review.couchbase.org/41899 Reviewed-by: Jeffry Morris <[email protected]> Tested-by: Matt Ingenthron <[email protected]>
|
I'd intentionally broken this branch out from master since master evolved to support Hortonworks. There are some subtle differences between the different repositories. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.