Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

force key encoding to binary to avoid UTF-8 issues #96

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

force key encoding to binary to avoid UTF-8 issues #96

wants to merge 4 commits into from

Conversation

aaronpk
Copy link

@aaronpk aaronpk commented Jan 26, 2015

Somehow I ended up with keys in redis that contained invalid UTF-8 characters, which is now causing problems trying to retrieve them.

My code is currently failing with this stacktrace:

ArgumentError: invalid byte sequence in UTF-8
        from /var/ruby/vendor/bundle/ruby/2.0.0/gems/redis-namespace-1.3.2/lib/redis/namespace.rb:380:in `gsub'
        from /var/ruby/vendor/bundle/ruby/2.0.0/gems/redis-namespace-1.3.2/lib/redis/namespace.rb:380:in `rem_namespace'
        from /var/ruby/vendor/bundle/ruby/2.0.0/gems/redis-namespace-1.3.2/lib/redis/namespace.rb:376:in `block in rem_namespace'
        from /var/ruby/vendor/bundle/ruby/2.0.0/gems/redis-namespace-1.3.2/lib/redis/namespace.rb:376:in `map'
        from /var/ruby/vendor/bundle/ruby/2.0.0/gems/redis-namespace-1.3.2/lib/redis/namespace.rb:376:in `rem_namespace'
        from /var/ruby/vendor/bundle/ruby/2.0.0/gems/redis-namespace-1.3.2/lib/redis/namespace.rb:338:in `method_missing'
        from /var/ruby/vendor/bundle/ruby/2.0.0/gems/redis-namespace-1.3.2/lib/redis/namespace.rb:227:in `keys'

pointing to the source of the problem being the .sub method of the key name.

I was able to get around this issue by forcing the string encoding to binary, which still successfully removes the namespace prefix from the keys.

@aaronpk
Copy link
Author

aaronpk commented Jan 27, 2015

Okay well it looks like my fix fails in ruby 1.9.3 and below for various reasons. Any better suggestions?

This fixes the frozen string issue from before.

Ruby 1.8.7 doesn't support string encoding, so just skip this fanciness for that version.

Updates the tests to demonstrate setting a key with invalid UTF-8 chars, getting a list of the keys (including the invalid string) and retrieving the data at those keys.
@aaronpk
Copy link
Author

aaronpk commented Jan 27, 2015

Okay I fixed this for all versions except 1.8.7 which doesn't have the concept of string encodings, so doesn't need fixing. I added a test case demonstrating how it failed in the first place.

@iloveitaly
Copy link
Contributor

@aaronpk I know this is an old once, but could you rebase on master so the latest tests run?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants