Accented characters being rejected in new versions

[collimarco]

I have noticed that URLs containing non-ASCII characters, like accented characters, were accepted in the past. However now this library rejects all of them.

This change is probably related to one of these recent commits:
1945ae4
3dde863

Is this change made on purpose or not?

We should probably add a test to clarify this if it is made on purpose.

I have read that the standard requires only ASCII characters, so this is probably correct.

On the other side, I have noticed that many users of my website were affected and started getting validation errors when they tried to post external URLs on our website... So, if this is not a security issue (I don't know), maybe we can consider accepting them as we did in the past?

[kritik]

Standard allows to use utf-8 characters

[collimarco]

@kritik Yes, I agree with you. However, on the other side, if users copy and paste some URLs opened in their browser, they may contain accented characters. Also, if you use client-side validation in forms (e.g. a URL field) most browsers allow accented characters (I don't know why).

[kritik]

can you give an example of wrong url?

[collimarco]

@kritik e.g. https://example.com/è is now rejected by the validation (since last version)

[kritik]

oh, then it should be fixed

[collimarco]

@kritik Based on the answers on SO only ASCII characters should be accepted. However that is not the common behavior of browsers, which currently accept also other characters (e.g. accented characters).

Accepting non ASCII characters is probably not compliant with the standard. However that would be more user-friendly and reflect the browser behavior.

In any case if we accept non ASCII characters we should make sure not to create security issues (for example when the link is included in a Rails link_to for example).

[kritik]

Let me put it in this way. non ASCII characters are accepted in the urls. If you network doesn't support it, possibly in US, then browser can encode it to ASCII. If domain name has non-ASCII characters then special prefix will be added (don't remember which one). Other parts of url will be encoded by url encode logic

[collimarco]

non ASCII characters are accepted in the urls

That is widespread, but not standard. Here's more information.

[kritik]

May be that website old? When we did at Perfectline update for Estonian domain system upgrade then we had to support ASCII and Non-ASCII characters. It works the same way as in Russian or Chinese domains. For more info you can check https://github.com/internetee/registry

[TimFletcher]

I am unable to validate a URL with Chinese characters:

https://www.redcloudoverall.com/collections/jeans/products/redcloud-赤芸-lot-r423-66-ow-washed-regular-cut-selvedge-denim

[Getmrahul]

@kritik Any update on this issue? I'm getting validation error for this url https://ro.linkedin.com/in/andrei-marcel-țiț-66b019a8

[xtagon]

My two cents is that it should be an option, maybe something like validates :my_url, url: {strict: true}. There are going to be people who want it one way or another.

[jfloff]

I also have a similar problem with a linkedin profile that has in its url the word césar which is not accepted by this gem. Any plans on this?