Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set charset name utf8mb4 by default #1157

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kamipo
Copy link
Contributor

@kamipo kamipo commented Jan 26, 2021

mysql2 gem dropped testing for MySQL 5.1 which was released in Dec 2008
and already EOL in Dec 2013 at efa47a9.

If we no longer support MySQL 5.1, utf8 (utf8mb3) is not appropriate
default encoding for MySQL 5.5 or higher, utf8mb4 should be set for
that.

FYI, Rails 6.0 no longer support MySQL 5.1 and set utf8mb4 by default.

rails/rails#33608
rails/rails#33853

@junaruga
Copy link
Contributor

@kamipo Thanks for the PR!

Though I am not a maintainer of this repository, let me share my opinion.

I am seeing the mysql2 still supports Ruby 2.0.0 which are already EOL in Feb 2016 at this announcement from the Ruby project. So, I think mysql2 might still want to support MySQL 5.1 as a possibility, though I see the oldest MySQL case on the CI is MySQL 5.5. Let's wait a response from a maintainer.

By the way, I see a test failure for possibly every test cases.
https://travis-ci.org/github/brianmario/mysql2/jobs/756231628#L1851

  1) Mysql2::Result#field_types should return correct types
     Failure/Error: expect(test_result.field_types).to eql(expected_types)
       expected: ["mediumint(9)", "varchar(10)", "bit(64)", "bit(1)", "tinyint(4)", "tinyint(1)", "smallint(6)", "medi...lob", "tinytext", "blob", "text", "mediumblob", "mediumtext", "longblob", "longtext", "enum", "set"]
        got: ["mediumint(9)", "varchar(13)", "bit(64)", "bit(1)", "tinyint(4)", "tinyint(1)", "smallint(6)", "medi...20)", "blob", "text(262140)", "mediumblob", "text(67108860)", "longblob", "longtext", "enum", "set"]
       (compared using eql?)

     # ./spec/mysql2/result_spec.rb:168:in `block (3 levels) in <top (required)>'

Could you rebase this PR on the latest mater branch? We migrated main cases into GitHub Actions. I would like to see if the test failure still happens, and how affects to additional CI cases that we added to GitHub Actions.

Thank you.

@junaruga
Copy link
Contributor

For MariaDB, this repository's oldest MariaDB version is 10.0. And I was able to confirm the MariaDB 10.0 and latest MariaDB 10.5 supports utf8mb4.

Here is the log on my environment using the MariaDB official container images.

$ podman ps
CONTAINER ID  IMAGE                                  COMMAND       CREATED       STATUS           PORTS                       NAMES
e5007e3619e8  docker.io/library/mariadb:10.0-xenial  --port=13307  36 hours ago  Up 36 hours ago  127.0.0.1:13307->13307/tcp  mariadb-10.0-xenial
1049a24b9f6c  docker.io/library/mariadb:10.5-focal   --port=13306  12 hours ago  Up 12 hours ago  127.0.0.1:13306->13306/tcp  mariadb-10.5-focal

For MairaDB 10.0 (port 13307)

$ mysql -h 127.0.0.1 -u root -P 13307 -e "SHOW CHARACTER SET LIKE 'utf8%'"
+---------+---------------+--------------------+--------+
| Charset | Description   | Default collation  | Maxlen |
+---------+---------------+--------------------+--------+
| utf8    | UTF-8 Unicode | utf8_general_ci    |      3 |
| utf8mb4 | UTF-8 Unicode | utf8mb4_general_ci |      4 |
+---------+---------------+--------------------+--------+

For MairaDB 10.5 (port 13306)

$ mysql -h 127.0.0.1 -u root -P 13306 -e "SHOW CHARACTER SET LIKE 'utf8%'"
+---------+---------------+--------------------+--------+
| Charset | Description   | Default collation  | Maxlen |
+---------+---------------+--------------------+--------+
| utf8    | UTF-8 Unicode | utf8_general_ci    |      3 |
| utf8mb4 | UTF-8 Unicode | utf8mb4_general_ci |      4 |
+---------+---------------+--------------------+--------+

mysql2 gem dropped testing for MySQL 5.1 which was released in Dec 2008
and already EOL in Dec 2013 at efa47a9.

If we no longer support MySQL 5.1, utf8 (utf8mb3) is not appropriate
default encoding for MySQL 5.5 or higher, utf8mb4 should be set for
that.

FYI, Rails 6.0 no longer support MySQL 5.1 and set utf8mb4 by default.

rails/rails#33608
rails/rails#33853
@kamipo kamipo force-pushed the set_charset_name_utf8mb4_by_default branch from 44f4b51 to 52994c6 Compare April 22, 2021 09:55
@yahonda
Copy link
Contributor

yahonda commented May 19, 2022

MySQL 8.0.29 reports warnings for utf8 character set.
https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-29.html

Character Set Support

Important Note: The server now uses utf8mb3 rather than utf8 in the following cases:

  • When reporting invalid strings.

Here is an example.

mysql> select @@version;
+-----------+
| @@version |
+-----------+
| 8.0.29    |
+-----------+
1 row in set (0.00 sec)

mysql> create table foo(id integer) character set utf8;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> show warnings;
+---------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                                                                     |
+---------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 3719 | 'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous. |
+---------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> show create table foo\G
*************************** 1. row ***************************
       Table: foo
Create Table: CREATE TABLE `foo` (
  `id` int DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3
1 row in set (0.00 sec)

@sodabrew sodabrew added this to the 0.6.0 milestone May 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants