Automatically detect character encoding of YAML files and ignore files

[Jayman2000]

This PR makes sure that yamllint never uses open()’s default encoding. Specifically, it uses the character encoding detection algorithm specified in chapter 5.2 of the YAML spec when reading both YAML files and files that are on the ignore-from-file list.

There are two other PRs that are similar to this one. Here’s how this PR compares to those two:

• This PR doesn’t have any merge conflicts.
• This PR has a cleaner commit history. You can run the tests and flake8 on each commit in this PR, and they’ll report no errors. I don’t think that you can do that with Detect encoding per yaml spec (fix #238) #240.
• This PR has longer commit messages. I really tried to explain why I think that my changes make sense.
• This PR detects the encoding of files being linted, config files, and files on the ignore-from-file list. Those two PRs only detects the encoding of files being linted.
• Detect encoding per yaml spec (fix #238) #240 PR adds a dependency on chardet. This PR doesn’t add any dependencies.
• This PR only supports UTF-8, UTF-16 and UTF-32. Both of those PRs support additional encodings.
• Unicode yaml #581 adds support for running tests on Windows. This PR doesn’t.
• The code that this PR adds to the yamllint package is simpler.
• The code that this PR adds to the test package is much more complicated, but hopefully it tests things more thoroughly.

Fixes #218. Fixes #238. Fixes #347.
Closes #240. Closes #581.

[coveralls]

coverage: 99.836% (+0.01%) from 99.824%
when pulling e14f401 on Jayman2000:auto-detect-encoding
into e427005 on adrienverge:master.

[Jayman2000]

I just noticed that one of the checks for this PR is failing. The coverage for yamllint/config.py went down, but that’s just because the total number relevant lines went down. There’s only two lines that aren’t covered, but those same two lines aren’t covered in the master branch. Is there anything that I need to do here?

[adrienverge]

Is there anything that I need to do here?

At the moment, no. I'm sorry, please excuse the delay, this is a big change with much impact, I need a large time slot to review this, which I couldn't find yet.

[adrienverge]

Hello Jason, please excuse the very long delay for reviewing this... This was a big piece and I needed time. I apologize.

The 6 commits are well splitted, well explained, and make the review much easier. Thanks a lot!

In my opinion this PR is good to go. I suspect it can solve problems in several cases (including the issues you pointed out), but I also see a small risk of breakage on exotic systems the day it's released. If this happens, will you be around to help find a solution?

A few notes:

• I notice that you used encoding names with underscores (e.g. utf_8 vs. utf-8). I just read on https://docs.python.org/fr/3/library/codecs.html#standard-encodings that not only are they valid, but they also seem to be the "right" notation:

  Notice that spelling alternatives that only differ in case or use a hyphen instead of an underscore are also valid aliases; therefore, e.g. 'utf-8' is a valid alias for the 'utf_8' codec.

• I feared that using open().decode() would put the whole contents of files in memory before even linting them, and affect performance. But this is already what yamllint does currently.

[Jayman2000]

Hello Jason, please excuse the very long delay for reviewing this... This was a big piece and I needed time. I apologize.

You shouldn’t be apologizing, I should be thanking you.

Thank you for taking the time to review this PR and to write valuable review comments. Sometimes, maintainers don’t take the time to thoroughly review my contributions. When that happens, my contributions end up being rejected without being understood which is frustrating and saddening. You’re review comments are different, though. They clearly demonstrate that you took the time to read, understand and think about this PR. I’d much rather receive a thoughtful review than a timely one.

In my opinion this PR is good to go. I suspect it can solve problems in several cases (including the issues you pointed out), but I also see a small risk of breakage on exotic systems the day it's released. If this happens, will you be around to help find a solution?

Sure! I had thought about adding a --force-encoding option that would disable encoding autodetection. The idea is that someone could use --force-encoding shift_jis to make yamllint decode everything using Shift JIS or --force-encoding cp1252 to make yamllint decode everything using code page 1252. I decided against adding the --force-encoding in this PR because this PR was already so big.

Now that you mention the small risk of breakage on exotic systems, it’s making me think about the --force-encoding idea again. It might be wise to implement --force-encoding after this PR gets merged and before there’s a new stable release of yamllint. That way, we can give users an easy workaround if encoding autodetection breaks something. If you think that that is a good idea, then I can open another PR that adds a --force-encoding option after this PR gets merged.

---

I just pushed a new version of this pull request with the following changes:

• I rebased it on 8513d9b (the tip of master, at the moment).
• I implemented the review suggestions (see the conversations that just recently got resolved).
• The variable in tests/common.py that used to be named test_strings is now called TEST_STRINGS_TO_ENCODE_AT_RUNTIME. I made this change for two reasons:
  1. I needed a add a new variable that contained a different type of test strings. I needed to add that variable in order to implement one of the suggestions from your review comments.
  2. The variable name should have been all uppercase anyway because the variable is a constant.
• For similar reasons, I also renamed the error1 and error2 variables to ERROR1 and ERROR2 in the function that used to be named test_detect_encoding().
• I added a repr() call to the msg="auto_decode({repr(input_bytes)}) returned the wrong value." line in tests/test_decoder.py. The repr() call helps ensure that the error message has proper syntax.
• I fixed a dead link in one of the commit messages (https://sourceware.org/glibc/manual/html_node/Extended-Char-Intro.html → https://www.gnu.org/software/libc/manual/html_node/Extended-Char-Intro.html).
• I revised the commit message for “decoder: Autodetect detect encoding of YAML files” to (hopefully) make it easier to understand for people who are unfamiliar with this issue.
• I made the commit message for “decoder: Autodetect encoding for ignore-from-file” slightly simpler.
• I improved the commit message for “CI: Fail when open()’s default encoding is used”. I reworded part of it in order to make it more clear why coverage.py is affected by default encoding errors.
• Updated copyright years for some files to account for the fact that I worked on some files in both 2023 and 2024.

[adrienverge]

Thanks for following up, again it's a great work.
Amended commits are still perfectly clear and anticipate some of my questions :)

Sure! I had thought about adding a --force-encoding option that would disable encoding autodetection. The idea is that someone could use --force-encoding shift_jis to make yamllint decode everything using Shift JIS or --force-encoding cp1252 to make yamllint decode everything using code page 1252. I decided against adding the --force-encoding in this PR because this PR was already so big.

Now that you mention the small risk of breakage on exotic systems, it’s making me think about the --force-encoding idea again. It might be wise to implement --force-encoding after this PR gets merged and before there’s a new stable release of yamllint. That way, we can give users an easy workaround if encoding autodetection breaks something. If you think that that is a good idea, then I can open another PR that adds a --force-encoding option after this PR gets merged.

If I understand correctly, YAML specification says that files should be Unicode-encoded, but in the real world not all YAML files are.
But I'm not sure whether today, with yamllint 1.35.1, it's possible to successfully run yamllint on non-Unicode-encoded YAML files. Do you know whether it's the case?
Here is what I tried on my system (a recent GNU/Linux):

• Python's sys.getdefaultencoding() always equals utf-8, even when setting environment variables PYTHONIOENCODING=cp1252, PYTHONUTF8=0 and LANG=fr_FR.CP1252.
• After creating a CP1252-encoded file (python -c 'with open("/tmp/cp1252.yaml", "wb") as f: f.write("- éçà".encode("cp1252"))'), running PYTHONIOENCODING=cp1252 yamllint /tmp/cp1252.yaml fails with UnicodeDecodeError: 'utf-8' codec can't decode ….

I'm not an expert of how Python handles encoding, so your input is more than welcome on this 🙂

• If it's not the case, then no need for an extra option --force-encoding.
• If it's the case, then I agree it would be better to add a way to avoid breakage for those users. However I'm not a fan of a new command line option (--force-encoding), because ideally this would be temporary and removed after a few yamllint versions (after users fixed their files to use UTF-8), so it would break usage the day when we remove this option.
  Instead, we could use an environment variable YAMLLINT_IO_ENCODING, that detect_encoding() would use to override the encoding if defined (then we could unit-test 2 or 3 cases in a new test_detect_encoding_with_env_var_override()). Removing support for this env var in the future won't break command-line options. In my opinion, this should be fairly simple and should go in the same commit.

I revised the commit message for “decoder: Autodetect detect encoding of YAML files”

I like the new version too, it's clearer about the Windows case. Did you mean Autodetect detect → Autodetect?

I revised the commit message for “decoder: Autodetect detect encoding of YAML files” to (hopefully) make it easier to understand for people who are unfamiliar with this issue.

Indeed, it's slightly clearer 👍 and will allow grepping warn_default_encoding or EncodingWarning inside Git history.

[adrienverge]

Good idea.

[dalito]

After creating a CP1252-encoded file (python -c 'with open("/tmp/cp1252.yaml", "wb") as f: f.write("- éçà".encode("cp1252"))'), running PYTHONIOENCODING=cp1252 yamllint /tmp/cp1252.yaml fails with UnicodeDecodeError: 'utf-8' codec can't decode ….

I believe you are expecting something wrong here: From the docs: PYTHONIOENCODING: If this is set before running the interpreter, it overrides the encoding used for stdin/stdout/stderr,
So setting "PYTHONIOENCODING" has no effect on the encoding used when opening files which uses what is returned by locale.getencoding which is still utf-8.

(I did not check if you do/did this.) It could be interesting to run tests with https://docs.python.org/3/library/io.html#io-encoding-warning to detect where the code relies on default encoding.

[Jayman2000]

I just pushed a new version of this PR. Here’s what’s new:

• I rebased this branch on top of e427005 (the tip of master, at the moment) which means that there are no more merge conflicts.
• The commit called “tests: Restore stdout and stderr” now called “tests: Restore stdout”. This was done to account for the changes from e427005.
• The YAMLLINT_FILE_ENCODING environment variable can now be used to override the encoding used to decode files (see below).
• I fixed the “Autodetect detect” thing in that one commit message’s subject line.
• I implemented the review suggestions (see the conversations that just recently got resolved).
• For files that already existed, already had a copyright notice and that I significantly contributed to, I added myself to the copyright notice (I added “Copyright (C) <years> Jason Yundt”).
• For the files that I created myself, I updated the copyright years in order to account for the fact that it’s now 2025.

---

@adrienverge

If I understand correctly, YAML specification says that files should be Unicode-encoded, but in the real world not all YAML files are. But I'm not sure whether today, with yamllint 1.35.1, it's possible to successfully run yamllint on non-Unicode-encoded YAML files. Do you know whether it's the case? Here is what I tried on my system (a recent GNU/Linux):

• Python's sys.getdefaultencoding() always equals utf-8, even when setting environment variables PYTHONIOENCODING=cp1252, PYTHONUTF8=0 and LANG=fr_FR.CP1252.

• After creating a CP1252-encoded file (python -c 'with open("/tmp/cp1252.yaml", "wb") as f: f.write("- éçà".encode("cp1252"))'), running PYTHONIOENCODING=cp1252 yamllint /tmp/cp1252.yaml fails with UnicodeDecodeError: 'utf-8' codec can't decode ….

Checking sys.getdefaultencoding() is a good idea, but I actually don’t think that that particular function is relevant here. Here’s what the Python documentation has to say about open() function’s encoding argument:

In text mode, if encoding is not specified the encoding used is platform-dependent: locale.getencoding() is called to get the current locale encoding.

That being said, I think that the documentation might actually be wrong here. locale.getencoding() seems to always return the actual locale encoding, regardless of whether UTF-8 mode is enabled or disabled. I think that the documentation for open() should probably say locale.getpreferredencoding() instead of locale.getencoding().

When I run python -c 'import locale; print(locale.getpreferredencoding())' with a plain-ASCII locale and UTF-8 mode disabled, it prints ANSI_X3.4-1968. When I run that same command with a plain-ASCII locale and UTF-8 mode enabled, it prints utf-8. When I run that same command with a UTF-8 locale, it prints UTF-8.

I agree with @dalito about PYTHONIOENCODING. It’s probably also a red herring.

That being said, I’m not sure why yamllint was still using UTF-8 when you tested it. I can think of a few possibilities, though:

1. fr_FR.CP1252 is not on glibc’s supported locales list. It’s possible that this is a bug with glibc and that it would work better if you tried a locale that’s on that list.
2. It’s possible that there isn’t actually a locale on your system that’s named fr_FR.CP1252. You could try running locale --all-locales to see if it’s on the list.
3. fr_FR.CP1252 is just a name. Technically, you could have accidentally created a locale that has CP1252 in its name, but actually uses UTF-8. You could try running LANG=fr_FR.CP1252 locale --keyword-name LC_CTYPE to verify that the charmap really is set to CP1252.
4. It’s also possible that your version of glibc doesn’t actually support the CP1252 character encoding. You could try running locale --charmaps to check and see if CP1252 is available on your system.

If you can’t get it to work on your host system, you can always try this Nix flake that I created. That flake will automatically create a virtual machine with a GB 18030 locale and then run yamllint in that virtual machine in order to demonstrate that yamllint 1.35.1 can indeed be used to lint GB 18030–encoded YAML files.

---

• If it's not the case, then no need for an extra option --force-encoding.

• If it's the case, then I agree it would be better to add a way to avoid breakage for those users. However I'm not a fan of a new command line option (--force-encoding), because ideally this would be temporary and removed after a few yamllint versions (after users fixed their files to use UTF-8), so it would break usage the day when we remove this option.
  Instead, we could use an environment variable YAMLLINT_IO_ENCODING, that detect_encoding() would use to override the encoding if defined (then we could unit-test 2 or 3 cases in a new test_detect_encoding_with_env_var_override()). Removing support for this env var in the future won't break command-line options. In my opinion, this should be fairly simple and should go in the same commit.

OK, that sounds like a good idea. I’ve implemented it, but I decided to call the environment variable YAMLLINT_FILE_ENCODING instead of YAMLLINT_IO_ENCODING. YAMLLINT_IO_ENCODING sounds too similar to PYTHONIOENCODING, and (as far as I know) the PYTHONIOENCODING variable affects stdin, stdout and stderr, not regular files. People who are familiar with PYTHONIOENCODING might see the name YAMLLINT_IO_ENCODING and assume that it has something to do with stdin, stdout and stderr.

---

I like the new version too, it's clearer about the Windows case. Did you mean Autodetect detect → Autodetect?

Yep. Thanks for pointing that out.

---

@dalito

(I did not check if you do/did this.) It could be interesting to run tests with https://docs.python.org/3/library/io.html#io-encoding-warning to detect where the code relies on default encoding.

I agree, -X warn_default_encoding is a really useful way to track down these kinds of bugs. I’m really glad that was added to Python. A lot of the changes in this PR come from me running python -X warn_default_encoding -W error::EncodingWarning -m unittest discover and seeing what broke. The last commit in this PR makes the CI tests run with -X warn_default_encoding -W error::EncodingWarning to help us avoid these kinds of mistakes in the future.

[adrienverge]

@dalito @Jayman2000 thanks for your help and explanations on system encodings! Indeed I misunderstood PYTHONIOENCODING.
@Jayman2000, https://jasonyundt.website/posts/terminal-in-ascii-on-linux is a nice and useful article 👍 For information, as you suspected, fr_FR.CP1252 was not in my glibc locales list. Since fr_FR.iso88591 is present, next time I'll use this one.

I rereviewed the code and it looks ready for merging. Before that, I have 2 remarks:

1. OK, that sounds like a good idea. I’ve implemented it, but I decided to call the environment variable YAMLLINT_FILE_ENCODING instead of YAMLLINT_IO_ENCODING

   Your argument makes sense, YAMLLINT_FILE_ENCODING sounds better to avoid confusion. But yamllint can also lint stdin (echo -e 'é: v\né: v' | yamllint -) and write reports to stdout (duplication of key "é" in mapping). Does your latest implementation with YAMLLINT_FILE_ENCODING also handle the encoding of stdin/stdout?

   By the way, good idea to properly warn about the temporariness of YAMLLINT_FILE_ENCODING.

2. The new documentation page on "Character Encoding Override" is a great addition. Thanks! To prepare for potential future additions, may I suggest to rename it to "Character Encoding", and put the last 2 paragraphs in a section "Override character encoding"?

[adrienverge]

Is it worth mentioning here? Why not, you decide!
By the way, the same also applies to the configuration file .yamllint.

If you keep it, maybe we can simplify it like this?

.. note:: Files on the ``ignore-from-file`` list should use either UTF-8, UTF-16
   or UTF-32. See :doc:`Character Encoding Override
   <character_encoding_override>` for details and wordarounds.

[adrienverge]

Idea:

Suggested change

	When yamllint reads a file, it will try to automatically detect that file’s
	When yamllint reads a file (either configuration or a file to lint), it will try
	to automatically detect that file’s …

(this goes along with my next comment ↓)