Python determine encoding of file

This method is not guaranteed to be accurate, but it can still be provided for our reference. If you have no chardet package in your environment, you can use the following command to install it:. To be careful is that you must select the " rb " mode to open thefile.

After all, python is read by default using unicode, so if we can't judge the encoding of such a file, if we open it directly, we will often get an error. Result pasted into Google Translate gives. Unfortunately, SO thinks that Chinese text in the result is a spam so I can't embed it here….

Anyway, abovementioned Subject: Re: I upgraded to or Re: My promotion suggested by Mark Tolonen looks to be more meaningful than this…. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams?

Collectives on Stack Overflow. Learn more. Asked 1 year ago. Active 1 year ago. Viewed times. I have an encoding issue. Is there a way I can detect encoding in these badly encoded UTF-8 files? For example you said some files are properly encoded whilst others are not. Accessing these files might help — Leonardo Maffei. LeonardoMaffei, I added an example txt file to my original post.

Apparently, it's a flagrant double mojibake case. The rest of your example file is total garbage. No it's not garbage. Anyways, I think I have a solution now, just testing it on multiple text files. Although it "looks fine" as Chinese, it is garbage text and doesn't translate correctly to anything. JosefZ is correct. The subject is double mojibake. Just because it looks like Chinese characters doesn't mean it makes sense.

That codec just makes chinese characters out of the garbage. Add a comment. Active Oldest Votes. Similar topics. NET Framework. Determining which encoding the browser used for a url. XmlTextWriter and Text. UTF8 issues. Visual Basic. Determining Charset used by system or software.

Determining encoding of a file. Determining possible encodings of a given text. Mobile Development. Software Development. Remote Development and Debugging Tools. Nested class structure question. How to use Nuget assembly as files with visual studio community. Ms access. Undesired Name?