keronnatural.blogg.se

In the above example, if you notice, the tag has been rewritten to reflect the generated document from BeautifulSoup is now in UTF-8 format. Below a document, where the polish characters are there in ISO-8859-2 format. The output from a BeautifulSoup is UTF-8 document, irrespective of the entered document to BeautifulSoup. > soup = BeautifulSoup(markup, exclude_encodings=) It can be used, when you don’t know the correct encoding but sure that Unicode, Dammit is showing wrong result. > soup = BeautifulSoup(markup, from_encoding="iso-8859-8")Īnother new feature added from BeautifulSoup 4.4.0 is, exclude_encoding. To resolve above issue, pass it to BeautifulSoup using from_encoding −

You can save some time and avoid mistakes, if you already know the encoding by passing it to the BeautifulSoup constructor as from_encoding.īelow is one example where the BeautifulSoup misidentifies, an ISO-8859-8 document as ISO-8859-7 −

As the document is searched byte-by-byte to guess the encoding, it takes lot of time. However, not all the time, the Unicode, Dammit guesses correctly. However, when you load that HTML/XML document into BeautifulSoup, it has been converted to Unicode.Ībove behavior is because BeautifulSoup internally uses the sub-library called Unicode, Dammit to detect a document’s encoding and then convert it into Unicode. All HTML or XML documents are written in some specific encoding like ASCII or UTF-8.