如何將HTML字符引用（ף）轉換爲常規UTF-8？

Question

我有一些包含字符引用的希伯來語網站，如：נוף

我只能查看這些字母，如果我將該文件保存爲.html並以UTF-8編碼進行查看。

如果我試圖打開它作爲常規文本文件，然後UTF-8編碼不顯示正確的輸出。

我注意到，如果我打開文本編輯器和寫入UTF-8希伯來語，每個字符使用兩個字節不是4字節線在本例中（ו）

任何想法，如果這是UTF-16或任何其他類型的字母UTF表示形式？

如果可能，我如何將它轉換爲普通字母？

使用最新的PHP版本。

Answer 1

那些是character references通過指定在小數點該字符的代碼點（&#n;）指的是字符在ISO 10646或十六進制（&#xn;）表示法。

您可以使用html_entity_decode解碼等字符引用以及對entities defined for HTML 4實體引用，所以其他的引用類似<，>，&也將得到解碼：如果你只是想解碼

$str = html_entity_decode($str, ENT_NOQUOTES, 'UTF-8'); 

數字字符引用，您可以使用此：

function html_dereference($match) { 
    if (strtolower($match[1][0]) === 'x') { 
     $codepoint = intval(substr($match[1], 1), 16); 
    } else { 
     $codepoint = intval($match[1], 10); 
    } 
    return mb_convert_encoding(pack('N', $codepoint), 'UTF-8', 'UTF-32BE'); 
} 
$str = preg_replace_callback('/&#(x[0-9a-f]+|[0-9]+);/i', 'html_dereference', $str); 

---

由於YuriKolovsky和thirtydot在another question中已經指出，似乎瀏覽器供應商對'字符引用映射'的某些事情'默默'達成了一致，這與規範不同，並且沒有相關文檔。

似乎有一些字符引用通常會映射到Latin 1 supplement上，但實際上映射到不同的字符上。這是因爲映射不是映射來自Windows-1252而是映射ISO 8859-1的字符，而Unicode字符集是在該映射上構建的。 Jukka Korpela寫了一個extensive article on this topic。

現在，這裏的一個擴展，上面提到的功能處理這個怪癖：

function html_character_reference_decode($string, $encoding='UTF-8', $fixMappingBug=true) { 
    $deref = function($match) use ($encoding, $fixMappingBug) { 
     if (strtolower($match[1][0]) === "x") { 
      $codepoint = intval(substr($match[1], 1), 16); 
     } else { 
      $codepoint = intval($match[1], 10); 
     } 
     // @see http://www.cs.tut.fi/~jkorpela/www/windows-chars.html 
     if ($fixMappingBug && $codepoint >= 130 && $codepoint <= 159) { 
      $mapping = array(
       8218, 402, 8222, 8230, 8224, 8225, 710, 8240, 352, 8249, 
       338, 141, 142, 143, 144, 8216, 8217, 8220, 8221, 8226, 
       8211, 8212, 732, 8482, 353, 8250, 339, 157, 158, 376); 
      $codepoint = $mapping[$codepoint-130]; 
     } 
     return mb_convert_encoding(pack("N", $codepoint), $encoding, "UTF-32BE"); 
    }; 
    return preg_replace_callback('/&#(x[0-9a-f]+|[0-9]+);/i', $deref, $string); 
} 

如果anonymous functions不可用（與5.3.0介紹），你也可以使用create_function：

$deref = create_function('$match', ' 
    $encoding = '.var_export($encoding, true).'; 
    $fixMappingBug = '.var_export($fixMappingBug, true).'; 
    if (strtolower($match[1][0]) === "x") { 
     $codepoint = intval(substr($match[1], 1), 16); 
    } else { 
     $codepoint = intval($match[1], 10); 
    } 
    // @see http://www.cs.tut.fi/~jkorpela/www/windows-chars.html 
    if ($fixMappingBug && $codepoint >= 130 && $codepoint <= 159) { 
     $mapping = array(
      8218, 402, 8222, 8230, 8224, 8225, 710, 8240, 352, 8249, 
      338, 141, 142, 143, 144, 8216, 8217, 8220, 8221, 8226, 
      8211, 8212, 732, 8482, 353, 8250, 339, 157, 158, 376); 
     $codepoint = $mapping[$codepoint-130]; 
    } 
    return mb_convert_encoding(pack("N", $codepoint), $encoding, "UTF-32BE"); 
'); 

---

這裏是另一個函數，它試圖符合behavior of HTML 5：

function html5_decode($string, $flags=ENT_COMPAT, $charset='UTF-8') { 
    $deref = function($match) use ($flags, $charset) { 
     if ($match[1][0] === '#') { 
      if (strtolower($match[1][0]) === '#') { 
       $codepoint = intval(substr($match[1], 2), 16); 
      } else { 
       $codepoint = intval(substr($match[1], 1), 10); 
      } 

      // HTML 5 specific behavior 
      // @see http://dev.w3.org/html5/spec/tokenization.html#tokenizing-character-references 

      // handle Windows-1252 mismapping 
      // @see http://www.cs.tut.fi/~jkorpela/www/windows-chars.html 
      // @see http://dev.w3.org/html5/spec/tokenization.html#table-charref-overrides 
      $overrides = array(
       0x00=>0xFFFD,0x80=>0x20AC,0x82=>0x201A,0x83=>0x0192,0x84=>0x201E, 
       0x85=>0x2026,0x86=>0x2020,0x87=>0x2021,0x88=>0x02C6,0x89=>0x2030, 
       0x8A=>0x0160,0x8B=>0x2039,0x8C=>0x0152,0x8E=>0x017D,0x91=>0x2018, 
       0x92=>0x2019,0x93=>0x201C,0x94=>0x201D,0x95=>0x2022,0x96=>0x2013, 
       0x97=>0x2014,0x98=>0x02DC,0x99=>0x2122,0x9A=>0x0161,0x9B=>0x203A, 
       0x9C=>0x0153,0x9E=>0x017E,0x9F=>0x0178); 
      if (isset($windows1252Mapping[$codepoint])) { 
       $codepoint = $windows1252Mapping[$codepoint]; 
      } 

      if (($codepoint >= 0xD800 && $codepoint <= 0xDFFF) || $codepoint > 0x10FFFF) { 
       $codepoint = 0xFFFD; 
      } 
      if (($codepoint >= 0x0001 && $codepoint <= 0x0008) || 
       ($codepoint >= 0x000E && $codepoint <= 0x001F) || 
       ($codepoint >= 0x007F && $codepoint <= 0x009F) || 
       ($codepoint >= 0xFDD0 && $codepoint <= 0xFDEF) || 
       in_array($codepoint, array(
        0x000B, 0xFFFE, 0xFFFF, 0x1FFFE, 0x1FFFF, 0x2FFFE, 0x2FFFF, 
        0x3FFFE, 0x3FFFF, 0x4FFFE, 0x4FFFF, 0x5FFFE, 0x5FFFF, 0x6FFFE, 
        0x6FFFF, 0x7FFFE, 0x7FFFF, 0x8FFFE, 0x8FFFF, 0x9FFFE, 0x9FFFF, 
        0xAFFFE, 0xAFFFF, 0xBFFFE, 0xBFFFF, 0xCFFFE, 0xCFFFF, 0xDFFFE, 
        0xDFFFF, 0xEFFFE, 0xEFFFF, 0xFFFFE, 0xFFFFF, 0x10FFFE, 0x10FFFF))) { 
       $codepoint = 0xFFFD; 
      } 
      return mb_convert_encoding(pack("N", $codepoint), $charset, "UTF-32BE"); 
     } else { 
      return html_entity_decode($match[0], $flags, $charset); 
     } 
    }; 
    return preg_replace_callback('/&(#(?:x[0-9a-f]+|[0-9]+)|[A-Za-z0-9]+);/i', $deref, $string); 
} 

我也注意到，在PHP 5.4.0的html_entity_decode function加入名爲ENT_HTML5對HTML 5的行爲的另一標誌。

Answer 2

這些是XML Character Reference s。你想用html_entity_decode()來解碼：

$string = html_entity_decode($string, ENT_QUOTES, 'UTF-8'); 

欲瞭解更多信息，您可以搜索谷歌有問題的實體。看到這幾個實例：

1. Hebrew Characters
2. HTML Entities for Hebrew Characters
3. UTF-8 Encoding Table with HTML entities