Improve binary detection for cases starting by chance by a BOM

The presence of a sequence of bytes resembling a BOM does not guarantee that the data is text. We can in those cases use the detection provided by Qt. If the codec matches the one selected, we can consider that text. See issue #2197
sqlitebrowser · Aug 22, 2020 · 221f1bc · 221f1bc
1 parent f5a2b10
commit 221f1bc
Showing 1 changed file with 6 additions and 3 deletions.
diff --git a/src/Data.cpp b/src/Data.cpp
@@ -16,9 +16,12 @@ static const QByteArray bom4b("\xFF\xFE\x00\x00", 4);
 
 bool isTextOnly(QByteArray data, const QString& encoding, bool quickTest)
 {
-    // If the data starts with a Unicode BOM, we always assume it is text
-    if(startsWithBom(data))
-        return true;
+    // If the data starts with a Unicode BOM, we can use detection provided by QTextCodec.
+    if(startsWithBom(data)) {
+        QTextCodec *codec = encoding.isEmpty()? QTextCodec::codecForName("UTF-8") : QTextCodec::codecForName(encoding.toUtf8());
+        QTextCodec *detectedCodec = QTextCodec::codecForUtfText(data, nullptr);
+        return detectedCodec == codec;
+    }
 
     // Truncate to the first few bytes for quick testing
     int testSize = quickTest? std::min(512, data.size()) : data.size();