Quantcast
Channel: Active questions tagged utf-8 - Stack Overflow
Viewing all articles
Browse latest Browse all 1029

GORM/SQL query to ignore invalid utf-8 characters

$
0
0

I'm using GORM to query a database with rows containing invalid utf8 characters and getting errors when comparing values:

db.Where("name ILIKE ?", value)

ERROR: invalid byte sequence for encoding "UTF8": 0xc6 0x61 (SQLSTATE 22021)

I've tried filtering out possible non utf8 characters by preceding my call with statements like these with no luck:

db.Where("name !~ ?", "[^\\\\x00-\\\\x7F]")

db.Where("octet_length(name) = char_length(name)")

db.Where("length(name) = length(encode(decode(name, 'escape'), 'escape'))")

db.Where("name not ILIKE '%[^%]%'")

Is there any way to filter out all rows with fields that are not utf8?

I can't fix the rows in the database because there is a constant flow of data coming in at the moment that could also be not utf8. For now, I just want to ignore those rows. Does anyone know the answer? Thanks


Viewing all articles
Browse latest Browse all 1029

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>