I'm using GORM to query a database with rows containing invalid utf8 characters and getting errors when comparing values:
db.Where("name ILIKE ?", value)
ERROR: invalid byte sequence for encoding "UTF8": 0xc6 0x61 (SQLSTATE 22021)
I've tried filtering out possible non utf8 characters by preceding my call with statements like these with no luck:
db.Where("name !~ ?", "[^\\\\x00-\\\\x7F]")
db.Where("octet_length(name) = char_length(name)")
db.Where("length(name) = length(encode(decode(name, 'escape'), 'escape'))")
db.Where("name not ILIKE '%[^%]%'")
Is there any way to filter out all rows with fields that are not utf8?
I can't fix the rows in the database because there is a constant flow of data coming in at the moment that could also be not utf8. For now, I just want to ignore those rows. Does anyone know the answer? Thanks