I forked this Next.JS-Boilerplate repository which uses drizzle-orm with node-postgres.
I created a database scheme
export const myTableSchema = pgTable('myTable', { id: serial('id').primaryKey(), name: text('name').notNull(),});and then wrote functionality to store and update entries within the database. This generally works.
await db .update(myTable) .set({ name: "ä" }) .where( eq(myTable.id, Number(id)), ) .returning();However, if I enter German umlauts (mutated vowels like 'ä' oder 'ü') I get
error: invalid byte sequence for encoding »UTF8«: 0xe4 0x68 0x73 at /home/testuser/test/node_modules/pg/lib/client.js:526:17 at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async eval (webpack-internal:///(rsc)/./node_modules/drizzle-orm/node-postgres/session.js:60:22) at async PUT$1 (webpack-internal:///(rsc)/./src/app/[locale]/(auth)/api/work_areas/route.ts:124:33) at async eval (webpack-internal:///(rsc)/./node_modules/@sentry/nextjs/build/cjs/common/wrapRouteHandlerWithSentry.js:58:36) at async /home/testuser/test/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:6:55755 at async eO.execute (/home/testuser/test/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:6:46523) at async eO.handle (/home/testuser/test/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:6:57089) at async doRender (/home/testuser/test/node_modules/next/dist/server/base-server.js:1359:42) at async cacheEntry.responseCache.get.routeKind (/home/testuser/test/node_modules/next/dist/server/base-server.js:1569:40) at async DevServer.renderToResponseWithComponentsImpl (/home/testuser/test/node_modules/next/dist/server/base-server.js:1489:28) at async DevServer.renderPageComponent (/home/testuser/test/node_modules/next/dist/server/base-server.js:1920:24) at async DevServer.renderToResponseImpl (/home/testuser/test/node_modules/next/dist/server/base-server.js:1958:32) at async DevServer.pipeImpl (/home/testuser/test/node_modules/next/dist/server/base-server.js:917:25) at async NextNodeServer.handleCatchallRenderRequest (/home/testuser/test/node_modules/next/dist/server/next-server.js:272:17) at async DevServer.handleRequestImpl (/home/testuser/test/node_modules/next/dist/server/base-server.js:813:17) at async /home/testuser/test/node_modules/next/dist/server/dev/next-dev-server.js:339:20 at async Span.traceAsyncFn (/home/testuser/test/node_modules/next/dist/trace/trace.js:154:20) at async DevServer.handleRequest (/home/testuser/test/node_modules/next/dist/server/dev/next-dev-server.js:336:24) at async invokeRender (/home/testuser/test/node_modules/next/dist/server/lib/router-server.js:173:21) at async handleRequest (/home/testuser/test/node_modules/next/dist/server/lib/router-server.js:350:24) at async requestHandlerImpl (/home/testuser/test/node_modules/next/dist/server/lib/router-server.js:374:13) at async Server.requestListener (/home/testuser/test/node_modules/next/dist/server/lib/start-server.js:141:13) { length: 167, severity: 'ERROR', code: '22021', detail: undefined, hint: undefined, position: undefined, internalPosition: undefined, internalQuery: undefined, where: 'unbenanntes Portal Parameter $1', schema: undefined, table: undefined, column: undefined, dataType: undefined, constraint: undefined, file: 'mbutils.c', line: '1665', routine: 'report_invalid_encoding'}I checked in postgres and it shows
postgres@myhost:~$ psql -l List of databases Name | Owner | Encoding | Collate | Ctype | ICU Locale | Locale Provider | Access privileges -----------+------------+-----------+--------------+-------------+------------+-----------------+----------------------------- my_db | postgres | UTF8 | de_DE.UTF-8 | de_DE.UTF-8 | | libc | =Tc/postgres + | | | | | | | postgres=CTc/postgres + | | | | | | | my_db=CTc/postgres...It also states CLIENT_ENCODING and SERVER_ENCODING are both set to UTF8.
I then went on and ran the exact same update directly via psql and it worked as expected.
I then tried switching from node-postgres to Postgres.JS as mentioned in the Drizzle ORM Documentation. It did not solve the problem.
I also tried to somehow convert the string to UTF-8 from what should be UTF-8 in the first place - with no luck.
myuser@myhost:~$ localeLANG=de_DE.UTF-8LANGUAGE=LC_CTYPE="de_DE.UTF-8"LC_NUMERIC="de_DE.UTF-8"LC_TIME="de_DE.UTF-8"LC_COLLATE="de_DE.UTF-8"LC_MONETARY="de_DE.UTF-8"LC_MESSAGES="de_DE.UTF-8"LC_PAPER="de_DE.UTF-8"LC_NAME="de_DE.UTF-8"LC_ADDRESS="de_DE.UTF-8"LC_TELEPHONE="de_DE.UTF-8"LC_MEASUREMENT="de_DE.UTF-8"LC_IDENTIFICATION="de_DE.UTF-8"LC_ALL=Is this an issue with drizzle?