I need to read data from a public link. Here is the link:https://webgate.ec.europa.eu/fsd/fsf/public/files/csvFullSanctionsList/content?token=dG9rZW4tMjAxNw
The original data is in UTF-8-BOM. I changed it to UTF-8 using ADF and saved it as CSV in Azure Data Lake Storage Gen2. The following are the dataset settings.Here
then need to read the file from ADLS with openrowset in Azure SQL Database.
SELECT *FROM OPENROWSET( BULK 'FullSanctionsList.csv' , DATA_SOURCE = 'myDataSource' , FORMATFILE = 'FullSanctionsList.fmt' , FORMATFILE_DATA_SOURCE = 'myDataSource' , MAXERRORS = 100 , FIELDQUOTE = '"' , FORMAT = 'CSV' , CODEPAGE = '65001' , FIRSTROW = 2) AS x
and the formatfile is like the following:Here
I tried different collates but still have problems with some free text columns like "Naal_wholename"