Im making an API for a company that inserts orders via this api into our DB, but I'm running into a problem:  Chinese characters don't get stored into my tables, how to fix this?  I have: - My test file which uses cURL to post set to utf8 via header* - The API code which inserts the data set to utf8 via header* - Stored my phpfile as an UTF-8 encoded file - mysqli connection set to utf8 ($connection-&gt;set_charset("utf8")) - mysql-database set to utf8_general_ci - mysql-table set to utf8_general_ci - mysql-columns set to utf8_general_ci - I insert the values into a VARCHAR and into a TEXT column (for test purposes) - Inserted the values directly into the query -> no result - Inserted the values via iconv("ISO-8859-1","UTF-8", $text)-> weird result - Inserted the values via iconv("UTF-8","UTF-8", $text)-> no result - Tested characters like áéíúó, which work perfectly fine - Set my postvalues for curl via foreach($post as $k=&gt;$v){ $data.= $k.'='.$v;} to send as data - Set my postvalues for curl via http_build_query($post) to send as data - Used a query before inserted doing SET CHARACTER SET utf8 - Used a query before inserted doing SET NAMES 'utf8'   When I echo the query before inserting it, I see the chinese characters   When I check the database, it's blank values   (not unexpected:) When I Select it and output it, it's blank   *PHP charset header:  header('Content-Type: text/html; charset=utf-8');   I'm out of ideas, anyone?
326
I'm setting up a new server and want to support UTF-8 fully in my web application. I have tried this in the past on existing servers and always seem to end up having to fall back to ISO-8859-1.  Where exactly do I need to set the encoding/charsets? I'm aware that I need to configure Apache, MySQL, and PHP to do this — is there some standard checklist I can follow, or perhaps troubleshoot where the mismatches occur?  This is for a new Linux server, running MySQL 5, PHP, 5 and Apache 2.
61253
I have this MySQL query using  that is the following:  SELECT *  FROM data_tbl WHERE AES_DECRYPT(data_point,'aBcDeF')='data_1';   I've figured out as much that the equivalent in Postgres 11 is  (F.25.4. Raw Encryption Functions) and the query would look something like this:  SELECT *  FROM data_tbl AND decrypt(data_point,'aBcDeF','aes')='data_1';   At least, the documentation for both leads me to believe that these are equivalent.  The data was dumped from MySQL and imported into PostgreSQL.   The data-point field in MySQL is type varbinary(255) and the values all show BLOB in MySQL Workbench.  The data_point field in Postgres is type bytea and the values all show [binary data] in pgAdmin.    Whether those data types are equivalent, I admit, I'm not sure.  The MySQL query works and finds a record based on he criteria. The PostgreSQL does not, although the record is in there. So it seems like one of three things:   The data types (varbinary(255) vs bytea) are not equivalent The data does not match (BLOB vs [binary data]) are not equivalent The functions (AES_DECRYPT vs decrypt()) are not equivalent   I did try to decrypt the value in PostgreSQL just to see what it is in a readable format by first trying:  SELECT decrypt(data_point, 'aBcDeF', 'aes')  FROM data_tbl;   But that just returned [binary data] for all the rows.  Then I came across  to use convert_from as in:  SELECT convert_from(decrypt(data_point, 'aBcDeF', 'aes'), 'SQL_ASCII') FROM data_tbl;   However, I just get this error:  ERROR:  invalid byte sequence for encoding "UTF8": 0xcf 0xf5 SQL state: 22021   So at this point I'm stumped as to how to resolve the primary issue. The primary issue being doing the equivalent of AES_DECRYPT but in PostgreSQL 11. The last error message isn't so much an issue, but it is likely related to decrypt() not returning the same results.  Any suggestions?
114665
When I want to execute a php file it's displayed as text in my browser.  I have a VPS at Hostinger I installed Ubuntu 18.04 with Webmin and LAMP(installed automatically).  I checked the installation see below.  apache2 - v and php -v gives me: Apache/2.4.29 and PHP 7.2.24-0ubuntu0.18.04.6  So everything seem to be installed correctly.  My default folder is /var/www/html/ I do not us short tag in my code.  I try to lunch a file called phpinfo.php with content:  &lt;?php phpinfo(); ?&gt;   And plain text appear on the page.  If the file begin with html code it's correctly rendered, even with a .php file. But the php part is not executed. e.g:  &lt;!DOCTYPE html&gt; &lt;html lang="en" dir="ltr"&gt;   &lt;head&gt;     &lt;meta charset="utf-8"&gt;     &lt;title&gt;&lt;/title&gt;   &lt;/head&gt;   &lt;body&gt;     &lt;h1&gt;Hello&lt;/h1&gt;   &lt;/body&gt; &lt;/html&gt;  &lt;?php phpinfo(); ?&gt;   Render :  Hello  But  &lt;?php phpinfo(); ?&gt;  &lt;!DOCTYPE html&gt; &lt;html lang="en" dir="ltr"&gt;   &lt;head&gt;     &lt;meta charset="utf-8"&gt;     &lt;title&gt;&lt;/title&gt;   &lt;/head&gt;   &lt;body&gt;     &lt;h1&gt;Hello&lt;/h1&gt;   &lt;/body&gt; &lt;/html&gt;   Render the code as plain text.
47900
In my little age with WordPress, I've seen WordPress itself and its friendly plugins are using PHP serialize() in storing data into db in many cases. But in a recent search I found a serious community support for the json_encode() over the serialize().     And I personally tested an associative array with both of 'em, that shows:  serialize() stores 342 chars json_encode() stores 285 chars  Why I'm asking this? I'm on a project while I'm going to store repeating meta fields to a post. Where:  Data would be basically in English, but sometimes can be Bengali Data would be associative array, 3 level deep (I hope I understood levels correctly):   array(     1 =&gt; array(         'key'=&gt;'value',         'key2'=&gt;'value'     ),     2 =&gt; array(         'key'=&gt;'value',         'key2'=&gt;'value'     ) )  I've checked the postmeta table's meta_value field it's a longtext,  a length of 4,294,967,295 chars (4GB). So I need a robust solution into storing things.
50152
Consider the following text (incidentally, part of a MySQL dump):   CREATE TABLE `table` (   `id` int(10) NOT NULL auto_increment,   `name` varchar(100) NOT NULL default '',   `description` text NOT NULL,   PRIMARY KEY  (`id`),   FULLTEXT KEY `full_index` (`name`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; /*!40101 SET character_set_client = @saved_cs_client */;   I would like to remove the FULLTEXT key, and I also want to remove the trailing comma on the line above so that the SQL remains valid.  Can anyone come up with (and explain) a sed recipe to do this?
121040
I expect the column to be a VARCHAR2, in my Oracle Database.  US Zips are 9.  Canadian is 7.  I am thinking 32 characters would be reasonable upper limit  What am I missing?  [EDIT] TIL: 12 is a reasonable answer to the question Thanks to everyone who contributed.
5845
I tried to use java.io.FileReader to read some text files and convert them into a string, but I found the result is wrongly encoded and not readable at all.  Here's my environment:   Windows 2003, OS encoding: CP1252 Java 5.0   My files are UTF-8 encoded or CP1252 encoded, and some of them (UTF-8 encoded files)  may contain Chinese (non-Latin) characters.  I use the following code to do my work:     private static String readFileAsString(String filePath)     throws java.io.IOException{         StringBuffer fileData = new StringBuffer(1000);         FileReader reader = new FileReader(filePath);         //System.out.println(reader.getEncoding());         BufferedReader reader = new BufferedReader(reader);         char[] buf = new char[1024];         int numRead=0;         while((numRead=reader.read(buf)) != -1){             String readData = String.valueOf(buf, 0, numRead);             fileData.append(readData);             buf = new char[1024];         }         reader.close();         return fileData.toString();     }   The above code doesn't work. I found the FileReader's encoding is CP1252 even if the text is UTF-8 encoded. But the JavaDoc of java.io.FileReader says that:     The constructors of this class assume   that the default character encoding   and the default byte-buffer size are   appropriate.   Does this mean that I am not required to set character encoding by myself if I am using FileReader? But I did get wrongly encoded data currently, what's the correct way to deal with my situtaion? Thanks.
93719
I'm struggling with the design of new database that has to use a different collation (Y Collation) than the SQL Server Instance default collation (X Collation). I'm looking for potential problems.  Temp Tables by default will be created with X Collation so any joins on text data types will fail unless I create temp table with Y collations on columns.  What about SQL Server string functions: REPLACE, SUBSTRING, etc.? Will they always be returned in X collation? What about when I use REPLACE(...) COLLATE Y? Do you see any potential data loss or maybe any different problems?
17325
I have a MySQL table with utf8 general ci collation. In the table, I can see two entries:  abad abád  I am using a query that looks like this:  SELECT *  FROM `words` WHERE `word` = 'abád'   The query result gives both words:  abad abád  Is there a way to indicate that I only want MySQL to find the accented word? I want the query to only return    abád    I have also tried this query:  SELECT *  FROM `words` WHERE BINARY `word` = 'abád'   It gives me no results. Thank you for the help.
115204
I want to import shapefile into PostGIS. There is one database postgres in it but no table added yet.   I am using this command:   shp2pgsql -I -s 4326 districts.shp districts | psql -d postgres -U postgres    and it's giving me some error:      Unable to convert data value to UTF-8 (iconv reports "Invalid or incomplete multibyte or wide character"). Current encoding is "UTF-8". Try "LATIN1" (Western European), or one of the values described at postgresql.org/docs/current/static/multibyte.html   Is my command wrong ??
18694
We have a simple T-SQL script file (.sql), which we execute through Powershell. Everything works fine, except the script portion, where we are using some special characters as string (example: 'ô' ).   The same script works fine, if I just execute it through SQL Server Management Studio, but through Powershell it's not rendering these special characters properly.  Here how we access the script file to execute on all databases on instance  Add-PSSnapin SqlServerCmdletSnapin100 Add-PSSnapin SqlServerProviderSnapin100 invoke-Sqlcmd -InputFile "C:\DailyPatch\run.sql" | Out-File -filePath "C:\DailyPatch\dbname.txt"  $text=Get-Content C:\DailyPatch\dbname.txt $a = $text.length $a=$a-3  $logFile = 'C:\DailyPatch\dblog.txt'  for ($i=3 ; $i -le $a ;$i++) {        write-host $text[$i]     $error.clear()      Invoke-sqlcmd -inputfile "C:\DailyPatch\Script4AllDBs.sql"-database $text[$i]-OutputSqlErrors $true|Out-File -filePath "C:\DailyPatch\dbname.txt"      foreach ($rr in $error) {         $text[$i] &gt;&gt; $logFile         $rr &gt;&gt; $logFile     } }   How to set collation in Powershell to avoid such kind of issues?  ps. Unicode thing is already handled on script side  Added: Special characters are being changed to � (Replacement Character)
57381
I'm having some brain failure in understanding reading and writing text to a file (Python 2.4).  # The string, which has an a-acute in it. ss = u'Capit\xe1n' ss8 = ss.encode('utf8') repr(ss), repr(ss8)      ("u'Capit\xe1n'", "'Capit\xc3\xa1n'")   print ss, ss8 print &gt;&gt; open('f1','w'), ss8  &gt;&gt;&gt; file('f1').read() 'Capit\xc3\xa1n\n'   So I type in Capit\xc3\xa1n into my favorite editor, in file f2.  Then:  &gt;&gt;&gt; open('f1').read() 'Capit\xc3\xa1n\n' &gt;&gt;&gt; open('f2').read() 'Capit\\xc3\\xa1n\n' &gt;&gt;&gt; open('f1').read().decode('utf8') u'Capit\xe1n\n' &gt;&gt;&gt; open('f2').read().decode('utf8') u'Capit\\xc3\\xa1n\n'   What am I not understanding here? Clearly there is some vital bit of magic (or good sense) that I'm missing. What does one type into text files to get proper conversions?  What I'm truly failing to grok here, is what the point of the UTF-8 representation is, if you can't actually get Python to recognize it, when it comes from outside. Maybe I should just JSON dump the string, and use that instead, since that has an asciiable representation! More to the point, is there an ASCII representation of this Unicode object that Python will recognize and decode, when coming in from a file?  If so, how do I get it?  &gt;&gt;&gt; print simplejson.dumps(ss) '"Capit\u00e1n"' &gt;&gt;&gt; print &gt;&gt; file('f3','w'), simplejson.dumps(ss) &gt;&gt;&gt; simplejson.load(open('f3')) u'Capit\xe1n'
82615
On PHP 5.3.13/MySQL 5.5.21 the following code doesn't work:  if($check_custom_fields_form!=1){     $sql = "CREATE TABLE IF NOT EXISTS ". $table_custom_fields_form ." (                 `form_name` longtext NOT NULL,                 `field_id` bigint(20) NOT NULL,                 FOREIGN KEY (`field_id`) REFERENCES $table_custom_fields (`ID`) ON DELETE CASCADE ON UPDATE CASCADE     ) CHARACTER SET utf8 COLLATE utf8_general_ci";     dbDelta($sql); } if($check_subscribe_cat!=1){     $sql = "CREATE TABLE IF NOT EXISTS ". $table_subscribe_cat ." (                 `subscribe_id` bigint(20) NOT NULL,                 `cat_id` bigint(20) NOT NULL,                 FOREIGN KEY (`subscribe_id`) REFERENCES ".$wpdb-&gt;prefix."tgt_subscription (`ID`) ON DELETE CASCADE ON UPDATE CASCADE,                 FOREIGN KEY (`cat_id`) REFERENCES ".$wpdb-&gt;prefix."terms (`term_id`) ON DELETE CASCADE ON UPDATE CASCADE     ) CHARACTER SET utf8 COLLATE utf8_general_ci";     dbDelta($sql);  }   The  suggested a downgrade to MySQL 5.1.37 (no, thanks) or the following update:  if($check_custom_fields_form!=1){         $sql = "CREATE TABLE IF NOT EXISTS ". $table_custom_fields_form ." (                     `form_name` longtext NOT NULL,                     `field_id` bigint(20) NOT NULL,                     KEY(field_id)         ) CHARACTER SET utf8 COLLATE utf8_general_ci";         dbDelta($sql);     } if($check_subscribe_cat!=1){       $sql = "CREATE TABLE IF NOT EXISTS ". $table_subscribe_cat ." (                   `subscribe_id` bigint(20) NOT NULL,                   `cat_id` bigint(20) NOT NULL,                   KEY(subscribe_id),                   KEY(cat_id)       ) CHARACTER SET utf8 COLLATE utf8_general_ci";       dbDelta($sql);    }   Which seems like a rather dirty way to work around the problem (no cascading deletes/updates). Therefore:   Do I really have to live with that ? Is it true that dbDelta only works with foreign key in a 3 year old MySQL version?
60896
If I execute the following simple script:  #!/bin/bash printf "%-20s %s\n" "Früchte und Gemüse"   "foo" printf "%-20s %s\n" "Milchprodukte"        "bar" printf "%-20s %s\n" "12345678901234567890" "baz"   It prints:  Früchte und Gemüse foo Milchprodukte        bar 12345678901234567890 baz   that is, text with umlauts (such as ü) is "shrunk" by one character per umlaut.  Certainly, I have some wrong setting somewhere, but I am not able to figure out which one that could be.  This occurs if the file's encoding is UTF-8.  If I change its encoding to latin-1, the alignment is correct, but the umlauts are rendered wrong:  Fr�chte und Gem�se   foo Milchprodukte        bar 12345678901234567890 baz
76738
I have a webserver administration question. In this website:   All PHP generated pages contain the following line:  &lt;meta http-equiv="Content-Type" content="text/html; charset=iso-8859-2" /&gt;   But this is somehow disregarded probably via the php-apache processing and the page displays in browsers with an UTF-8 header. As a result of that question marks (�) are shown in the page text instead of accented characters (éáöőóüűúí). I tested this in Firefox, IE, Chrome and Seamonkey.  The strangest in this phenomenon, that this symptom started only yesterday, after I upgraded my server to Debian 8.0 Jessie from 7.0 Wheezy. During the upgrade I also upgraded all other packages as well, including apache, php, and so on, and selected "yes" for overwriting config files with factory default ones. After this I fine-tuned my config files to have everything the way I like, but I did not find a way to fix this. Before the upgrade, the page displayed just fine.  Here is a screenshot, where you can see that Firefox sees the "charset=iso-8859-2" definition, but still displays the page with UTF-8 encoding.    My suspicion is, that this is a server configuration issue, but it could also be, that one part of the processing component (Apache, php) changed due to the upgrade in some way, resulting this strange behavior. The problem is, I can't pinpoint, what could possibly cause this problem.  Can anyone solve this mistery? What could be possibly going wrong during the processing of the page?
115935
I have a form with a textarea. Users enter a block of text which is stored in a database.  Occasionally a user will paste text from Word containing smart quotes or emdashes. Those characters appear in the database as: â€“, â€™, â€œ ,â€  What function should I call on the input string to convert smart quotes to regular quotes and emdashes to regular dashes?   I am working in PHP.  Update: Thanks for all of the great responses so far. The page on Joel's site about encodings is very informative:   Some notes on my environment:  The MySQL database is using UTF-8 encoding. Likewise, the HTML pages that display the content are using UTF-8 (Update:) by explicitly setting the meta content-type.  On those pages the smart quotes and emdashes appear as a diamond with question mark.  Solution:  Thanks again for the responses. The solution was twofold:   Make sure the database and HTML files were explicitly set to use UTF-8 encoding. Use htmlspecialchars() instead of htmlentities().
6735
I've got a working PHP script that gets Longitude and Latitude values and then inputs them into a MySQL query. I'd like to make it solely MySQL. Here's my current PHP Code:  if ($distance != "Any" &amp;&amp; $customer_zip != "") { //get the great circle distance      //get the origin zip code info     $zip_sql = "SELECT * FROM zip_code WHERE zip_code = '$customer_zip'";     $result = mysql_query($zip_sql);     $row = mysql_fetch_array($result);     $origin_lat = $row['lat'];     $origin_lon = $row['lon'];      //get the range     $lat_range = $distance/69.172;     $lon_range = abs($distance/(cos($details[0]) * 69.172));     $min_lat = number_format($origin_lat - $lat_range, "4", ".", "");     $max_lat = number_format($origin_lat + $lat_range, "4", ".", "");     $min_lon = number_format($origin_lon - $lon_range, "4", ".", "");     $max_lon = number_format($origin_lon + $lon_range, "4", ".", "");     $sql .= "lat BETWEEN '$min_lat' AND '$max_lat' AND lon BETWEEN '$min_lon' AND '$max_lon' AND ";     }   Does anyone know how to make this entirely MySQL? I've browsed the Internet a bit but most of the literature on it is pretty confusing.