Posts

Showing posts from January, 2014

mysql tips - export csv file from MYSQL database in remote server

1.  Export csv file in client server directly This is the situation that I think most beginners will have for doing development or backup of the mysql database. You have a remote mysql server to hold the data, and a local client server to develope. As solution 2 provided by MYSQL official csv export solution will export the file only on the database server (here is on remote server), that is not what we want here. mysql -h hostname -u user -p'password' -P port dbname -e "SELECT * FROM tablename WHERE Date<='2013-07-14';" | sed "s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g" >  tablename .csv Using variables and in shell script (which easily cause problems): mysql -h $hostname -u $user -p $pass -P $port $dbname -e "SELECT * FROM $tablename WHERE Date<= '$oldDate'; " | sed "s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g" >  tablename .csv For the sed...

xml validation and invalid characters solution in PHP/Python

Validate xml file, the website below can do the trick and show you invalid xml characters: http://www.xmlvalidation.com/ When you use xml parser in PHP like simplexml_load_string()  of version 1.0 to process the xml file based on version 1.1, then it is possible that you will have this issue. You may got an error from your script like below: simplexml_load_string(): Entity: line 166858: parser error : xmlParseCharRef: invalid xmlChar value 0 or  simplexml_load_string(): Entity: line 166858: parser error : xmlParseCharRef: invalid xmlChar value 14 This is because in the version 1.0, the characters like &#x0, &#xE is not allowed. The rule is that all characters composition /&#x[0-1]?[0-9A-E]/  is not allowed. So we can replace the xml file like below to fix this problem: $string = file_get_content($xmllink); preg_replace('/&#x[0-1]?[0-9A-E]/', ' ', $string); The xml validation is similar in python, to process a big .xml file, we do not ready it on...