TooManyCooks TooManyCooks - 1 year ago 48
SQL Question

How to remove htmlentities() values from the database?

Long before I knew anything - not that I know much even now - I desgined a web app in php which inserted data in my mysql database after running the values through

. I eventually came to my senses and removed this step and stuck it in the output rather than input and went on my merry way.

However I've since had to revisit some of this old data and unfortunately I have an issue, when it's displayed on the screen I'm getting values displayed which are effectively htmlentitied twice.

So, is there a mysql or phpmyadmin way of changing all the older, affected rows back into their relevant characters or will I have to write a script to read each row, decode and update all 17 million rows in 12 tables?


Thanks for the help everyone, I wrote my own answer down below with some code in, it's not pretty but it worked on the test data earlier so barring someone pointing out a glaring error in my code while I'm in bed I'll be running it on a backup DB tomorrow and then on the live one if that works out alright.

Answer Source

I ended up using this, not pretty, but I'm tired, it's 2am and it did its job! (Edit: on test data)

$tables = array('users', 'users_more', 'users_extra', 'forum_posts', 'posts_edits', 'forum_threads', 'orders', 'product_comments', 'products', 'favourites', 'blocked', 'notes');
foreach($tables as $table)
        $sql = "SELECT * FROM {$table} WHERE data_date_ts < '{$encode_cutoff}'";
        $rows = $database->query($sql);
        while($row = mysql_fetch_assoc($rows))
                $new = array();
                foreach($row as $key => $data)
                        $new[$key] = $database->escape_value(html_entity_decode($data, ENT_QUOTES, 'UTF-8'));
                $new_string = "";
                $i = 0;
                foreach($new as $new_key => $new_data)
                        if($i > 0) { $new_string.= ", "; }
                        $new_string.= $new_key . "='" . $new_data . "'";
                $sql = "UPDATE {$table} SET " . $new_string . " WHERE id='" . $row['id'] . "'";
                // plus some code to check that all out