Last night we installed an SSL Certificate on WritingForums.org. This updated forces all traffic to https://www.writingforums.org instead of the http:// version. We've chosen to install the SSL Certificate largely due to Google's strong recommendation. Beginning in 2017, Google began showing user login forms as insecure if the website did not use HTTPS, and noted that this insecure warning was the first of several future steps. All pages should now show the "Secure" green lock next to our URL in the browser, though there are still a number of checks and configurations that are needed to complete this process. This post will act as a checklist to ensure HTTPS is completely and properly configured on WritingForums.org SSL Checklist Install Certificate Force HTTPS Ensure Server has HTTPS Turned On Convert Smiley URLs Configure & Test Image/Link Proxies Validate 301 Redirects (three tools confirmed no errors) Review .htaccess File Test Facebook, Twitter, Google Integration (Xenforo test passed, should test actual login/registration) Update Facebook, Twitter, Google Application Accounts Update Facebook, Twitter, Google Profiles Review Queries on Xenforo Tables (queries ran, should confirm none were missed in locations url types) Check Template References for HTTP gravtar.com - account_avatar_overlay facebook.com/profile/ - account_external_accounts (changed) [bd] Medals (changed) data-vocabulary.com/breadcrumbs - breadcrumbs elseusernotes_memberview EWR_block facebook.com - Page_Container twitter.com - page_container_js_head Update Phrases Update Notices, Help Pages, User Upgrade Descriptions, Warnings, Style Property Images, FAQ Review Ads Add Google Analytics Note/Flag Test Google Search Results, Ensure Old Links Redirect Review Port 80 v. Port 443 Access Review cPanel Certificate Choice Respond to LiquidWeb Support Ticket (expiration, certificate choice, .htaccess, bugs) Re-Write Hard-Coded Internal Links Use Fetch & Render to Test Leave Old Sitemaps to Process 301 Redirects Research Strict Transport Security Only Access Identify How to Transfer Social Shares Update Media Site Extensions (removed unused, still needs reviewed) Update Tracking Codes Review robots.txt (review later) Change robots.txt Sitemap Address Rebuild & Submit Sitemap to Search Engines (rebuilt, need to check against robots.txt) Test Old vBulletin Redirects (still works except blog posts) Create New Google Search Console Property Update Old Google Search Property (not necessary - continue to monitor) Review Site Speed Seek Support for Better Analytics Update External Links to New URL Confirm SSL Analytics Configuration Ensure Google Can Index New URLs (See Google Guidelines) Monitor Webmaster Tools & Analytics Daily to Spot Potential Issues Review Analytics for Alerts and Configuration Issues Test Paypal Integration Update other External Account Integrations Known Bugs Solved Broken mobile navigation design on horizontal mobile Broken authentication when using Smilies Unable to post via Safari
Last night we installed an SSL Certificate on WritingForums.org. This updated forces all traffic to https://www.writingforums.org instead of the http:// version. We've chosen to install the SSL Certificate largely due to Google's strong recommendation. Beginning in 2017, Google began showing user login forms as insecure if the website did not use HTTPS, and noted that this insecure warning was the first of several future steps. All pages should now show the "Secure" green lock next to our URL in the browser, though there are still a number of checks and configurations that are needed to complete this process. This post will act as a checklist to ensure HTTPS is completely and properly configured on WritingForums.org SSL Checklist Install Certificate Force HTTPS Ensure Server has HTTPS Turned On Convert Smiley URLs Configure & Test Image/Link Proxies Validate 301 Redirects (three tools confirmed no errors) Review .htaccess File Test Facebook, Twitter, Google Integration (Xenforo test passed, should test actual login/registration) Update Facebook, Twitter, Google Application Accounts Update Facebook, Twitter, Google Profiles Review Queries on Xenforo Tables (queries ran, should confirm none were missed in locations url types) Check Template References for HTTP gravtar.com - account_avatar_overlay facebook.com/profile/ - account_external_accounts (changed) [bd] Medals (changed) data-vocabulary.com/breadcrumbs - breadcrumbs elseusernotes_memberview EWR_block facebook.com - Page_Container twitter.com - page_container_js_head Update Phrases Update Notices, Help Pages, User Upgrade Descriptions, Warnings, Style Property Images, FAQ Review Ads Add Google Analytics Note/Flag Test Google Search Results, Ensure Old Links Redirect Review Port 80 v. Port 443 Access Review cPanel Certificate Choice Respond to LiquidWeb Support Ticket (expiration, certificate choice, .htaccess, bugs) Re-Write Hard-Coded Internal Links Use Fetch & Render to Test Leave Old Sitemaps to Process 301 Redirects Research Strict Transport Security Only Access Identify How to Transfer Social Shares Update Media Site Extensions (removed unused, still needs reviewed) Update Tracking Codes Review robots.txt (review later) Change robots.txt Sitemap Address Rebuild & Submit Sitemap to Search Engines (rebuilt, need to check against robots.txt) Test Old vBulletin Redirects (still works except blog posts) Create New Google Search Console Property Update Old Google Search Property (not necessary - continue to monitor) Review Site Speed Seek Support for Better Analytics Update External Links to New URL Confirm SSL Analytics Configuration Ensure Google Can Index New URLs (See Google Guidelines) Monitor Webmaster Tools & Analytics Daily to Spot Potential Issues Review Analytics for Alerts and Configuration Issues Test Paypal Integration Update other External Account Integrations Known Bugs Solved Broken mobile navigation design on horizontal mobile Broken authentication when using Smilies Unable to post via Safari
Last Wednesday (4/27) the Community Interaction category of the forum disappeared. I know many of you have been wondering what happened, what is currently going on, and when it will be back. This post is to explain the situation - and does so in detail - so as to be transparent about the problem, since it's still unresolved. It may get somewhat technical. If you want the short explanation, jump to the end. I would have made an announcement sooner, but I had to gather information and I had expected it to be resolved by now. What Happened? On 4/27 I was updating forum software add-ons by deleting old add-ons and updating existing add-ons to their latest version. Add-ons provide the forum additional functionality, such as our Resources, Medals, Blogs, FAQ, Gallery, and others. My investigation suggests that this process triggered the deletion of the Community Interaction forum category and it's child forums. In my investigation process I've shut down the forum a few times to take database backups. Fixing this entire issue had become more complicated, and thus taking longer, because when I attempted to work on the database I encountered technical server-sided issues that couldn't be explained. This prevented me from accessing or backing up the database. After working with our web host, they suggested the data directory was damaged. The data directory stores database information for all databases on a server, not a specific site's content, and allows databases to operate (a database for databases, if you will). We spent three days mapping out a technical plan to fix the issue with the data directory (taking backups, resizing the server's disk space, etc). At the last minute, the server's support team told me the issue had been resolved without the steps we'd been planning. This was because the cause wasn't corruption/damage of the data directory, but that the password contained a character the server could not read, and thus threw the error. This put restoration process of the Community Interaction category back several days, but tells me that the issue was probably not caused by the issue on the server. In short: the category and forums are missing because a software add-on removed them unexpectedly. Resolution of this issue has taken longer because of an significant unrelated issue with the web host, which has now been resolved. What Can We Do? In examining this issue, I've also spoken with Xenforo support staff. They strongly suggest restoring the forum from a database backup. The problem here is it would remove all posts made after the backup, and our most recent backup was from 4/19 when we upgraded to the latest version of Xenforo 1.4.x. Not wanting to pursue this solution, I found a similar situation on a different forum that was able to restore forums and posts that went missing by repairing the database. This only works if the cause was triggered by the software add-ons as suspected above, that the database queries are implemented correctly, and that the information wasn't deleted from the database in the process. Another option was to merge the old backup with the current version of the site, which was discouraged by Xenforo Staff. This is because it's a very technical process (you need an expert) and you risk causing other issues in the future (damaged functionality, incompatibility issues, inconsistent data, etc.). This brought the options to the following: 1) restore the database from a backup and lose data since 4/19, 2) confirm the data still exists and repair the database, 3) merge the old and current database and risk peripheral issues (serious technical expertise required). Option #1 is the least appealing to me, and becomes less appealing every day this isn't resolved, because we'd lose more information. Option #3 risks unknown future issues and requires an database professional (expert). Fortunately, I found a professional who is willing to help with this process, but even he does not like the merge idea. He's willing to attempt it but has not been available the past few days. Option #2 is the best solution, but may not work. It requires some level of technical expertise to run the queries and make sure everything is how it should be. I've consulted with a few people and they've outlined possible queries that should fix it if we're right about the cause. What We're Doing After fixing the second issue with the server's data directory, I've been preparing the server to run a private test installation of the site. This process basically includes taking backups of other database on the server and deleting old content from the server to make room for the test site. Our forum database is huge, and approaches 30 gigabytes, so we need more space. I believe I now have enough disk space to set up a test installation. What I am doing today is setting up a test installation of the site based on a backup I took two days ago. After successfully duplicating the database, I'm going to make some adjustments in the database and then run the queries given to me by someone who's had a similar issue happen on their forum. Essentially, if the data wasn't deleted from the database, it's still there, it just isn't assigned to a forum or category - it's orphaned. These queries should reassigned the orphaned threads to a new forum/category, which should restore the missing content. I've taken an early look and we have a huge number of orphaned threads. Our database size now is also approximately the same size as the backup I took two weeks, which indicates our data is safe in the database, but simply has to be repaired. I will run this test repair today. If I'm successfully, I will consult with the database professional and have him confirm the process. We will the implement the same process on the live installation of the site. If we're not successful, the database professional has agreed to attempt Option #3, merging the two databases. Final Thoughts I apologize for being so detailed and technical about the process, but I want to be transparent about the process because it's been long enough that I owe you an explanation, and because it is a serious issue. It's complicated and unfortunate. This post isn't intended to freak anyone out about losing content, and I realize that if we lost the entire Community Interaction category it'd be devastating. I don't think this will happen. I'm confident this problem will be fixed and will be by the end of this week (hopefully sooner). Please be patient. I'll make updates as the process continues. After this is behind us, I am making some changes to prevent anything like this happening again in the future: Scheduling automatic daily database backups Testing changes to the software on a test installation before a live update Updating our server to the latest hardware and software Implementing additional security protocols to protect the site from other risks, such as spam and hacking Optimizing the forum's technical operation so it runs faster and more efficiently TL;DR An upgrade to a small part of the forum's software may have deleted or hidden the Community Interaction category. There were other issues that caused a delay in a fix. This is a serious issue with some risk but it looks like we can fix it. Options include restoring from an outdated backup, finding a way to repair the database, or merging the backup with the current site (all have benefits/risks). Today, I'm testing a fix on a test installation of the website and I am consulting with a database professional who can help us repair and restore the data. I expect this to be solved earlier than the end of the week. We're taking steps to prevent anything like this happening again in the future. Thank you for your patience! - @Daniel
Welcome to the WF Development Blog. We are currently working on major long-term and short-term plans for WritingForums.org. The intent of this blog is to provide previews of planned improvements, to gather feedback on proposed changes, and to engage the community in the development of this forum. This blog will act as a development roadmap, and as a way to vet new ideas before implementation. This blog may also feature tutorials, information on behind-the-scene operations, and policy changes. Currently, We already have a small development team that proposes, discusses, and helps implement new ideas. This blog is an extension of that development process. If you’re interested in helping shape WritingForums.org over the next year, please follow this blog. There will be sneak peaks, but you can also have a voice in deciding the future of this website. Expect more details on what we're working on next week. Since this is the first post, here’s a small glimpse of what we’re working on and considering: [Implementing] Optimizing forum design for better usability, speed, and new user registration [Planned] Overhaul of the Article and Blogging sections [Design & Development] Implementation of a Private and Group Critiquing System Expect more details on a weekly/bi-weekly basis. Thank you for reading.