Mobile Website Design: How We Did It

May 17, 2013 by davidmoeller Leave a comment
Twitter Facebook

The Mission: Mobile Site!

Here at CodeGuard things tend to move pretty quickly. We’re made up of a team of highly driven individuals that like to build things that we know will positively impact our users and build them fast! We soon realized that not only do we move fast, but our users do too. More and more we realized that we were getting a lot of traffic to our website from mobile devices, and our users were increasingly requesting features that were optimized for mobile (if not built to be on mobile first – desktop second).

We have been steadily optimizing the CodeGuard website into a responsive design for a while now, but it became clear to us that it was time for a different strategy: A mobile-first design. It’s not that there is anything seriously wrong with a responsive design. RWD (Responsive Web Design) is an increasingly popular way to serve mobile users a version of your site that can fit their devices and allow them to access the same content that they would on a desktop website. There are some major caveats though. When you serve the same exact content to desktop and mobile visitors in the same exact way, regardless of what device they are on, you risk slow load times and giving your mobile users the content that maybe they don’t want to see.

For example, a restaurant may have large images of their food at the top of their desktop website but a user visiting that same website from their phone may not want to be served large images that are slow to load. Maybe all they wanted was to find that one tiny link that says “Directions” so they could get to that restaurant. If you don’t think about what your users want from you out of your mobile experience then you are bound to give them an experience that is unsatisfactory. One needs to think about why users are visiting your site from a mobile device, and here at CodeGuard we set out to find those answers and create a multi-device mobile-first design that best suited our users needs.

The Timeline/Deadline: Two Weeks?!?!?!?!

As always, we strive to push ourselves to new limits at CodeGuard. We gave ourselves two weeks to plan, research, ideate, wireframe, design, prototype, develop, test, and launch a completely new mobile homepage for CodeGuard.com – all while still taking care of the other on-going projects that we had going on at the time.

The Research/Planning:

So how did I get the idea to try a mobile-first design? A lot of my inspiration for this project came from one spectacular event I went to in Atlanta called “An Event Apart.”

Not only were the speakers amazing at what they presented on, but the community that showed up was really fun to interact with and meet. I highly recommend that you attend one in your city if you get the chance! It was here that I got a lot of inspiration and ideas for how to re-imagine CodeGuard’s mobile experience. I learned that a mobile-first design where you started from the smallest screen size and worked your way up was the best way ensure that your site would look good on any screen size. I learned that you should let the content decide where the layout “breaks” into a new layout and not the device it is on (which is a great way to make sure your design is device agnostic) and I learned a bunch of creative and inventive ways on how to best optimize your content and layout for these small screen sizes.

Mobile website - An Event Apart

With this newfound inspiration I took to researching good design and development practices. For my design research I sought out to answer the questions: How is navigation typically laid out on mobile websites? Who does it well? What do I like about this mobile website? What do I not like about this other mobile website?

Development planning was a slightly larger task. Because we were going with a mobile-first design this meant that we were going to be creating a separate mobile website – while still maintaining our desktop website for a time period. I did a lot of research on this topic, and going to An Event Apart helped me reach this decision in the end. The cons involved us supporting two codebases for a short period of time, and figuring out a graceful solution for how we would handle device detection and redirects when they landed on our homepage, but the pros far outweighed the cons in the end. We would be able to start fresh with our content and give our users the content we know they want, instead of just serving them what is the same on the Desktop site. We would also be eliminating a lot of potential “code bloat” by starting over with a new codebase that is purely optimized for mobile from the start, instead of the other way around. It also has the potential to be much faster than our existing website.

In the end we decided to create our mobile homepage as a stand alone static website built on a front-end tool called Middleman while creating our html/css using haml and sass, but I’ll get into more detail on how I developed the site in moment!

The Wireframing/Designing/Prototyping:

So to begin, what was our goal with this new mobile homepage? What did we want to accomplish? What do we want our users to see? What do we think they want to see? To answer these questions, we decided in the end that our goal would be to “start a relationship with the mobile user.”

I started sketching wireframes on my whiteboard at home and broke up the homepage into a few sections that I thought would help us achieve this.

Mobile website - Planning

The top section would answer the question that every homepage – mobile or not – should answer, “Who are we, and what do we do?” Right away there were some challenges though, because the way our current homepage does this is by having some text and then a large video. While everyone here at CodeGuard loves our homepage video, it takes up a lot of space when translated directly into a small viewport size. Space is already very precious on a mobile website. I tried to keep the navigation and logo area just a small strip at the top of the screen so that it only takes up about 10% of the screen on load, and then the rest of the 90% of the screen can be all content answering the “Who are we, and what do we do?” question.

Mobile website - Planning
Our video could have easily taken up all of that 90% though, so I had to think differently about displaying it. The end result is that it is hidden until requested to be watched. This ended up saving us precious bytes as well, since we are not loading it until it is asked for (but I’ll get to speed optimization and other development wizardry later in the post).

The next section would answer the question, “What can we do for you?” I went back and forth with this section on whether I wanted to show a story of how CodeGuard works, or what our three main features were. I decided to go with our three main features (Backup, Monitor, Restore) in the end, hoping that if people liked what we had to say there, they could find out the “behind the scenes” stuff on a different page.

The third main section (one that cannot be seen in my whiteboard sketches because I added it later) is a client testimonial section. We can tell our mobile users all day long about how perfect we think we are for them, and how important our services are (and they are!), but hearing it a second time from a source other than us is always a good thing too!

The final section of our mobile homepage is a section where the user can enter their email address and get a special discount code emailed to them that they can use on any plan. We liked the idea of our mobile users checking out our website from their mobile device, or wherever they may be, but also getting the opportunity to browse our desktop site at a later time. We thought the email would be a great way to remind the user that they visited us, while also giving them a great incentive to try us out!

After deciding on all of this it was time to hit up Photoshop. Below I’ve included a few screenshots of things that I was designing, scraping, re-designing and polishing throughout this phase.

ThreeScreenView2

The Development/Collaboration:

Now that I was ready to develop it was time to start playing with my fun new tool Middleman. Middleman is a static site generator that came with a lot of nice features out of the box for those familiar with Ruby (which is what we primarily wield here at CodeGuard). It already had support for haml and sass, which were two tools I’ve come to love here at CodeGuard, and it had a wicked fast server for building your project and compiling your tools into whatever output you needed (html, css, javascript). It also did a lot of neat compression things for you when you start the project/server up so it would keep our site fast. When it came to the client testimonial section of the homepage I had a lot of cool ideas to involve jQuery mobile swipe functionality and other neat touch interactions, but I was fast approaching on my two week deadline. In the end I decided to save some of the more “bells and whistles” type coding for version 2.0 and focus on the nuts and bolts haml and sass coding that needed to be done. For testing the site as I was developing I used a Ruby gem called localtunnel to be able to view the site on my mobile devices and a neat Chrome plugin called OSX Resizer to let me resize my browser window on my laptop smaller than 400px.

The Testing/Finishing Touches:

I saved the last two to three days of my two week deadline for testing because I wanted to be as thorough as possible. The most popular mobile browser being used today is Safari, followed by Android default browser, and then Chrome. The fourth most popular was Opera Mini (my personal mobile browser of choice at the moment) but the drop off from Chrome to Opera in use was so severe that it was almost not worth testing for (but I did anyway). The Android default browser had the most issues to work out, mainly around the video functionality and pop-up messages surrounding the form, but overall testing was not nearly as painful as it was when I was doing responsive design testing. This made me believe even stronger that I had done things right this time around, and had approached this project the right way.

Launch:

With help from our Director of Engineering Jonathan Manuzak, we were able to solve all server redirect details quickly and get our mobile website out and in front of our eager users. Two weeks went by very quickly, but with all the planning and research that went into this project I was very confident that we were doing things very right. I think that we stayed true to our goal in the end while never forgetting our users in the process. In two weeks we were able to critically think about the content our users want from our homepage and give it to them in a way that is easy to use, fast to access, and fun to interact with. But don’t take my word for it, go see for yourself! Visit codeguard.com on your mobile device of choice and make sure to let us know what you think!

- Natalie

 

→ Leave a comment

Posted in Behind the Scenes

404 Redirect Hosted Scam Sites: WordPress Impacted

May 9, 2013 by davidmoeller 2 Comments
Twitter Facebook

We have reason to believe that a new type of attack – WordPress 404 Redirect Hosted Scam Sites – could be impacting millions of WordPress websites. Several weeks ago we posted on a Yahoo! Mail XSS or CSRF attack that hijacked Yahoo! Mail accounts and sent emails to address books without the user’s knowledge. We explained that the emails were just the first step in an elaborate phishing or scam attack. The crucial second step, we explained, was the landing page on a compromised website that entices the victim.

In attempting to book a dinner reservation, I stumbled across a compromised WordPress Site (3.5) that seems to be a part of the same scam. But this compromise is different, in that any URL input will return the scam landing page. We think that this type of website compromise – the 404 Redirect Hosted Scam Page may be related to the WordPress Brute Force attacks from mid-April. A WordPress site with compromised credentials could be violated at any point in time, and the website owner would not know it.

The worst type of web compromise, or any compromise, is one that you don’t know is occurring. If you are not aware of the compromise, you will not take steps to end it. There is no site scanning system on the market that is catching these hacks. I will repeat this as it is important. McAfee SiteAdvisor, Symantec’s Safe Web, and Google Safe Browsing do not detect and will not help you. You will not know if your site has been compromised. Slightly disconcerting?

Why should you care?

This is important because if you own a website, you are responsible for how it is being used or abused. You are liable for the business asset that is your website. Your website is intended to help your business in some manner, or perhaps it is your personal space on the interwebs to blog and share with the world. Either way, your website could be a part of scam. Right now. Just like a high-end Atlantan restaurant that is redirecting customers to weight loss pills.

WordPress 404 Redirect Hosted Scam Sites - Top Floor Restaurant Website

But the site is fine if I view it and don’t see anything weird, right? Maybe not. Look at what happens if we add anything to the URL that directs us to a page not explicitly defined by the webserver (meaning the page isn’t there). Normally, we would expect to see a 404 page stating that the page we are looking for can’t be found. When we do the same thing with topflr.com, and enter “www.topflr.com/test” the following screenshot is what is returned. Notice that the title of the page has changed to “ConsumerLifestyles” and the content looks completely foreign and irrelevant to fine dining in Atlanta.

WordPress 404 Redirect Hosted Scam Sites - Compromised Website Page

Look carefully at the URL – “http://topflr.com/test”. Hmmmmm. What’s happened? What has happened is that someone has taken advantage of a highly profitable corner of the internet – URL typos and missing pages. John Biggs explains best, referring to the outcry when Network Solutions attempted to monetize these pages circa 2004:

“These DNS servers are absolutely vital to the Internet. When Network Solutions, the arbiters of US websites, changed the nature of DNS translations by adding a ‘search page’ when users typed in incorrect addresses, the tech world protested loudly. This simple change destroyed hundreds of hours of work in the community and commercialized the Internet to a degree that made many gurus uncomfortable. Network Solutions quickly removed the search page, meaning that visitors who typed the URL www. groosble. com by mistake will get an error instead of a Network Solutions-sponsored web page and advertisement.”

John Biggs. Black Hat: Misfits, Criminals, and Scammers in the Internet Age (Kindle Locations 850-853). Kindle Edition.

What is happening is that miscreants are taking advantage of a very known and profitable portion of internet real estate: 404 pages. They are using random chance to aid in driving unsuspecting website visitors to diet pill scam sites. Have you ever typed in the wrong address to a website and seen a 404 page? How can you figure out if your site is infected. Let’s take a look at some of the tools you might try to use.

Garcinia Cambogia: Is your site infected?

Google has some handy functions that allow you to search on content within your site, using the “site:” command. The key here is that these pages have been indexed by Google. Unfortunately, they might not work with this hack, if Google hasn’t indexed a particular page, or if the page does not yet exist – meaning the php file has not rendered it for Google to index, because the unwary site visitor has not yet mistyped the address? Confusing? I’ll explain how the 404 Redirect Hosted Scam Page works in just a moment.

WordPress 404 Redirect Hosted Scam Sites - Google site content search

How does it work?

What is known right now, thanks to the Sucuri team, is that two key files are necessary for this hack to work. “Wpppm.php” and “.k”. The wpppm.php file, when any 404 requests are made, loads the content in the “.k” directory. Right now, that content points to Garcinia Cambogia diet pills, but it could change.

What is the impact?

Something has caused the extreme spike in websearches for “Garcinia Cambogia”. Press releases, and Dr. Oz mentions would fuel increased interest, but nothing like what is seen in the Google Trends chart below. Dr. Oz talked about Cambogia first around October 2012. The small bump in the below chart can be most likely attributed to the mention on his show. But what, then, has caused the massive surge in interest? A press release? Anyone who has published a press release can tell you that simply publishing one will not generate the type of interest seen below. It is likely that some type of mass spam email and website hacking has taken place to fuel this surge.

Google Trends Garcinia Cambogia

WordPress 404 Redirect Hosted Scam Sites Could Impact Millions

WordPress websites, visited by hundreds of millions of people globally, are being compromised. The exact vector is unknown at this point. I’m using “vector” to mean the way that the site was compromised – if someone guessed your WordPress password & username, for instance, it can be said that they exploited a credentials management vulnerability.

What should you do? Search your wp-content (and the rest of your WordPress installation) via FTP for “wpppm.php” and “.k”. And keep searching for those files everyday. Or you can try CodeGuard, which will notify you via a ChangeAlert if anything on your website has changed.

-David 

→ 2 Comments

Posted in Backup, ChangeAlerts, Hacked websites

Website Backup (Files) Server Load Testing

April 30, 2013 by davidmoeller Leave a comment
Twitter Facebook

CodeGuard Host Server Load Testing
Version: 1.0 / Date: 5 April, 2013

Overview

The goal of this test is to determine what impact the CodeGuard website backup service has on a host server during a backup. The results contained in this document were gathered after a series of four tests (described later). They are by no means conclusive and real-world results will vary based on the composition of the websites being backed up and the hardware and software running the underlying host server.

Test Methodology

There are two phases of a backup that are of particular interest with respect to host server performance: file listing and downloading. The first, file listing, entails inspecting the metadata for each file and folder within the scope of the backup. Practically, this means recursing through all of the folders in the target website and inspecting every file. The downloading phase simply transfers all new or modified files from the host system to the CodeGuard service. For the initial backup, this means downloading every file in the site, but for subsequent backups only new or changed files are downloaded. These two steps are always performed sequentially.

Definitions

Activation: The first backup performed by CodeGuard. All files within the backup scope will be inspected and all files will be downloaded.

Pull: All backups performed after the initial Activation. All files within the backup scope will be inspected, but only new or changed files will be downloaded.

Test Website

Only one website was used for this suite of tests. In cases where concurrent backups were taking place, multiple copies of the site were used to avoid simultaneous requests for the same files on disk.

Website Statistics
Size: 538MB
File and Folder Count: 8,774
Type: Real, active WordPress blog

Test Host
Server: The host used for testing was a RackSpace CloudServer.
OS: CentOS 6.3
FTP Server: Pro-FTP
Memory: 512MB
CPU Cores: 1

Website Backup Testing Results

The graphs below illustrate the results of each test. Following each are notes discussing the findings.

Metrics and Definitions

  • CPU usage: System: Percentage of CPU usage by system processes.
  • CPU usage: User: Percentage of CPU usage by user processes.
  • % Memory Used: Percentage of system memory used.
  • eth0 in: Network transfer in from the public network connection in KB/s.
  • eth0 out: Network transfer out to the public network connection in KB/s.
  • Server Load (Last 5 Minutes): The numeric representation of the load on the system for the last five minutes. This is an amalgamation of different metrics but, for this system, loads less than 1.0 are acceptable. More information can be found here: http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages
  • I/O Average Queue Size: Weighted number of milliseconds spent doing I/Os. This can provide a measure of both I/O completion time and the backlog that may be accumulating.
  • I/O Wait: Time in milliseconds spent waiting to perform I/O operations.
  • I/O Reads / second: Number of file system reads per second.
  • I/O Writes / second: Number of file system writes per second.

1. One Activation – FTP

Website Backup 1a
Figure 1. Server load graph.

Website Backup 1b
Figure 2. Server I/O graph.

Notable Times
File Listing Start/End: 6:50AM/7:05AM
Download Start/End: 7:05AM/8:00AM

As shown in Figure 1, CPU usage is slightly elevated for the duration of the process, but never exceeds 10% for this single core system. Server load oscillates at the beginning of the file listing process as well as during and after the file downloading phase of the backup, but remains relatively low (< .1). We can see that the eth0 out metric corresponds to the file download times, peaking at ~750 KB/s of outbound network traffic. Memory usage increases marginally during both phases, presumably due to the FTP server managing the listing and file transfer operations. The I/O profile in Figure 2 shows an increase in latency during the download phase of the backup. At the peak (7:45am), there is a > 4.5ms I/O wait time. The baseline for this system before the test began was between 0.5ms and 1ms, so there is an obvious increase, but the real-world impact of an increase of this magnitude is unknown. Finally, Reads increase during the download phase, but largely stay below 10 reads/sec, peaking briefly at 20 reads/sec.

2. Five Concurrent Activations – FTP

Website Backup 2a
Figure 1. Server load graph.

Website Backup 2b
Figure 2. Server I/O graph.

Notable Times
File Listing Start/End: 12:15PM/12:30PM
Download Start/End: 12:30PM/1:25PM

The profile for five concurrent FTP activations follows the same general trends as the single FTP activation discussed previously. The metrics peak at slightly higher values, but multiple concurrent FTP activations do not appear to cause a linear increase in resource consumption.

3. One Activation – SFTP

Website Backup 3a
Figure 1. Server load graph.

Website Backup 3b
Figure 2. Server I/O graph.

Notable Times
File Listing Start/End: 7:05AM/7:30AM
Download Start/End: 7:30AM/10:00AM

In Figure 1, we can see that CPU usage is negligible for the duration of the process. Server load oscillates during both the download and listing phases, but remains low overall (< 0.1). As expected, eth0 out is elevated for the majority of the downloading phase, peaking at more than 1,500 KB/s. There is an anomaly from 8:05am to 9:55am, where the download speed is very low. Further investigation is necessary here to determine the root cause of this delay.

In Figure 2, we can see that there is an increase in I/O Wait Time during the file listing process but, like the FTP activation, it stays below 5ms. The Average Queue Size peaks over 200 during the download phase which is higher than the FTP activation tests. It’s difficult to say if this is a result of our testing or if we were competing with another VM on this host for disk resources.

4. Five Concurrent Activations – SFTP

Website Backup 4a
Figure 1. Server load graph.

Website Backup 4b
Figure 2. Server I/O graph.

Notable Times
File Listing Start/End: 2:55PM/3:10PM
Download Start/End: 3:10PM/4:05PM

As shown in Figure 1, the CPU, Memory and server load metrics follow the same profile as a single SFTP activation. Aside from the Average Queue Size, all of the other metrics are right inline with the single SFTP activation. The Average Queue Size for multiple concurrent SFTP activations peaks lower (< 125) than the single activation (which was > 200). It’s difficult to say if this is a result of the backups running slower because of the concurrency or if there was another VM on this same host using more disk resources during this test.

As with the single SFTP activation, it does not appear to cause a linear increase in resource consumption.

Conclusion

These tests produced some interesting results, but they are fairly high-level and were only performed once. As a result, they should be viewed with some amount of skepticism.

The three biggest takeaways that were found during this analysis are:

  1. Multiple concurrent activations do not produce a linear increase in server resource consumption for either SFTP or FTP.
  2. The profile of SFTP and FTP backups are very similar in terms of network, memory, CPU and I/O load.
  3. The most load on the server is during the download phase. For CodeGuard, that should mean the Pulls should not put nearly as much strain on the servers as Activations.

After this round of testing, there are still some other open questions, like:

  • What is the real world (i.e. user) impact of the elevated load and I/O wait times?
  • Would using dedicated hardware change the outcome?
  • Do the performance characteristics change for much larger sites and longer-running jobs?
  • Would using a different FTP server (Pure FTP vs Pro FTP) have made a difference?

- Jonathan Manuzak, Director of Engineering

→ Leave a comment

Posted in Backup, Behind the Scenes, Website Backup

Tagged ,

Database backup load analysis

April 26, 2013 by admin Leave a comment
Twitter Facebook

CodeGuard Database Backup Load Testing
Version: 1.1 / Date: 25 April, 2013

Overview

The goal of this test is to determine what impact the CodeGuard service has on a host server during a MySQL database backup. The results contained in this document were gathered after a series of four tests paralleling the tests previously done on FTP and SFTP backups by Jonathan Manuzak (see other document). They are by no means conclusive and real-world results will vary based on the composition of the websites being backed up and the hardware and software running the underlying host server.

Test Methodology

From the perspective of the CodeGuard backup service, there is a single phase that occurs on a remote server: running the ‘mysqldump’ command. Running this command includes using the MySQL client, either remotely or tunneled over SSH, to simultaneously extract the database tables to a flat file format and transfer that file from the remote server to the CodeGuard service. Currently ‘mysqldump’ commands are run with the options ‘–quick’, ‘–single-transaction’, and ‘–skip-extended-insert’. These configure the operations one row at a time, attempt to ensure consistent state of some types of tables, and avoid multiple-row insert statements, respectively. Executing a mysqldump with different options may result in different performance characteristics.

Pull Definition

Pull: All database backups are of this type. The entire database is downloaded in via the ‘mysqldump’ command, committed to a git repository, and uploaded to Amazon S3. Unlike website file backups, incremental downloads are not supported by mysqldump, so the entire database is downloaded from the remote server each time, although the backup is ultimately incremental since it is stored as a commit in a git repository. Test Database Only one database was used for this suite of tests. In cases where concurrent backups were taking place, the same database was downloaded simultaneously. While a somewhat contrived test setup, this to some approximation simulates other processes accessing database tables concurrently with backups.

Test Database

Only one database was used for this suite of tests. In cases where concurrent backups were taking place, the same database was downloaded simultaneously. While a somewhat contrived test setup, this to some approximation simulates other processes accessing database tables concurrently with backups.

Database Statistics
Size: 2723 MB
Row Count: 856262 (21 tables)
Type: Real MySQL database from language-learning website.

Test Host
Server: The host used for testing was a RackSpace CloudServer.
OS: CentOS 6.3
MySQL Server: 5.1.67
Memory: 512MB CPU Cores: 1

Database Backup Testing Results

The graphs below illustrate the results of each test. Following each are notes discussing the findings.

Metrics and Definitions

  • CPU usage: System: Percentage of CPU usage by system processes.
  • CPU usage: User: Percentage of CPU usage by user processes.
  • % Memory Used: Percentage of system memory used.
  • eth0 in: Network transfer in from the public network connection in KB/s.
  • eth0 out: Network transfer out to the public network connection in KB/s.
  • Server Load (Last 5 Minutes): The numeric representation of the load on the system for the last five minutes. This is a unitless amalgamation of different metrics but, for this system with a single core processor, loads less than 1.0 are acceptable. Loads above 1.0 indicates that processes are waiting for CPU access. More information can be found here: http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages
  • I/O Average Queue Size: Weighted number of milliseconds spent doing I/Os. This can provide a measure of both I/O completion time and the backlog that may be accumulating.
  • I/O Wait: Time in milliseconds spent waiting to perform I/O operations.
  • I/O Reads / second: Number of file system reads per second.
  • I/O Writes / second: Number of file system writes per second.

1. One Backup – MySQL Direct Connection

Database Backup Load 1
Figure 1a. Server load graph. View on ScoutApp

Database Backup 1b
Figure 1b. Server I/O graph. View on ScoutApp

Notable Times
MySQL Dump Start/End: 6:55AM/7:22AM

As shown in Figure 1, mysqldump is neither CPU-intensive or memory-intensive for a single database backup. Server load remains below 0.1. I/O is affected more significantly, as mysqldump is a read-intensive operation. Reads peak at 55/s with latency topping 7 ms. Ethernet output (eth0 out) averages between 1500 and 2300 kB/s. The entire process completes in 27 minutes.

2. Five Concurrent Backups – MySQL Direct Connection

Database Backup 2a
Figure 2a. Server load graph. View on ScoutApp

Database Backup 2b
Figure 2b. Server I/O graph.  View on ScoutApp

Notable Times
MySQL Dump Start/End: 10:28AM/1:33PM

Concurrent backups show a similar pattern with the main impact of concurrency causing all backups to take more time. Peak latency peaks at 13 ms, higher than the single backup, but peak reads/s are actually lower likely due to inefficiencies introduced by multiple processes competing for I/O access leading to more HD seeking. Network output also varies between 1500 and 2300 kB/s with a sharp break at half an hour into the backup lasting for 5 minutes. I do not yet have a good explanation for the pause in the backup process. Network output also lags the start of the backups by about 20 minutes. This means that transferring of the mysqldump file to CodeGuard is not occurring during this time.

The entire backup process completes in 3:05. This is 1.37 times the length of 5 consecutive 27 minute backups, giving an idea of the performance impact of the competition for the shared database resource. However, server performance does not appear to be significantly impacted.

3. One Backup – MySQL Tunneled over SSH

Database Backup 3a
Figure 3a. Server load graph.View on ScoutApp

Database Backup 3b
Figure 3b. Server I/O graph.View on ScoutApp

Notable Times
MySQL Dump Start/End: 9:28AM/9:53AM

A single backup tunneled over SSH mirrors the performance of a MySQL direct backup closely for all metrics. Completion time is very slightly faster at 25 minutes, although this was not tested for reproducibility. See discussion of the single MySQL direct backups for more information of the overall metrics.

4. Five Concurrent Backups – MySQL tunneled over SSH

Database Backup 4a
Figure 4a. Server load graph. View on ScoutApp

Database Backup 4b
Figure 4b. Server I/O graph.View on ScoutApp

Notable Times
MySQL Dump Start/End: 1:47PM/4:37PM

Multiple concurrent MySQL backups tunneled over SSH perform slightly better than concurrent MySQL direct backups. Notably, the pause in the process seen at 40 minutes in direct backups does not occur.  Peak I/O wait time is 6.3 ms vs 13.2 ms for MySQL direct. Network average transfer speeds are lower, possibly due to compression on the SSH tunnel as well as the lack of a break in the process. Despite the lower network speeds the backups still completed slightly faster at 2:55 vs. 3:05 for MySQL direct.  Similarly to the MySQL direct test, the concurrent backups completed in 1.4x the time of 5 sequential 25-minute backups.

Conclusion

These tests of database backup performance produced solid high-level information on the performance metrics that predict performance of a mysqldump, as well as the load it places on a server, but given the simulated nature of the test system they must be viewed with some skepticism.

Given the above caveat, analysis of the test results shows that:

  • MySQL database backups are almost entirely I/O bound with very little CPU or memory usage.
  • MySQL direct and SSH tunnel behavior is almost identical for single backups.
  • Concurrent MySQL backups tunneled over SSH are slightly faster than MySQL direct backups and appear to have better sustained performance and lower impact on remote server performance.
  • There is a slight penalty to running concurrent backups of the same database vs. sequential backups.

Further questions not fully addressed by this testing:

  • What is the real world (i.e. user) impact of the elevated load and I/O wait times?
  • Would using dedicated hardware change the outcome?
  • What is the impact of backups to real world MySQL database performance. Does backing up certain tables cause significant slowdown in database performance that would impact user experience?
  • Would different MySQL dump options cause different performance profiles and are there ways to optimize the process further with compression of the data either over SSH tunnel or with MySQL?
  • It is possible that the limiting factor is network bandwidth in some phases of the backup. It is unknown what the impact to I/O performance would be in a system with a higher network bandwidth to I/O bandwidth ratio. 

-Randall McPherson, Sr. Engineer

→ Leave a comment

Posted in Backup, Behind the Scenes, CodeGuard How It Works

Tagged , ,

Website Hacked? Would You Know It?

April 25, 2013 by davidmoeller 1 Comment
Twitter Facebook

Is Your Website Hacked?

Your website might be hacked right now. “My website hacked?”, you think. Yes, your website. “But I’m not a major media outlet, I don’t process billions of e-commerce transactions, and I’m not a federal agency”, you say. If you have a website, it may have been compromised. “But if my site was hacked, it would show up on Google’s blacklist, right?” Wrong. “But if my site was hacked, the hackers would deface it and I would see it, right?” Wrong. “But if my site was hacked, my web developer would know it, right?” Wrong. “But my host protects me, right?” Wrong.

Website Hacked

You are alone and your website is your responsibility. Your website is valuable because of the reputation you have established with online registries and because it sits atop a webserver that can be manipulated. To explain why someone would want to hack your quiet corner of cyberspace, I will use an analogy, comparing your website to a cement plant. I’m not joking.

Your Website is Like a Cement Plant

Think about it this way – your website is like a cement factory connected to a power plant. Please follow along – I promise it won’t be too painful.

The cement factory is known in the community and has established relationships. It is trusted. People have bought cement from it, it is in the yellow pages, and therefore, viewed as a legitimate business entity – because it is one.

Powering the cement factory is a power plant, on the premises. This power plant produces electricity that is used for crushing rocks and keeping the lights on at the plant, but that electricity could be used for anything, if it was accessible.

If a criminal broke into the plant and found a way to harness the electricity of the power plant, would he want to be found? If that criminal figured out a way to sell that electricity on the open market, would he ever want the cement plant to know? Of course not – he would want to utilize those assets for his purposes as long as possible.

Cement Plant Similar to Hacked Website

Should a cement plant be commandeered, it could look similar to a website hacked. I realize this is a tough analogy – but give me just a bit more to explain. The reputation of the cement plant and the goodwill it had built in the community would allow the criminals to continue to operate it, and as long as they didn’t post signs that they had taken over the plant, no one would know. Contrast this with the criminals building a new fake cement plant. Permitting, construction, time, energy, and money would be poured into the new facility. And even once it was finished, if it didn’t produce cement, people would doubt its authenticity. Building and running a cement plant is hard work, and it is tough to fake!

The crucial first step to repurposing the all important power plant at the cement plant is taking over the plant and tricking everyone in the community into believing the plant has not been taken over. Imagine if cement plant could be overthrown without the employees even knowing. This happens everyday to a website hacked. The server administrator doesn’t know. The webmaster doesn’t know. Website visitors don’t know. The problem is that no one knows it has been compromised other than the criminals.

Now the cement plant has been taken over – but surreptitiously. And now the criminals run cables to the power plant on site, and start selling that energy on the open market. Buyers of the energy use it to power factories to illegally construct weapons, illicit drugs, etc. The point is that if you buy energy from an overthrown cement plant, you are most likely up to no good.

What are the criminals doing with my hacked website?

Right now, millions of websites are compromised. And their owners do not have a clue. This is because the website is valuable only as long as its owner does not realize it has been compromised.

Criminals Are Using Your Reputation

Establishing a trusted reputation for a URL & IP address doesn’t happen overnight, so spammers rely on websites with trusted reputations to elude spam filters and deceive recipients into clicking on the links within them. By using your website to redirect victims of phishing attacks, you become part of the problem. Your reputation is used as part of the scam, and all without you knowing it. The recent WordPress brute force attacks are an example of the first step – compromising your site without your knowledge. Once the brute force attacker figures out your username and password, do you think they are going to change it? Not a chance.

Criminals Are Using Your Webserver

The server that powers your website isn’t that different than the laptop or desktop you use everyday. It is just dedicated for a single purpose, which is responding to requests as visitors to your website make them with clicks of the mouse or form submissions. Your webserver can be used in a multitude of ways; think how many different ways your laptop or desktop can be used – and extend this logic to your webserver. To list a few: your hacked website could be used to execute a DDOS (distributed denial of service) attack on your government, companies, or individuals. DDOS isn’t complex; just imagine Amazon’s website around Christmas. Lots of visitors, and lots of requests. DDOSing isn’t nice. It is an attack, and your hacked website could be part of a DDOS right now. Your hacked website could also be used for URL redirecting.

URL redirecting is when a criminal uses your hacked website to take visitors to another website of the criminal’s choice. An example is below.

Hacked Website Used in Gmail Scam Spam

This is happening millions of times everyday. HST Egypt is a security products company, and right now it has a hacked website that is a part of a weight-loss diet pill scam.

What Can I Do?

At CodeGuard, we are sick and tired of this virtual blight, and aim to stop it. What can you do? Sign up for CodeGuard, right now. We are the only way you can know if your site has been compromised via our ChangeAlerts. And we don’t leave you high and dry at that point. If your site is compromised, we fix it with the click of a button, and then we will patch whatever security vulnerability your site has. Just. that. simple.

Ignorance is bliss. But now you know. Do your duty, and ensure that your website isn’t part of the problem, but part of the solution.

→ 1 Comment

Posted in Hacked websites, Small Business

Tagged