Your guide to a better future

From cordless vacs to self-emptying robot vacuums, we test them all to help you find the best fit. Here's how we do it.

Labs Manager / Senior Technical Project Manager

I am the Labs Manager for CNET's Home Division based in Louisville, KY. My interest in technology began in the early '90s, and soon after I began my double major in computer science and computer engineering. I've worked in many areas, including computer hardware, software, technology, networking, graphic design, instruction, music and even ballroom dancing! 65% Ron Swanson, 25% Ben Wyatt, 10% Andy Dwyer.

We test all sizes and shapes of vacuums here at CNET -- full-sized models , lightweight cordless alternatives , and the most high-tech of the bunch, robot vacuums . Our testing, which takes place at the CNET test lab in Louisville, Kentucky, helps us recommend the best vacuums for your home -- and it can also be pretty fun. So here's a peek behind the curtain at our setup and process for testing the wide variety of different vacuums available. 

The main test we use on all vacuums is known as a straight-line test. Yes, it's mostly just vacuuming, but we do a lot of calculations to make sure it's a fair test for all the models that come through our lab. Our process is closely aligned with the standard established by the International Electrotechnical Commission. The goal of a straight-line test is to measure what percentage of dirt the vacuum is capable of picking up. We use three types of dirt (we call it soil when testing): black rice, sand and pet hair. We run the test on three different types of flooring: hardwood, high-pile carpet and low-pile carpet.

Our pet hair test is the simplest. We use 2 grams of humanely collected pet hair spread across each test bed and take before and after photos for visual comparison. We call this an anecdotal test -- we're really just comparing how many clumps of hair each vac can pick up. 

For black rice and sand, we start by weighing the vacuum's empty dustbin. Then we adjust the width of this rig we made based on the unit's nozzle size. The goal is to equalize the soil density, which is the amount of mass per surface area of floor, for all of the different nozzle widths. The length of our test bed stays the same. We change the width so that it's one inch less than the vacuum's nozzle length. That is so that the vacuum has plenty of room to go over the entire width of the soil in one pass. Think of it like giving it a half inch margin of error on either side. 

Our rig to distribute soil across the test bed.

Then, we weigh out a certain amount of soil based on a simple equation with width as the variable. For each vacuum, we want a soil density of 125 grams per meter squared of soil on carpet -- a number established by the IEC's dust removal test. We use 40% of that for hardwood. 

Then, we vacuum! We have each machine do one pass in a straight line, covering our test area. Then we weigh the dustbin again, and calculate the percentage of the initial soil that has been picked up by the vacuum. To make sure the dustbin, and hence the vacuum, is not keeping soil inside, we clean it and weigh it after each run, making sure that this amount equals our initial dustbin weight. 

Rinse and repeat three times per floor type for rice and sand, but only once per floor type for pet hair -- again, it's more anecdotal. But all of that adds up. Between all the cleaning and measuring, just testing one unit equals a total of 21 runs.  

We do all of that for every single vacuum we test, and that serves as our main form of testing for any type of user-operated vacuums. 

For robot vacuums, we do all of the above testing, but we're just getting started. Our second test for robot vacuums is a navigation test. This test is important because you need to know which model is going to clean your living room the quickest and which is going to cover every spot while it works. Different models use wildly different techniques to navigate, and it's one of the ways that higher-end models separate themselves. 

Cheaper models navigate with random bumping. They run in a straight line until they bump into something and then change direction. They basically just keep going until the battery dies. 

We watch what robot vacuums do inside the test room closely.

Higher-end vacuums use lasers that shoot out of a turret to map out their surroundings. Then the navigation software uses this info to tell the vac where to go and where not to go. You can usually see the maps these models create in corresponding apps. Some models use images of the ceiling as well, but in general, the pricier models work with an actual plan and it results in much more efficient work. 

We can see all of that in action in this room that we built to mimic a real-life home layout from the point of view of a robot vacuum. It likely looks strange to you, but to a vacuum, these represent table legs, chairs and other furniture to navigate around. The size and dimensions of the room also reflect what other national and worldwide test bodies use for robot vacuum navigation testing. 

Our goal is to use long-exposure shooting to assess and compare the navigation efficiency of each unit. We mount an LED bar on top of the robot vacuum in the same location and the same size as the cleaning nozzle on the bottom, and turn off all of the lights in the lab. We have a high-resolution camera attached to the ceiling that captures all movements of the robot vacuum as it navigates across the room, avoiding obstacles, stopping, turning and doing its best to reach into every spot. 

We end up with images like these. Below is an example of a low-end vac using bump navigation. On it, not only can you see how much of the room the robot vacuum has covered, but also its navigation pattern and the duration of its run, all of which are taken into account when comparing units to one another. Notice that the vac has pretty good coverage, but there's no discernible pattern to how it works. 

You can see that this robot vacuum navigates by bumping into walls and moving in another direction. 

Compare that to this image below of a higher-end vac at work. Here, you can clearly see a systematic pattern. Coverage is the most important thing we assess. You want your vac to get to every spot, but systematic navigation makes a big difference. The first vac completed the test in 55 minutes. The second one covered the same area in 24 minutes. 

This higher-end robot vacuum maps the floor with lasers to clean more efficiently. 

These tests represent the total of our standardized vacuum tests. Occasionally products will offer unique features that encourage us to do additional testing. Read our more in-depth look at how we test robot vacuums for more information on our pet waste detection features. Or, watch the affectionately titled poop test in action in the video below. 

That's our vacuum testing, complete with lots of math and lasers. We turn all of the data we obtain in the lab into performance charts, so we can see results side-by-side and understand which vacs are the best at which tests. And we play with plastic poop so you don't have to mess with smears of the real stuff, all to help you make an informed purchase decision when it's time to drop the big bucks on a new piece of tech.