Why Website accessibility tests aren't perfect

April 14, 2025

When you have a website that you think is accessible, you want to test it to see what you may have missed. There are numerous ways to do that, however none of the tools catch absolutely everything.

How to do a test?

First, let’s rewind a bit and talk about how to test a website for accessibility. Usually these tests happen when you have completed a website and want to fix any accessibility issues, or you are constantly testing as the website is being made. No matter which way you do it, there are tons of companies which offer the ability to run a test. Currently, the tools I use to test a website are WAVE by WebAIM, WebAIM’s contrast checker, and WhoCanUse.

At first glance, you may think that all of these are for websites that are already viewable by anyone. However, that’s not the case, WAVE has a thing you can add to Firefox, Chrome, or Edge that will provide results if you can view the website in said browser. The contrast checker and WhoCanUse are ones that are where you can input the colour and see how it looks, which allows you to see how it looks and make adjustments as needed just on that site.

The results

After you run the test and get the results, you go through them to see what needs to be changed. You may see some easy fixes and some you are unsure how to solve. While you can go through the list of issues and fix them, I think it’s better to read into each issue so that you can know why it’s coming in, and how to work to ensure that error doesn’t come up in your future projects.

If you are unsure how to solve an issue, and the tool you are using doesn’t provide an understandable explanation, then consider searching online, or asking someone who may know more than you. While searching online can be hard to know exactly what to search for, there can be tons of results, written by many people. If you consider asking someone who may know more than you, be sure to mention that you’ve tried searching it, and don’t expect a response. The main reason for not getting a response is that the person most likely has many people asking them questions and that they aren’t able to answer everyone.

False positives

Some results could come up as false positives, where the tests say something should be fixed. However, someone working on the site knows why it’s the current way, and why it shouldn’t be changed in order to satisfy some test.

One great example of this is having an image just for decoration, then having text underneath that says where the image is from (because the license requires you to do so). The W3C Web Accessibility Initiative says because the image is for visual, there shouldn’t be any alt text on it. There’s also the option of also hiding the text underneath from a screen reader. As the image is decoration, so a screen reader would skip over the image, and without knowing there’s an image, the text underneath would be confusing.

There are many opinions on this and that could lead to a false positive, where the test tool says that both of those should be available to the screen reader, and there should be alt text. But, someone creating the website wants to ensure the assistive technology user isn’t confused when on the site.

That doesn’t mean you should ignore every error and say it’s a false positive. In fact, you should almost never think there’s a false positive until someone whose worked on the site has let you know it is and why they have done so.

What accessibility guidelines are being used?

The first web content accessibility guidelines (WCAG) become endorsed by W3C and encouraged everyone to follow in May 1999, and since then there have been many updates and versions. As of the time of writing, there is version 2.2. There are also many other guidelines depending on where in the world your company is based. Such guidelines include European Accessibility Act, or Accessible Canada Act. If you are unsure what guidelines to follow, then it’s worth it to do some research.

What guidelines are your tools using? Many times that information is either not publicly available or hidden. For example, I had to go past a couple headings on WAVE’s help page in order to see what they use. Other tools might just say they “using the latest guidelines”, then you look at it and see it hasn’t been updated in years.

Years old guidelines isn’t necessarily incorrect, from what I’ve read, the newer versions organize the content and update it to reflect the feedback from accessibility experts. There are laws or acts’ that reference certain versions. While I understand that is done because that may have been the current version at the time it was created, I believe we should always reflect what the latest version says or is.

I’ll admit, I don’t often check to see what guidelines a tool is using, however we should get into the habit of doing so.

Tests don’t catch this

What do you put in the alt text of images? There’s no perfect alt text, which makes it hard for both people and tools to say if the alt text work for those who need it or not. While there are many helpful resources that can provide good directions (some of which I’ve listed in the additional links), the best way to get better is to continually write it.

Most tests from a tool just involve looking at the page, like what happens when the user pushes the submit button without filling in the form, or submits information that is invalid. An automated accessibility test won’t catch that. Most of the time this is caught when manual testing is done.

Does the location of everything make sense on the page? Like, should the menu be in a different order to make it more accessible? Results from a tool won’t tell you, but manual testing may pick it up.

From my research, it doesn’t look like tools such as WAVE and Google Lighthouse test to ensure everyone can see and click a button or link. Why this is so important? Because it allows more people to be able to use the website. Let’s say someone has sight issues, or is using a smaller device. With small button’s it’s hard to be able to submit, which means people can get frustrated and go elsewhere. While WCAG does have a minimum size for buttons, the best guidance is the bigger, the better.

While big is good, it can really help to have space between each button and link. This not only allows someone to be able to clearly click on it, but also see and be sure they click want they want to. For some sites this will result in having to put less on the page, however that’s not always a bad thing.

One more thing related to links and buttons that tools may not catch, allow the customer to choose if a website opens in the same tab or a new one. Forcing a new tab or window can confuse someone and if they decide to use the back button and a link opens in a new tab, it won’t result in going back to your website.

Taking a look at what others have written about accessibility tests, I notice one thing is either briefly mentioned or not mentioned at all, and I don’t see on accessibility tools, that is the writing. Not only making sure it can be seen in terms of size, but also that it’s understandable by your audience.

Your audience could be everyone, or it could be narrowed down, like the audience for this blog post that I’m aiming for is those who use accessibility tools for whatever website. Which means I’m writing in a way that I hope everyone who fits that understands what is being said and doesn’t have to stop reading to search something because they don’t understand a point.

If your website is aimed for a more general audience, you may be asked to ensure what’s on site is at a certain grade reading level. That is difficult, as there are many ways that reading level is tested, from at least 12 listed on Wikipedia. How then should we test to ensure everyone can understand it? What I like to do is get someone who is completely unfamiliar with the company, and get them to read the site, and summarize to someone what the site or contents is about. All without doing any other research or visiting any other pages. Both the Australian Government and the Province of British Columbia have good guides for plain language.

How about dark mode with a black background and white text? This is also known as dark mode, and there are people who use it to make your website more accessible to them. They could have light sensitivity, want to reduce eye strain, or simply prefer it over a white background. While turning on dark mode usually results in no issues, there may be certain sections that aren’t readable. This can be due to either the site forcing certain colours with no changes, or changing the colour to a darker one is unreadable. I would encourage you to test your site in dark mode and do your best to make it just as accessible as those who don’t use it. You may be thinking that the site should be in dark mode by default, however as Stéphanie Walter says so well in her blog post, let users choose.

Other ways to test

Besides testing with the accessibility tool you use most often, there are other ways you can test websites.

I know what you’re thinking, I’ll just learn and use another tool. While that can help to ensure one tool doesn’t become the dominate one, it may test against the same thing’s resulting in the same results. Even if you use multiple or every tool, they won’t catch everything.

Another tool that you may not think about is all the different browsers and devices that exist. While I test on the latest version of Firefox on my desktop, how does your site look on a mobile device, or a different browser like Iridium Browser? If you are able to use a different operating system or screens, then that can also help to spot any potential accessibility issues that may need a fix. Sometimes there are no issues, but sometimes there are issues that need to be fixed.

Many companies focused on accessibility mention usability or user testing, which does sound a bit weird when you first read it. That’s when someone who fits your target audience (someone who does and doesn’t have have any accessibility issues is usually best) comes in, and you give them a specific goal to accomplish on your website. Why this is something to consider? It’s to ensure see about any confusion and know how to fix it, and that any other issues are solved.

Yale University has some great steps for this, such as having a goal-based task and writing up a scenario with layman’s terms. Of course there are many other ways to make the experience of doing this better, the United States Government Section 508 also has a number of tips. It isn’t expected for every website owner to make their own team of people that does this for them, and there are many companies which can do this testing for you, I would highly suggest finding a company either local to you or local to where your target audience is. If you need some idea of what to search for to find that, then may I suggest seeing the Equalize Digital website.

Challenge to the accessibility and website community

Accessibility is never complete, and that applies to websites as well. It can get frustrating continuing to make a website accessible, but think about how someone with a disability feels. If someone can’t properly use a website, then they get frustrated that yet another website won’t put in the time to fix it, and they will go to another website, which could have been a potential customer.

If you’re serious about fixing the accessibility issues, and have the time to do so, then I suggest using multiple tools to test and fix all the errors that each one shows. This can take more time depending on how the tool checks your site, what needs to be fixed, and ensure that it’s actually accessible.

Keep learning. I know this can either be obvious or something you don’t want to do since you spend all day doing it. Yes there’s people you can follow on social media, sites you can add to your rss feed, or even challenging yourself to do something outside your usual day-to-day work.

Verify information against multiple websites. It’s easy to take what one website says as fact, but it could actually be worst for accessibility. While some say that’s confirming your own bias, I believe it’s ensuring the best experience for everyone. This also ensures that the source of the information is accurate. One thing about accuracy that isn’t common, is to check who owns the site, or what other sites are owned by the company that runs the one site. For example, I’ve stopped reading The Bureau of Internet Accessibility because it’s owned by AudioEye, which is known to make a website worst for accessibility. As well as Level Access because they acquired UserWay which is very similar to AudioEye in making a website less accessible.

If those companies are already known to have inaccessible products, why should we trust what they have to say about accessible? We can’t. They may publish incorrect information, and that’s why we have to verify against multiple sites, and ensure the information is correct.

Signal that we are trying with accessibility?

Now that you know how tools aren’t 100% effective, how do you signal on your website that your company is trying and welcomes feedback?

Usually you would put an accessibility page on your site, mention what accessibility guidelines you are aiming for, with a link to it, and that you are continuously working to make the site accessible, along with multiple contact methods that someone can use if they have any feedback or something is inaccessible to them (and you’ll provide it to them in an accessible way).

That is what most businesses do on their website, and I encourage you to write it in a friendly way that everyone can understand, and if the website is for a business, get multiple people such as the legal department to read through it to ensure there are no issues.