Bot blocking

A number of websites use automated bot blocking technologies to protect themselves against malicious attacks. Unfortunately, harmless bots can be caught in the crossfire. Cloudflare is a particularly widely-used bot blocking technology that will often automatically block automated activity of any kind.

Cloudflare and other such services detect activity that is fully automated, but will not block activity that it sees as coming from a legitimate user. By attaching your axiom run to an existing Chrome window which has already validated you as a real user, you can reasurre the bot blocker that the traffic is not malicious in nature.

# Download and install the axiom desktop application

To use this method of bypassing bot detection, you will need to download and install the axiom desktop application.

You can find it here (opens new window).

# Select the method to bypass bot blocking

By default, axiom does not attempt to bypass bot detection. This is because the checks slow down automations.

To turn on bot blocking, open the axiom which is being blocked from the dashboard.

Click the three dots menu in the top-right of the Axiom builder, then click on the gear icon ‘Settings’ and select the 'Bypass bot detection' tab in the left panel.

There are two options to choose from:

  • Use automatic bot detection bypassing
  • Attach to existing Chrome

# Use automatic bot detection bypassing

When this mode is switched on, axiom will automatically detect and solve bot challenges where possible.

Although this works for many sites, it is not infallible; sometimes you may need to solve the captcha yourself. There are two ways, depending on the Captcha type.

The first is to use the captcha solving step. This is required for more complicated captchas. See more information here.

The best way to do this is to use the "Press Key(s)" step:

  • Record Tab the correct number of times to highlight the checkbox
  • Record 'Space' to complete the challenge

# Attach to existing Chrome

When this method is selected, you need to give Chromium permission to use your window, or axiom will not be allowed to connect.

A button to quickly open such a window can be found in the "Bypass bot detection" section of the settings page.

Alternatively, you can start your own version of Chromium manually.

To do this, an extra argument needs to be passed to Chromium when starting it. For example, you can enter this in your terminal (altering the path to your Chromium executable as required):

/path/to/Chrome/chrome --remote-debugging-port=21222

You can also create a shortcut or symbolic link with this parameter specified, for easier access.

# Solve the bot blocker's challenges

After you've started your new window, open it and navigate to the website which is deploying bot blocking, and navigate to the page or section of the website where the bot blocking is taking place.

The website will now validate whether you are a real user or not. Often this happens automatically; sometimes you will need to solve a Captcha.

Once you've been validated, we're ready to let axiom take over the browser!

# Adjust your axiom for bypassing bot detection

Depending on the automation, you may need to make some adjustments to your axiom so that it starts in the correct place. The best way to maximise the chances of being allowed in is to make sure your automation is set to start on the page you have bypassed bot detection on.

For example, if this is after login (a common occurrence) then perform the login manually on your open browser window, then adjust your automation so that it performs only the steps after the login.

Once you click the "Run" button, Axiom should begin to run these steps in the window which has already solved the challenges.

Note: axioms running in this mode do not close the Chrome / Chromium window when they are stopped; instead the automation will finish what it is doing and then stop running.