How to scrape product details from Amazon Books with a bot
If you're looking to improve your product offerings or marketing strategies, it's important to gather data on bestselling books, popular genres, and emerging trends. Additionally, keeping tabs on your competitors' product listings, pricing strategies, and customer reviews can give you a competitive edge.
In this guide, we show you how to extract data from Amazon Books using a bot. You’ll learn how to scrape URLs from a list of top-selling books, and then filter them by ranking, and extract their Best Sellers Rank.
Don't worry if this example doesn't match your use case, because what you will learn in this guide will enable you to scrape any data from Amazon. For example, you can use the basis of this bot to scrape data from other product types, such as electrical goods or home furnishings.
# What exactly is a bot?
Axiom’s bots are powerful tools that you can delegate browser tasks to. Think of them as your very own digital helpers. They’re built using a simple step-by-step process - without the need for code - that replicates your actions in the Chrome browser, allowing you to automate a whole range of data gathering tasks in the Amazon store.
The beauty of learning how to create bots is that they can be tailored to your specific requirements and scheduled to run independently, freeing up valuable time. They can even automate Amazon posts.
# How does the Amazon Book scraper bot work?
The Amazon Book scraper bot mimics your actions in the browser. It navigates to a URL and extracts the links to books before proceeding to loop through the links to gather information on the best-selling rankings. Finally, it outputs the data and saves it to a Google Sheet. This guide teaches you specifically how to scrape Best Sellers book guides on Amazon.
# Let’s learn how to build an Amazon Book scraper bot with Axiom.ai
Creating bots for Amazon scraping has never been easier thanks to Axiom's no-code bot builder. With our user-friendly no-code interface, you can create as many bots as you need without any coding knowledge. By following the simple steps below, you can make a fully functional bot that extracts Amazon data within minutes.
# 1. Start from blank
To build your bot from scratch, click on 'Add first step’. This will open the step selector and you can start adding steps to your bot.
# 2. Add your first step: ‘Get data from a URL’
Use the Step Finder to search for ‘Get data from a URL’ and click on it. The step will be added to Axiom for you to configure.
In the field called 'URL', enter the URL of the Amazon Book listing page you wish to scrape data from. In this guide, we will be scraping best sellers in Women's Fantasy Fiction.
Next, we need to select the data we want to scrape. In this case, we require two columns of data: column A will contain the book listing number, and column B will contain the link to the book page.
For column A, by clicking ‘Select’ select the book ranking number on the first book, then repeat the action on subsequent books. Tip: don’t just select books 1, 2 and 3. Instead, scroll down the page and select Books 1, 5 and 20. This helps our AI understand the selectors.
Then click on ‘Add column’ to add column B. Use the ‘Link selector’ to click on the box around the books, again repeating the selection as above.
Press ‘Complete’ to save your selections. Your data output should look something like this:
Now we need to set up the pager by clicking ‘Find pager’. Scroll down the Amazon page and select the pager button ‘Next page’ then click ‘Complete’.
Then click on ‘Confirm selector’ and then ‘Complete’. Lastly, set ‘Max results’ to 100.
# 3. Remove results which do not contain certain words
In this example, we only want to scrape books listed at numbers 1, 20 and 50, so we’re going to pass the URLs we just scraped through a filter to remove all unwanted URLs.
Use the Step Finder to find and add the step ‘Remove results which do not contain certain words’.
In the ‘Data’ field, select ‘Scrape data’.
In ‘Words (comma separated)’ type 1,20,50.
In ‘Word matching mode’, select ‘Any’.
Select the ‘Match on word boundary’ box.
Finally, we only want to use the first column for our filter as the URLs also have numbers in. In ‘Columns (optional)’ switch the toggle on and set to 1.
# 34. Add the ‘Loop through data’ step
Use the Step Finder to find and add the step ‘Loop through data’. This step allows your bot to loop through the links we just scraped. Be sure to add the steps you want to repeat inside the loop as substeps.
# 5. Add the ‘Go to page’ sub step
Staying within the ‘Loop through data’ step, it’s time to add a sub step (click the white ‘Add step’ button).
Use the Step Finder to search for ‘Go to page’ and click on it. The step will be added for you to configure.
In the 'Enter URL' field, click ‘Insert data’ and select ‘[matching-word-data?*&1]’.
In the data preview that appears, select only column ‘B’ with the URLs.
# 6. Add the ‘Current URL’ sub step
No configuration is needed for this step. It simply retrieves the current URL.
# 7. Add the ‘Get data from bot's current page’ sub step
For this guide, we want to scrape the best seller ranking, but of course you can make different data selections. Start by clicking ‘Select’ and then scroll the Amazon page until you see the Bestseller ranking. Then click on it, and click complete.
Leave Max results set to ‘1’.
To learn more about the selector tools, follow this link.
# 8. Add the ‘Write data to a Google Sheet’ sub step
Now we’re going to write the data that we extracted from Amazon to a Google Sheet.
Use the Step Finder to search for 'Write data to a Google Sheet' and click to add it.
In the 'Spreadsheet URL' field, search for the Google Sheet you wish to use.
In the 'Sheet name' field, select 'Data'. In the 'Data' field, select '[interact-data]'.
Finally, in the ‘Write options’ field, click the ‘Add to existing Data’ option.
If you do not wish to use a Google Sheet you could output to a CSV or Display Message - just add the relevant step.
# 9. Ready to test
Almost there! Your Amazon bot should resemble the screenshot below.
Now you’re ready to test your Amazon scraper with the desktop app, by clicking ‘Run w/desktop app’.
# 10. Running the bot
You can run this Amazon bot in both the cloud and the desktop app. If you want to learn more about scheduling, click here. (opens new window)
# Issues you may encounter with the Amazon bot
If you’re experiencing any problems with running the bot, a great way to debug is by using ‘Display messages’ to examine the data at various stages of its transition through the bot.
For example, you could add a ‘Display messages’ step after a scraper to check that the data being scraped is output correctly.
Here are some other issues you may experience:
- ‘Remove results’ step not returning any data? Check that ‘Columns (optional)’ is set to 1.
- The bot isn’t looping through the book page links? Check that the ‘Go to page’ step has the data inserted properly.
- Amazon page doesn’t load when running the bot? Try using just the desktop app and running it locally.
Don't forget we offer excellent customer support. If you need help, get in touch. (opens new window)
Congratulations, you've learned how to build a bot to extract data from Amazon. With this newly acquired skill, you know how to scrape data, loop through actions, and extract data to a Google Sheet. You can make bots to scrape any website - the sky's the limit with your new AI super powers.