How to Scrape Amazon Product Data: Names, Pricing, ASIN, etc.
Amazon offers numerous services on their ecommerce thing they do not offer though, is easy access to their product ’s currently no way to just export product data from Amazon to a spreadsheet for any business needs you might have. Either for competitor research, comparison shopping or to build an API for your app scraping easily solves this Amazon Web ScrapingWeb scraping will allow you to select the specific data you’d want from the Amazon website into a spreadsheet or JSON file. You could even make this an automated process that runs on a daily, weekly or monthly basis to continuously update your this project, we will use ParseHub a free and powerful web scraping that can work with any website. Make sure to download and install ParseHub for free before getting raping Amazon Product DataFor this example, we will scrape product data from ’s results page for “computer monitor”. We will extract information available both on the results page and information available on each of the product tting StartedFirst, make sure to download and install ParseHub. We will use this web scraper for this ParseHub, click on “New Project” and use the URL from Amazon’s result page. The page will now be rendered inside the raping Amazon Results PageOnce the site is rendered, click on the product name of the first result on the page. In this case, we will ignore the sponsored listings. The name you’ve clicked will become green to indicate that it’s been rest of the product names will be highlighted in yellow. Click on the second one on the list. Now all of the items will be highlighted in green.
On the left sidebar, rename your selection to product. You will notice that ParseHub is now extracting the product name and URL for each product.
On the left sidebar, click the PLUS(+) sign next to the product selection and choose the Relative Select command.
Using the Relative Select command, click on the first product name on the page and then on its listing price. You will see an arrow connect the two selections.
Expand the new command you’ve created and then delete the URL that is also being extracted by default.
Repeat steps 4 through 6 to also extract the product star rating, the number of reviews and product image. Make sure to rename your new selections Tip: The method above will only extract the image URL for each product. Want to download the actual image file from the site? Read our guide on how to scrape and download images with have now selected all the data we wanted to scrape from the results page. Your project should now look like this:Scraping Amazon Product PageNow, we will tell ParseHub to click on each of the products we’ve selected and extract additional data from each page. In this case, we will extract the product ASIN, Screen Size and Screen, on the left sidebar, click on the 3 dots next to the main_template your template to search_results_page. Templates help ParseHub keep different page layouts separate.
Now use the PLUS(+) button next to the product selection and choose the “Click” command. A pop-up will appear asking you if this link is a “next page” button. Click “No” and next to Create New Template input a new template name, in this case, we will use product_page.
ParseHub will now automatically create this new template and render the Amazon product page for the first product on the list.
Scroll down the “Product Information” part of the page and using the Select command, click on the first element of the list. In this case, it will be the Screen Size item.
Like we have done before, keep on selecting the items until they all turn green. Rename this selection to labels.
Expand the labels selection and remove the begin new entry in labels command.
Now click the PLUS(+) sign next to the labels selection and use the Conditional command. This will allow us to only pull some of the info from these items.
For our first Conditional command, we will use the following expression:
$(“Screen Size”)
We will then use the PLUS(+) sign next to our conditional command to add a Relative Select command. We will now use this Relative Select command to first click on the Screen Size text and then on the actual measurement next to it (in this case, 21. 5 inches).
Now ParseHub will extract the product’s screen size into its own column. We can copy-paste the conditional command we just created to pull other information. Just make sure to edit the conditional expression. For example, the ASIN expression will be:$(“ASIN”)
Lastly, make sure that your conditional selections are aligned properly so they are not nested amongst themselves. You can drag and drop the selections to fix this. The final template should look like this:Want to scrape reviews as well? Check our guide on how to Scrape Amazon reviews using a free web, you might want to scrape several pages worth of data for this project. So far, we are only scraping page 1 of the search results. Let’s setup ParseHub to navigate to the next 10 results the left sidebar, return to the search_results_page template. You might also need to change the browser tab to the search results page as on the PLUS(+) sign next to the page selection and choose the Select command.
Then select the Next page link at the bottom of the Amazon page. Rename the selection to next_button.
By default, ParseHub will extract the text and URL from this link, so expand your new next_button selection and remove these 2 commands.
Now, click on the PLUS(+) sign of your next_button selection and use the Click command.
A pop-up will appear asking if this is a “Next” link. Click Yes and enter the number of pages you’d like to navigate to. In this case, we will scrape 9 additional pages. Running and Exporting your ProjectNow that we are done setting up the project, it’s time to run our scrape the left sidebar, click on the “Get Data” button and click on the “Run” button to run your scrape. For longer projects, we recommend doing a Test Run to verify that your data will be formatted the scrape job is completed, you will now be able to download all the information you’ve requested as a handy spreadsheet or as a JSON ThoughtsAnd that’s it! You are now ready to scrape Amazon data to your heart’s why stop there? With the skills you’ve just learned, you could scrape almost any other out our guides you may be interested in:How to scrape data from Yellow Pages How to scrape data from to use a data extraction tool to scrape AutoTraderScraping Rakuten dataBetter yet, become a certified Web Scraping expert with our free courses! Enroll for free today and get your certificates! Download ParseHub for freeThis post was originally published on August 29th, 2019 and last updated on November 9th, 2020.
How to Scrape Amazon Product Data: Names, Pricing, ASIN, etc.
Amazon offers numerous services on their ecommerce thing they do not offer though, is easy access to their product ’s currently no way to just export product data from Amazon to a spreadsheet for any business needs you might have. Either for competitor research, comparison shopping or to build an API for your app scraping easily solves this Amazon Web ScrapingWeb scraping will allow you to select the specific data you’d want from the Amazon website into a spreadsheet or JSON file. You could even make this an automated process that runs on a daily, weekly or monthly basis to continuously update your this project, we will use ParseHub a free and powerful web scraping that can work with any website. Make sure to download and install ParseHub for free before getting raping Amazon Product DataFor this example, we will scrape product data from ’s results page for “computer monitor”. We will extract information available both on the results page and information available on each of the product tting StartedFirst, make sure to download and install ParseHub. We will use this web scraper for this ParseHub, click on “New Project” and use the URL from Amazon’s result page. The page will now be rendered inside the raping Amazon Results PageOnce the site is rendered, click on the product name of the first result on the page. In this case, we will ignore the sponsored listings. The name you’ve clicked will become green to indicate that it’s been rest of the product names will be highlighted in yellow. Click on the second one on the list. Now all of the items will be highlighted in green.
On the left sidebar, rename your selection to product. You will notice that ParseHub is now extracting the product name and URL for each product.
On the left sidebar, click the PLUS(+) sign next to the product selection and choose the Relative Select command.
Using the Relative Select command, click on the first product name on the page and then on its listing price. You will see an arrow connect the two selections.
Expand the new command you’ve created and then delete the URL that is also being extracted by default.
Repeat steps 4 through 6 to also extract the product star rating, the number of reviews and product image. Make sure to rename your new selections Tip: The method above will only extract the image URL for each product. Want to download the actual image file from the site? Read our guide on how to scrape and download images with have now selected all the data we wanted to scrape from the results page. Your project should now look like this:Scraping Amazon Product PageNow, we will tell ParseHub to click on each of the products we’ve selected and extract additional data from each page. In this case, we will extract the product ASIN, Screen Size and Screen, on the left sidebar, click on the 3 dots next to the main_template your template to search_results_page. Templates help ParseHub keep different page layouts separate.
Now use the PLUS(+) button next to the product selection and choose the “Click” command. A pop-up will appear asking you if this link is a “next page” button. Click “No” and next to Create New Template input a new template name, in this case, we will use product_page.
ParseHub will now automatically create this new template and render the Amazon product page for the first product on the list.
Scroll down the “Product Information” part of the page and using the Select command, click on the first element of the list. In this case, it will be the Screen Size item.
Like we have done before, keep on selecting the items until they all turn green. Rename this selection to labels.
Expand the labels selection and remove the begin new entry in labels command.
Now click the PLUS(+) sign next to the labels selection and use the Conditional command. This will allow us to only pull some of the info from these items.
For our first Conditional command, we will use the following expression:
$(“Screen Size”)
We will then use the PLUS(+) sign next to our conditional command to add a Relative Select command. We will now use this Relative Select command to first click on the Screen Size text and then on the actual measurement next to it (in this case, 21. 5 inches).
Now ParseHub will extract the product’s screen size into its own column. We can copy-paste the conditional command we just created to pull other information. Just make sure to edit the conditional expression. For example, the ASIN expression will be:$(“ASIN”)
Lastly, make sure that your conditional selections are aligned properly so they are not nested amongst themselves. You can drag and drop the selections to fix this. The final template should look like this:Want to scrape reviews as well? Check our guide on how to Scrape Amazon reviews using a free web, you might want to scrape several pages worth of data for this project. So far, we are only scraping page 1 of the search results. Let’s setup ParseHub to navigate to the next 10 results the left sidebar, return to the search_results_page template. You might also need to change the browser tab to the search results page as on the PLUS(+) sign next to the page selection and choose the Select command.
Then select the Next page link at the bottom of the Amazon page. Rename the selection to next_button.
By default, ParseHub will extract the text and URL from this link, so expand your new next_button selection and remove these 2 commands.
Now, click on the PLUS(+) sign of your next_button selection and use the Click command.
A pop-up will appear asking if this is a “Next” link. Click Yes and enter the number of pages you’d like to navigate to. In this case, we will scrape 9 additional pages. Running and Exporting your ProjectNow that we are done setting up the project, it’s time to run our scrape the left sidebar, click on the “Get Data” button and click on the “Run” button to run your scrape. For longer projects, we recommend doing a Test Run to verify that your data will be formatted the scrape job is completed, you will now be able to download all the information you’ve requested as a handy spreadsheet or as a JSON ThoughtsAnd that’s it! You are now ready to scrape Amazon data to your heart’s why stop there? With the skills you’ve just learned, you could scrape almost any other out our guides you may be interested in:How to scrape data from Yellow Pages How to scrape data from to use a data extraction tool to scrape AutoTraderScraping Rakuten dataBetter yet, become a certified Web Scraping expert with our free courses! Enroll for free today and get your certificates! Download ParseHub for freeThis post was originally published on August 29th, 2019 and last updated on November 9th, 2020.
Scrape product information from Amazon | Octoparse
The latest version for this tutorial is available here. Go to have a check now!
In this tutorial, we are going to show you how to scrape the product information from
To follow through, you may want to use this URL in the tutorial:
We will enter each detail page of Bluetooth Headphones and scrape the details including the product title, brand, rating, and price.
This tutorial will also cover:
Deal with AJAX for pagination
Here are the main steps in this tutorial: [Download task file here]
“Go To Web Page” – to open the targeted web page
Create a pagination loop – to scrape all the results from multiple pages
Create a “Loop Item” – to loop click into each item on each list
Extract data – to select the data for extraction
Start extraction – to run the task and get data
1. “Go To Web Page” – to open the targeted web page
Click “+ Task” to start a new task with Advanced Mode
Advanced Mode is a highly flexible and powerful web scraping mode. For people who want to scrape from websites with complex structures, like, we strongly recommend Advanced Mode to start your data extraction project.
Paste the URL into the “Extraction URL” box and click “Save URL” to move on
Turn on the “Workflow Mode” by switching the “Workflow” button in the top-right corner in Octoparse
We strongly suggest you turn on the “Workflow Mode” to get a better picture of what you are doing with your task, just in case you mess up with the steps.
2. Create a pagination loop – to scrape all the results from multiple pages
Click “Next” button
Click “Loop click next page” on “Action Tips”
Set up AJAX Load for the “Click to paginate” action
applies the AJAX technique to the pagination button. Therefore, we need to set up AJAX Load for the “Click to paginate” action.
Uncheck the box for “Retry when page remains unchanged (use discreetly for AJAX loading)”
Check the box for “Load the page with AJAX” and set up AJAX Timeout as 10 seconds
Click “OK” to save
3. Create a “Loop Item” – to scrape all the items on each page
Click “Go To Web Page” to go back to the first page
When extracting data throughout multiple pages, you should always begin your task building on the first page.
Click the name of the first product on the current page
Click “Select all” on the “Action Tips” panel
Octoparse will automatically select all the links to the detail pages on the current page. The selected links will be highlighted in green while other links to the detail pages will be highlighted in red.
Click “Loop click each element” to create a “Loop Item”
Octoparse will click through each link captured in the “Loop Item”, and open the detail page.
Tips!
If you want to learn more about AJAX, here is a related tutorial you might need：
Deal with AJAX
4. Extract data – to select the data for extraction
After you click “Loop click each element”, Octoparse will open the detail page of the first hotel.
Click on the data you need on the page
Select “Extract text of the selected element” from the “Action Tips”
Rename the fields by selecting from the pre-defined list or inputting on your own
When the content of the page has already shown out, but it is still loading, you could click the “X” button at the right end of the navigating bar to stop loading.
5. Save and start extraction – to run the task and get data
Click “Start Extraction” on the upper left side
Select “Local Extraction” to run the task on your computer, or select “Cloud Extraction” to run the task in the Cloud (for premium users only)
Here is the sample output. You can see some blank fields in the column “Price”. This is because these products are out of stock and thus they don’t have the price information.
By default, if Octoparse cannot find the element of the defined pattern on the page, the field will be left blank. However, Octoparse may fail to find the element of the defined pattern even if the element needed is shown on the website. If you encounter this problem, here are a related tutorial you might need：
What to do with those blank fields I got in the extracted result?
Happy data hunting!
Was this article helpful? Contact us at any time if you need our help!
Frequently Asked Questions about how to scrape amazon
Can Amazon be scraped?
Free Amazon Web Scraping Web scraping will allow you to select the specific data you’d want from the Amazon website into a spreadsheet or JSON file. You could even make this an automated process that runs on a daily, weekly or monthly basis to continuously update your data.Nov 9, 2020
How do I scrape an item on Amazon?
Scrape product information from Amazon”Go To Web Page” – to open the targeted web page.Create a pagination loop – to scrape all the results from multiple pages.Create a “Loop Item” – to loop click into each item on each list.Extract data – to select the data for extraction.More items…•Jul 15, 2021
Is it hard to scrape Amazon?
It is no news that Amazon has been at the forefront of the e-commerce industry, for quite some time now. Retailers fight tooth and nail to scrape data from Amazon. However, Amazon data scraping is not easy! Let us go through a few issues you may face while scraping data from Amazon.Oct 27, 2020
