Q: How to improve the speed of data extraction?

 

 

     A:

Actually, the speed of data extraction is mainly determined by several factors.

1. The speed of your computer's CPU(Central Processing Unit), especially when you are using Local Extraction.

2. The amount of the data you plan to crawl

3. The time to load the web pages

4. Free trial or paid accounts (Standard Edition/Professional Edition)

5. Local Extraction or Cloud Extraction

6. ...

If the data you want to extract is not that much, Octoparse can finish your job very quickly even though you’ re using local extraction.

But if the amount of the data you want is very huge and you would like to speed up the extraction task, Cloud Extraction would be strongly recommended.

Besides, you can also improve the speed of data extraction by choosing the right way to configure your rule. That is, sometimes it's better to separate one rule into two to speed up the extraction. For some websites, we'd suggest that you use the “URL list” loop to extract information out of multiple web-pages with similar layout. So the task will be split into many sub-tasks when you use Cloud Extraction and our cloud server will scrape the web pages and collect the data for you much quicker than using Local Extraction.

 

Check out this article: To scrape data from multiple web pages/URLs

 

 

 

If you have any question, we'd happy to help.

Octoparse Support Team

btn_sidebar_use.png
btn_sidebar_form.png
当社ウェブサイトは、利便性、品質維持・向上を目的に、Cookieを使用しております。詳しくはプロキシーをご確認ください。Cookieの利用に同意頂ける場合は、「同意する」ボタンを押してください。同意頂けない場合は、ブラウザを閉じて閲覧を中止してください。
同意する 閉じる