Q: How to scrape detail page data with pagination?




When the task executes, the action (the blue textbox) of the workflow are executed in order.

If only one action is in a loop/cycle, then all the items extracted in the loop/cycle will execute the action before going to the next action. Similarly, if there are two actions in a loop/cycle, Octoparse will execute these actions orderly for all the items extracted in the loop/cycle.

So when we drag the second "Loop Item List" box into the first "Cycle Pages" box and place it right above the "Click to paginate" action in the Workflow Designer, Octoparse will get all the data required from the current page, then click the "Click to paginate" action to go to the next page, and will scrape the data of all the items from that page.



同意する 閉じる