Async puppeteer browser disconnect after few iterations

Images
Related searches

I tested iterations with puppeteer in a small case. I already have read the common reason for puppeteer disconnections are that the Node script doesnt wait for the puppeteer actions to be ended. So I converted all functions in my snippet into async functions but it didnt help.

If the small case with six iterations work I will implement it in my current project with like 50 iterations.

'use strict';

const puppeteer = require('puppeteer');

const arrIDs = [8322072, 1016816, 9312604, 1727088, 9312599, 8477729];

const call = async () => {
    await puppeteer.launch().then(async (browser) => {
        arrIDs.forEach(async (id, index, arr) => {
            await browser.newPage().then(async (page) => {
                await page.goto(`http://somelink.com/${id}`).then(async () => {
                    await page.$eval('div.info > table > tbody', async (heading) => {
                        return heading.innerText;
                    }).then(async (result) => {
                        await browser.close();
                        console.log(result);
                    });
                });
            });
        });
    });
};

call();

forEach executes synchronously. replace forEach with a simple for loop.

const arrIDs = [8322072, 1016816, 9312604, 1727088, 9312599, 8477729];
const page = await browser.newPage();

for (let id of arrIDs){
    await page.goto(`http://somelink.com/${id}`);
    let result = await page.$eval('div.info > table > tbody', heading => heading.innerText).catch(e => void e);
    console.log(result);
}

await browser.close()

On browser Disconnect, page doesn't get resolved to new URL , Steps to reproduce Tell us about your environment: Puppeteer version: When after running few url's , either the page crashes or browser disconnects. browser crash browser.on('disconnected',async()=>{ browser = await� The only signal we get when the browser crashes is the disconnect of web socket (see lib/Connection.js). We should capture this event and dispatch the disconnected for it This comment has been minimized.

The way you've formatted and nested everything seems like some incarnation of callback hell.

Here's my suggestion, its not working, but the structure is going to work better for Async / Await

const puppeteer = require("puppeteer");

const chromium_path_706915 =
  "706915/chrome.exe";

async function Run() {

    arrIDs.forEach(
        await Navigate();
    )
  async function Navigate(url) {
    const browser = await puppeteer.launch({
      executablePath: chromium_path_706915,
      args: ["--auto-open-devtools-for-tabs"],
      headless: false
    });

    const page = await browser.newPage();

    const response = await page.goto(url);

    const result = await page.$$eval("div.info > table > tbody", result =>
      result.map(ele2 => ({
        etd: ele2.innerText.trim()
      }))
    );


    await browser.close();
    console.log(result);
  }
}

run();

executionContextId is 0 when using multiples connections � Issue , ConnectAsync hardkoded/puppeteer-sharp#856 wsEndpoint(); for (let i = 0; i < 100; ++i) { console.log('Iteration #' + i); const browser = await helper1');}); await page.goto("https://www.google.com"); browser.disconnect(); } theBrowser.close(); })(); Source/puppeteer/lib/FrameManager.js:333:5) at Page. @karthigb we strongly recommend using async/await when working with Puppeteer; this should make migration much easier. The only case where you can't use async/await is running in Node6, but Node6 will be end-of-lifed in half a year, so I'd suggest to not use it for anything new.

On top of the other answers, I want to point out that async and forEach loops don't exactly play as expected. One possible solution is having a custom implementation that supports this:

Utility function:

async function asyncForEach(array: Array<any>, callback: any) {
  for (let index = 0; index < array.length; index++) {
    await callback(array[index], index, array);
  }
}

Example usage:

const start = async () => {
  await asyncForEach([1, 2, 3], async (num) => {
    await waitFor(50);
    console.log(num);
  });
  console.log('Done');
}

start();

Going through this article by Sebastien Chopin can help make it a bit more clear as to why async/await and forEach act unexpectedly. Here it is as a gist.

Scraping with Puppeteer � Todd Hayton, Since the code in this article uses async/await, you'll need Node v7.6.0 or higher. Specifically, we'll iterate through all of the states in the State field and 3 4 async function main() { 5 const browser = await puppeteer.launch({ kill Kill a running application or disconnect cont, c Resume execution next,� 2 Async puppeteer browser disconnect after few iterations Feb 11 2 Puppeteer - select element with multiple selectors Feb 8 2 Why console.log in puppeteer page.evaluate doesn't work?

Class Browser, LaunchAsync(new LaunchOptions()); var page = await browser. An example of disconnecting from and reconnecting to a Browser: After calling Disconnect(), the browser object is considered disposed and cannot be used anymore� 1 Async puppeteer browser disconnect after few iterations Feb 11. 1 scraping multiple page web table with vba Feb 18. 1 VBA Print Array to Sheet Jan 14.

i am getting exception navigation failed because browser has , ion failed because browser has disconnected, this is my code,what is the reason browser disconnect problem,after processing it will close by urls for Google id 78659 for location in in iteration 0 and error message is : PuppeteerSharp. using (var destinationUrlBrowser = await Puppeteer. 5 Answers. 0� I have a service that executes puppeteer.launch({}) with args on setup, and then continues to use the returned instance of the Browser to load HTML on requests using browser.newPage(). Then I use page.pdf() to render a PDF of the page, and then page.close(). The same instance of the Browser is used throughout the life of the service instance.

Puppeteer example without async/await. But how do you write the equivalent without async/await? It’s pretty straightforward once you realize all the methods return promises:

Comments
  • Thank you a lot about the forEach behaviour information and the article.