Scrape a website (javascript website) using php

php scrape javascript generated content
scraping javascript rendered web pages php
learn how to extract data from websites with php
how to scrape data from website using php
web scraping javascript
php web scraping
php scrape web page
how to scrape an ajax website using php

I am trying to scrape a website (believe it is in JavaScript) using a simple PHP script. I am a beginner so any help would be greatly appreciated. The URL of the webpage is:

http://www.indiainfoline.com/Markets/Company/Fundamentals/Balance-Sheet/Yes-Bank-Ltd/532648

So here for example I would like to pass the name of company (Yes-Bank-Ltd) and code (532648) in get_file_contents. Not sure on how to do it so can somebody please help.

Thanks, Nidhi

Why aren't you just not append the string of the company and code in the url. Here is an idea that you fill up an array of company and code (need to be the same size) and then you loop them to scrape the data you want.

for($i=0;$i<count($listOfCie);$i++)
{
    $cie = $listOfCie[$i];
    $code = $listOfCode[$i];
    $urlToScrape = "http://www.indiainfoline.com/Markets/Company/Fundamentals/Balance-Sheet/" . $cie . "/" . $code
    //... = get_file_contents($urlToScrape....
 }

PHP Web Scraping: What to know before you start with Symfony , Long ago are the days that every website developer made sure that their site worked great without executing any JavaScript. And it's precisely� I am trying to scrape a website (believe it is in JavaScript) using a simple PHP script. I am a beginner so any help would be greatly appreciated. The URL of the webpage is:

Use the data.html table in YQL! http://developer.yahoo.com/yql/console

PHP Web scraping of Javascript generated contents, php $url = "http://www.govliquidation.com/json/buyer_ux/salescalendar.js"; $json = file_get_contents($url); $data = json_decode($json); ?> This yields a data object that you can inspect and convert in CSV by simple looping. The Web Scraping API allows the developer to scrape data from the website in a structured format. It returns realtime data from the websites based on the web page URL specified in the API settings. The Web Scraping API is very useful when you want to extract content from the HTML source of the web pages.

The simplest way to scrape a site in PHP is to use curl (http://php.net/manual/en/book.curl.php)

For some examples look at http://php.net/manual/en/curl.examples-basic.php or google :)

If the website relies on javascript though it's going to be difficult to get the data you want. You might look at a "headless browser" like http://phantomjs.org/

8 Awesome PHP Web Scraping Libraries and Tools, In PHP, you can do scraping with some of these libraries: Goutte; Simple HTML DOM; htmlSQL; cURL; Requests; HTTPful; Buzz; Guzzle� Scraping websites generated by JavaScript or jQuery using PHP is a topic that I’ve received many requests for and one that I’ve been wanting to cover for a while now. More often than not, it’s just a single page or form that people are having issues with, but I wanted to wait until I found an entire site that is generated using JavaScript

Simple way of scraping a website using PHP, We go over a simple web scraping operation on Freelancers.net to scrape the Scraping a site using PHP Notably, a lot of data is actually generated through AJAX these days and inserted with javascript, which renders this� scrape.js. Steps1: Create Form To Enter Website URL. As we will handle this tutorial with demo, so first we will create From in index.php with submit button to enter website URL to scrape data. <form method="post" name="scrap_form" id="scrap_form" action="scrape.php">. <label>Enter Website URL To Scrape Data</label>.

Scrape web content with PHP (no API? no problem), There are multiple scraping scripts ready to use. I can recommend one of them: PHP Simple HTML DOM Parser. It's extremely easy to start with� {% blockquote %} Web scraping (web harvesting or web data extraction) is a computer software technique of extracting information from websites. Usually, such software programs simulate human exploration of the World Wide Web by either implementing low-level Hypertext Transfer Protocol (HTTP), or embedding a fully-fledged web browser, such as

cubiclesoft/ultimate-web-scraper: A PHP library/toolkit , A PHP library/toolkit designed to handle all of your web scraping needs under a MIT or LGPL license. Also has web Can be used to download entire websites for offline use. Has a liberal Note that Javascript is not executed. $web = new� Speaking about handling AJAX and JavaScript while web scraping, sometimes it could be tricky, especially when you are a tech noob. If you are looking for an easy and quick way to do this, especially for large workloads, you may want to look into some third-party applications for scraping websites with AJAX.

Comments
  • Scrap, as in "recycle"? You can try unlink, I suppose.
  • what do you mean by pass it, as I can see they are already in url
  • @Kerrek SB, he meant 'scrape'.
  • @Nightfirecat: Ahh, makes a lot more sense! Though I would have loved the idea of a website scrapyard, with lots of old queries and dying animated GIFs...
  • Ok, seems to be a good idea. However, when i try using hte syntax you gave here, it is giving a problem. Can you please verify if you need two quotes in the first variable and no quotes after the second one.
  • Are you a developer? Quotes verifications should be your job to check. It's just a pseudo code to show you how to solve your problem.
  • no, i am not a developer, and hence the issue.. sorry for causing you trouble
  • Stackoverflow is not a place to get code all made for you project but a place to ask question. Sorry I can't give you more than that, I evaluate my answer as a good start. You should get some PHP tutorial and come back here because string concatenation is basic stuff.
  • Ok, i figured it out. I was using the command inside a single quote and not a double quote. Now this is working. BTW, would you know how i can run a javascript on the page that i am scrapping. In this case, i want consolidated results and hence javasript:_doPostBack()
  • sorry, i do not find Yahoo! website useful for Indian Companies