Afzaal Afzaal - 1 month ago 19
MySQL Question

how to build web crawler/spider Like Google php mysql


I am wondering that how to build a superfast Web crawler or spider that works same as google's bot did,
that is pretty difficult but where to get started any idear will be appreciated

Answer

You have to use this script for php search engine.

<script src="jquery.js"></script>
<script language="javascript">

function getCrawl(x)
{

$.get("json.php", {site:x}, function(data){

var ans = data;
var b = $("#output").html();

for(var y=0;y!=ans.length;y++)
{
b += "<a href='javascript:getCrawl(\""+ans[y]+"\")'>"+ans[y]+"</a><br />";
}

$("#output").html(b);

}, "json")

}

</script>

<div id="output">
<?php
include_once('simple_html_dom.php');


$target_url = "http://website.com/";

$html = new simple_html_dom();
$html->load_file($target_url);
foreach($html->find('a') as $link){
echo "<a href='javascript:getCrawl(\"".$link->href."\")'>".$link->href."</a><br />";
}
?>
</div>