Afzaal Afzaal - 1 year ago 124
MySQL Question

how to build web crawler/spider Like Google php mysql

I am wondering that how to build a superfast Web crawler or spider that works same as google's bot did,
that is pretty difficult but where to get started any idear will be appreciated

Answer Source

You have to use this script for php search engine.

<script src="jquery.js"></script>
<script language="javascript">

function getCrawl(x)

$.get("json.php", {site:x}, function(data){

var ans = data;
var b = $("#output").html();

for(var y=0;y!=ans.length;y++)
b += "<a href='javascript:getCrawl(\""+ans[y]+"\")'>"+ans[y]+"</a><br />";


}, "json")



<div id="output">

$target_url = "";

$html = new simple_html_dom();
foreach($html->find('a') as $link){
echo "<a href='javascript:getCrawl(\"".$link->href."\")'>".$link->href."</a><br />";

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download