Constantin Schreiber Constantin Schreiber - 3 years ago 86
Javascript Question

Script to check for keyword on website

I want to write a script that goes trough a list of url's checking wheter they are valid or not.

The page does not redirect to a 404 but rather displays the sentence 'Sorry, not found! if the URL is invalid.

So if the script finds this sentence, the URL is invalid. If it does not it should most likely be valid.

Any idea on how to realize that in JS? Pointers to possible methods in other languages are welcome too!

Thanks!

Answer Source

A simple Python way would be:

import requests

urls = ['https://www.google.com'] # Fill this however
for url in urls:
    resp = requests.get(url)
    if 'Sorry, not found!' in resp.text:
        print(url + ' had no page') # or something
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download