shoaljumper shoaljumper - 4 months ago 16
Perl Question

remove URL's of a certain length

I have a flat file with a list of URL's I want to spider. I only want to check the ones of a certain length though. I am unaware of any way to cut the shorter urls from the list via bash. Would it be possible to create a simple for do loop to remove lines in the file that contain less than 5 forward slashed? "/"

Answer

You can simply to:

grep '://.*/.*/.*/.*/.*/' urls.txt

This will only match lines that have at least 5 slashes.