shoaljumper shoaljumper - 1 year ago 72
Perl Question

remove URL's of a certain length

I have a flat file with a list of URL's I want to spider. I only want to check the ones of a certain length though. I am unaware of any way to cut the shorter urls from the list via bash. Would it be possible to create a simple for do loop to remove lines in the file that contain less than 5 forward slashed? "/"

Answer Source

You can simply to:

grep '://.*/.*/.*/.*/.*/' urls.txt

This will only match lines that have at least 5 slashes.