I have a flat file with a list of URL's I want to spider. I only want to check the ones of a certain length though. I am unaware of any way to cut the shorter urls from the list via bash. Would it be possible to create a simple for do loop to remove lines in the file that contain less than 5 forward slashed? "/"
You can simply to:
grep '://.*/.*/.*/.*/.*/' urls.txt
This will only match lines that have at least 5 slashes.