I'm working on a script to crawl a page of search results and I've created a loop to find the correct search result and that works. The issue I'm running into is that every permutation of trying to click on the link within the LI element doesn't work. At first I tried finding the anchor within the LI but it's a nokogiri object, I've tried going out into
@agent = Mechanize.new do |agent|
agent.user_agent_alias = 'Mac Safari'
@page = @agent.get('https://www.milwaukeetool.com/search-results?FullTextKeywords=48-42-5540')
def resultsPageSearcher (searchText)
@page.search('#results ul > li').each do |li|
results_sku = li.css('span.results-sku').text
if results_sku == searchText
link = @page.link_with(:text => results_sku)
@page = link.click
<a href="/accessories/cutting/48-42-5540" class="results-image">
<img src="//cdn.milwaukeetool.com/~/media/Images/Accessories/Cutting/48-42-5540/21867_48-42-5540.jpg?h=100&thn=1&w=100" alt=""></a>
<a class="results-title" href="/accessories/cutting/48-42-5540">4" 10 TPI High-Carbon Steel Jig Saw Blade (5 PK)</a>
So I found a pretty cool library the other day that I think will help you with this. It will go to the link you give it and then search for a link with the text that you want and click it. If you are having issues with this let me know.
require 'watir' require 'headless' require 'csv' headless = Headless.new headless.start browser = Watir::Browser.new browser.goto('https://www.milwaukeetool.com/search-results?FullTextKeywords=48-42-5540') browser.link(text: searchText).when_present.click