Ryan Stone Ryan Stone - 2 months ago 23
Ruby Question

Click Link within .each loop in Ruby Mechanize

I'm working on a script to crawl a page of search results and I've created a loop to find the correct search result and that works. The issue I'm running into is that every permutation of trying to click on the link within the LI element doesn't work. At first I tried finding the anchor within the LI but it's a nokogiri object, I've tried going out into

@page
and drilling down to the correct element but it's coming back nil.

Is there a way to search within the LI element that I already drilled down to?

require 'mechanize'
require 'nokogiri'
require 'csv'

@agent = Mechanize.new do |agent|
agent.user_agent_alias = 'Mac Safari'
end
@page = @agent.get('https://www.milwaukeetool.com/search-results?FullTextKeywords=48-42-5540')

def resultsPageSearcher (searchText)
@page.search('#results ul > li').each do |li|
results_sku = li.css('span.results-sku').text
if results_sku == searchText
link = @page.link_with(:text => results_sku)
@page = link.click
end
end
end

resultsPageSearcher("48-42-5540")


Relevent LI Element (I cut out extraneous code that didn't seem relevant, I can add more if necessary I just didn't want to over post)

<li>
<a href="/accessories/cutting/48-42-5540" class="results-image">
<img src="//cdn.milwaukeetool.com/~/media/Images/Accessories/Cutting/48-42-5540/21867_48-42-5540.jpg?h=100&amp;thn=1&amp;w=100" alt=""></a>
<a class="results-title" href="/accessories/cutting/48-42-5540">4" 10 TPI High-Carbon Steel Jig Saw Blade (5 PK)</a>
<span class="results-sku">48-42-5540</span>
</li>


Thanks for being patient with me.

Answer

So I found a pretty cool library the other day that I think will help you with this. It will go to the link you give it and then search for a link with the text that you want and click it. If you are having issues with this let me know.

require 'watir'
require 'headless'
require 'csv'

headless = Headless.new
headless.start
browser = Watir::Browser.new
browser.goto('https://www.milwaukeetool.com/search-results?FullTextKeywords=48-42-5540')
browser.link(text: searchText).when_present.click