Ruslan Ruslan - 3 months ago 13
Python Question

Match everything until optional string (Python regex)

I've pounded my head against this issue, and it just seems like I am missing something uber-trivial, so apologies in advance. I have a url, which may, or may not, contain some POST values. I want to match the the entire url UNTIL this optional part (not inclusive). So for example:

import re
myurl = r'http://myAddress.com/index.aspx?cat=ThisPartChanges&pageNum=41'
matchObj = re.match(r'(.*?)(&pageinfo=\d+){0,1}', myurl)
print matchObj.groups()
>> ('', None)

# Putting the non-greedy ? outside
matchObj = re.match(r'(.*)?(&pageinfo=\d+){0,1}', myurl)
print matchObj.groups()
>> ('http://myAddress.com/index.aspx?cat=ThisPartChanges&pageNum=41', None)

# The url might also be without the last part, that is
myurl = r'http://myAddress.com/index.aspx?cat=ThisPartChanges'
# I'd like the regex to capture the first part. "ThisPartChanges" might
# be different every time


What I would like is to get the everything until pageNum=\d+, not inclusive.
That is

http://myAddress.com/index.aspx?cat=ThisPartChanges


I am only interested in the part before &pageNum, and don't care if it exists or not, just want to filter it out somehow so that I can get the real address until cat=....

I've tried all sorts of non-greedy acrobatics, but the part that fails me is that the 2nd part is optional, so there's nothing to 'anchor' the non-greedy match.
Any ideas how to elegantly do this? Only the first part is important. Non-regex solutions are also welcome

Thanks!

BPL BPL
Answer

I'd recommend you to avoid regular expressions when it comes to url parsing, use this module instead, here's a working example for your problem:

import urlparse

myurl = 'http://myAddress.com/index.aspx?cat=ThisPartChanges&pageNum=41'

parsed = urlparse.urlparse(myurl)

print 'scheme  :', parsed.scheme
print 'netloc  :', parsed.netloc
print 'path    :', parsed.path
print 'params  :', parsed.params
print 'query   :', parsed.query
print 'fragment:', parsed.fragment
print 'username:', parsed.username
print 'password:', parsed.password
print 'hostname:', parsed.hostname, '(netloc in lower case)'
print 'port    :', parsed.port

print urlparse.parse_qs(parsed.query)
Comments