Nicolae Surdu Nicolae Surdu - 28 days ago 6
Python Question

Python: How to resolve URLs containing '..'

I need to uniquely identify and store some URLs. The problem is that sometimes they come containing ".." like

http://somedomain.com/foo/bar/../../some/url
which basically is
http://somedomain.com/some/url
if I'm not wrong.

Is there a Python function or a tricky way to resolve this URLs ?

Answer

There’s a simple solution using urlparse.urljoin:

>>> import urlparse
>>> urlparse.urljoin('http://www.example.com/foo/bar/../../baz/bux/', '.')
'http://www.example.com/baz/bux/'

However, if there is no trailing slash (the last component is a file, not a directory), the last component will be removed.

This fix uses the urlparse function to extract the path, then use (the posixpath version of) os.path to normalize the components. Compensate for a mysterious issue with trailing slashes, then join the URL back together. The following is doctestable:

import urlparse
import posixpath

def resolveComponents(url):
    """
    >>> resolveComponents('http://www.example.com/foo/bar/../../baz/bux/')
    'http://www.example.com/baz/bux/'
    >>> resolveComponents('http://www.example.com/some/path/../file.ext')
    'http://www.example.com/some/file.ext'
    """

    parsed = urlparse.urlparse(url)
    new_path = posixpath.normpath(parsed.path)
    if parsed.path.endswith('/'):
        # Compensate for issue1707768
        new_path += '/'
    cleaned = parsed._replace(path=new_path)
    return cleaned.geturl()