PhilipB PhilipB - 2 months ago 15
Python Question

Remove duplicates of IP addresses by netmask

I have got an Array:

[192.0.0.3, 0.0.0.0 , 192.0.10.24, ...]


With IP addresses and i want to remove duplicates for the /16 netmasks, so i got 192.0.0.3 but 192.0.10.24 will be removed (i don't mind which one of them, it would also be okay if the first one is removed).

My first thoughts were to use a regex to cast the netmask and remove every IP address which matches the then generated patttern.

Is there an easier way?

Answer

You could remove duplicates using a set, with the keys being tuples of the first two parts:

>>> ips = ["192.0.0.3", "0.0.0.0", "192.0.10.24"]
>>> seen = set()
>>> for ip in ips:
...     key = tuple(ip.split(".")[:2])
...     if key not in seen:
...         print(ip)
...         seen.add(key)
... 
192.0.0.3
0.0.0.0

Or alternatively using the ipaddress module:

>>> from ipaddress import ip_network
>>> ips = ["192.0.0.3", "0.0.0.0", "192.0.10.24"]
>>> seen = set()
>>> for ip in ips:
...     key = ip_network(ip + "/16", strict=False)
...     if key not in seen:
...         print(ip)
...         seen.add(key)
... 
192.0.0.3
0.0.0.0