IPSET. Creating white-list for apt-get updates.

Today we've learned about IPSET, an extension for iptables firewall.
The homework was to write a script on python, that will do:

1. Find out all apt-get sources.
2. Get IP's from domain names
3. Create/Update IPSET set.

It is too painful to do it in python. I've done it in bash:

#!/bin/bash
name="apt-white"      #change it for your needs
function makeset(){
 check=$(sudo ipset list $name |grep -o $name)
 if [[ ! "$check" == "$name" ]]
 then
  sudo ipset create $name iphash
  append_set
 else
  append_set
 fi
}
function append_set(){ 
 ipset flush $name
 ip_list=$(for line in $(echo "$(grep -Ril "http" /etc/apt/ |xargs cat)" |grep http|cut -d":" -f2|cut -d'/' -f3 |sort -u |grep '.'); do host $line;done |grep 'has address' | rev |cut -d' ' -f1 |rev |sort -u)
 for line in $ip_list; do
  ipset -A $name $line
 done
 exit 0
}
makeset

It is only 21 lines. Will see how much code will be in python, next lesson...

UPDATE:

At today's lesson, everyone wrote a script on Python. Here's my: 

#!/usr/bin/env python

from subprocess import Popen,PIPE
from urlparse import urlparse
import sys,re,os
import socket

# Name your SET & Global variables
name = 'apt-white'
srv_list = []
srv_ip = []

# Search in '/etc/apt/' directory for files that includes 'http'
p = Popen('grep -Ril http /etc/apt/'.split(' '),stdout=PIPE)
file_list = p.stdout.read().split()

# Look inside every file in list and extract every domain-name into list(srv_list)
for one_file in file_list:
   with open(one_file,'r') as f:
      for line in f.readlines():
         line = line.lower()
         line = re.findall(r'(https?://\S+)', line)
         if line:
             parsed = urlparse(line[0])
             srv_list.append(parsed.hostname)

# Make DNS lookup for every domain-name in set(srv_list), and return list of IP's in list(srv_ip), IGNORING IPv6 addresses (':' not in string)
for srv in set(srv_list):
   srv_ip += list(set([i[-1][0] for i in socket.getaddrinfo(srv, 80) if not ':' in i[-1][0]]))

# Checking if your SET already exists
command = 'ipset list '+name
p = Popen(command.split(' '),stdout=PIPE)
check = p.stdout.read()

# If not, creating a new one
if not name in check:
   print 'Creating new SET %s' % name
   os.system('ipset create '+name+' iphash')

# If yes, flushing all data in your SET
else:
   print 'Flushing all data in %s' % name
   os.system('ipset flush '+name)

# Appending to the SET all IP addresses we found
for i in set(srv_ip):
   os.system('ipset -A '+name+' '+i)
print '\ndone.'

# Profit! :)

It's a little larger, but also not too big. Only ~35 lines.

Do you know a better way to communicate with shell CLI on Python?