Sunday, March 13, 2016

Pihole, squid, squidguard on a $5 Pi Zero

My plan was to get the most use possible from the $5 raspberry Pi Zero that I managed to get my hands onto by buying a full kit that included a couple cables and a power supply.  It was actually not that bad a deal.  One pain that I was having is the fact that my mobile devices and android machines do not block ads.  At all.  Can't even load a plugin to the browser to block ads.  Not to mention all the ads that are strewn through all the apps on the phones and pad computers. So when I saw Pihole again on reddit a couple of weeks ago, I knew that this was the project to use for my Pi Zero.

I first loaded raspberian lite onto a 16 GB class 10 SD card.  I loaded this into a raspberry pi 2 for the initial setup.  At this point you should set up a static ip address, either handed out by the router, or outside the range handed out by the router.

Also expand out the file system and set the gpu_mem=16MB in /boot/config.txt to free up as much memory as possible.

I loaded the script from https://pi-hole.net by copying it from the web page and pasting it into a terminal as root.  At one point the install nuked /etc/resolve.conf and I had to add the line
nameserver 8.8.8.8
to the bottom of the file. After this one problem I reran the script and finished up.  I changed the DNS server that the router used and rebooted my desktop machine so it would get all the changes.

The testing showed that it all worked. It worked to block about 18% of the links my browser was trying to load. It used less than 1% on the Pi Zero CPU for this and a tiny amount of memory. Web pages began loading in half the time.

The grey case is the Pi Zero,
the white plug is the usb network card,
and the sd card is there for scale.

I added a few local host names into the /etc/hosts file on the Pihole machine and reran gravity.sh command to reload everything, and then every machine was able to resolve those hostnames.  This means that I no longer have to maintain a hosts table on all my clients, which is nice.

There is also a very nice admin console for the Pihole so you can see how effective it is.  http://piholeserveraddress/admin/index.php



I powered down the RasPi 2 and put the card into the Pi Zero with an old 100MB usb network dongle attached to a usb on the go cable.  I also swapped the 2 amp power supply with a smaller 1.5 amp power supply.  The pi zero was in a small sleeve case. Everything worked just as fast with the $5 computer as it did with the $25 computer.

Watching the performance at less than 1% usage I realized that I could install a squid proxy server and found another ad blocker called squidguard that works as a pattern matching blocker. I had wanted a squid server for a decade now and had never gotten around to making one.

I followed this guide to set up squid and squid guard. The first thing you need to do is install, configure, and test squid.

Once that works you have to install, configure, tie into squid, restart squid, and test with a blacklist that just blocks a handful of sites.

Once that was done I downloaded the easylist that ad block plus uses, found a sed script that would translate easylist.txt into a blacklist that squidguard could use, and then created the blacklist.db file. I found the script and a write up of how to do that here. The script in the article does not work, but in the comments is a link to a script that does work on the github site.

At this point every device on my network is ad blocked.  Phones, pad computers, both iphone and android, desktop machines, servers are all protected now, both browsers and apps, and are loading web pages twice as fast.

One last thing, I had to run a command on the blacklist.db file that was created to set the group and owner to proxy.
cd /var/lib/squidguard/db
chown proxy blacklist.db
chgrp proxy blacklist.db 

Nagios

Now that I have all this nifty infrastructure and I have all sorts of available space on the pihole I am adding in a monitor to check the hosts on my local network and to check on a few web servers I am responsible for.

I installed Nagios following a guide I found on the internet.

The only thing I had to change to get this to work was adding
server.modules += ( "mod_alias" )
to the top of 10-nagios3.conf 

Other than just getting a blank page like every other ad on the internet because of pihole, the only clue I had to this error was one line in the error.log for lighttpd that said
WARNING: unknown config-key: alias.url (ignored)
And it took an hour to fix that.

I reran gravity.sh after I installed Nagios3.  But in hindsight I think it was another network error that made me think I needed to run this, but no harm done.

Tomorrow I am going to start monitoring some sites. :D

-- --

15 Mar 2016

Today I got pings happening to everything on my network except my iphone, which is up and down all the time, for some strange reason.  Not even nmap can find it on the network when the iphone goes to sleep. My android phone and two pad computers are up all the time, even when the screen is in standby.

I created a hosts folder in /etc/nagios3/  and configured /etc/nagios/nagios.cfg to read that directory using a line that said
cfg_dir=/etc/nagios3/hosts
then I used this template:
define host {  host_name hostname  alias plainenglishname  address 192.168.0.xxx   max_check_attempts 3  check_period 24x7  check_command check-host-alive  contacts root  notification_interval 60  notification_period 24x7}
placing the files in the /etc/nagios3/hosts directory. The filename doesn't matter, but I used ipaddress-hostnam.cfg for all these files.

I had one problem on localhost for the pihole box running the nagios3 localhost disk monitor command.  I had to do these commands to give it permission to work correctly.  
chmod 755 /sys
chmod 755 /sys/kernel/
chmod 755 /sys/kernel/debug/
chmod 755 /sys/kernel/debug/tracing/
I worked on setting up nagios agent on one of my ubuntu desktop machines, following this pdf guide, doing the following:
  add-apt-repository ppa:nagiosinc/ppa
  echo "deb http://ppa.launchpad.net/nagiosinc/ppa/ubuntu lucid main" > /etc/apt/sources.list.d/nagiosinc.list
  gpg --keyserver keyserver.ubuntu.com --recv-keys B18637BB5175BC68
  gpg --export --armor B18637BB5175BC68 | apt-key add -
  apt-get update
  apt-get install nagios-agent
This adds a client that will let the nagios3 server run a lot of commands on remote machines.

On the server side I had to add in some plugins to give me the check_nrpe command that the guide talked about.  I used this guide to figure that out.

-- -- --

Services.

There were already checks for http and ssh built into the /etc/nagios3/conf.d/services_nagios2.cfg file.  Turning these on for clients just involved adding the host name that you made above to the comma seperated with no spaces list in hostgroups_nagios2.cfg


Where I found the smb script.  Change line 11 to where basepath is on your system.

https://exchange.nagios.org/directory/Plugins/System-Metrics/File-System/SMB/check_smb/details

You have to put check_smb in  /usr/lib/nagios/plugins/

Configure a file to tie the plugin to a nagios command here: /etc/nagios-plugins/config/smb.cfg

This is what I put in the file:
# 'check_smb' command definitiondefine command{ command_name check_smb command_line /usr/lib/nagios/plugins/check_smb -H '$HOSTADDRESS$' }
In /etc/nagios3/conf.d/services_nagios2.cfg file append these lines at the end:

# check that smb services are running
define service {        hostgroup_name                  smb-servers        service_description             SMB check_command                   check_smb        use                             generic-service notification_interval           0 ; set > 0 if you want to be renotified}
Then you can tie hosts to that service in /etc/nagios3/conf.d/hostgroups_nagios2.cfg  with a block of text added to the end like this:

# A list of your smb-accessible serversdefine hostgroup {        hostgroup_name  smb-servers alias           SMB servers members         serverA,server2,serverIII        }

And tie it to the hosts you want to run the check against on the members line. remember that these hosts all have to be predefined. 


--

Where I found raspberry pi logos for nagios.

https://exchange.nagios.org/directory/Graphics-and-Logos/Images-and-Logos/Raspberry-Pi/details

put that directory here:

/usr/share/nagios3/htdocs/images/logos

edit nagios to use them in the /etc/nagios3/conf.d/extinfo_nagios2.cfg file.

My hosts list.

The three services I am currently looking for, smb, http, and ssh.


---

Tomorrow I get nrpe client running against a few hosts.



3 comments:

  1. Good ads blacklist --> http://www.squidblacklist.org/downloads/squidblacklists/squid-ads.tar.gz

    ReplyDelete
    Replies
    1. Thanks! I will have to work on a way to merge lists together for the squid-guard part.

      Delete
  2. I came up with a similar idea, and doing some research came across your blog. Love the tutorial. Thanks for everything. Keep up the blog!

    ReplyDelete