Skip to content

Commit 313e734

Browse files
tbdstbds
committed
add dns resolvers
1 parent 637f59f commit 313e734

File tree

5 files changed

+1079
-78
lines changed

5 files changed

+1079
-78
lines changed

‎README.md

+96-17
Original file line numberDiff line numberDiff line change
@@ -15,37 +15,41 @@ TL:DR `cat domains-{ads,tracking,malware} > /dev/null`
1515

1616
This bash script intends to extract domains lists from various sources.
1717
It is a replacement for ad blocking extensions in your browser.
18-
It blocks ads, malware, trackers at DNS level.
18+
It [blocks ads, malware, trackers at DNS level](https://en.wikipedia.org/wiki/DNSBL).
1919

2020
## Why
2121

2222
- [Major sites including New York Times and BBC hit by 'ransomware' malvertising](http://www.theguardian.com/technology/2016/mar/16/major-sites-new-york-times-bbc-ransomware-malvertising)
2323
- [Adblocking: advertising 'accounts for half of data used to read articles'](http://www.theguardian.com/media/2016/mar/16/ad-blocking-advertising-half-of-data-used-articles)
24+
- [The Verge's web sucks](http://blog.lmorchard.com/2015/07/22/the-verge-web-sucks/) and [The web is Doom](https://mobiforge.com/research-analysis/the-web-is-doom)
2425

2526
## What the scripts does?
2627

2728
- Backup the original configuration file
2829
- Download and merge domains lists from various sources.
2930
- Create a cron job to automaticly update the hosts file, default every week (optional)
3031

31-
## Benefits
32+
## Benefits and Features
3233

3334
- Low CPU and RAM usage.
34-
- Speeds up your Internet use since the local dnsmasq file is checked first, before send a DNS request.
35-
- Data savings since the ad content is never downloaded.
36-
- Not just for browsers, it blocks ads and malware across the entire operative system.
35+
- **Speeds up your Internet** use since the local file is checked first, before send a DNS request.
36+
- **Data savings** since the ad content is never downloaded.
3737
- Stops ad tracking.
3838
- Blocks spyware and malware. That increases the safety of your networking experience.
39+
- Not just for browsers, it blocks ads and malware across the entire operative system.
3940

40-
## Dependencies
4141

42+
## Dependencies
4243

4344
- [GNU bash](http://www.gnu.org/software/bash/bash.html)
4445
- [GNU sed](http://www.gnu.org/software/sed)
4546
- [GNU grep](http://www.gnu.org/software/grep/grep.html)
4647
- [GNU coreutils](http://www.gnu.org/software/coreutils)
47-
- [GNU wget](https://www.gnu.org/software/wget/) or [curl](http://curl.haxx.se/)
48-
- [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html)
48+
- [GNU wget](https://www.gnu.org/software/wget/) or [cURL](http://curl.haxx.se/) (default)
49+
- DNS cacher: [Dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) (default), [Unbound](https://unbound.net/) or [Pdnsd](http://members.home.nl/p.a.rombouts/pdnsd/index.html)
50+
- Caching web proxy: [Polipo](https://www.irif.univ-paris-diderot.fr/~jch//software/polipo/) (optional)
51+
- Filtering web proxy: [Privoxy](http://www.privoxy.org/) (optional)
52+
4953

5054
## Install
5155

@@ -55,8 +59,9 @@ cd FreeContibutor/src
5559
sudo ./installer.sh
5660
```
5761

62+
FreeContributor has some scripts, such as, exporting uBlock or uMatrix rules to dnsmasq format
5863

59-
#### Sources
64+
## Sources
6065

6166
FreeContributor downloads external files; each has its own license, detailed in the list below.
6267

@@ -70,21 +75,95 @@ FreeContributor downloads external files; each has its own license, detailed in
7075
|[Dan Pollock’s hosts file](http://someonewhocares.org/hosts/) | non-commercial |
7176
|[CAMELEON](http://sysctl.org/cameleon/) | ? |
7277
|[StevenBlack/hosts](https://github.com/StevenBlack/hosts/) | ? |
73-
|[quidsup/notrack](https://github.com/quidsup/notrack) | ? |
74-
|[gorhill's uMatrix Blocklist](https://github.com/gorhill/uMatrix) | ? |
78+
|[Quidsup/notrack](https://github.com/quidsup/notrack) | ? |
79+
|[Gorhill's uMatrix Blocklist](https://github.com/gorhill/uMatrix) | ? |
7580
|[Malware Domain List](http://www.malwaredomainlist.com/hostslist/hosts.txt) | |
7681
|[AdBlock Manager](http://adblock.gjtech.net/?format=unix-hosts) | CC Attribution 3.0 |
77-
|[hostfile project](http://hostsfile.org/hosts.html) | LGPL as GPLv2 |
82+
|[Hostfile project](http://hostsfile.org/hosts.html) | LGPL as GPLv2 |
7883
|[Airelle's host file](http://rlwpx.free.fr/WPFF/hosts.htm) | CC Attribution 3.0 |
7984
|[The Hosts File Project](http://hostsfile.mine.nu) | LGPL |
8085
|[Mahakala](http://adblock.mahakala.is/) | ? |
81-
|[Secure Mecca](http://securemecca.com/Downloads/hosts.txt) | LGPL as GPLv2 |
82-
|[spam404scamlist](http://spam404bl.com/spam404scamlist.txt) | |
83-
|[Malwaredomains](http://malwaredomains.lehigh.edu/files/domains.txt) | |
84-
|[Adzhosts](http://optimate.dl.sourceforge.net/project/adzhosts/HOSTS.txt) | |
86+
|[Secure Mecca](http://securemecca.com/) | LGPL as GPLv2 |
87+
|[Spam404scamlist](http://spam404bl.com/) | |
88+
|[Malwaredomains](http://malwaredomains.lehigh.edu/) | |
89+
|[Adzhosts](https://sourceforge.net/projects/adzhosts/) | |
8590
|[Zeustracker](hhttps://zeustracker.abuse.ch/blocklist.php) | |
8691
|[hosts.eladkarako.com](http://hosts.eladkarako.com/) | |
87-
|[Malekal](http: //www.malekal.com/HOSTS_filtre/HOSTS.txt) | |
92+
|[Malekal](http://www.malekal.com/) | |
93+
94+
## DNS 101
95+
96+
Without an custom DNS Server
97+
98+
----
99+
+----+ +------------+ +------------------+ +------------------------+
100+
| PC | <==> | DNS Server | <==> | Other DNS Server | <==> | example.tld = ip adress|
101+
+----+ +------------+ +------------------+ +------------------------+
102+
103+
then
104+
105+
+----+ +-------------------------- +
106+
| PC | <==> | ip adress of example.tld |
107+
+----+ +---------------------------+
108+
----
109+
110+
With a local DNS resolver
111+
112+
----
113+
+----+ +----------------+ +------------------+ +------------------+
114+
| PC | | DNS Server | <==> | Other DNS Server | <==> | goodwebsite.tld |
115+
+----+ +----------------+ +------------------+ +------------------+
116+
^^ ^^ ||
117+
|| || ||
118+
vv || ||
119+
+--------------------+ +----------------------------------------------------+
120+
| local DNS resolver | <==> | ads.example.tld = 127.0.0.1 or 0.0.0.0 or NXDOMAIN |
121+
+--------------------+ +----------------------------------------------------+
122+
+------------+ ||
123+
| DNS cache | <= /
124+
+------------+
125+
126+
future requests of goodwebsite.tld
127+
128+
+----+ +--------------------+ +------------------------------------------+
129+
| PC | <==> | local DNS resolver | <==> | DNS cache of goodwebsite.tld = ip adress |
130+
+----+ +--------------------+ +------------------------------------------+
131+
----
132+
133+
## Hosts vs DNS resolver
134+
135+
The hosts blocking method can not use wildcards (*) and and therefore someone must keep track
136+
of each subdomain that should be blocked. Some DNS caching servers can block the domain and
137+
subdomains with just one rule. For example `/etc/hosts`
138+
139+
127.0.0.1 example.tld
140+
0.0.0.0 example.tld
141+
142+
Will redirect example.tld to the localhost, but not ads.example.tld. With a dns caching server,
143+
such as Dnsmasq, for example `/etc/dnsmasq.conf`
144+
145+
address=/example.tld/127.0.0.1
146+
address=/example.tld/0.0.0.0
147+
server=/example.tld/
148+
149+
Will redirect example.tld and all subdomains to 127.0.0.1 or 0.0.0.0. Better yet, it can
150+
return NXDOMAIN.
151+
152+
153+
## Comparasion
154+
155+
156+
| Program | Language | Adblocking Method |
157+
| :------------- | :-------------| :----------------------------------------------|
158+
| FreeContributor | Bash | DNS caching server (Dnsmasq, Unbound or Pdnsd) |
159+
| Pi-Hole | Bash, Php | Hosts with Dnsmasq (for cache only) |
160+
| NoTrack | Bash, Php | Hosts with Dnsmasq (for cache only) |
161+
| Hostsblock | Bash | Hosts with Dnsmasq (for cache only) |
162+
| dnsgate | Python | Hosts or Dnsmasq |
163+
| StevenBlack/hosts | Python | Hosts |
164+
| adsuck | C | DNS server |
165+
| pfBlockerNG | Sh, PHP | DNS caching server: Unbound |
166+
88167

89168
## License
90169

‎conf/pdnsd.conf

+143
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
// Sample pdnsd configuration file. Must be customized to obtain a working pdnsd setup!
2+
// Read the pdnsd.conf(5) manpage for an explanation of the options.
3+
// Add or remove '#' in front of options you want to disable or enable, respectively.
4+
// Remove '/*' and '*/' to enable complete sections.
5+
6+
global {
7+
perm_cache=1024;
8+
cache_dir="/var/cache/pdnsd";
9+
# pid_file = /var/run/pdnsd.pid;
10+
run_as="pdnsd";
11+
server_ip = 127.0.0.1; # Use eth0 here if you want to allow other
12+
# machines on your network to query pdnsd.
13+
status_ctl = on;
14+
# paranoid=on; # This option reduces the chance of cache poisoning
15+
# but may make pdnsd less efficient, unfortunately.
16+
query_method=udp_tcp;
17+
min_ttl=15m; # Retain cached entries at least 15 minutes.
18+
max_ttl=1w; # One week.
19+
timeout=10; # Global timeout option (10 seconds).
20+
neg_domain_pol=on;
21+
udpbufsize=1024; # Upper limit on the size of UDP messages.
22+
}
23+
24+
# The following section is most appropriate if you have a fixed connection to
25+
# the Internet and an ISP which provides good DNS servers.
26+
server {
27+
label= "myisp";
28+
ip = 192.168.0.1; # Put your ISP's DNS-server address(es) here.
29+
# proxy_only=on; # Do not query any name servers beside your ISP's.
30+
# This may be necessary if you are behind some
31+
# kind of firewall and cannot receive replies
32+
# from outside name servers.
33+
timeout=4; # Server timeout; this may be much shorter
34+
# that the global timeout option.
35+
uptest=if; # Test if the network interface is active.
36+
interface=eth0; # The name of the interface to check.
37+
interval=10m; # Check every 10 minutes.
38+
purge_cache=off; # Keep stale cache entries in case the ISP's
39+
# DNS servers go offline.
40+
edns_query=yes; # Use EDNS for outgoing queries to allow UDP messages
41+
# larger than 512 bytes. May cause trouble with some
42+
# legacy systems.
43+
# exclude=.thepiratebay.org, # If your ISP censors certain names, you may
44+
# .thepiratebay.se, # want to exclude them here, and provide an
45+
# .piratebay.org, # alternative server section below that will
46+
# .piratebay.se; # successfully resolve the names.
47+
}
48+
49+
/*
50+
# The following section is more appropriate for dial-up connections.
51+
# Read about how to use pdnsd-ctl for dynamic configuration in the documentation.
52+
server {
53+
label= "dialup";
54+
file = "/etc/ppp/resolv.conf"; # Preferably do not use /etc/resolv.conf
55+
proxy_only=on;
56+
timeout=4;
57+
uptest=if;
58+
interface = ppp0;
59+
interval=10; # Check the interface every 10 seconds.
60+
purge_cache=off;
61+
preset=off;
62+
}
63+
*/
64+
65+
/*
66+
# The servers provided by OpenDNS are fast, but they do not reply with
67+
# NXDOMAIN for non-existant domains, instead they supply you with an
68+
# address of one of their search engines. They also lie about the addresses of
69+
# of the search engines of google, microsoft and yahoo.
70+
# If you do not like this behaviour the "reject" option may be useful.
71+
server {
72+
label = "opendns";
73+
ip = 208.67.222.222, 208.67.220.220;
74+
reject = 208.69.32.0/24, # You may need to add additional address ranges
75+
208.69.34.0/24, # here if the addresses of their search engines
76+
208.67.219.0/24; # change.
77+
reject_policy = fail; # If you do not provide any alternative server
78+
# sections, like the following root-server
79+
# example, "negate" may be more appropriate here.
80+
timeout = 4;
81+
uptest = ping; # Test availability using ICMP echo requests.
82+
ping_timeout = 100; # ping test will time out after 10 seconds.
83+
interval = 15m; # Test every 15 minutes.
84+
preset = off;
85+
}
86+
*/
87+
88+
/*
89+
# This section is meant for resolving from root servers.
90+
server {
91+
label = "root-servers";
92+
root_server = discover; # Query the name servers listed below
93+
# to obtain a full list of root servers.
94+
randomize_servers = on; # Give every root server an equal chance
95+
# of being queried.
96+
ip = 198.41.0.4, # This list will be expanded to the full
97+
192.228.79.201; # list on start up.
98+
timeout = 5;
99+
uptest = query; # Test availability using empty DNS queries.
100+
# query_test_name = .; # To be used if remote servers ignore empty queries.
101+
interval = 30m; # Test every half hour.
102+
ping_timeout = 300; # Test should time out after 30 seconds.
103+
purge_cache = off;
104+
# edns_query = yes; # Use EDNS for outgoing queries to allow UDP messages
105+
# larger than 512 bytes. May cause trouble with some
106+
# legacy systems.
107+
exclude = .localdomain;
108+
policy = included;
109+
preset = off;
110+
}
111+
*/
112+
113+
source {
114+
owner=localhost;
115+
# serve_aliases=on;
116+
file="/etc/hosts";
117+
}
118+
119+
/*
120+
include {file="/etc/pdnsd.include";} # Read additional definitions from /etc/pdnsd.include.
121+
*/
122+
123+
rr {
124+
name=localhost;
125+
reverse=on;
126+
a=127.0.0.1;
127+
owner=localhost;
128+
soa=localhost,root.localhost,42,86400,900,86400,86400;
129+
}
130+
131+
/*
132+
neg {
133+
name=doubleclick.net;
134+
types=domain; # This will also block xxx.doubleclick.net, etc.
135+
}
136+
*/
137+
138+
/*
139+
neg {
140+
name=bad.server.com; # Badly behaved server you don't want to connect to.
141+
types=A,AAAA;
142+
}
143+
*/

0 commit comments

Comments
 (0)