Last night oisd.nl started to return HTML responses (specifically a login form of some kind) on all known oisd.nl urls like https://big.oisd.nl and https://dbl.oisd.nl and even https://oisd.nl itself. No blocklists were returned, just the login form HTML. All responses were served with a HTTP status code 200 OK
. This means that pihole purges all entries from the given oisd.nl url and tries to import the invalid "blocklist" (i.e. the login form HTML). Since this response is not a valid syntax list, the net result is that all oisd.nl entries are purged and you're left with a lot less blocked domains.
This PHP script intelligently caches oisd.nl responses; the response is validated to be a valid blockfile (see Support for other lists) and only then it is written to a local directory and thus 'cached'. The file is stored in GZipped format to a) save space but, more importantly, b) not have to compress the output every time the script is invoked. For big lists being requested many times this saves a lot of CPU cycles.
When oisd.nl responds with somethig other than a valid list the cached value will be kept. When the list is requested the old(er) version is returned. It may be outdated, but it's better than nothing (see the purging issue described).
- A webserver
- PHP 7+
- A directory to put the PHP file and a directory to store the cached files
- Put the script in a webdirectory
- Create a directory where you want to store the cached files. Make sure the script can write to that directory
- Point the
$fileroot
variable to the cache directory from step 2 - (OPTIONAL) Update the
$lists
variable; this script works with oisd.nl but should work with other blocklists too - Check the script: Open a webbrowser, go to
https://myserver.local/oisd/index.php?list=big
(wherebig
is the name of any of the defined lists) and make sure a file is written to the cache directory and a list is returned to your browser.
- Go to Pihole -> Adlists
- Add the url to this script (e.g.
https://myserver.local/oisd/index.php?list=big
) - Disable the original oisd.nl list
- Go to Tools -> Update Gravity -> Click update
As you can see in the $lists
variable, by default the big
, small
and nsfw
lists are supported, but you can add custom lists which may be from any url as long as they pass the IsValidBlocklist
check which, currently, is very basic and simple (it simply checks wether the response starts with any of [Adblock Plus]
, [Adblock Plus 2.0]
, # Version:
or ; Version:
). This method will need to be improved to support more lists. All lists from oisd.nl should work. So you should be able to add:
$lists = [ // Known lists
'small' => 'https://small.oisd.nl/',
'big' => 'https://big.oisd.nl/',
'nsfw' => 'https://nsfw.oisd.nl/',
'small-domainswild' => 'https://small.oisd.nl/domainswild',
'big-domainswild' => 'https://big.oisd.nl/domainswild'
];