Cyril Chaboisseau and GitHub
f8d71e067e
improved way to generate user-agent file
2021-05-01 19:04:18 +02:00
Cyril Chaboisseau and GitHub
02ae3b6bd3
change IFS before subshell
...
There needs to be a change in IFS before the 2 curl commands in order to keep line by line formatting
2021-05-01 15:48:33 +02:00
Cyril Chaboisseau and GitHub
2fb0e7c473
deduplicate list of user-agents
2021-05-01 15:08:52 +02:00
Cyril Chaboisseau and GitHub
9adcc2f1a7
more optimized way to generate map referrer file
2021-05-01 14:51:28 +02:00
Cyril Chaboisseau and GitHub
7b98db4d14
improve the generation of blocking file (abusers)
2021-05-01 12:29:15 +02:00
Cyril Chaboisseau and GitHub
ddb2b85916
improve generation of block file (Tor exit nodes)
2021-05-01 12:25:43 +02:00
Cyril Chaboisseau and GitHub
da1a460a64
huge improvement to generate blocking file
...
process the file in 2 commands (grep + sed) instead of a loop running on each line
the time to generate the file takes 0.235 seconds instead of one hour, making it roughly 15,000 times quicker
the output file is exactly the same as with the former method
2021-05-01 11:42:07 +02:00
Cyril ChaboisseauandGitHub