...
- Download the file crawlertrapsCollection.txt and upload it as a global-crawler-trap list.
- Download the list again from the GUI and compare it with original list. If necessary use "sort -u" to remove any duplicates and ensure that the two versions are ordered identically. A sample Unix command line:
diff <(sort -u crawlertrapsCollection.txt ) <(sort -u crawlertrap ) (note that any empty lines are also removed during upload)
Update Byte Limits
Update the "Maximum number of bytes" for the defaultconfig for six domains as follows:
...