aboutsummaryrefslogtreecommitdiff
path: root/www/p5-WWW-RobotRules-Parser
Commit message (Collapse)AuthorAgeFilesLines
* - stage supportAndrej Zverev2014-01-053-4/+2
| | | | | | | Approved by: portmgr (blanket infrastructure) Notes: svn path=/head/; revision=338771
* Add NO_STAGE all over the place in preparation for the staging support (cat: ↵Baptiste Daroussin2013-09-201-5/+2
| | | | | | | www) Notes: svn path=/head/; revision=327776
* - convert to the new perl5 frameworkAndrej Zverev2013-09-031-1/+2
| | | | | | | Approved by: portmgr (bapt@, blanket) Notes: svn path=/head/; revision=326139
* - Get Rid MD5 supportMartin Wilke2011-03-191-1/+0
| | | | Notes: svn path=/head/; revision=271305
* WWW::RobotRules::Parser allows you to simply parse robots.txt filesMartin Wilke2008-02-164-0/+40
as described in http://www.robotstxt.org/wc/norobots.html. WWW: http://http://search.cpan.org/dist/WWW-RobotRules-Parser/ PR: ports/120160 Submitted by: Masahiro Teramoto <markun at onohara.to> Notes: svn path=/head/; revision=207323