Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | HTML::RobotsMETA is a simple HTML::Parser subclass | Cheng-Lung Sung | 2008-01-30 | 4 | -0/+42 |
that extracts robots exclusion information from meta tags. Author: Daisuke Maki <dmaki@cpan.org> WWW: http://search.cpan.org/~dmaki/HTML-RobotsMETA/ PR: ports/120117 Submitted by: Masahiro Teramoto <markun at onohara.to> Notes: svn path=/head/; revision=206438 |