HTML::RobotsMETA is a simple HTML::Parser subclass
that extracts robots exclusion information from meta tags.

WWW: http://search.cpan.org/dist/HTML-RobotsMETA/