v2.1
Release notes:
- use only English language for README.md (#45)
- added more phpunit tests for RobotsTxtParser, RobotsTxtValidator (#44, #49)
- fixed bug in RobotsTxtParser with processing of list of user-agents (#47)
For example:
User-Agent: ahrefs User-Agent: SurdotlyBot Disallow: /
Now will be disallowed path "/
" for both user-agents (earlier Disallow: /
has been applied only for the latest agent SurdotlyBot)
4. fixed bug with handle character '+
' in allow,disallow directives in RobotsTxtValidator (#50)