initial codebase

2018-05-28 21:01:27 +05:30
commit 5eedfa13bd
136 changed files with 19812 additions and 0 deletions
@@ -0,0 +1,5 @@
+# www.robotstxt.org/
+
+# Allow crawling of all content
+User-agent: *
+Disallow: