Google Dorking
User-agent
Specify the type of "Crawler" that can index your site (the asterisk being a wildcard, allowing all "User-agents"
Allow
Specify the directories or file(s) that the "Crawler" can index
Disallow
Specify the directories or file(s) that the "Crawler" cannot index
Sitemap
Provide a reference to where the sitemap is located (improves SEO as previously discussed, we'll come to sitemaps in the next task)
user-agent: *
disallow: /*.ini$
sitemap: www.mywebsite.com/sitemap.xml
.ini files useful for Windows configuration .CONF are useful for *nix configuration
filetype
Search for a file by its extension (e.g. PDF)
cache
View Google's Cached version of a specified URL
intitle
The specified phrase MUST appear in the title of the page
-inurl:htm -inurl:html intitle:”index of” apk
Google Proxy:
Last updated