#State Google appliance, disallowing past sessions User-agent: gsa-crawler Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /billtracking/bills95/ Disallow: /billtracking/bills96/ Disallow: /billtracking/bills97/ Disallow: /billtracking/bills98/ Disallow: /billtracking/bills99/ Disallow: /billtracking/bills00/ Disallow: /billtracking/bills01/ Disallow: /billtracking/spec01/ Disallow: /billtracking/bills02/ Disallow: /billtracking/bills03/ Disallow: /billtracking/bills033/ Disallow: /billtracking/bills034/ Disallow: /billtracking/bills041/ Disallow: /billtracking/bills051/ Disallow: /billtracking/bills053/ Disallow: /billtracking/bills061/ Disallow: /billtracking/bills071/ Disallow: /billtracking/bills073/ Disallow: /billtracking/bills081/ Disallow: /BillTracking/PDFViewer/ User-agent: * Disallow: /*.axd$ Disallow: /DXR.axd Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /javascript:false Disallow: /BillTracking/PDFViewer/ Disallow: /*? Crawl-delay: 120 #Yahoo. User-agent: Slurp Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Crawl-delay: 120 #MSN. User-agent: MSNBot Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #Google. User-agent: Googlebot Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #Google Images. User-agent: Googlebot-Image Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #MSN Images. User-agent: psbot Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #SingingFish. User-agent: asterias Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #Yahoo Multimedia. User-agent: yahoo-mmcrawler Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #Ask.com. *Supports delay User-agent: Teoma Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #Searchme.com. User-agent: Charlotte Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #Lexis-Nexis User-agent: lnbot Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #Zibb.com. Has preset crawl delay of 30 seconds. Appears to be following a 60 second delay instead. User-agent: zibber-v0.1(www.zibb.com/crawler/) Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ #Others. User-agent: Gigabot Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 User-agent: Scrubby Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 User-agent: Robozilla Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 User-agent: Nutch Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 User-agent: ia_archiver Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 User-agent: baiduspider Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 User-agent: FiscalNote/1.0 Disallow: /billcontent.aspx Disallow: /bill.aspx Disallow: /billcentral.aspx Disallow: /BillTracking/PDFViewer/ Crawl-delay: 120 #If you are trying to get legislative data, use https://documents.house.mo.gov/ instead of crawling the site. Contact the webmaster@ house.mo.gov for more information.
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)