Age	Commit message (Collapse)	Author	Lines
9 days	Stop robots.txt blocking files named blame	Bryce Kwon	-2/+2
	Without the trailing slash the rules also matched any path segment that starts with blame or snapshot, such as /repo/tree/blame.c. Every link cgit makes to the two pages ends the name with a slash.
2026-07-19	Widen the robots.txt rules to extensionless paths	Bryce Kwon	-3/+2
2026-07-15	Reorganise into source, assets and libraries	Bryce Kwon	-0/+0
	Everything the build generates now lands in build/, so a clean is a single remove and the source tree stays free of build output.
2022-12-19	ui-tree,ui-blame: bail from blame if blob is binary	June McEnroe	-0/+1
	This avoids piping binary blobs through the source-filter. Also prevent robots from crawling it, since it's expensive. Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2013-08-12	robots.txt: disallow access to snapshots	Jason A. Donenfeld	-0/+3
	My dmesg is filled with the oom killer bringing down processes while the Bingbot downloads every snapshot for every commit of the Linux kernel in tar.xz format. Sure, I should be running with memory limits, and now I'm using cgroups, but a more general solution is to prevent crawlers from wasting resources like that in the first place. Suggested-by: Natanael Copa <ncopa@alpinelinux.org> Suggested-by: Julius Plenz <plenz@cis.fu-berlin.de> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>