ProjectSegfault/go-away
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
249 Commits 6 Branches 11 Tags
3abdc2ee5b8efcb32aa3709bbeb1a684db2212fe
Commit Graph

60 Commits

Author SHA1 Message Date
WeebDataHoarder
3abdc2ee5b examples: add private / localhost networks to snippets and forgejo/generic examples 2025-05-13 03:06:23 +02:00
Alan Orth
c6a1d50f39 examples/config.yml: fix YAML syntax 2025-05-04 12:25:44 +03:00
WeebDataHoarder
fb6c5c3eb4 examples/forgejo: remove standard-bots rule, it's redundant 2025-05-03 22:43:09 +02:00
WeebDataHoarder
44c9114ae5 challenges: add refresh via JavaScript window.location 2025-05-03 21:35:12 +02:00
WeebDataHoarder
4b1878f1ac examples/forgejo: exclude fetchers from suspicious crawler 2025-05-03 21:21:13 +02:00
WeebDataHoarder
3d73ee76c4 state: add more meta tags onto cached tags, add missing txt and xml resources to well-known snippet 2025-05-03 05:59:32 +02:00
WeebDataHoarder
606f8ec3a0 templates: explicitly allow overriding logo via cmdline/override in config, have bundled templates support it 2025-05-03 04:14:11 +02:00
WeebDataHoarder
1ea19c5a6c state context: Added proxy-safe-link-tags to proxy <link> tags, use specific LinkTags ranger on templates instead of raw elements 2025-05-03 04:12:58 +02:00
WeebDataHoarder
736c2708e9 examples/forgejo: exclude fetchers from TLS Fingerprint rule 2025-05-02 22:21:40 +02:00
WeebDataHoarder
e8e072286e challenge: lower preload-early-hint-deadline to 2 seconds by default 2025-05-02 20:42:25 +02:00
pwgen2155
2ab45983e9 feat: all betterstack ip ranges and useragent (#16) 
ref: https://betterstack.com/docs/uptime/frequently-asked-questions/#what-ips-does-uptime-use

I believe this is how you do it. Will test later on. Unfortunately their playwrite contains a generic user agent...

Co-authored-by: WeebDataHoarder <weebdatahoarder@noreply.gammaspectra.live>
Reviewed-on: https://git.gammaspectra.live/git/go-away/pulls/16
Co-authored-by: pwgen2155 <pwgen2155@noreply.gammaspectra.live>
Co-committed-by: pwgen2155 <pwgen2155@noreply.gammaspectra.live>
 2025-05-02 11:00:39 +00:00
WeebDataHoarder
3047dcfd4b examples/forgejo: Restrict meta tag fetching for likely bots 2025-05-01 16:15:28 +02:00
WeebDataHoarder
868c76eeb9 examples/forgejo: add commit graph endpoint to heavy resources 2025-05-01 14:20:03 +02:00
WeebDataHoarder
d6c29846df condition: generalize AST compilation, hot load network prefix blocks as needed, walk the AST and detect and preload networks 2025-05-01 02:40:43 +02:00
WeebDataHoarder
6e47cec540 examples/forgejo: allow releases summary-card fetch 2025-05-01 02:34:14 +02:00
WeebDataHoarder
a9f03267b6 settings: allow transparent backends that don't set all values 2025-04-30 20:54:50 +02:00
WeebDataHoarder
b3cd741bee readme: note that port is necessary in case of non-standard port usage 2025-04-30 03:08:18 +02:00
WeebDataHoarder
62277aac64 examples: modify spa to allow cookie fallback on other endpoints 2025-04-28 17:30:23 +02:00
WeebDataHoarder
6db839e23f examples: add spa.yml for single page application examples 2025-04-28 17:25:49 +02:00
WeebDataHoarder
e49c4ae72f action/context: add capability to set response headers 2025-04-28 12:40:03 +02:00
WeebDataHoarder
3234c4e801 feature: Implement <meta> tag fetcher from backends with allow-listed entries to prevent unwanted keys to pass 2025-04-27 21:40:59 +02:00
WeebDataHoarder
957303bbca examples: Do not block generic tools on generic.yml by default 2025-04-27 21:19:17 +02:00
WeebDataHoarder
d36d8354a2 examples: clarify rules order, default action and standard-tools rule 2025-04-27 20:53:30 +02:00
WeebDataHoarder
d665036d98 examples: move desired-crawlers before undesired-networks 2025-04-25 17:59:16 +02:00
WeebDataHoarder
398675aa3c config: Add string replacement for templates, add example config.yml (close #10) 2025-04-25 17:32:45 +02:00
Alan Orth
6f3d81618c examples: add TikTokSpider 
Requests using this user agent are coming from the same Amazon net-
works as Bytespider.
 2025-04-25 11:02:48 +03:00
WeebDataHoarder
1f84f5e981 examples: forgejo: Add branches/tags listing on repo to API endpoints 2025-04-24 20:51:15 +02:00
Alan Orth
fc7d67ad70 Add examples/snippets/bot-uptimerobot.yml 
Add network prefixes and user agent for UptimeRobot.

Source: https://uptimerobot.com/help/locations/
 2025-04-24 13:39:33 +03:00
WeebDataHoarder
a0224cb21c policy: allow fetching ASN directly via RADb WHOIS service 2025-04-23 20:35:20 +02:00
WeebDataHoarder
3b11792594 Implement policy snippets 2025-04-23 20:35:20 +02:00
WeebDataHoarder
d83fe3653a examples: update bot matches, allow badges to be fetched 2025-04-23 20:35:20 +02:00
WeebDataHoarder
ead41055ca Condition, rules, state and action refactor / rewrite 
Add nested rules
Add backend action, allow wildcard in backends
Remove poison from tree, update README with action table

Allow defining pass/fail actions on challenge,

Remove redirect/referer parameters on backend pass

Set challenge cookie tied to host

Rewrite DNSBL condition into a challenge

Allow passing an arbitrary path for assets to js challenges

Optimize programs exhaustively on compilation

Activation instead of map for CEL context, faster map access, new network override

Return valid host on cookie setting in case Host is an IP address.
bug: does not work with IPv6, see https://github.com/golang/go/issues/65521

Apply TLS fingerprinter on GetConfigForClient instead of GetCertificate

Cleanup go-away cookies before passing to backend

Code action for specifically replying with an HTTP code
 2025-04-23 20:35:20 +02:00
Alan Orth
5189dd41f9 examples: fix duckduckbot regex 
Each IP entry in the table is wrapped in a div.
 2025-04-23 00:40:12 +03:00
Alan Orth
14c6a42e88 examples/generic.yml: fix duckduckbot URL 2025-04-23 00:39:17 +03:00
WeebDataHoarder
629e6b40b4 Expand homesite match for forgejo template 2025-04-19 13:32:51 +02:00
WeebDataHoarder
6202f5a642 Remove http-cookie-check on generic.yml 2025-04-19 07:41:07 +00:00
WeebDataHoarder
f04d801a43 Add user/org profile to homesite rule 2025-04-18 11:03:19 +02:00
WeebDataHoarder
2866cf8e80 Add undocumented Google-PageRenderer to Googlebot preset on forgejo 2025-04-16 23:34:28 +02:00
WeebDataHoarder
6458e6d019 Create negative match based on Forgejo reserved word list for embedding homesites 2025-04-16 18:48:34 +02:00
WeebDataHoarder
f690cfaac3 Move bots early 2025-04-16 18:12:00 +02:00
WeebDataHoarder
ce8bc52d94 Explicitly support plaintext browsers (Lynx) and serve challenges that they can solve 2025-04-15 21:03:22 +02:00
WeebDataHoarder
c84c67439e Only error when target network list is not reachable 2025-04-15 19:23:04 +02:00
WeebDataHoarder
a5be4faa8a Default to forgejo-auto on forgejo 2025-04-13 19:06:37 +02:00
WeebDataHoarder
b1620e4d92 Target alpn-less scrapers 2025-04-13 17:31:06 +02:00
WeebDataHoarder
1c0e015c69 Added JA4 sample to forgejo example 2025-04-13 16:43:27 +02:00
WeebDataHoarder
6d5aaddd03 New README 2025-04-13 13:10:56 +02:00
WeebDataHoarder
06e8556d68 Add generic template, update README 2025-04-13 11:50:03 +02:00
WeebDataHoarder
cb0a3732bd Update forgejo template 2025-04-13 11:24:58 +02:00
WeebDataHoarder
15472b00b8 Add older clients, change standard-tools challenge 2025-04-09 00:08:53 +02:00
WeebDataHoarder
ce111f6ae9 Add DNSBL querying in conditions 2025-04-08 22:11:58 +02:00