Collection design · September 2026
Discover. Correlate. Review.
Existing candidates receive first-pass analysis before new searches. The 124 detailed-case repositories receive a weekly branch check; the other 576 public repositories are outside that watchlist. The public register remains a dated, reviewed release; raw search hits stay private.
What runs automatically
Editable GitHub queries collect candidates on their configured intervals. An hourly enrichment job processes up to 30 candidates. Every six hours, discovery first processes up to six candidates, then saves returned search matches when first-pass and decision-recovery backlogs permit. Repository, path and Git blob identify a file version, while every matching rule retains its own provenance.
Six reviewed task byte patterns for 500-font and 400-font targets can lead to their payload automatically: resolve a public branch, verify the same task bytes in that commit tree, then inspect the referenced blob as inert text. Unknown templates remain unsupported. The persistent backlog drains even when no query is due, and repeated blobs reuse verified metadata across jobs, preserving the original byte inspection time and source repository. Each repository still gets its own task/tree check.
Search requests remain seven seconds apart; core metadata requests are one second apart. Each discovery job allows at most 42 GitHub requests and 12 blob reads. Hourly enrichment allows up to 180 GitHub requests and 60 blob reads. The weekly job supports up to 150 watchlist repositories and reports unavailable rows.
How collection runs
Cloudflare hosts the public site, an Access-protected rule editor and the D1 evidence database. An isolated GitHub Actions job runs our analysis code, retrieves bounded public GitHub text into memory, and sends allowlisted metadata to the ingestion API. Target projects are never checked out or executed.
- Hourly enrichment: up to 30 candidates and 60 source inspections. Discovery every six hours: up to six search pages and twelve source inspections, with separate search and core request spacing.
- Detailed-case watchlist branch metadata: weekly. Historical before/after tree recovery remains analyst-driven.
- Rules: edit label, query, campaign, enablement, interval and pagination depth without a deployment. Every query revision is retained.
- Correlation: wallets, hashes, normalized loader bodies, version markers, literal IPs and URL hosts retain extraction roles and source observations.
- Review: exact known bytes can be confirmed privately; manual holds take precedence. A private preview identifies task/payload records eligible for future publication. Public snapshots still require a reviewed change. Private Discord alerts are available when configured; repository notices are not sent.
A schedule is not proof of successful collection. The public collection status page shows recorded successes, scheduled-trigger evidence and stale rules. The private run history adds errors, request counts and search limits. Public evidence includes separate 500-font and 400-font query audits.
Different ways to expand collection
| Approach | What it finds | Tradeoff |
|---|---|---|
| Repeated code-search query packs | Fresh indexed occurrences across unrelated repositories | Default-branch indexing, result caps and known index gaps; partition broad hunts |
| Rare co-change templates | Tasks, ignore-file changes and config tails that travel together | Needs tree comparison; a filename or extension recommendation is weak alone |
| Public push archives | Historical heads and changes missing from current Git history | Archive coverage and event schemas vary; recover before/after IDs where available |
| Wallet publication monitoring | Infrastructure advertised by a known sender, including repeated publications | Not every transaction rotates C2; an advertised address is not reachability evidence |
| Existing urlscan and report lookups | Cross-platform IP, URL-path and response-hash pivots | Search existing records only; no new scans or visits to payload servers |
| Registry advisory metadata | npm/PyPI package names, versions and source-reported behavior | Keep advisory claims separate from independently inspected repository artifacts |
| Authorized GitHub App webhooks | Low-latency events for repositories that install the app | Cannot subscribe to arbitrary third-party repositories without their authorization |
Limits shape the design
GitHub code search permits 10 API requests per minute and returns at most 1,000 results per query. Each page costs a request. Running on GitHub infrastructure does not remove those API limits. The collector keeps a lower local budget and stops on denied or rate-limited requests. GitHub search reference ↗
Authenticated core REST requests generally have a separate 5,000/hour allowance; workflow tokens normally have 1,000/hour per repository. Actual response headers take precedence. REST limits ↗
Cloudflare Workers Free has 100,000 requests/day, 10 ms CPU per invocation and 50 external subrequests. Paid Workers have different allowances. Heavy decoding and future YARA evaluation fit the isolated runner better than the free CPU budget. Workers limits ↗
This repository is private, so hosted runner time uses the owner's Actions allowance. GitHub Free includes 2,000 minutes/month; Pro includes 3,000. As a planning example, one two-minute run every hour is 1,440 minutes in a 30-day month, before weekly checks and other workflows. This is an estimate, not measured usage. Actions billing ↗
Data that supports defensible connections
Search matches, immutable observations and canonical entities have separate records. A wallet includes its chain namespace. IP observations retain whether the value was an unclassified literal, a source-reported address or decoded from a dated transaction. Every edge has a role, confidence, extraction method and source.
Collection time, commit author/committer dates and server event time are separate fields. “First observed here” is not “first infected.” Missing input, oversized files and incomplete results retain their uncertainty. No raw malware is stored in the evidence database.
Private rule editor →Reviewed connections →Try manual searches →