Filter traffic with IP lists

An IP filter source is a list of addresses and networks that Core downloads on a schedule and Edge enforces. A request from a listed address is rejected with 403 and counted under the IP filter block source. Every source is created disabled and starts in Observe mode, so nothing is blocked until you promote it.

Core downloads each list directly from its provider. Clearplane does not host or redistribute any list, and no provider receives a request until you enable that source.

Sources

Pick a provider when you create a source, then a list and, where the provider publishes several formats, a format. Clearplane offers Spamhaus DROP, FireHOL, Emerging Threats, Blocklist.de, CINS Army, Feodo Tracker, Tor exit nodes, GreenSnow, Binary Defense and IPsum, each with its published lists, plus a custom plain-text source for any HTTPS URL that returns one IP address or CIDR range per line. IP filter providers and lists shows every list with its download URL, format and the reason it records.

One source can combine several lists from Spamhaus DROP, FireHOL, Emerging Threats, Blocklist.de or IPsum, such as FireHOL levels 1, 3 and 4. Core downloads them together and merges them into one list, so the source has one health, one mode and one indicator count, and the indicator and storage limits apply to the merged list. An address on more than one of the lists is stored once, with the reason of one of them. If any of the lists fails to download or parse, the refresh fails and the previous version stays active. The Adapter column shows the provider and the lists it combines.

A custom source takes comment prefixes to skip, plus an optional reason and category. Its URL must be an absolute HTTPS URL on the default port, with a DNS hostname (no IP literals, no embedded credentials, no fragment).

Large lists need room. The FireHOL anonymous and proxies lists hold millions of entries, so raise the Edge container and memory settings before enabling them next to other large lists (see Capacity and Edge memory).

Each source also has a priority (unique; the lower number wins when several sources list the same address), a refresh interval (60 minutes to 7 days, default 6 hours) and a maximum stale age (default 24 hours, at least the refresh interval; see Expiry and staleness).

Safe workflow

  1. Create the source. Pick a provider, then one or more lists and, where the provider publishes several formats, the format. The remaining fields appear once you have chosen. It starts disabled in Observe mode.
  2. Test it. Core downloads and parses the list without changing what Edge enforces, and reports the result. A Test is valid for 15 minutes and only for the settings it ran with; changing the URL or other source settings invalidates it.
  3. Enable it. Enabling is refused unless a valid Test exists. The source is enabled in Observe mode and the first refresh runs immediately.
  4. Review. In Observe mode Edge records what the list would have blocked and lets the request through. Open the Observed tab to see which addresses it matched.
  5. Allowlist any legitimate address you find. The Observed tab's Allowlist action opens the entry for that address and source.
  6. Promote the source to Block. Switch to Block first shows the last seven days of the source's Observe activity: requests it would have blocked, distinct addresses, allowlisted requests and the routes affected. It warns when the source has observed for less than that or when Edge dropped records, and Review matches opens the Observed tab instead. Promotion requires an enabled source with Healthy health. Returning to Observe is always allowed.

Disabling a source removes its downloaded data and stops it from affecting any request.

Route scope

A source is either Global or Route. A route-scoped source applies only to the routes you select. Choose routes before promoting a source to Block if the list should protect only part of your estate.

Allowlist

An allowlist entry is an IP address or a CIDR range with no host bits set, a required reason, and an optional expiry. Scope it as:

  • Global: every route.
  • Route: only the routes you select.
  • Source: optionally limited to one source; without a source, the entry suppresses every source.

An allowlisted address is never blocked by an IP filter, never banned automatically, and cannot be banned manually (creating the ban returns 409). An expired entry stops protecting the address immediately.

The IP filters page opens on the Allowlist tab, followed by Sources and Observed.

Observed tab

The Observed tab shows what Observe-mode sources would have blocked over the last 24 hours, 7 days or 30 days, limited to the matched-address retention. A card per source shows its matches, distinct addresses, the routes they reached and the requests an allowlist entry let through, with the time the source started observing when that falls inside the range. Below the cards, the addresses each source matched are listed with country, reason, route, first and last seen, and hits since first seen; filter them by source, route and country. Allowlist on a row opens an allowlist entry for that address, limited to that source.

On the Sources tab, Matches (7d) counts each source's matches in its current mode over the last seven days, and Review matches opens the Observed tab filtered to that source.

Edge keeps a bounded number of matched addresses for each mode between reports to Core, so a busy Observe list can never crowd out the Block records you need to resolve a report. When the registry fills, the Observed tab and the promotion preview say how many records Edge dropped, so missing data is never shown as zero matches. With address recording off, the tab keeps the counts and hides the address list.

The allowlist only exempts addresses from bans and IP filter sources. To block a single address or a range yourself, use an access control policy or alias; access control runs after bans and IP filters, so an allowlisted address is still subject to it.

Expiry and staleness

Core keeps the last good download of a source. Each successful refresh, including one where the provider reports the list unchanged, extends that download's validity by the source's maximum stale age (default 24 hours, never shorter than the refresh interval). When refreshes keep failing, the source becomes Degraded while the old data remains valid, and then Expired. An expired source stops observing and blocking. Edge applies expiry on its own, even if Core is unreachable, so a source that stops refreshing cannot block forever.

A refresh that fails validation, exceeds a capacity limit or cannot be parsed never replaces the active data.

Capacity and Edge memory

Settings under IP filter settings bound the feature:

Setting Default
Maximum indicators per source 10,000,000
Maximum active indicators across sources 10,000,000
Maximum Edge memory (MB) 1,536
Maximum Core storage (MB) 3,072
Record matched addresses On
Matched address retention (days) 30 (1 to 365)

Edge compiles each update into in-memory indexes and swaps them in only when the whole candidate fits the memory ceiling. A candidate that does not fit is rejected, and Edge keeps its previous data. Edge keeps about 32 bytes per indicator and estimates 40. While it builds a new generation it also holds the downloaded list, so it needs room for the active generation, the candidate and about 80 bytes per candidate indicator at the same time. A full 10,000,000 indicators therefore peaks at roughly 1.5 GB, which is what the default 1,536 MB ceiling allows.

Edge also keeps one shared memory budget for everything that holds large data in memory. It admits at most 85% of the Edge container's memory limit, sets 512 MB of that aside as a fixed baseline for the runtime and request handling, and shares the rest between the WAF rulesets, the IP filter lists, the response cache and the country data. A list is admitted only if the active and candidate generations fit in what the other consumers leave free, in addition to the Maximum Edge memory ceiling. The response cache gives way first: it counts as reclaimable, so Edge trims the oldest cache entries to make room for a new list or ruleset before it refuses one, and the cache never grows past what the other consumers leave free. Responses Edge is still holding to store them count as reserved and cannot be trimmed, so they reduce the room for a new list until they finish. Before Core activates a list it asks Edge for its current budget, so a list that Edge could not hold fails the refresh with a message that says how much memory it needs and how much is left. Edge checks again before it applies the data and keeps the previous generation if the budget changed in between. On the stock 2 GiB container the pool is about 1.2 GiB for all four, so raise the Edge container's limit if you run large lists next to many WAF rulesets. The Memory budget card under System → Services → Edge shows the split, and the IP filter settings dialog warns when the Edge memory setting is larger than the pool. Changing the container limit in your Compose file takes effect after Edge restarts.

The default Compose limits are 2 GiB for Edge and 1 GiB for Core. The ceiling leaves Edge about 500 MB for everything else, so give Edge more memory and raise the ceiling together if you enable large lists next to many WAF rulesets. Core does not hold a list in memory: it reads the download as a stream and writes each address to its database as it goes, so Core's memory stays flat however long the list is, and its storage grows with the list instead. Core keeps up to two copies of each source (the active one and the one being replaced). The Maximum Core storage setting bounds the estimated size of all stored lists, about 130 bytes per indicator plus the length of its reason and category, and a refresh that would exceed it fails and keeps the previous list. A full 10,000,000 indicators takes about 1.3 GB per copy, which is why the default is 3,072 MB. The setting counts live data only; SQLite does not shrink its file after a list shrinks, so make sure the disk behind clearplane-core-data has room beyond it. The limits accept up to 1,000,000,000 indicators and the Edge ceiling up to 1,048,576 MB. Clearplane trusts the values you set, so size the containers to match. Core accepts a downloaded document of up to 64 bytes per allowed indicator, with a minimum of 64 MB, and a list that takes longer than 30 minutes to read fails.

The same settings control matched-address recording for Block and Observe sources.

Licensing and attribution

Each provider sets its own terms, including who may use a list and for what. You are responsible for respecting them. Provider lists need no key or settings, and addresses in private, loopback, link-local and other reserved ranges are skipped so a list can never block internal clients. Spamhaus DROP is published under Spamhaus's terms of use for its free threat feeds; see the DROP page.

REST API and permissions

Permission Allows
IpFilters.Read List and view sources, allowlist entries and settings
IpFilters.Write Create and edit sources and allowlist entries; test, enable, disable, change mode, refresh
IpFilters.Manage Delete sources and allowlist entries; change settings

Routes: api/ip-filter-sources (plus /{id}/test, /enable, /disable, /mode, /refresh, and activity, /{id}/activity and observations/search for the Observed tab), api/ip-filter-allowlist-entries (search, create, get, update, delete) and api/ip-filter-settings. See the REST API reference for authentication.