Skip to main content

Setup guide

Elastic Security threat intelligence over TAXII 2.1, for indicator match rules

The Custom Threat Intelligence integration polls an isMalicious TAXII 2.1 collection with your API key and keeps one document per indicator, ready for indicator match rules. Without a TAXII plan, a list loads as a value list.

No credit card required · Free API key

Data path
  1. isMalicious

    api.ismalicious.com/taxii/

    c2-indicators

  2. Elastic Security

    Step 3

    Point it at one collection

  3. Create an indicator match rule

    Step 5
On this page08

What you get

One integration polls one collection. Start with the category collections below; the broad ones, such as malicious-ips and malicious-domains, are not filtered by category, so read a collection’s size before you add it (Verify). The lists load as value lists instead.

ListEntriesRebuiltPlans
c2-indicatorsStart here: IPs and domains in the C2 and botnet categories.pagedon each pollPro and Enterprise
ransomware-iocsDomains, IPs, URLs and file hashes in the ransomware category.pagedon each pollPro and Enterprise
phishing-indicatorsDomains and URLs in the phishing category.pagedon each pollPro and Enterprise
blocklist-ips-critical.txtWithout TAXII: a value list of type IP addresses. IPs listed by 6 or more threat sources, or 3 or more with a critical category.about 89,000every 12 hBasic, Pro, and Enterprise
blocklist-domains-c2.txtWithout TAXII: a value list of type Keywords, with the command-and-control domains.about 22,000every 12 hBasic, Pro, and Enterprise
/app/integrations/elasticIOCs pushed to an Elasticsearch index through the Bulk API. Not a feed: see Limits.up to 1,000 per syncwhen you syncPro and Enterprise

Entries rounded from the build of ; each list is rebuilt every 12 hours. Today’s counts are public and need no key.

What each plan receives

  • FreeFree account, or no key: the first 10% of each list, marked X-Blocklist-Version: lite.
  • Basic, Pro, and EnterpriseBasic, Pro, and Enterprise: every list in full.
  • Pro and EnterpriseTAXII 2.1 collections: Pro and Enterprise.
  • Pro and EnterpriseSIEM destinations in the dashboard (Splunk, Elastic, Sentinel): Pro and Enterprise.

Compare plans

What a download returnsHTTP

GET https://api.ismalicious.com/blocklist/download/blocklist-ips-critical.txt

Full list or 10% sample

FieldBasic, Pro, and EnterpriseFree
X-Blocklist-Version:fulllite
X-Blocklist-Percentage:10010
Total entries:<COUNT><COUNT> (Lite Version - 10% of <TOTAL>)

First lines of the file

# IsMalicious.com Blocklist - IPs (Critical)
# Format: Plain
# Generated: <BUILD_TIME>
# Total entries: <COUNT>
# Update frequency: every 12 hours
# Category: All
# Threat level: Critical
# Website: https://ismalicious.com
# © <YEAR> IsMalicious (compilation). Licensed to the downloading account under https://ismalicious.com/terms; redistribution of the compilation prohibited. Third-party entries remain under their providers' licences — see https://ismalicious.com/sources.
#

Values in angle brackets are set by each build.

Prerequisites

  • Elastic Stack with Fleet and Elastic Agent, and the Custom Threat Intelligence integration 1.8.2 or later, which needs Kibana 8.19.16 or later on 8.19, or 9.3.5 or later on 9.x: Kibana 9.0 to 9.3.4 only gets 1.7.2. Agentless runs (now Elastic Managed Integration) are GA on Serverless and on Elastic Cloud Hosted from 9.5, a technical preview on 9.0 to 9.4.
  • DNS, proxy or network events with ECS fields such as dns.question.name, source.ip and destination.ip, for the rules to match.
  • An isMalicious API key and secret, from Account › API access.
  • Outbound HTTPS (TCP 443) over IPv4 from the Elastic Agent, or Elastic Cloud in agentless mode, to api.ismalicious.com, which has no IPv6 address.

Set it up

  1. Copy your API key and secret

    Open Account › API access and copy the API Key and the API Secret. The TAXII collections need a Pro or Enterprise plan; other plans get a 403. The value lists need Basic, Pro, or Enterprise.
  2. Add the Custom Threat Intelligence integration

    In Kibana, open Integrations, search for Custom Threat Intelligence and click Add Custom Threat Intelligence. Use version 1.8.2 or later: before it, the pages after the first lose added_after, and the server then sends every indicator on those pages.
  3. Point it at one collection

    • Turn on Collect STIX data via RESTful API, which is off when the integration is added, and fill in the values below; the credentials, Limit and Feed name sit under the advanced options.
    • The integration sends the key and the secret as HTTP Basic on every request.
    • Leave API Key empty (when set, it is sent as a Bearer token instead), the Accept and Content-Type headers at their defaults, and never add a query string to the URL: the integration appends its own.
    • Keep Initial Interval set: without it the integration never sends added_after.
    • Here the first poll reads 30 days.
    Custom Threat Intelligence settingsKibana
    URL API endpoint
    https://api.ismalicious.com/taxii/api-root/collections/c2-indicators/objects/
    Enable TAXII 2.1
    on
    Interval
    1h
    Initial Interval
    720h
    IOC Expiration Duration
    90d

    # Advanced options:

    Basic Auth Username
    <API_KEY>
    Basic Auth Password
    <API_SECRET>
    Limit
    1000
    Feed name
    isMalicious c2-indicators
    URL API endpoint ........ https://api.ismalicious.com/taxii/api-root/collections/c2-indicators/objects/
    Enable TAXII 2.1 ........ on
    Interval ................ 1h
    Initial Interval ........ 720h
    IOC Expiration Duration . 90d
    
    # Advanced options:
    Basic Auth Username ..... <API_KEY>
    Basic Auth Password ..... <API_SECRET>
    Limit ................... 1000
    Feed name ............... isMalicious c2-indicators
  4. Save it, then repeat per collection

    Choose the agent policy, or agentless on Elastic Cloud or Serverless, then Save and continue. The URL names the collection, so each collection needs its own integration, with a feed name that starts with isMalicious: the rule below matches them all.
  5. Create an indicator match rule

    • In Detection rules (SIEM), click Create new rule, then Indicator Match.
    • Read the indicators from logs-ti_custom_latest.indicator, the alias that keeps one document per indicator and drops it when its stix.ioc_expiration_date passes.
    • Replace the default indicator index query, @timestamp > "now-30d/d", with the one below, which has no date filter: the integration sets @timestamp to the STIX created date, and Elastic keeps the dates of an indicator’s first delivery, so a filter on either date drops indicators that sources still report.
    • The integration stores domains in threat.indicator.url.original; for IPs, a second rule on your network indices maps destination.ip, or source.ip in a separate group, to threat.indicator.ip.
    Indicator match rules for domains and IPsKibana
    Rule type
    Indicator Match
    Index patterns
    <DNS_INDEX_PATTERN>
    Custom query
    dns.question.name:*
    Indicator index patterns
    logs-ti_custom_latest.indicator
    Indicator index query
    threat.feed.name:isMalicious* and threat.indicator.type:"domain-name"
    Indicator mapping
    dns.question.name MATCHES threat.indicator.url.original

    # The IP rule, on your network indices:

    Indicator index query
    threat.feed.name:isMalicious* and threat.indicator.type:(ipv4-addr or ipv6-addr)
    Indicator mapping
    destination.ip MATCHES threat.indicator.ip
    OR
    source.ip MATCHES threat.indicator.ip
    Rule type ................ Indicator Match
    Index patterns ........... <DNS_INDEX_PATTERN>
    Custom query ............. dns.question.name:*
    Indicator index patterns . logs-ti_custom_latest.indicator
    Indicator index query .... threat.feed.name:isMalicious* and threat.indicator.type:"domain-name"
    Indicator mapping ........ dns.question.name MATCHES threat.indicator.url.original
    
    # The IP rule, on your network indices:
    Indicator index query .... threat.feed.name:isMalicious* and threat.indicator.type:(ipv4-addr or ipv6-addr)
    Indicator mapping ........ destination.ip MATCHES threat.indicator.ip
                        OR ... source.ip MATCHES threat.indicator.ip
  6. Or load a list as a value list

    • Without a TAXII plan, download a list, compare its two counts, check that it is not the 10% sample, and strip its # header lines.
    • In Detection rules (SIEM) › Manage value lists, choose the type IP addresses (Keywords for domains), select the file and click Import value list: the list takes the file name as its id.
    • In an indicator match rule, use .items-default as the indicator index (in another space, .items- followed by the space id) and query that id; a Keywords list maps to keyword.
    • To refresh, remove the value list and import it again: importing a file with the same name adds to the list instead of replacing it.
    Download and prepare a value listsh
    # ~/.ismalicious.netrc (mode 600) holds the three netrc lines of your key.
    curl -fsS --netrc-file ~/.ismalicious.netrc -o list.txt \
      https://api.ismalicious.com/blocklist/download/blocklist-ips-critical.txt
    # The header's count, the entries' count, and the 10% sample marker:
    grep -m1 'Total entries' list.txt
    grep -c '^[^#]' list.txt
    grep -q 'Lite Version' list.txt && echo 'lite list: check the key and the plan'
    grep -v '^#' list.txt > ismalicious-ips-critical.txt
    Indicator match rule on a value listKibana
    Indicator index patterns
    .items-default
    Indicator index query
    list_id : "ismalicious-ips-critical.txt"
    Indicator mapping
    destination.ip MATCHES ip

    # A Keywords list of domains maps to keyword:

    Indicator mapping
    dns.question.name MATCHES keyword
    Indicator index patterns . .items-default
    Indicator index query .... list_id : "ismalicious-ips-critical.txt"
    Indicator mapping ........ destination.ip MATCHES ip
    
    # A Keywords list of domains maps to keyword:
    Indicator mapping ........ dns.question.name MATCHES keyword
  7. Push IOCs to an index (optional, Pro or Enterprise)

    • Create an Elasticsearch API key with the index and auto_configure privileges on ismalicious-* and copy its encoded value.
    • In /app/integrations/elastic, enter the Elastic base URL (the Elasticsearch endpoint, not Kibana), the encoded value as API key, and ismalicious-iocs as IOC index.
    • Click Test, then Sync now: each sync sends up to 1,000 IOCs (100 by default) from where the last one stopped.
    • Then run GET ismalicious-iocs/_count in Dev Tools: the Bulk API answers 200 even when it rejects every document, so a key without auto_configure or a mapping conflict still reads as delivered in the dashboard.

Verify it works

  • In Fleet, the agent policy shows the integration healthy. A failed request appears as an event whose error.message starts with GET: and holds the server’s answer.
  • In Discover, create a data view on logs-ti_custom_latest.indicator and widen the time range to several years: @timestamp is the date a source first reported each indicator. The query below then shows the indicators, with threat.indicator.type, threat.indicator.ip or threat.indicator.url.original, and stix.ioc_expiration_date.
  • The collection resource gives its size in x_ismalicious_total, counted every 12 hours and absent until the first count. The first poll reads only the Initial Interval, and an indicator held by two collections is kept once, so expect fewer.
  • The rule’s execution results list its runs. An event with a listed value raises an alert enriched with the matched indicator.
  • Manage value lists opens the list from its file name; Total items there matches the prepared file’s line count.
ChecksKibana · sh
# Discover, on a data view for logs-ti_custom_latest.indicator, with a time
# range of several years (@timestamp is the date a source first reported it):
threat.feed.name:isMalicious*

# The collection's size, counted every 12 hours (x_ismalicious_total):
curl -fsS --netrc-file ~/.ismalicious.netrc \
  -H 'Accept: application/taxii+json;version=2.1' \
  https://api.ismalicious.com/taxii/api-root/collections/c2-indicators/

# Pushed documents, in Dev Tools:
GET ismalicious-iocs/_count

Troubleshooting

401 Unauthorized

The key or the secret is wrong, or missing: re-enter Basic Auth Username and Basic Auth Password. Regenerating the key in Account › API access invalidates the old pair.

403 UPGRADE_REQUIRED

The TAXII collections need a Pro or Enterprise plan. With another plan, use the value lists.

429 Too Many Requests

A Pro plan allows 60 requests per 60 seconds across the account, and every page is one request, so integrations polling at the same time add up. Elastic Agent retries a 429 up to 5 times, waiting the Retry-After delay. If the last retry fails, the walk stops, and the next interval starts a new walk after the last page read, which can skip indicators (see Limits). Keep Limit at 1000, poll fewer collections at once, or move to Enterprise, which allows 5,000 requests per 60 seconds.

“exceeding maximum number of CEL executions”

The walk needed more pages than Maximum Pages Per Interval. Elastic Agent 8.19 logs a warning only, 9.0 to 9.3 mark the integration Degraded, and 9.4 and later log “reached maximum number of CEL executions” without a status change; the walk continues at the next interval. Raise that setting, or shorten Initial Interval.

400 Bad Request

On every request: the URL carries its own query string, such as ?min_score=; remove it. Once, with “Invalid or expired next pagination token”: the walk stops, and the next interval starts a new walk after the last page read, which can skip indicators (see Limits).

503 or 504

A transient error on our side. Elastic Agent retries it up to 5 times; if every retry fails, the next interval starts a new walk after the last page read.

No alerts while indicators exist

The indicator index query still filters on a date, @timestamp or threat.indicator.last_seen, or domains are mapped to threat.indicator.url.domain instead of threat.indicator.url.original.

A value list holds about a tenth of the list

No credential reached the server, or the plan is Free or lapsed: past due, unpaid, canceled, incomplete or paused counts as Free. The file’s Total entries line then says Lite Version. TAXII has no sample: a plan without TAXII gets a 403.

The push test fails

The base URL must be public HTTPS: private, loopback, link-local and reserved addresses are refused, and a delivery times out after 5 seconds.

Limits

  • Incremental polls can miss indicators. The integration takes the next added_after from the last page read, whose X-Taxii-Date-Added-Last header reports the latest date on that page only, and pages follow storage order, not date order. A walk cut short, by failed retries, an expired next token or an agent restart, therefore skips most unread pages, and an indicator added during a long walk, behind its position, can be skipped by later polls. The first 30-day walk is the most exposed; small collections narrow the window. It has not been measured.
  • Elastic keeps each indicator as first delivered: a later report does not resend it, so it leaves logs-ti_custom_latest.indicator 90 days after the last-seen date it had then, even if sources still report it.
  • An indicator held by two collections is kept once, under one of the feed names: the alias keys on the STIX id, which is the same in every collection. Match on threat.feed.name:isMalicious*, never on one feed name.
  • Domains land in threat.indicator.url.original, a choice of the integration. The prebuilt indicator match rules map no DNS field to it: write the domain rule yourself.
  • No score filter: the integration appends its own query string to the URL, so min_score and max_score cannot be set.
  • The push is not a feed: it runs when you click Sync now and sends at most 1,000 IOCs per sync. Its documents carry isMalicious fields, not @timestamp or ECS threat.indicator fields, so an indicator match rule cannot read them without an ingest pipeline of your own.
  • Value lists are refreshed by hand, accept files up to 9 MB by default, and a re-import with the same name adds to the list.
  • No CSV, JSON, CIDR or wildcard lists are served. The TAXII patterns hold one value each, which the integration parses fully.

Questions

Which version of Custom Threat Intelligence do I need?

1.8.2 or later, which needs Kibana 8.19.16 or later on 8.19, or 9.3.5 or later on 9.x. Before 1.8.2, the pages after the first lose added_after, so the server sends every indicator again on those pages.

Why change the default indicator index query?

The default, @timestamp > "now-30d/d", keeps indicators ingested in the last 30 days, and the integration sets @timestamp to the date a source first reported the indicator. Elastic also keeps the dates of an indicator’s first delivery, so any date filter drops indicators that sources still report. Query on the feed name and the type instead: the latest-indicator alias already drops an indicator when its expiration date passes.

How fast can a Pro plan poll?

A Pro plan allows 60 TAXII requests per 60 seconds across the account, one per page, and a page holds up to 1,000 indicators with Limit at 1000. Enterprise allows 5,000 requests per 60 seconds. Polls do not count against the monthly request quota.

Can I use isMalicious in Elastic Security without a TAXII plan?

Yes, with value lists. A Basic, Pro, or Enterprise plan downloads the full lists, which you import as value lists and refresh by hand. The TAXII collections need Pro or Enterprise.

Get Started

Ready to get started?

No credit card required · Free API key