Skip to main content

Setup guide

QRadar reference set of malicious IPs and domains, refilled through the REST API

QRadar downloads no list itself, so a script downloads the full isMalicious list with your API key, empties a reference set and refills it through the QRadar REST API. Rules then test event and flow properties against the set.

No credit card required · Free API key

Data path
  1. isMalicious

    api.ismalicious.com

    blocklist-ips-critical.txt

    Rebuilt every 12 h

  2. IBM QRadar

    Step 4

    Install the script and run it once

  3. Use the sets in rules

    Step 6
On this page08

What you get

Each list fills one reference set: type IP for the IP lists, ALNIC for the domains. Start with the lists below; IBM’s pages read for this guide state no maximum set size.

ListEntriesRebuiltPlans
blocklist-ips-critical.txtDefault: set isMalicious IPs critical, type IP, IPv4 entries only. IPs listed by 6 or more threat sources, or 3 or more with a critical category.about 89,000every 12 hBasic, Pro, and Enterprise
blocklist-ips-c2.txtSet isMalicious IPs C2, type IP: command-and-control IPs reported by C2 trackers.about 44,000every 12 hBasic, Pro, and Enterprise
blocklist-domains-c2.txtSet isMalicious domains C2, type ALNIC: command-and-control domains.about 22,000every 12 hBasic, Pro, and Enterprise
blocklist-domains-ransomware.txtSet isMalicious domains ransomware, type ALNIC: domains in the ransomware category.about 3,600every 12 hBasic, Pro, and Enterprise
c2-indicatorsOver TAXII 2.1, with the Threat Intelligence app: IPs and domains in the C2 and botnet categories, not the whole corpus. A feed reads one observable type, so the collection is added twice: IPs into an IP reference set, domains into an ALNIC one.pagedon each pollPro and Enterprise

Entries rounded from the build of ; each list is rebuilt every 12 hours. Today’s counts are public and need no key.

What each plan receives

  • FreeFree account, or no key: the first 10% of each list, marked X-Blocklist-Version: lite.
  • Basic, Pro, and EnterpriseBasic, Pro, and Enterprise: every list in full.
  • Pro and EnterpriseTAXII 2.1 collections: Pro and Enterprise.

Compare plans

What a download returnsHTTP

GET https://api.ismalicious.com/blocklist/download/blocklist-ips-critical.txt

Full list or 10% sample

FieldBasic, Pro, and EnterpriseFree
X-Blocklist-Version:fulllite
X-Blocklist-Percentage:10010
Total entries:<COUNT><COUNT> (Lite Version - 10% of <TOTAL>)

First lines of the file

# IsMalicious.com Blocklist - IPs (Critical)
# Format: Plain
# Generated: <BUILD_TIME>
# Total entries: <COUNT>
# Update frequency: every 12 hours
# Category: All
# Threat level: Critical
# Website: https://ismalicious.com
# © <YEAR> IsMalicious (compilation). Licensed to the downloading account under https://ismalicious.com/terms; redistribution of the compilation prohibited. Third-party entries remain under their providers' licences — see https://ismalicious.com/sources.
#

Values in angle brackets are set by each build.

Prerequisites

  • IBM QRadar SIEM 7.5.0, with REST API 16.0 or later for the bulk update.
  • A host with Python 3 that reaches both api.ismalicious.com and the QRadar console.
  • An isMalicious API key and secret, from Account › API access.
  • Outbound HTTPS (TCP 443) over IPv4 from the script host to api.ismalicious.com, which has no IPv6 address.

Set it up

  1. Copy your API key and secret

    Open Account › API access and copy the API Key and the API Secret. The full lists need a Basic, Pro, or Enterprise plan; with a Free key the script receives the 10% sample, and refuses it.
  2. Create a QRadar authorized service

    In QRadar, create an authorized service with a role allowed to manage reference data, and copy its token: the script sends it in the SEC header.
  3. Create the reference sets once

    Create one set per list through the REST API, which answers with the set and its id: the script needs it. The token goes in a mode-600 curl config file, ~/.qradar.curlrc, not on the command line. Set the Version header to an API version listed at /api_doc/ on your console; without it, QRadar uses the latest version, which IBM warns can break integrations after an upgrade.
    Create a reference setsh
    # ~/.qradar.curlrc (mode 600) holds the token, off the command line:
    #   header = "SEC: <QRADAR_TOKEN>"
    curl -sS -K ~/.qradar.curlrc -X POST "https://<QRADAR_CONSOLE>/api/reference_data_collections/sets" \
      -H "Version: <API_VERSION>" -H "Content-Type: application/json" \
      --data '{"name": "isMalicious IPs critical", "entry_type": "IP", "description": "isMalicious blocklist-ips-critical"}'
    
    # Domain sets: "entry_type": "ALNIC" (alphanumeric, case-insensitive).
  4. Install the script and run it once

    • Put the credentials in /root/.ismalicious-qradar.env, mode 600, and save the script as /usr/local/sbin/ismalicious-qradar.py, mode 700.
    • It downloads the list first and refuses an error, a body that is not a list, the 10% sample and a file whose entry count differs from its header, then drops the # header lines, and IPv6 addresses for IP sets.
    • Only then does it empty the set, wait until it is empty, and load the values in batches of 10,000 through the asynchronous bulk update, waiting up to 30 minutes for each task.
    /root/.ismalicious-qradar.envsh
    export ISM_KEY='<API_KEY>'
    export ISM_SECRET='<API_SECRET>'
    export QRADAR_URL='https://<QRADAR_CONSOLE>'
    export QRADAR_TOKEN='<QRADAR_TOKEN>'
    export QRADAR_API_VERSION='<API_VERSION>'
    # Optional: the console's CA bundle.
    # export QRADAR_CA=/etc/ssl/qradar-ca.pem
    /usr/local/sbin/ismalicious-qradar.pypython
    #!/usr/bin/env python3
    """Refresh a QRadar reference set from an isMalicious list.
    
    Usage: ismalicious-qradar.py <isMalicious file> <set id> <ip|domain>
    Environment: ISM_KEY, ISM_SECRET, QRADAR_URL (https://console), QRADAR_TOKEN,
    QRADAR_API_VERSION (a version the console lists at /api_doc/), optional
    QRADAR_CA (CA bundle path).
    """
    import base64, json, os, re, ssl, sys, time, urllib.error, urllib.request
    
    LIST, SET_ID, KIND = sys.argv[1], int(sys.argv[2]), sys.argv[3]
    BATCH = 10000
    
    def fetch_list(name):
        token = base64.b64encode(f"{os.environ['ISM_KEY']}:{os.environ['ISM_SECRET']}".encode()).decode()
        req = urllib.request.Request(
            f"https://api.ismalicious.com/blocklist/download/{name}",
            headers={"Authorization": f"Basic {token}"})
        try:
            with urllib.request.urlopen(req, timeout=300) as resp:
                text = resp.read().decode("utf-8")
        except urllib.error.HTTPError as err:
            sys.exit(f"{name}: HTTP {err.code}")
        lines = text.splitlines()
        if not lines or not lines[0].startswith(("# IsMalicious.com Blocklist", "! IsMalicious.com Blocklist")):
            sys.exit(f"{name}: not an isMalicious list")
        if "Lite Version" in text:
            sys.exit(f"{name}: lite list received, check the API key and the plan")
        if not text.endswith("\n"):  # every entry ends with a newline: cut inside a line
            sys.exit(f"{name}: cut short, no final newline")
        values = [l.strip() for l in lines if l.strip() and not l.startswith(("#", "!"))]
        totals = [l.split(":", 1)[1].strip().replace(",", "") for l in lines
                  if l.startswith(("# Total entries:", "! Total entries:"))]
        # One header, and as many entries as it says: never empty the set for an
        # empty, cut or doubled file.
        if len(totals) != 1 or not re.fullmatch(r"[0-9]+", totals[0]) \
                or int(totals[0]) == 0 or len(values) != int(totals[0]):
            sys.exit(f"{name}: {len(values)} entries, the header says {' / '.join(totals) or 'none'}")
        if KIND == "ip":
            values = [v for v in values if ":" not in v]  # IP sets take dotted IPv4
        return values
    
    CTX = ssl.create_default_context(cafile=os.environ.get("QRADAR_CA"))
    
    def qradar(method, path, body=None):
        req = urllib.request.Request(
            os.environ["QRADAR_URL"].rstrip("/") + "/api" + path, method=method,
            data=None if body is None else json.dumps(body).encode(),
            headers={"SEC": os.environ["QRADAR_TOKEN"],
                     "Version": os.environ["QRADAR_API_VERSION"],
                     "Content-Type": "application/json", "Accept": "application/json"})
        with urllib.request.urlopen(req, context=CTX, timeout=120) as resp:
            return json.loads(resp.read() or b"null")
    
    def wait(task):
        deadline = time.time() + 1800  # a task left PAUSED or QUEUED must not hang the run
        while task["status"] not in ("COMPLETED", "EXCEPTION", "CONFLICT", "CANCELLED", "INTERRUPTED"):
            if time.time() > deadline:
                sys.exit(f"bulk update task {task['id']} still {task['status']} after 30 minutes")
            time.sleep(5)
            task = qradar("GET", f"/reference_data_collections/set_bulk_update_tasks/{task['id']}")
        if task["status"] != "COMPLETED":
            sys.exit(f"bulk update task {task['id']} ended {task['status']}: {task.get('error_message')}")
    
    values = fetch_list(LIST)                     # download first: never empty the set on a failed fetch
    qradar("POST", f"/reference_data_collections/sets/{SET_ID}", {"delete_entries": True})
    for _ in range(120):                          # the docs do not say whether emptying is synchronous
        if qradar("GET", f"/reference_data_collections/sets/{SET_ID}").get("number_of_entries") == 0:
            break
        time.sleep(5)
    else:
        sys.exit(f"reference set {SET_ID} was not emptied within 10 minutes")
    for i in range(0, len(values), BATCH):
        chunk = [{"collection_id": SET_ID, "value": v} for v in values[i:i + BATCH]]
        wait(qradar("PATCH", "/reference_data_collections/set_entries", chunk))
    print(f"{LIST}: {len(values)} values loaded into reference set {SET_ID}")
    First runsh
    . /root/.ismalicious-qradar.env
    /usr/local/sbin/ismalicious-qradar.py blocklist-ips-critical.txt <SET_ID> ip
  5. Schedule it every 12 hours

    Add one cron line per list and set: the lists are rebuilt every 12 hours.
    /etc/cron.d/ismalicious-qradarcron
    17 3,15 * * *  root  . /root/.ismalicious-qradar.env && /usr/local/sbin/ismalicious-qradar.py blocklist-ips-critical.txt <SET_ID> ip
  6. Use the sets in rules

    In the Rules wizard, write rules that test event or flow properties, such as the source or destination IP, or the DNS or URL host property your log sources provide, against these reference sets.
  7. Or import a file by hand

    Download a list with a netrc file, compare the header’s count with the entries’ count, check that it is not the 10% sample, and strip its header. Then, in Admin › System Configuration › Reference Set Management, select the set, click View Contents, then Import on the Content tab, and select the file: one value per line.
    Prepare a file for the importsh
    # ~/.ismalicious.netrc (mode 600) holds the three netrc lines of your key.
    curl -fsS --netrc-file ~/.ismalicious.netrc -o list.txt \
      https://api.ismalicious.com/blocklist/download/blocklist-ips-critical.txt
    # The header's count, the entries' count, and the 10% sample marker:
    grep -m1 'Total entries' list.txt
    grep -c '^[^#]' list.txt
    grep -q 'Lite Version' list.txt && echo 'lite list: check the key and the plan'
    # One value per line, no header; IP sets take dotted IPv4 only:
    grep -v '^#' list.txt | grep -v ':' > ismalicious-ips-critical.csv
  8. Or poll a TAXII 2.1 collection with the Threat Intelligence app (Pro or Enterprise)

    • The QRadar Threat Intelligence app reads TAXII 2.1 from version 2.5.0, which needs QRadar 7.5.0 UP7 or later. IBM’s setup page still describes TAXII 1.x and 2.0 only.
    • In the app’s Feeds Downloader, click Add Threat Feed, then Add TAXII Feed.
    • On the Connection tab, enter the values below, then click Discover. The endpoint is the collections URL, the Get Collections endpoint IBM’s page refers TAXII 2 servers to.
    • On the Parameters tab, pick c2-indicators and an Observable Type. IBM: only observables of that type are used, all others are ignored. Add the feed twice: once with the IP address type, into an IP reference set, and once with the domain type, into an ALNIC one. Set Polling Intervals (hourly by default) and Poll Initial Date, and select the reference set created beforehand. The app polls one TAXII feed at a time.
    • This path has not been tested in QRadar itself.
    Add TAXII Feed, Connection tabGUI
    TAXII Endpoint
    https://api.ismalicious.com/taxii/api-root/collections/
    Version
    TAXII 2.1
    Authentication Method
    HTTP Basic: <API_KEY> / <API_SECRET>

    # After Discover, on the Parameters tab. A feed uses one observable type and

    # ignores the others, so add the collection twice:

    Collections
    c2-indicators (IP addresses, into an IP reference set)
    Collections
    c2-indicators (domains, into an ALNIC reference set)
    TAXII Endpoint ......... https://api.ismalicious.com/taxii/api-root/collections/
    Version ................ TAXII 2.1
    Authentication Method .. HTTP Basic: <API_KEY> / <API_SECRET>
    
    # After Discover, on the Parameters tab. A feed uses one observable type and
    # ignores the others, so add the collection twice:
    Collections ............ c2-indicators   (IP addresses, into an IP reference set)
    Collections ............ c2-indicators   (domains, into an ALNIC reference set)

Verify it works

  • The set’s element count, in Reference Set Management or as number_of_entries through the API, equals the script’s printed count: the list’s count in /blocklist/stats, minus the IPv6 entries for IP sets.
  • Each run prints one line with the count loaded. A non-zero exit means the download failed or was refused, or a bulk task did not complete in 30 minutes; in the last case the set stays empty or partial until the next good run. A completed task can still list per-entry failures at set_bulk_update_tasks/ followed by its id and /results.
  • A test event or flow with a listed address fires the rule.
  • The first lines of a direct download show Total entries without Lite Version.
Checkssh
# Entries in the set (number_of_entries):
curl -sS -K ~/.qradar.curlrc -H "Version: <API_VERSION>" \
  "https://<QRADAR_CONSOLE>/api/reference_data_collections/sets/<SET_ID>"

# The key reaches isMalicious: no "Lite Version" in Total entries.
curl -sS --netrc-file ~/.ismalicious.netrc https://api.ismalicious.com/blocklist/download/blocklist-ips-critical.txt | head -12

Troubleshooting

“HTTP 401” from the script

The key or the secret is wrong or incomplete. The script stops before it empties the set, so the old contents stay. The response body reads “Blocklist not found or empty”; trust the status code. Regenerating the key in Account › API access invalidates the old pair.

“lite list received”

No credential reached the server, or the plan is Free or lapsed: past due, unpaid, canceled, incomplete or paused counts as Free. Check the key, the secret and the plan.

“entries, the header says” or “cut short, no final newline”

The file was empty, cut short or not one of the lists, as a proxy or captive portal answers. The script stops before it empties the set, so the old contents stay; the next run tries again.

Timeouts or a 502

A download that takes longer than 30 seconds on our side ends in a timeout or a 502: use a tier or category list from the table.

401 or 403 from QRadar

The SEC token is wrong or expired, or its role cannot manage reference data.

422 from QRadar

A value does not match the set’s entry type, for example a domain sent to an IP set: check the type and the last argument of the script.

A bulk task ends in EXCEPTION

Read its error_message from set_bulk_update_tasks, then run the script again. A task still queued or paused after 30 minutes also stops the run.

Certificate errors

Towards the console, give its CA bundle in QRADAR_CA. Towards api.ismalicious.com, they mean TLS inspection or an old CA bundle.

“Unexpected Response. Got Content-Type” in the Threat Intelligence app

The feed’s Version is TAXII 2.0, or the app is older than 2.5.0: a TAXII 2.0 client refuses a TAXII 2.1 answer. Upgrade the app, then set Version to TAXII 2.1.

Limits

  • The TAXII 2.1 path needs the Threat Intelligence app 2.5.0 or later and has not been tested in QRadar. IBM documents the TAXII setup for 1.x and 2.0 only, and not how the app maps STIX 2.1 indicators to reference sets. A feed reads one collection and one observable type.
  • The script empties a set before refilling it, so rules miss matches while the bulk tasks run.
  • IPv6 entries are dropped for IP sets: IBM’s import documentation gives dotted IPv4 addresses.
  • Reference sets hold literal values. No CIDR or wildcard lists are served, so the CIDR entry type stays unused.
  • IBM documents no maximum set size or bulk request size in the pages read for this guide: the batch of 10,000 values is our choice.
  • The 10% sample is the first tenth of an unsorted file, not the riskiest tenth: the script refuses it.

Questions

Can the QRadar Threat Intelligence app poll the isMalicious TAXII feed?

From app version 2.5.0, which reads TAXII 2.1, with a Pro or Enterprise plan: add c2-indicators twice with the collections URL and HTTP Basic, one feed per observable type (IP addresses, domains), as in the TAXII step. The pairing has not been tested in QRadar itself; the reference-set script works whatever the app version.

Can QRadar download a blocklist into a reference set by itself?

No. A reference set takes a file imported by hand or entries sent through the REST API. The script downloads the list with your API key and loads it through the API.

Why does the script drop IPv6 addresses?

IBM’s import documentation gives valid IP addresses as dotted IPv4, so the script keeps IPv4 entries for sets of type IP.

How often should the reference sets be refreshed?

Every 12 hours, when the lists are rebuilt. Each run empties the set before refilling it, so rules miss matches while the load runs.

Get Started

Ready to get started?

No credit card required · Free API key