Advanced Search

This API endpoint allows you to perform complex searches for threat intelligence entities using advanced filters and aggregations. It provides more flexibility and power than the simple search endpoint, allowing you to construct sophisticated queries using a structured query syntax. If you need a more straightforward search interface, consider using the Simple Search instead.

Endpoint: https://apis.threatwinds.com/api/search/v1/entities/advanced

Parameters

Headers

Header	Type	Required	Description
Authorization	string	Optional*	Bearer token from an active session.
api-key	string	Optional*	Your API key
api-secret	string	Optional*	Your API secret

Note: The user-id and groups headers are added automatically by the API gateway when required and should not be provided by the client.

Query Parameters

Parameter	Type	Required	Description
limit	integer	No	Maximum number of results to return. Default is 10, maximum is 1000
page	integer	No	Page number for pagination. Default is 1. Values ≤ 1 default to 1
sort	string	No	Field to sort results by. Default is @timestamp
order	string	No	Sort order, either “asc” (ascending) or “desc” (descending). Default is desc

Request Body

The request body is required. Pass an empty object ({}) to return all entities.

The advanced search query supports the following fields:

Note: Authentication is optional. Unauthenticated requests work with reduced quotas. For more details on authentication, see the Authentication documentation.

Request

To perform an advanced search, use a POST request with a JSON body containing your search parameters:

curl -X 'POST' \
  'https://apis.threatwinds.com/api/search/v1/entities/advanced?limit=10&page=1' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <YOUR_BEARER_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": {
    "must": [
      {
        "term": {
          "type": {
            "value": "ip"
          }
        }
      },
      {
        "range": {
          "reputation": {
            "gte": 1
          }
        }
      }
    ],
    "should": [
      {
        "match": {
          "attributes.ip": {
            "query": "8.8.8.8"
          }
        }
      }
    ],
    "must_not": [
      {
        "term": {
          "tags": {
            "value": "malicious"
          }
        }
      }
    ]
  },
  "aggs": {
    "types": {
      "terms": {
        "field": "type",
        "size": 10
      }
    },
    "reputation_ranges": {
      "range": {
        "field": "reputation",
        "ranges": [
          { "from": -3, "to": 0 },
          { "from": 0, "to": 1 },
          { "from": 1, "to": 3 }
        ]
      }
    }
  },
  "source": {
    "includes": ["id", "type", "attributes", "reputation", "accuracy", "tags"],
    "excludes": []
  }
}'

The request body parameters include:

Parameter	Type	Required	Description
query	object	No	The query object for filtering and matching
aggs	object	No	Aggregations to perform on the data
source	object	No	Fields to include or exclude in the response

The query object supports the clauses listed under Query Clauses (must, should, must_not, filter, minimum_should_match).

Note: In the example preceding, a term query filters by the type field with a value of "ip". For a comprehensive list of all possible entity types, see the Entity Types page.

Query Syntax

The advanced search endpoint uses a structured query syntax for filtering and querying. Queries are expressed as JSON objects with support for boolean logic and various match types.

Note: For a list of all searchable fields and their search types, see the Entity Mapping documentation.

Query Clauses

Clause	Description	Logic
must	All conditions must match	AND
should	At least one condition should match	OR
must_not	None of these conditions should match	NOT
filter	Exact-match conditions that don’t affect scoring	AND (without scoring)
minimum_should_match	Minimum number of should clauses to satisfy	Threshold

Note on should: In OpenSearch boolean queries, a should clause is optional unless minimum_should_match is set — it only adds to the score and ranks matches higher. So the example query above (must: type=ip and reputation≥1, should: match 8.8.8.8) returns both 8.8.8.8 and 8.8.4.4: the 8.8.8.8 document scores higher because it also satisfies the should, but 8.8.4.4 is not excluded because the should is not required. If you want a should clause to act as a required filter, add "minimum_should_match": 1 alongside it.

Optional Query Features

Parameter	Type	Description
collapse	Object	Collapse results to unique values on a specified field for deduplication
search_after	Array	Deep pagination. Pass the sort values from the last result of the previous page to fetch the next page (more stable than page for large offsets)
script_fields	Object	Runtime-computed fields added to each result, returned under a top-level fields object

search_after — worked example

Results are sorted by @timestamp (descending) by default, and each result carries a sort array holding that timestamp as epoch-milliseconds. To fetch the next page, send the last result’s sort array back as search_after:

# Page 1
curl -X 'POST' 'https://apis.threatwinds.com/api/search/v1/entities/advanced?limit=2' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <YOUR_BEARER_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "query": { "must": [ { "term": { "type": { "value": "ip" } } } ] }
  }'

Page 1 returns (abridged — note each result’s sort):

{
  "items": 10000,
  "pages": 5000,
  "results": [
    { "id": "ip-01640373cf90bceb6349889cb7711edb73a80e97b23fd43c0d4e5b518bcde74e", "type": "ip",
      "attributes": { "ip": "35.214.170.240" }, "reputation": 0,
      "sort": [1790493868462] },
    { "id": "ip-8dce26995f46e896caa98c69461d63e2bb91b60d592af4e768cc2afbab04359b", "type": "ip",
      "attributes": { "ip": "142.250.126.113" }, "reputation": 0,
      "sort": [1790493855015] }
  ]
}

Take the sort of the last result ([1790493855015]) and send it as search_after for page 2:

curl -X 'POST' 'https://apis.threatwinds.com/api/search/v1/entities/advanced?limit=2' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <YOUR_BEARER_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "query": { "must": [ { "term": { "type": { "value": "ip" } } } ] },
    "search_after": [1790493855015]
  }'

Page 2 continues immediately after, with no overlap and no page offset:

{
  "results": [
    { "id": "ip-1dd16343c6b59c46d320784ea7b1b488ff1e8a2f7994480de354eb70a37a9f45", "type": "ip",
      "attributes": { "ip": "172.67.189.36" }, "reputation": 0, "sort": [1790493804617] },
    { "id": "ip-2cb86c9136a154a06eb27223d156d79f880ce1aabefdbf1d03bf39d76399bebb", "type": "ip",
      "attributes": { "ip": "104.21.65.70" }, "reputation": 0, "sort": [1790493804560] }
  ]
}

This is stable pagination that does not depend on page/offset arithmetic, and is the recommended approach when walking far into a large result set.

script_fields — worked example

script_fields computes a per-result value server-side from the document (doc.<field>). The computed value is returned in a top-level fields object on each result — each value is wrapped in an array, so read fields.<name>[0]:

curl -X 'POST' 'https://apis.threatwinds.com/api/search/v1/entities/advanced?limit=1' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <YOUR_BEARER_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "query": { "must": [ { "term": { "type": { "value": "ip" } } } ] },
    "script_fields": {
      "reputation_x2": {
        "script": { "source": "doc.reputation.value * 2", "lang": "painless" }
      }
    }
  }'

{
  "results": [
    {
      "id": "ip-01640373cf90bceb6349889cb7711edb73a80e97b23fd43c0d4e5b518bcde74e",
      "type": "ip",
      "reputation": 0,
      "fields": { "reputation_x2": [0] }
    }
  ]
}

Supported Query Operators

Operator	Description	Example
term	Exact match on a field value	{ "term": { "type": { "value": "ip" } } }
range	Numeric or date range comparison	{ "range": { "reputation": { "gte": -2 } } }
match	Full-text search on a field	{ "match": { "attributes.text": { "query": "malware" } } }
exists	Check if a field has a value	{ "exists": { "field": "attributes.ip" } }
prefix	Match values starting with a prefix	{ "prefix": { "attributes.domain": "evil." } }
wildcard	Pattern matching with * and ?	{ "wildcard": { "attributes.hostname": "srv-*" } }

Aggregations

Use the aggs field to group and summarize results:

Aggregation	Description
terms	Group by field values
range	Group by value ranges
date_histogram	Group by date intervals
avg/sum/min/max	Calculate statistics
cardinality	Count unique values

Sample Requests

Here are some sample requests that demonstrate various query types and aggregation types:

Term Query

Search for entities of type “ip”:

{
  "query": {
    "must": [
      {
        "term": {
          "type": {
            "value": "ip"
          }
        }
      }
    ]
  }
}

Terms Query

Search for entities with specific tags:

{
  "query": {
    "must": [
      {
        "terms": {
          "tags": ["malicious", "suspicious"]
        }
      }
    ]
  }
}

Range Query

Search for entities with a reputation between 1 and 3:

{
  "query": {
    "must": [
      {
        "range": {
          "reputation": {
            "gte": 1,
            "lte": 3
          }
        }
      }
    ]
  }
}

Match Query

Full-text search for entities:

{
  "query": {
    "must": [
      {
        "match": {
          "attributes.ip": {
            "query": "8.8.8.8"
          }
        }
      }
    ]
  }
}

Query with Multiple Clauses

Search for IP entities with a high reputation that aren’t tagged as malicious:

{
  "query": {
    "must": [
      {
        "term": {
          "type": {
            "value": "ip"
          }
        }
      },
      {
        "range": {
          "reputation": {
            "gte": 2
          }
        }
      }
    ],
    "must_not": [
      {
        "term": {
          "tags": {
            "value": "mail-server"
          }
        }
      }
    ]
  }
}

Exists Query

Search for entities that have a specific field:

{
  "query": {
    "must": [
      {
        "exists": {
          "field": "attributes.country"
        }
      }
    ]
  }
}

Prefix Query

Search for entities with a field value starting with a specific prefix:

{
  "query": {
    "must": [
      {
        "prefix": {
          "attributes.aso": "Google"
        }
      }
    ]
  }
}

Wildcard Query

Search for entities with a field value matching a wildcard pattern:

{
  "query": {
    "must": [
      {
        "wildcard": {
          "attributes.text": {
            "value": "Google*DNS"
          }
        }
      }
    ]
  }
}

Terms Aggregation

Group results by entity type:

{
  "query": {
    "must": [
      {
        "range": {
          "reputation": {
            "gte": 0
          }
        }
      }
    ]
  },
  "aggs": {
    "entity_types": {
      "terms": {
        "field": "type",
        "size": 10
      }
    }
  }
}

Date Histogram Aggregation

Group entities by the month they were last seen:

{
  "query": {
    "must": [
      {
        "term": {
          "type": {
            "value": "ip"
          }
        }
      }
    ]
  },
  "aggs": {
    "entities_over_time": {
      "date_histogram": {
        "field": "lastSeen",
        "interval": "month"
      }
    }
  }
}

Nested Aggregations

Group by entity type and then calculate an average reputation for each type:

{
  "query": {
    "must": [
      {
        "range": {
          "reputation": {
            "gte": 3
          }
        }
      }
    ]
  },
  "aggs": {
    "entity_types": {
      "terms": {
        "field": "type",
        "size": 10
      },
      "aggs": {
        "avg_reputation": {
          "avg": {
            "field": "reputation"
          }
        }
      }
    }
  }
}

Response

A successful response will return a JSON object containing the search results and aggregations:

{
  "items": 2,
  "pages": 1,
  "results": [
    {
      "id": "ip-ad0c2ed9a0a9b23822e5907b0d009bcaf8f969db793cd1d94c40e17e0287c04b",
      "type": "ip",
      "attributes": {
        "ip": "8.8.8.8"
      },
      "reputation": 3,
      "accuracy": 3,
      "tags": ["dns", "google", "public"]
    },
    {
      "id": "ip-3f8a1c6e9d2b4a70c5e8f1a3b6d9c2e4a7f0b3c8d1e5a9f2c4b6e0d8f3a",
      "type": "ip",
      "attributes": {
        "ip": "8.8.4.4"
      },
      "reputation": 3,
      "accuracy": 3,
      "tags": ["dns", "google", "public"]
    }
  ],
  "aggregations": {
    "types": [
      {
        "value": "ip",
        "count": 2
      }
    ],
    "reputation_ranges": [
      {
        "range": "-3.0 to 0.0",
        "from": -3,
        "to": 0,
        "count": 0
      },
      {
        "range": "0.0 to 1.0",
        "from": 0,
        "to": 1,
        "count": 0
      },
      {
        "range": "1.0 to 3.0",
        "from": 1,
        "to": 3,
        "count": 2
      }
    ]
  }
}

The response includes:

Field	Type	Description
items	integer	Total number of items matching the query
pages	integer	Total number of pages available
results	array	Array of entities matching the search criteria
aggregations	object	Results of any aggregations requested in the query

Each entity in the results array includes fields specified via source filtering.

Note: Each result may include a sort array when search_after is being used — that’s the value to pass into the next request. Other extra metadata fields (version, score, fields) can be safely ignored.

Error Response Headers

Responses with status codes 400, 401, and 403 include the following custom headers:

Header	Description
x-error	Human-readable error message describing what went wrong
x-error-id	Unique identifier for error tracking and support

Error Codes

Status Code	Description	Possible Cause
400	Bad Request	Invalid request parameters or malformed JSON
401	Unauthorized	Missing or invalid authentication credentials
403	Forbidden	Authenticated user lacks permission for this operation
404	Not Found	No results found matching your query criteria
500	Internal Server Error	Server-side error; please contact support if persistent