ioc-extractor-without-sort
v8.0.4-p1
Published
ioc-extractor fork that keeps indicator order from the input document + other patches
Downloads
10
Readme
IoC extractor
This is a fork of https://github.com/ninoseki/ioc-extractor with the following changes
- sorting is removed
- IPv4 and IPv6 address also accepts optional CIDR mask (can be disable by setting
enableOptionalMask: false
) - new
only
option for extractIOC to only extract certain types of IoCs
I'm too lazy to make this configurable and send PR, so here we are.
IoC extractor is an npm package for extracting common IoC (Indicator of Compromise) from a block of text.
Note: the package is highly influenced by cacador.
Installation
npm install -g ioc-extractor
# or if you want to use ioc-extractor as a library in your JS/TS project
npm install ioc-extractor
Usage
As a CLI
$ ioc-extractor --help
Usage: ioc-extractor [options]
Options:
-ns, --no-strict Disable strict option
-nr, --no-refang Disable refang option
-p, --punycode Enable punycode option
-h, --help display help for command
$ echo "1.1.1.1 8.8.8.8 example.com" | ioc-extractor | jq
{
"asns": [],
"btcs": [],
"cves": [],
"domains": [
"example.com"
],
"emails": [],
"eths": [],
"gaPubIDs": [],
"gaTrackIDs": [],
"ipv4s": [
"1.1.1.1",
"8.8.8.8"
],
"ipv6s": [],
"macAddresses": [],
"md5s": [],
"sha1s": [],
"sha256s": [],
"sha512s": [],
"ssdeeps": [],
"urls": [],
"xmrs": []
}
As a library
import { extractIOC } from "ioc-extractor";
const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const ioc = extractIOC(input);
console.log(ioc.md5s);
// => ['f6f8179ac71eaabff12b8c024342109b']
console.log(ioc.ipv4s);
// => ['1.1.1.1']
console.log(ioc.domains);
// => ['google.com']
extractIOC
takes the following options:
If you want to extract a specific type of IoC, you can use extract function.
import {
refang,
extractDomains,
extractIPv4s,
extractMD5s,
} from "ioc-extractor";
const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const refanged = refang(input);
// => 1.1.1.1 google.com f6f8179ac71eaabff12b8c024342109b
const ipv4s = extractIPv4s(refanged);
// => ['1.1.1.1']
const domains = extractDomains(refanged);
// => ['google.com']
const md5s = extractMD5s(refanged);
// => ['f6f8179ac71eaabff12b8c024342109b']
Network related extract functions (e.g. extractDomains
) can take the following options:
See docs for more details.
IoC Types
This package supports the following IoCs:
- Hashes: MD5, SHA1, SHA256, SHA512, SSDEEP
- Networks: domain, email, IPv4, IPv6, URL, ASN
- Hardwares: MAC address
- Utilities: CVE (CVE ID)
- Cryptocurrencies: BTC (BTC address), ETH (ETH address), XMR (XMR address)
- Trackers: GA track ID (Google Analytics tracking ID), GA pub ID (Google Adsense Publisher ID)
Refang Techniques
For Networks IoCs, the following refang techniques are supported:
| Techniques | Defanged | Refanged |
| ------------------------------------ | -------------------------------------- | ------------------------------- |
| .
in spaces | 1.1.1 . 1
| 1.1.1.1
|
| .
in brackets, parentheses, etc. | 1.1.1[.]1
| 1.1.1.1
|
| dot
in brackets, parentheses, etc. | example[dot]com
| example.com
|
| Back slash before .
| example\.com
| example.com
|
| /
in brackets, parentheses, etc. | http://example.com[/]path
| http://example.com/path
|
| ://
in brackets, parentheses, etc. | http[://]example.com
| http://example.com
|
| :
in brackets, parentheses, etc. | http[:]//example.com
| http://example.com
|
| @
in brackets, parentheses, etc. | test[@]example.com
| [email protected]
|
| at
in brackets, parentheses, etc. | test[at]example.com
| [email protected]
|
| hxxp
| hxxps://example.com
| https://example.com
|
| Partial | 1.1.1[.1
| 1.1.1.1
|
| Any combination | hxxps[:]//test\.example[.)com[/]path
| https://test.example.com/path
|
Options
strict
Whether to do strict TLD matching or not. Defaults to true
.
refang
Whether to do refang or not. Defaults to false
.
punycode
Whether to do Punycode conversion or not. Defaults to false
.