# Transform functions

In Mindflow, you have the capability to perform a wide range of data transformations as data flows through the application. This enables you to utilize arithmetic functions, manipulate arrays, apply cryptographic and hashing techniques, extract valuable insights, and much more.

To access these transformation options, simply click on the `fx` icon and then choose the specific operation you wish to execute.

   <div class="docs-screenshot-compact">

   ![Access transformation options simply click fx](./assets/shot-01.png)

   </div>

   <div class="docs-screenshot-compact">

   ![Access transformation options simply click fx](./assets/shot-02.png)

   </div>

## Example

Extracting URLs from a previously received email message.

   <div class="docs-screenshot-compact">

   ![Extracting urls previously received email](./assets/example-2.png)

   </div>

You can use the **`Extract URLs`** function found in the Extractors category. This function will produce an array containing the urls extracted from the previous message.

   <div class="docs-screenshot-compact">

   ![You can use extract urls function](./assets/example.png)

   </div>

# Arithmetic

Full parameter tables and examples: [Arithmetic](./arithmetic/).

The **Arithmetic** module is the calculation engine of your Mindflow workflows. It allows you to perform mathematical operations, apply logical conditions, and manage complex data sets without writing code.

## Why?

While simple data mapping moves data from point A to B, Arithmetic functions allow you to **analyze** and **validate** that data mid-flight:

- **Metrics & Stats:** Calculate average response times or total threat scores.
- **Security Logic:** Use bitwise operations or rotations (like ROT13) to decode or obscure strings.
- **Comparison:** Use "Set" operations to compare lists (e.g., comparing "detected IPs" against a "whitelist").

## How?

1. **Input:** Provide a number or a list of numbers/strings.
2. **Operation:** Drag your desired function into the Recipe.
3. **Result:** The output is automatically calculated for use in the next step.

## Operation

- **Sum / Divide / Substract / Multiply / Mean / Median / Standard Deviation:** Basic math for calculating totals, percentages, statistics or quotas.

   <div class="docs-screenshot-compact">

   ![Sum divide substract multiply mean median](./assets/sum-divide-substract-multiply-mean-median.png)

   </div>

- **ADD / AND / OR / NOT:** These functions operate at the bitwise (logic gate) level.

   <div class="docs-screenshot-compact">

   ![Add not](./assets/add-not.png)

   </div>

- **Bit shift (Left/Right) & Rotate:** Used for low-level data manipulation, often found in custom encryption or legacy protocol decoding.
- **ROT13:** A simple substitution cipher often used in security puzzles or to hide text.
- **Set Union:** Merges two lists and removes duplicates.
- **Set Intersection:** Returns only the items that appear in **both** lists.
- **Set Difference:** Returns items found in the first list but **not** the second.
- **Cartesian Product:** Creates every possible pair between two sets.
- **Power Set:** Generates all possible subsets of a list.

:::note
The result is output as raw bytes and displayed using Latin-1 / ASCII encoding. To decode Latin-1 / ASCII you can use the function `To Binary`
:::

# Array

Full parameter tables and examples: [Array](./array/).

Within the tool, you have access to an array of formula functions that enhance user flexibility and efficiency when working with arrays. These functions streamline the tasks of manipulating and transforming arrays, simplifying the process of data management and analysis.

Within the list feature, you can perform a range of operations on an array, such as adding elements, retrieving its size, removing elements, sorting, searching for specific elements, and more.

- Array concat: Merges multiple arrays into one

   <div class="docs-screenshot-compact">

   ![Array concat](./assets/array-concat.png)

   </div>

- Array flat: Flattens nested arrays into a single array

   <div class="docs-screenshot-compact">

   ![Array flat](./assets/array-flat.png)

   </div>

- Array get item: Returns the element at a specific index

   <div class="docs-screenshot-compact">

   ![Array get item](./assets/array-get-item.png)

   </div>

- Array join: Converts an array into a string

   <div class="docs-screenshot-compact">

   ![Array join](./assets/array-join.png)

   </div>

- Array length: Returns the number of elements

   <div class="docs-screenshot-compact">

   ![Array length](./assets/array-length.png)

   </div>

- Array pop: Removes the last element of the array

   <div class="docs-screenshot-compact">

   ![Array pop](./assets/array-pop.png)

   </div>

- Array push: Adds an element to the end of the array

   <div class="docs-screenshot-compact">

   ![Array push](./assets/array-push.png)

   </div>

- Array range: Generates a range of numbers

   <div class="docs-screenshot-compact">

   ![Array range](./assets/array-range.png)

   </div>

- Array slice: Extracts a portion of an array

   <div class="docs-screenshot-compact">

   ![Array slice](./assets/array-slice.png)

   </div>

- Array sort: Sorts the array

   <div class="docs-screenshot-compact">

   ![Array sort](./assets/array-sort.png)

   </div>

- Array split: Splits a string into an array

   <div class="docs-screenshot-compact">

   ![Array split](./assets/array-split.png)

   </div>

- Array unique:

   <div class="docs-screenshot-compact">

   ![Array unique](./assets/array-unique.png)

   </div>

# Cryptography

Full parameter tables and examples: [Cryptography](./cryptography/).

It includes tools to generate unique identifiers, time-based tokens (TOTP), as well as to encode and decode certificates and JWTs. These functions also facilitate conversion between cryptographic formats, such as hexadecimal, PEM, and object identifiers.

- **Atbash Cipher:** A simple substitution cipher for the alphabet. It works by "flipping" the alphabet so that A becomes Z, B becomes Y, and so on.

   <div class="docs-screenshot-compact">

   ![Atbash cipher](./assets/atbash-cipher.png)

   </div>

- **SM4 Encrypt / Decrypt:** A block cipher standard used primarily in China for wireless networks and data encryption. It is functionally similar to AES but uses a different algorithm.

- **Substitute:** A general-purpose tool that allows you to replace specific characters or strings with others. It’s often used for manual decoding or fixing broken data.

   <div class="docs-screenshot-compact">

   ![Substitute](./assets/substitute.png)

   </div>

- **JWT Decode:** Takes a JWT (often found in web cookies or headers) and breaks it down into its three parts: Header, Payload, and Signature, making the data human-readable.

   <div class="docs-screenshot-compact">

   ![Jwt decode](./assets/jwt-decode.png)

   </div>

- **JWT Sign:** Used to create a signature for a JWT using a secret key or a private key, ensuring the token's integrity.

   <div class="docs-screenshot-compact">

   ![Jwt sign](./assets/jwt-sign.png)

   </div>

- **JWT Verify:** Checks if a JWT is valid by verifying its signature against a provided secret or public key.
- **Hex to Object Identifier (OID) / Object Identifier to Hex:** Converts between raw hexadecimal data and OID notation (e.g., `1.2.840.113549`). OIDs are used in digital certificates to identify things like encryption algorithms.

- **Hex to PEM / PEM to Hex:** Converts raw hex data into PEM format (the text files that start with `----BEGIN CERTIFICATE-----`) or vice versa.

   <div class="docs-screenshot-compact">

   ![Hex pem pem hex](./assets/hex-pem-pem-hex.png)

   </div>

- **Parse ASN.1 hex string:** ASN.1 is a standard for describing data structures. This tool takes a hex string and visualizes the tree structure of the data (often used for certificates or public keys).
- **Parse X.509 certificate:** Specifically designed to read digital certificates. It extracts info like the **Issuer**, **Subject**, **Expiry Date**, and **Public Key**.
- **Generate TOTP:** Creates a Time-based One-Time Password. This is the 6-digit code you get from apps like Google Authenticator or Authy.
- **Hex to Object Identifier:** (Mentioned above) Specifically translates hex bytes into the dot-separated OID format.

# Custom

Full parameter tables and examples: [Custom](./custom/). Temporary credentials:
[Create temporary credential](./create-temporary-credential/).

It provides tools to compare values, concatenate strings, calculate the Levenshtein distance for text similarity, and pad text to ensure equal length. Additionally, it offers functionalities to convert data formats, such as transforming XML into JSON, and evaluating XPath expressions

- **CSV to JSON / XML to JSON:** They take structured text (like an Excel-style CSV or a tagged XML file) and turn it into **JSON.**

   <div class="docs-screenshot-compact">

   ![Csv json xml json](./assets/csv-json-xml-json.png)

   </div>

- **MSG to JSON:** It takes an Outlook email file (`.msg`) and extracts the metadata (sender, recipient, date) and body into a searchable JSON format.

- **Extract Email Addresses:** This will instantly list every email address found in the text.

   <div class="docs-screenshot-compact">

   ![Extract email addresses](./assets/extract-email-addresses.png)

   </div>

- **XPath:** It allows you to write a specific query to grab exactly one piece of information from a massive, complex XML document

   <div class="docs-screenshot-compact">

   ![Xpath](./assets/xpath.png)

   </div>

- **Assert:** You use it to tell: "If the data doesn't contain the word 'Password', stop the recipe and show an error." It's great for debugging complex automation.

- **Compare:** This tool takes two pieces of data and tells you if they are identical or different. It is often used to see if a file has been modified.

   <div class="docs-screenshot-compact">

   ![Compare](./assets/compare.png)

   </div>

- **Concatenate:** It takes multiple lines or pieces of data and glues them together into one long string.

   <div class="docs-screenshot-compact">

   ![Concatenate](./assets/concatenate.png)

   </div>

- **Levenshtein Distance:** Is a string metric used to measure the difference between two sequences. Informally, it is the minimum number of single-character edits (insertions, deletions, or substitutions) required to change one word into another. It can detecting **deception** and **automation**.

   <div class="docs-screenshot-compact">

   ![Levenshtein distance](./assets/levenshtein-distance.png)

   </div>

   <div class="docs-screenshot-compact">

   ![Levenshtein distance](./assets/levenshtein-distance-2.png)

   </div>

- **Pad text to equal length:** This adds extra characters to the beginning or end of lines so they all have the exact same character count.

   <div class="docs-screenshot-compact">

   ![Pad text equal length](./assets/pad-text-equal-length.png)

   </div>

- **Random Number Generator:** Generates a sequence of random numbers.

   <div class="docs-screenshot-compact">

   ![Random number generator](./assets/random-number-generator.png)

   </div>

- **UUID V4:** Generates a "Universally Unique Identifier". You can use it when you are in need of an identifier!

   <div class="docs-screenshot-compact">

   ![Uuid v4](./assets/uuid-v4.png)

   </div>

# Data format

These functions encode, decode, and convert text and structured data. Full parameter tables, returned values, and
examples live under [Data format](./data-format/).

## Basic text and alphabet

- **[Convert to NATO alphabet](./data-format/convert-to-nato-alphabet/):** Turns text into the phonetic radio alphabet.

   <div class="docs-screenshot-compact">

   ![Convert nato alphabet](./assets/convert-nato-alphabet.png)

   </div>

- **[Encode text](./data-format/encode-text/) / [Decode text](./data-format/decode-text/):** Change the character
  encoding (such as UTF-8) of a string.

   <div class="docs-screenshot-compact">

   ![Encode text decode text](./assets/encode-text-decode-text.png)

   </div>

- **[Escape Unicode Characters](./data-format/escape-unicode-characters/):** Converts special characters into their code
  equivalents.

   <div class="docs-screenshot-compact">

   ![Escape unicode characters](./assets/escape-unicode-characters.png)

   </div>

- **[Normalise Unicode](./data-format/normalise-unicode/):** Fixes text where the same character might be represented in
  different ways internally (important for digital signatures).
- **[Remove Diacritics](./data-format/remove-diacritics/):** Strips accents from letters.

   <div class="docs-screenshot-compact">

   ![Remove diacritics](./assets/remove-diacritics.png)

   </div>

- **[To Upper case](./data-format/to-upper-case/) / [To Lower case](./data-format/to-lower-case/):** Change letter case.
- **[Unicode Text Format](./data-format/unicode-text-format/):** Adds Unicode combining characters for formatting.
- **[Text Encoding Brute Force](./data-format/text-encoding-brute-force/):** Tries supported encodings to recover text.

## Base encodings

- **[To Base64](./data-format/to-base64/) / [From Base64](./data-format/from-base64/):** The most common encoding for web
  data and email attachments. Multiple alphabets (URL-safe is important) are available.
- **[Show Base64 offsets](./data-format/show-base64-offsets/):** Lists possible Base64 encodings of a string at different
  offsets.
- Other restricted alphabets:
  [Base32](./data-format/to-base32/),
  [Base45](./data-format/to-base45/),
  [Base58](./data-format/to-base58/),
  [Base62](./data-format/to-base62/),
  [Base85](./data-format/to-base85/),
  and [To Base](./data-format/to-base/) / [From Base](./data-format/from-base/).

   <div class="docs-screenshot-compact">

   ![Other restricted alphabets](./assets/other-restricted-alphabets.png)

   </div>

## Number systems and hardware formats

- **[To Binary](./data-format/to-binary/) / [From Binary](./data-format/from-binary/):** Convert between text and
  `010101` strings.
- **[To Hex](./data-format/to-hex/) / [From Hex](./data-format/from-hex/):** Convert hexadecimal byte strings.
- **[To Decimal](./data-format/to-decimal/) / [From Decimal](./data-format/from-decimal/)** and
  **[To Octal](./data-format/to-octal/) / [From Octal](./data-format/from-octal/):** ordinal and octal forms.
- **[To BCD](./data-format/to-bcd/) / [From BCD](./data-format/from-bcd/):** Binary-Coded Decimal.
- **[To Charcode](./data-format/to-charcode/) / [From Charcode](./data-format/from-charcode/):** Unicode character codes.
- **[Swap endianness](./data-format/swap-endianness/):** Switch big-endian and little-endian word order.

## Web, markup, and email formats

- **[URL Encode](./data-format/url-encode/) / [URL Decode](./data-format/url-decode/):** Percent-encoding for URIs.
- **[To HTML Entity](./data-format/to-html-entity/) / [From HTML Entity](./data-format/from-html-entity/):** HTML
  entities.
- **[Strip HTML tags](./data-format/strip-html-tags/):** Remove markup and keep plain text.
- **[To Quoted Printable](./data-format/to-quoted-printable/) /
  [From Quoted Printable](./data-format/from-quoted-printable/):** Email QP encoding.
- **[To Punycode](./data-format/to-punycode/) / [From Punycode](./data-format/from-punycode/):** Internationalized domain
  names.
- **[To Hex Content](./data-format/to-hex-content/) / [From Hex Content](./data-format/from-hex-content/)** and
  **[To Hexdump](./data-format/to-hexdump/) / [From Hexdump](./data-format/from-hexdump/):** hex-in-text and hexdump
  views.
- **[Unescape Unicode Characters](./data-format/unescape-unicode-characters/):** Reverse unicode escapes.

## Structured data

- **[JSON Path](./data-format/json-path/):** Select and extract values from JSON (data retrieval). See the reference for
  parameters, return values, and examples.

  <div class="docs-screenshot-compact">

  ![Json path](./assets/json-path.png)

  </div>

- **[JSONata](./data-format/jsonata/):** Query and transform JSON (filter, aggregate, reshape). See the reference for
  parameters, return values, and examples.

  <div class="docs-screenshot-compact">

  ![Jsonata](./assets/jsonata.png)

  </div>

### JSON Path and JSONata syntaxes

| Feature          | JSONPath          | JSONata    | Description                                       |
| ---------------- | ----------------- | ---------- | ------------------------------------------------- |
| Root document    | `$`               | `$`        | The start of the JSON document.                   |
| Field access     | `$.field`         | `field`    | Navigate to a child property.                     |
| Current context  | `@`               | Implied    | Reference to the current object in a loop/filter. |
| Recursive search | `..`              | `**`       | Deep scan for a key at any level.                 |
| Equality         | `==`              | `=`        | Checks if two values are equal.                   |
| Inequality       | `!=`              | `!=`       | Checks if two values are different.               |
| Filters          | `[?(@.id > 1)]`   | `[id > 1]` | Selects elements matching a condition.            |
| Wildcard (keys)  | `.*`              | `.*`       | Selects all properties of an object.              |
| Wildcard (array) | `[*]`             | `*`        | Selects all elements in an array.                 |
| Array slicing    | `[0:5]`           | `[[0..4]]` | Selects a range of elements.                      |
| Last element     | `[-1:]`           | `[-1]`     | Accesses the end of an array.                     |
| Logical AND      | `&&`              | `and`      | Combines two conditions.                          |
| Logical OR       | `\|\|`            | `or`       | Either condition can be true.                     |
| Regex match      | Specific to impl. | `=~`       | Match string against a pattern.                   |

- **[Parse TLV](./data-format/parse-tlv/):** Decode Type-Length-Value (or KLV) payloads.
- **[To MessagePack](./data-format/to-messagepack/) / [From MessagePack](./data-format/from-messagepack/):** Compact
  binary JSON-like encoding.

# Date & time

Full parameter tables and examples: [Date & Time](./date-time/).

Computers often store time as a single long number (a timestamp), which is impossible for humans to read without these tools.

- From/To UNIX Timestamp: Convert UNIX TimeStamp to a DateTime string (or contrary)

   <div class="docs-screenshot-compact">

   ![Unix timestamp](./assets/unix-timestamp.png)

   </div>

- Windows Filetime to UNIX Timestamp: Microsoft Windows uses a different "starting point" for time (January 1, 1601) and measures in 100-nanosecond intervals.

- Translate DateTime Format: Lets you convert dates between time zones and adapt them to the desired format. This is used to change how a date looks (e.g., changing US format to European format).

   <div class="docs-screenshot-compact">

   ![Translate datetime format](./assets/translate-datetime-format.png)

   </div>

- Parse DateTime: Tries to figure out what a date string means, even if it’s messy, and turns it into a standardized ISO format.

- Modify date with duration: This allows you to do date math easily adjust dates by adding or subtracting months, days, hours, and more.

   <div class="docs-screenshot-compact">

   ![Modify date duration](./assets/modify-date-duration.png)

   </div>

# Extractors

Full parameter tables and examples: [Extractors](./extractors/).

These functions allow extracting dates, domain names, email addresses, file paths, IP addresses, MAC addresses, URLs, and using regular expressions to extract specific patterns.

- Extract Dates / Domains / IP addresses / URLs: / File paths Ignore the noise in a file and list only the specific entities you are looking for.

   <div class="docs-screenshot-compact">

   ![Extract dates domains ip addresses urls](./assets/extract-dates-domains-ip-addresses-urls.png)

   </div>

- Extract MAC addresses: Finds unique hardware identifiers for network devices.

- Regular expression: A "build-your-own" search tool where you define the pattern yourself to find custom data.

   <div class="docs-screenshot-compact">

   ![Regular expression](./assets/regular-expression.png)

   </div>

# Regex Survival cheat sheet

| **Category**          | **Token** | **Meaning**                         | **Example** | **Matches**                |
| --------------------- | --------- | ----------------------------------- | ----------- | -------------------------- |
| **Basics**            | `.`       | Wildcard (any character)            | `h.t`       | **hat**, **hot**, **h7t**  |
| —                     | `\`       | Escape character                    | `\.`        | Finds a literal **.**      |
| **Character Classes** | `\d`      | Digit                               | `\d\d`      | **42**, **07**             |
| —                     | `\w`      | Word character (alphanumeric + `_`) | `\w+`       | **Hello**, **user_1**      |
| —                     | `\s`      | Whitespace (space, tab, newline)    | `\s`        | " "                        |
| **Quantifiers**       | `*`       | 0 or more                           | `ab*`       | **a**, **ab**, **abbb**    |
| —                     | `+`       | 1 or more                           | `ab+`       | **ab**, **abbb** (not "a") |
| —                     | `?`       | 0 or 1 (Optional)                   | `apple?`    | **apple**, **appl**        |
| —                     | `{n}`     | Exactly _n_ times                   | `\d{3}`     | **123**, **999**           |
| **Anchors**           | `^`       | Start of string/line                | `^Start`    | "Start here..."            |
| —                     | `$`       | End of string/line                  | `end$`      | "...the end"               |
| **Sets & Logic**      | `[]`      | Character set (any of these)        | `[aeiou]`   | **a**, **e**, **i**...     |
| —                     | `[^]`     | Negated set (none of these)         | `[^0-9]`    | Any non-digit              |
| —                     | `\|`      | OR operator                         | `cat\|dog`  | **cat** or **dog**         |
| **Groups**            | `()`      | Capturing group                     | `(abc)+`    | **abc**, **abcabc**        |

- Slice**:** Allows you to "cut" a specific portion of the data (e.g., "give me only the first 100 characters").

   <div class="docs-screenshot-compact">

   ![Slice](./assets/slice.png)

   </div>

# Hashing

Full parameter tables and examples: [Hashing](./hashing/).

In the world of cybersecurity and data analysis, hashing is used for integrity checks, malware identification, and digital forensics. Here are the most commonly used **Hashing** functions in CyberChef.

\*\*\*\*The Industry Standards
These are the most reliable and widely adopted functions today.

- SHA-256: The current "gold standard." It produces a 64-character hex string. It is used for SSL certificates, Blockchain transactions, and identifying malware samples in databases like VirusTotal.
- SHA-512: Provides a higher level of security than SHA-256. It is often used in sensitive systems where computational power is not a constraint.
- SHA-1: Once the standard, it is now considered cryptographically weak against collisions. However, it is still everywhere in legacy systems and for file versioning (like in Git).

---

Legacy Functions
While no longer secure for passwords, these are essential for digital forensics and identifying known files.

- MD5: Extremely fast. It creates a 32-character hex fingerprint. In incident response, the MD5 hash is the most common IOC (Indicator of Compromise) used to share information about a specific malicious file.
- MD4: An older version of MD5, primarily seen when dealing with legacy Microsoft authentication protocols (NTLM).

1. Forensic & Specialized Hashes
   CyberChef includes unique hashing methods used specifically for Malware Analysis:

- SSDEEP (Fuzzy Hash): Unlike MD5 (where changing 1 bit changes the whole hash), SSDEEP allows you to find files that are "similar" but not identical. This is used to detect modified versions of the same malware.
- Imphash (Import Hash): It hashes the list of functions a program imports. It helps analysts group different pieces of malware that were compiled by the same author or toolkit.
- SHA-3: The latest member of the SHA family (based on the Keccak algorithm). It is used in modern applications that require maximum security resistance.

Summary TableOperationOutput Length (Hex)Primary Use CaseSecurity LevelMD5**32 charsFile identification (IOCs)Low (Vulnerable)**SHA-1

# Networking

Full parameter tables and examples: [Networking](./networking/).

These functions allow modifying IP address formats, dechunking HTTP responses, encoding and decoding NetBIOS names, defanging IP addresses and URLs, formatting MAC addresses, grouping IP addresses, and checking IP addresses within CIDR ranges. They also include tools for analyzing protocol fingerprints (HASSH, JA3) and verifying SSH host keys, facilitating network data management and security.

- Change IP format : Change IP format
- Dechunk HTTP response: Reassembles HTTP data that was sent using "chunked transfer encoding" into a single, readable body.
- Defang IP Addresses/ Defang URL: A security practice that modifies IPs or URLs (e.g., changing `http://` to `hXXp://`) so they aren't "clickable," preventing accidental navigation to malicious sites during analysis.
- Encode NetBIOS Name / Decode NetBIOS Name: Handles the unique 16-character naming convention used by older Windows networking protocols (often seen in legacy internal network traffic).
- Format MAC addresses: Standardizes the look of MAC addresses (e.g., adding colons or dashes: `001A2B...` → `00:1A:2B...`).
- Group IP addresses (Grouper des adresses IP)
- HASSH Client Fingerprint / HASSH Server Fingerprint: Similar to JA3, but specifically for SSH (Secure Shell) traffic. It fingerprints the key exchange and algorithms used during the SSH connection.
- IP Address In CIDR Range Check: Tests whether a specific IP address falls within a given CIDR (Classless Inter-Domain Routing) network block.
- JA3 Fingerprint / JA3S Fingerprint: Creates a fingerprint of a TLS Client (JA3) or TLS Server (JA3S). This helps identify specific browsers or malware families even when the traffic is encrypted.
- Parse IP range: Expands a range (like `192.168.1.1-192.168.1.5`) into a full list of individual IP addresses.
- Parse IPv4 header / Parse IPv6 address : Breaks down raw packet headers or complex IPv6 strings into their constituent parts (source/destination, TTL, flags, etc.).
- Parse UDP / Parse SSH Host Key : Extracts structured information from raw UDP datagrams or SSH public keys.
- Parse URI: Breaks a URL/URI into its components: scheme, host, port, path, query parameters, and fragment.
- Parse User Agent: Decodes a "User-Agent" string to identify the browser, operating system, and device type used by a visitor
- Strip HTTP headers: Removes the metadata (headers) from an HTTP request or response, leaving only the payload/body.
- VarInt Decode / VarInt Encode: Handles "Variable length Integers," a method used in protocols like Google Protocol Buffers to save space by using fewer bytes for smaller numbers.

# Others

Full parameter tables and examples: [Others](./others/).

These functions include calculating the Chi-square, disassembling x86, frequency distribution, generating HOTP (HMAC-based one-time password), and creating random text with Lorem Ipsum. They also allow calculating the Haversine distance between two geographical points, analyzing the index of coincidence of characters, rendering content in Markdown, and using YARA rules to identify and classify malicious files.

- Chi Square: Performs a statistical test to determine how much the observed data differs from expected data. In cryptography, it's often used to detect if a piece of text is random or follows the frequency of a known language.
- Frequency distribution: Counts the occurrence of each character (or byte) in the input. This is the first step in breaking simple substitution ciphers or identifying the nature of unknown data.
- Index of Coincidence: A measure of how similar a text is to a random distribution. It helps cryptanalysts determine the type of cipher used (e.g., if it's a polyalphabetic cipher like Vigenère) and find the length of the encryption key.
- Disassemble x86: Converts raw machine code (binary/hex) into human-readable assembly language for Intel/AMD x86 processors. This is essential for understanding what a piece of malware is actually doing at the CPU level.
- YARA Rules: A powerful tool for malware researchers. This function allows you to run YARA "rules" (descriptions of malware families based on textual or binary patterns) against your input to identify malicious files.
- Generate HOTP: Creates a HMAC-based One-Time Password. This is a standard algorithm used in 2-factor authentication (2FA) apps, where the code changes every time you press a button or log in.
- Generate Lorem Ipsum: A classic utility that generates "placeholder" text used in design and layout testing.
- Haversine distance: Calculates the "as-the-crow-flies" distance between two points on a sphere using their latitude and longitude. Useful for geolocation data analysis.
- Render Markdown: Takes text written in Markdown format and converts it into a visually formatted HTML preview.

# Utils

Full parameter tables and examples: [Utils](./utils/).

These functions include adding line numbers, converting units (area, distance, mass, speed, etc.), counting occurrences, filtering and find/replace text, and managing strings with options like escaping or expanding alphabet ranges. They also provide tools for fuzzy matching, offset checking, calculating Hamming distance, and formatting text with padding and line formatting options.

- Add / Remove line number: Add a number at the beginning of each line.

   <div class="docs-screenshot-compact">

   ![Add remove line number](./assets/add-remove-line-number.png)

   </div>

- Convert area / co-ordinate format / data-units / mass / speed / distance: Versatile utility used to translate measurements.

---

   <div class="docs-screenshot-compact">

   ![Convert area co ordinate format data](./assets/convert-area-co-ordinate-format-data.png)

   </div>

- Drop bytes / Take bytes: Allows you to either remove or keep a specific number of bytes from the start or end of the data. Useful for stripping headers or isolating specific payloads.

   <div class="docs-screenshot-compact">

   ![Drop bytes take bytes](./assets/drop-bytes-take-bytes.png)

   </div>

- Remove whitespace: Strips out spaces, tabs, and newlines. Essential before decoding formats like Base64 which can be sensitive to extra spaces.

   <div class="docs-screenshot-compact">

   ![Remove whitespace](./assets/remove-whitespace.png)

   </div>

- Count occurences: Scans the input for a specific string or character and returns the total number of times it appears. Very useful for determining if a specific byte pattern is frequent in a piece of malware.

   <div class="docs-screenshot-compact">

   ![Count occurences](./assets/count-occurences.png)

   </div>

- Escape / Unescape string: Converts special characters into "escaped" versions (like changing a newline to `\n`) or back again.

   <div class="docs-screenshot-compact">

   ![Escape unescape string](./assets/escape-unescape-string.png)

   </div>

- Expand alphabet range: Converts a shorthand range like `a-z0-9` into the full string of characters `abcdef...789`.

   <div class="docs-screenshot-compact">

   ![Expand alphabet range](./assets/expand-alphabet-range.png)

   </div>

- Filter: Acts as a "search and keep" or "search and remove" tool for lines. You can set it to only show lines that contain (or don't contain) a specific keyword or regex.

- Find / Replace: A powerful search tool that supports Regex (Regular Expressions). It's the most common way to mass-edit text.

   <div class="docs-screenshot-compact">

   ![Find replace](./assets/find-replace.png)

   </div>

- From / To Case Insensitive Regex: These are specific versions of Find/Replace or Match that ignore whether letters are uppercase or lowercase. It ensures you catch `Malware.exe`, `malware.exe`, and `MALWARE.EXE` all at once.
- Fuzzy Match: Finds strings that are _approximately_ equal to your search term, which is helpful if the data contains typos or small variations.
- Get all Casings: Generates every possible variation of upper and lower case for a string. If you search for `abc`, it outputs `abc, abC, aBc, aBC, Abc...` etc. This is often used to generate wordlists for brute-forcing passwords or hidden directories.
- Hamming Distance: A cryptographic metric that counts how many positions at which the corresponding symbols are different. It is used to measure how much two strings of equal length differ.

- Head / Tail: These return only the first few lines (Head) or last few lines (Tail) of the input.

   <div class="docs-screenshot-compact">

   ![Head tail](./assets/head-tail.png)

   </div>

- Offset checker: Helps you find the exact position (offset) of a specific byte or string within a larger file.

- Trim / Pad lines: Trim removes extra spaces from the beginning/end of each line, while Pad lines adds characters to make every line the same length.

   <div class="docs-screenshot-compact">

   ![Trim pad lines](./assets/trim-pad-lines.png)

   </div>

- Parse ObjectID timestamp: Extracts the creation time from a MongoDB BSON ObjectID.
- Parse UNIX file permissions: Converts numerical or symbolic permissions (like `755` or `rwxr-xr-x`) into a readable format.
- Remove null bytes: Removes `0x00` characters, which are often used as padding in binary files or shellcode.

- Sort / Reverse: Alphabetizes your data or reverses the current order of lines.
  Forensics & Timestamps

   <div class="docs-screenshot-compact">

   ![Sort reverse](./assets/sort-reverse.png)

   </div>

- To table: Takes delimited data (like CSV or tab-separated text) and renders it into a clean, visual table. It makes messy log files much easier to read.

   <div class="docs-screenshot-compact">

   ![Table](./assets/table.png)

   </div>

- Unique: Removes all duplicate lines from text, perfect for cleaning up massive lists of harvested IP addresses or emails.

---

## Table of tokens for DateTime

Please see below the table of tokens you can use as the input and output format of the functions above. Examples below are given for `2014-08-06T13:07:04.054` considered as a local time in America/New-York.

| **Standalone token** | **Description**                                                   | **Example**                 |
| -------------------- | ----------------------------------------------------------------- | --------------------------- |
| S                    | millisecond, no padding                                           | `54`                        |
| SSS                  | millisecond, padded to 3                                          | `054`                       |
| u                    | fractional seconds, (5 is a half second, 54 is slightly more)     | `54`                        |
| uu                   | fractional seconds, (one or two digits)                           | `05`                        |
| uuu                  | fractional seconds, (only one digit)                              | `5`                         |
| s                    | second, no padding                                                | `4`                         |
| ss                   | second, padded to 2 padding                                       | `04`                        |
| m                    | minute, no padding                                                | `7`                         |
| mm                   | minute, padded to 2                                               | `07`                        |
| h                    | hour in 12-hour time, no padding                                  | `1`                         |
| hh                   | hour in 12-hour time, padded to 2                                 | `01`                        |
| H                    | hour in 24-hour time, no padding                                  | `13`                        |
| HH                   | hour in 24-hour time, padded to 2                                 | `13`                        |
| Z                    | narrow offset                                                     | `+5`                        |
| ZZ                   | short offset                                                      | `+05:00`                    |
| ZZZ                  | techie offset                                                     | `+0500`                     |
| z                    | IANA zone                                                         | `America/New_York`          |
| a                    | meridiem                                                          | `AM`                        |
| d                    | day of the month, no padding                                      | `6`                         |
| dd                   | day of the month, padded to 2                                     | `06`                        |
| E                    | day of the week, as number from 1-7 (Monday is 1, Sunday is 7)    | `3`                         |
| EEE                  | day of the week, as an abbreviate localized string                | `Wed`                       |
| EEEE                 | day of the week, as an unabbreviated localized string             | `Wednesday`                 |
| M                    | month as an unpadded number                                       | `8`                         |
| MM                   | month as an padded number                                         | `08`                        |
| MMM                  | month as an abbreviated localized string                          | `Aug`                       |
| MMMM                 | month as an unabbreviated localized string                        | `August`                    |
| y                    | year, 1-6 digits, very literally                                  | `2014`                      |
| yy                   | two-digit year, interpreted as > 1960 by default (also accepts 4) | `14`                        |
| yyyy                 | four-digit year                                                   | `2014`                      |
| yyyyy                | four- to six-digit years                                          | `10340`                     |
| yyyyyy               | six-digit years                                                   | `010340`                    |
| G                    | abbreviated localized era                                         | `AD`                        |
| GG                   | unabbreviated localized era                                       | `Anno Domini`               |
| GGGGG                | one-letter localized era                                          | `A`                         |
| kk                   | ISO week year, unpadded                                           | `17`                        |
| kkkk                 | ISO week year, padded to 4                                        | `2014`                      |
| W                    | ISO week number, unpadded                                         | `32`                        |
| WW                   | ISO week number, padded to 2                                      | `32`                        |
| o                    | ordinal (day of year), unpadded                                   | `218`                       |
| ooo                  | ordinal (day of year), padded to 3                                | `218`                       |
| q                    | quarter, no padding                                               | `3`                         |
| D                    | localized numeric date                                            | `9/6/2014`                  |
| DD                   | localized date with abbreviated month                             | `Aug 6, 2014`               |
| DDD                  | localized date with full month                                    | `August 6, 2014`            |
| DDDD                 | localized date with full month and weekday                        | `Wednesday, August 6, 2014` |
| t                    | localized time                                                    | `1:07 AM`                   |
| tt                   | localized time with seconds                                       | `1:07:04 PM`                |
| T                    | localized 24-hour time                                            | `13:07`                     |
| TT                   | localized 24-hour time with seconds                               | `13:07:04`                  |
| f                    | short localized date and time                                     | `8/6/2014, 1:07 PM`         |
| ff                   | less short localized date and time                                | `Aug 6, 2014, 1:07 PM`      |
| F                    | short localized date and time with seconds                        | `8/6/2014, 1:07:04 PM`      |
| FF                   | less short localized date and time with seconds                   | `Aug 6, 2014, 1:07:04 PM`   |
| '                    | literal start/end, characters between are not tokenized           | `'T'`                       |
