The Advanced Syslog Parser (ASP) extracts (parses) data out of syslog messages, based on user-defined rules.
ASP uses rules to identify where data resides in message-specific events, such as signature IDs, IP addresses, ports, user names, and actions.
When the system receives an ASP log, it compares the time format in the log with the format specified in the ASP rule. If the time format doesn't match, the system doesn't process the log.
To increase the likelihood of matching time formats, add multiple custom time formats.
With Policy Administrator rights, you can define the order for running ASP rules.
Custom ASP rules
You can write rules to sort parse complex log sources.
Note
This functionality requires knowledge of regular expressions.
The first regular expression determines if a message is parsed, so write the first rule to look for a pattern that is present in all message you want the rule to parse. More regular expressions can be written to capture values from the messages and map them to custom types in the Trellix ESM. Subsequent regular expressions do not determine the rule match, and are used for parsing only.
While it is possible to test regular expression results on a few log lines in the Trellix ESM console itself, we recommend using a graphical tool. There are many free web-based tools that can be used in addition to standalone installable tools. Optionally, another useful tool would be a text editor that supports regular expression searches. Any tools used to test regular expressions need to support pcre expressions.
Important
Make sure regular expressions are written to maximize efficiency. Poorly written expressions can adversely affect parsing performance.
Optimize your rules by:
Thoroughly understanding the value that a log can provide to your organization.
Making sure that captured values align with the intended use of the specific custom type fields.
Avoiding indexing fields that contain unique and random or high cardinality data (such as URLs).
Making sure that rules mapping event messages directly from the log do not map unique, random, or high cardinality strings as messages. Trellix ESM creates a data source rule for each unique event message, and numerous unique strings can reduce Trellix ESM performance.
Categorizing events by adding a normalized category to the rule. Data source rules, generated by parsing rules, inherit the normalization assigned to the main parsing rule. If the main parsing rule is left normalized to "Uncategorized," then the parsed events are also normalized as "Uncategorized," making a search for "Uncategorized" events to find unparsed events inaccurate.
ASP rule format
Define rules as follows.
Section | Field | Description |
|---|---|---|
Rule Header |
| The rule header contains the Alert action and the 'any any any' format. The rule is:
|
Action | Option of what to do with the event when a match occurs:
| |
Protocol | If the event defines a protocol, filter the effective match based on the protocol. | |
Src/Dst IP | If the event defines a source or destination IP address, filter the effective match based on that address. | |
Src/Dst Port | If the event defines a source or destination port, filter the effective match based on that port. | |
Rule Body |
| The rule body contains most the match criteria and defines how the data must be parsed and logged into the database. Elements of the Rule Body are defined in keyword-option pairs. Some keywords have no following option. |
msg | (Required) The message to associate with this rule. This is the string displayed in the Trellix ESM Thin Client for reporting purposes unless overridden with a pcre/setparm detected message (see below). The first work of the msg is the category name followed by actual message (msg: "category rule message"). | |
content | (Optional — one or more) The content keyword is a non-wildcard text qualifier to pre-filter events as they pass through the rule set, which can also contain spaces (for example, content: "search 1"; content "something else") | |
procname | On many UNIX and Linux systems, the process name (and process ID) is part of a standardized syslog message header. The procname keyword can be used to filter Event matches for the Rule. Used to exclude or filter Event matches where two processes on a Linux or UNIX server might have similar or the same message text. | |
adsid | The data source ID to use. This value overrides the Default Rule Assignment in the data source editor. | |
sid | Signature ID of the Rule. This is the match ID used in the Trellix ESM Thin Client unless overridden with a pcre/setparm detected sid. | |
rev | Rule revision. Used to track changes. | |
severity | Value between 1 (least severe) and 100 (most severe) assigned to events matching the rule. | |
pcre | The PCRE keyword is a Perl Compatible Regular Expression match against incoming events. The PCRE is quote delimited and all occurrences of "/" is treated as a normal character. Content in parentheses is held for the use of the setparm keyword. You can change the PCRE keyword by nocase, nomatch, raw and setparm keywords. | |
nocase | Causes the PCRE content to be matched whether the case matches or not. | |
nomatch | Inverts the PCRE match (equivalent to !~ in Perl). | |
raw | Compare the PCRE to the entire syslog message including header data (Facility, daemon, date, host/IP, process name, and process ID). Normally the header is not used in the PCRE match. | |
setparm | Can occur more than once. Each set of parentheses in the PCRE is assigned a number in order of occurrence. Those numbers can be assigned to data tags (for example: setparm:username=1). This takes the captured text in the first set of parentheses and assigns it to the user name data tag. Recognized tags are listed in the table below. |
Tag | Description | ||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
* sid | This captured parameter overrides the matched rule's sid. | ||||||||||||||||||||||||||||||||||||||||||
* msg | This captured parameter overrides the matched rule's message or name. | ||||||||||||||||||||||||||||||||||||||||||
* action | This captured parameter indicates what action the third-party device took. | ||||||||||||||||||||||||||||||||||||||||||
* protocol |
| ||||||||||||||||||||||||||||||||||||||||||
* src_ip | This replaces the syslog source's IP address which is the default source IP address of an event. | ||||||||||||||||||||||||||||||||||||||||||
* src_port |
| ||||||||||||||||||||||||||||||||||||||||||
* dst_ip |
| ||||||||||||||||||||||||||||||||||||||||||
* dst_port |
| ||||||||||||||||||||||||||||||||||||||||||
* src_mac |
| ||||||||||||||||||||||||||||||||||||||||||
* dst_mac |
| ||||||||||||||||||||||||||||||||||||||||||
* dst_mac |
| ||||||||||||||||||||||||||||||||||||||||||
* genid | This is used to change the sid as stored in the database, used for non-Trellix snort matches in snort preprocessors. | ||||||||||||||||||||||||||||||||||||||||||
* url | Reserved, but not used yet. | ||||||||||||||||||||||||||||||||||||||||||
* src_username | First/source user name. | ||||||||||||||||||||||||||||||||||||||||||
* username | Alternate name for src_username. | ||||||||||||||||||||||||||||||||||||||||||
* dst_username | Second/destination user name. | ||||||||||||||||||||||||||||||||||||||||||
* domain |
| ||||||||||||||||||||||||||||||||||||||||||
* hostname |
| ||||||||||||||||||||||||||||||||||||||||||
* application |
| ||||||||||||||||||||||||||||||||||||||||||
* severity | Must be an integer. | ||||||||||||||||||||||||||||||||||||||||||
* action map | Allows you to map specific actions of your product to the Trellixactions. The action map is case sensitive. Example: alert any any any -> any any (msg:"OpenSSH Accepted Password"; content:"Accepted password for "; action_map:Accepted=8, Blocked=3; pcre:"(Accepted)\s+password\s+for\s+(\S+)\s+from\s+(\d+\.\d+\.\d+\.\d+)\s+port\s+(\d+)"; setparm:action=1; sid:31; rev:1;)). See Severity and Action Map for details. | ||||||||||||||||||||||||||||||||||||||||||
* severity map | Allows you to map specific severities of your product to the Trellixseverity. Like the action map, the severity map is case sensitive. Example: alert any any any -> any any (msg:"OpenSSH Accepted Password"; content:"Accepted password for "; severity_map:High=99, Low=25, 10=99, 1=25; pcre:"(Accepted)\s+password\s+for\s+(\S+)\s+from\s+(\d+\.\d+\.\d+\.\d+)\s+port\s+(\d+)"; setparm:action=1; sid:31; rev:1;))pri(?:\x3d|\x3a)\s*(?:p\x5f)?([^\x2c]+). See Severity and Action Map for details. | ||||||||||||||||||||||||||||||||||||||||||
* var | This is another way to use setparms. The beneficial use is the use of creating one value from multiple captures of multiple PCREs. You can create more than one PCRE that captures only a small portion of your string rather than one large PCRE with multiple captures. Here's an example of capturing a user name, domain, and creating an email address to store in the objectname field.
| ||||||||||||||||||||||||||||||||||||||||||
* sessionid | This is an integer. | ||||||||||||||||||||||||||||||||||||||||||
* commandname | This is a string value. | ||||||||||||||||||||||||||||||||||||||||||
* objectname | This is a string value. | ||||||||||||||||||||||||||||||||||||||||||
* event_action | This tag is used to set a default action. You can't use event_action and action_map in the same rule. For example, if you had an event for a Successful Login you could use the event_action tag and default the action to success (for example, event_action:8;). | ||||||||||||||||||||||||||||||||||||||||||
* firsttime_fmt | Used to set the first time of the event. See list of formats. | ||||||||||||||||||||||||||||||||||||||||||
* lasttime_fmt | Used to set the last time of the event. See list of formats. You can use this with a setparm or a var (var:firsttime="${1:1}" or setparm:lasttime="1"). For example:
For current formats supported, see http://pubs.opengroup.org/onlinepubs/009695399/functions/strptime.html for more detail.
|
This is an example of a rule that identifies a password based on OpenSSH logon and pulls from the event’s source IP address, source port, and user name: alert any any any -> any any (msg:"OpenSSH Accepted Password";content:"Accepted password for ";pcre:"Accepted\s+password\s+for\s+(\S+)\s+from\s+(\d+\.\d+\.\d+\.\d+)\s+port\s+(\d+)";setparm:username=1;setparm:src_ip=2;setparm:src_port=3;sid:31;rev:1;)