Settings

Delete batch size

This setting is used to control the number of events the module deletes when the local database has stored the maximum number of events. The module deletes a batch of events when the database is full to reduce the number of database transactions. By increasing this value, the module deletes more events when it becomes full and allow more time until the next deletion is required. However, increasing this value will also increase the amount of data lost when the database is full. Decreasing this value will delete fewer items when the database is full causing less data loss and deletions will occur more frequently. Depending on the number of events being generated, this can cause an increase in CPU usage. The value is only used when the database is full

100

Data transmission frequency (seconds)

The module does not send the data immediately as received where this can put a substantial load on servers (thousands of endpoints connecting sending a small number of events). Instead, the module waits for a period and sends batches of events to the destination server thus reducing the number of simultaneous connections on the server and allowing greater scaling. By increasing this value, the module sends events less frequently to the server which can be useful if the destination server is struggling to scale to the required capacity. However, by increasing this frequency will also introduce a longer delay between event and potential server alert. By decreasing this value, the module sends the data more frequently and will introduce additional load on the destination server. The advantage of increasing the value is a decrease in time from event generation to potential server alert.

20

Maximum number of events sent in a batch

When the module sends stored data to the destination server to reduce load on the server, the data is sent in batches. This value specifies the maximum number of events to be sent in a batch to the server. By increasing this value, the overall message size being sent to the server is increased. While this increases the number of events being sent to the server, it can cause difficulties with the destination server being able to handle the substantial load. Decreasing this value will reduce the load on the destination server but may also delay events being processed. Modification of this value can cause Helix to fail to accept and ingest the data.

250

Database vacuum frequency

The module caches the collected events in a local database. Over time, as events are written, read then deleted the overall database size on disk may be larger than required. Vacuuming the database reduces the amount of drive space used and can improve the overall performance. While the size on disk may be reduced, vacuuming the database is a CPU intensive operation that may take a fair amount of time to complete. This is the number of events that are deleted before the module vacuums the database. Increasing this value delays the frequency of the vacuuming process however, may increase the amount of time required for the vacuum process to complete. Decreasing this value can cause the vacuum process to run more frequently but for a shorter overall duration. This value should only be modified if the system is producing a large number of events per second and module is consuming a substantial amount of CPU.

1000

Maximum number of events stored in the database

The module caches the events in a local database for transmission when possible. This is the maximum number of events the database will store before reaching maximum capacity and starting to delete older events. By increasing this value, the module will store more events before reaching maximum capacity and starting to delete however this will also increase the amount of disk space required. By reducing this value the module will store less events in the database and therefore use less space on disk but may increase the amount of data lost as events are deleted.

10000

Store events in memory only

In some situations, it may be advantageous to cache the events in memory only to reduce the disk I/O the module performs. By setting this value to true, the module will not cache events in a database on disk instead will only store the events in memory. This will significantly reduce the disk I/O used by the module however, could result in data loss due to a system unexpected reboot, power failure or a software fault (crash). This setting is most useful when the module is installed and running on a server that is generating a large number of events per second and power interruptions are unlikely.

false

Send on number of events in memory and on time

This setting is only used when events are stored in memory and not on disk. When set to "True", the module will attempt to send events to the destination server when the specified number of events are collected and stored in memory or when the specified timeout is reached. For example, when set to true, the number of events set to 100 and the data transmission frequency set to 20, the module will attempt to send the events to the destination server when 100 events are stored in memory or 20 seconds has elapsed since the last send (even if less than 100 events are stored). This setting is intended to be used on servers producing many events per second allowing the module to support this rate without causing data loss. When enabled, this can cause additional load on the destination server and potentially cause requests to be rejected due to overload.

true

Number of events to send on

This setting is only used if events are stored in memory and the option to send number of events in memory and on time is set to true. When used, this value is the number of events that must be reached before the timeout occurs for the module to attempt to send the data to the destination server. If this value is set too low the module will attempt to make frequent connections to the destination server which can increase the load on the server and cause requests to be rejected therefore delaying the ingestion of the data into the system.

100

Reset policy back to default settings

This action will reset ALL values back to the original values; including clearing out all custom event logs, syslog server, filters and custom settings.

false