transforms.conf basics Flashcards
What is transforms.conf?
It is a .conf file that allows us to make changes to the data that we onboarding, such as:
- Sending events to NULL queue
- Separating a single source type into multiple ones
- Host ans sources overrides based on regex
- Delimiter based field extractions
- Anonymizing data
What is a Splunk’s pipeline?
Pipeline is the route that the data takes when it comes through Splunk, and depending on the phase of that process, different things happen to the data, such as line breaking, line merging, time extraction.
What is a queue?
It is a memory space to store data between pipelines
What kind of pipelines there is, and what do they do?
a) Parsing pipeline:
- parsing of utf8 decoding
- linebreaker
- header
- First place to split data stream into a single line event
b) Merging pipeline:
- aggregator
- line merging for multiline events
- time extraction
c) Typing pipeline:
- regex replacement
- annotator
d) Index pipeline:
- syslog out
- tcp out to another Splunk component
- indexer
What kind of queues there is? (in order)
- parsing queue
- aggregator queue
- typing queue
- index queue
What is the transforms.conf stanza for discarding specific events?
[]
REGEX=
DEST_KEY=queue
FORMAT=nullQueue
What is the transforms.conf stanza for deleting all events BUT ones with xyz?
[]
REGEX=.
DEST_KEY=queue
FORMAT=nullQueue
[]
REGEX=xyz
DEST_key=queue
FORMAT=indexQueue
What is the transforms.conf stanza for splitting a log into multiple sourcetypes?
[]
REGEX=
DEST_KEY=Metadata:Sourcetype
FORMAT=sourcetype::xyz
[]
REGEX=
DEST_KEY=Metadata:Sourcetype
FORMAT=sourcetype::xyz2
What is the transforms.conf stanza for changing host name?
[]
REGEX=
DEST_KEY=Metadata:Host
FORMAT=
What is the transforms.conf and props.conf for delimiter based field extraction?
props.conf:
[]
REPORT-=xyz
transforms.conf:
[xyz]
DELIMS=”,”
FIELDS=”saleTime”,”productName”,”somethingElse”,
What is the transforms.conf for masking data?
[xyz]
REGEX=
DEST_KEY=_raw
FORMAT=
How to anonymize data with props.conf?
[]
SEDCMD-= s///g