Module 5: Getting Data into Splunk Enterprise/Cloud Flashcards
What is a Splunk intermediate forwarder?
- Last hop on the on-prem network
- Need to UF credentials from Splunk Cloud
- Gather data and parse if necessary
- Send data across the internet to Splunk Cloud for indexing
What is a Splunk Universal Forwarder?
Specifically designed to run on production servers o Lightweight o Minimal CPU and mem usages o No web interface A separate install binary Use UF for defining Splunk inputs
What is a Splunk Heavy Forwarder?
Splunk Enterprise binary configured as a forwarder (outputs.conf)
o Requires forwarder license
o Does everything except indexing
o Creates more network traffic than UF
Accepts all input types and can parse raw data
Can be used as IF for other UF
o As a mid-tier component
o Aggregate data from UFs, parse, and route to Splunk cloud
o Can be a single point of failure
What are the forwarder best practices?
• UFs are the preferred method of getting data in to Splunk cloud
• Use a HF only when necessary
o Parsing data before going into cloud
o When apps are not allowed on Splunk cloud
• Recommend ratio of forwarders to indexers is 2 UFs to 1 indexer
What are sourcetypes?
- Splunk uses source types to categorise data
- Can be used while searching to filter data
- Defined in props.conf on the indexer or heavy-forwarder
- Automatically assigned by Splunk or explicitly assigned by the data administrator.
What are the three phases of indexing?
Input phase
Parsing phase
Indexing phase
What is the input phase?
o data is read from data sources such as files and network
o Metadata such as host, source, and sourcetype are applied
o Most configuration is in inputs.conf (some in props.conf)
o Operates on the entire data stream
What is the parsing phase?
o Data is broken in to events o Timestamp extracted. o Most configuration in props.cnf (some in transforms.conf) o Operates on individual events o Event level transformations
What is the indexing phase?
o Segment events that then can be searched
o Build index
o Licence meter runs before the data is writing to disk
o Raw data and index files are written to disk (in buckets)
The Indexing pipeline can be viewed from the ___.
monitoring console (monitoring console > indexing > performance > indexing performance:instance)
What is the parsing pipeline?
Responsible for character encoding and line breaking
What is the merging pipeline?
Responsible for timestamp extraction and merges multi-line events
What is the typing pipeline?
Responsible for punctuations extraction and regex replacement
What is the indexing pipeline?
Responsible for indexing.
What are three use cases for using a HF?
- Parse and route data before sending to indexers
- Firewall rule simplification, can allow just the heavy forwarder through the firewall as opposed to all UF
- Use addons such as DBconnect and HTTP event collector.
How do you install a UF in Unix?
- Download the tar ball and unzip in desired location
- Install UF in a separate file system (example /opt/splunkforwarder)
- The command ‘Splunk’ is located in $SPLUNK_HOME/bin that can be used to start/stop UF
- Default admin password is changeme
- Employ config management tools such as ansible or puppet
How do you install a UF in Windows?
- Can use MSI wizard or msiexec.exe CLI
- Can be installed as a domain admin
- Use msiexec.exe with /quite an AGREETOLICENSE=yes for complete silent install
- Can enable windows event logs and perfmon ingestion (recommended)
- Runs as a windows service
What must be done to set up forwarding?
- Set up receiving on the indexers. Primary config file is inputs.conf
- Set up forwarding on the UF. Primary config file is outputs.conf
How do you set up receiving?
- Must be done before forwarders can send data
- Using gui – Settings > forwarding and receiving > configure receiving
- CLI – Splunk enable listen
- Default port is 9997.
How do you set up forwarding?
- Use CLI or outputs.conf to set up forwarding
- CLI - ./Splunk add forward-server
- Edit outputs.conf
What are the four ways to add inputs?
• Using deployment server to distribute inputs.conf
• Using Splunk CLI]
o $SPLUNK_HOME/bin/slunk add monitor [location & filename]
• Editing inputs.conf manually
• Using Splunk add-ons
What are some additional forwarder configurations?
Forward data selectively to multiple indexers Load balancing Compression SSL Buffering (queueing) Index acknowledgement
How do you configure forwarding data selectively to multiple indexers?
- Define two tcpout destinations in outputs.conf
* Use _TCP_ROUTING in inputs.conf to selectively send data
How do you configure compression?
- Enable compression on both indexer and forwarder
- Reduces network traffic, but slightly increase CPU usage
- Compressed=true on the forwarder outputs.conf and indexer inputs.comf
How do you configure SSL?
- Enabling SSL automatically enables compression
- Can increase CPU usage
- Specify cert files in the conf files
How do you configure buffering?
- Configure maxQueueSize in outputs.conf to enable queuing
- Default is 500kb when useACK is false. 7mb when useACK is true
- When load balancing is enabled, queueing occurs only when all indexers are not available
How do you configure index acknowledgement?
- Ensures data is received by indexers
- Protects against data loss
- Forwarders resend data not acknowledged by indexer
- A wait queue is created with 3x size of maxQueuSize
- Enabled by using useACK = true in outputs.conf
How do you configure load balancing accross multiple indexers?
- Specify a list of indexers in the server list in outputs.conf
- Every 30 seconds, forwarder will randomly select an available indexer from the list
- Frequency can be updated using autoLBFrequency setting (not recommended)
Name the troubleshooting steps when troubleshooting a forwarder?
Use SPL • index=_internal host= Check if forwarder is running • $SPLUNK_HOME/bin/Splunk status • Ps -ef | grep Splunk Check forwarder configuration • $SPLUNK_HOME/bin/Splunk list forward-server Check indexer configuration • $SPLUNK_HOME/bin/Splunk display listen Check forwarder splunkd log • $SPLUNK_HOME/var/log/Splunk/splunkd.log
