Splunk

Question 1

Compare Splunk with Spark.

Answer 1

Criteria
Deployment area
Splunk: Collecting large amounts of
machine-generated data
Spark: Iterative applications and in-memory processing

Nature of tool
Splunk: Proprietary
Spark: Open-source

Working mode
Splunk: Streaming mode
Spunk: Both streaming and batch modes

Question 2

What is Splunk?

Answer 2

Splunk is ‘Google’ for our machine-generated data. It’s a software/engine that can be used for searching, visualizing, monitoring, reporting, etc. of our enterprise data. Splunk takes valuable machine data and turns it into powerful operational intelligence by providing real-time insights into our data through charts, alerts, reports, etc.

Question 3

What are the common port numbers used by Splunk?

Answer 3

Splunk Web port : 8000
Splunk Management port : 8089
Splunk Indexing port : 9997
Splunk Index Replication port: 8080
Splunk Network port: 514
KV Store : 8191

Question 4

4. What are the components of Splunk? Explain Splunk architecture.

Answer 4

Search Head: Provides the GUI for searching
Indexer: Indexes the machine data
Forwarder: Forwards logs to the Indexer.
Deployment Server: Manages Splunk components in a distributed environment.

Question 5

6. What is a Splunk indexer? What are the stages of Splunk indexing?

Answer 5

A Splunk indexer is the Splunk Enterprise component that creates and manages indexes. The primary functions of an indexer are mentioned below:

Indexing incoming data
Searching the indexed data
Picture

Question 6

What is a Splunk forwarder? What are the types of Splunk forwarders?

Answer 6

There are two types of Splunk forwarders, which are mentioned below:

Universal Forwarder (UF): the Splunk agent installed on a non-Splunk system to gather data locally; it can’t parse or index data.
Heavyweight Forwarder (HWF): A full instance of Splunk with advanced functionalities.
It generally works as a remote collector, intermediate forwarder, and possible data filter, and since it parses data, it is not recommended for production systems.

Question 7

Can you name a few most important configuration files in Splunk?

Answer 7

props.conf
indexes.conf
inputs.conf
transforms.conf
server.conf

Question 8

What are the types of Splunk Licenses?

Answer 8

Enterprise license
Free license
Forwarder license
Beta license
Licenses for search heads (for distributed search)
Licenses for cluster members (for index replication)

Question 9

What is the Splunk app?

Answer 9

The Splunk app is a container or directory of configurations, searches, dashboards, etc. in Splunk.

Question 10

Where is the Splunk default configuration stored?

Answer 10

1
$splunkhome/etc/system/default

Question 11

What are the features not available in Splunk Free?

Answer 11

Splunk Free does not include below features:

Authentication and scheduled searches/alerting
Distributed search
Forwarding in TCP/HTTP (to non-Splunk)
Deployment management

Question 12

What happens if the license master is unreachable?

Answer 12

If the license master is not available, the license slave will start a 24-hour timer, after which the search will be blocked on the license slave (though indexing continues). However, users will not be able to search for data in that slave until it can reach the license master again.

Question 13

What is a summary index in Splunk?

Answer 13

A summary index is the default Splunk index (the index that Splunk Enterprise uses if we do not indicate another one).

If we plan to run a variety of summary index reports, we may need to create additional summary indexes.

Question 14

What is Splunk DB Connect?

Answer 14

Splunk DB Connect is a generic SQL database plugin for Splunk that allows us to easily integrate database information with Splunk queries and reports.

Question 15

How do I troubleshoot Splunk performance issues?

Answer 15

The answer to this question would be very wide, but, mostly, an interviewer would be looking for the following keywords:

Check splunkd.log for errors
Check server performance issues, i.e., CPU, memory usage, disk I/O, etc.
Install the SOS (Splunk on Splunk) app and check for warnings and errors in its dashboard
Check the number of saved searches currently running and their consumption of system resources
Install and enable Firebug, which is a Firefox extension. Log into Splunk (using Firefox) and open Firebug’s panels. Then, switch to the ‘Net’ panel, which we will have to enable. The Net panel will show us the HTTP requests and responses, along with the time spent on each. This will give us a lot of information quickly, such as which requests are hanging Splunk, which requests are blameless, etc.

Question 16

What are buckets? Explain the Splunk bucket lifecycle.

Answer 16

Splunk places indexed data in directories, which are called ‘buckets.’ It is physically a directory containing events from a certain period.

A bucket moves through several stages as it ages. Below are the various stages it goes through:

Hot: A hot bucket contains newly indexed data. It is open for writing. There can be one or more hot buckets for each index.
Warm: A warm bucket consists of data rolled out from a hot bucket. There are many warm buckets.
Cold: A cold bucket has data that is rolled out from a warm bucket. There are many cold buckets.
Frozen: A frozen bucket is comprised of data rolled out from a cold bucket. The indexer deletes frozen data by default, but we can archive it. Archived data can later be thawed (data in a frozen bucket is not searchable).
By default, the buckets are located in the following location:

1
$SPLUNK_HOME/var/lib/splunk/defaultdb/db
We should see the hot-db there and any warm buckets we have. By default, Splunk sets the bucket size to 10 GB for 64-bit systems and 750 MB for 32-bit systems.

Question 17

What is the difference between stats and eventstats commands?

Answer 17

The stats command generates summary statistics of all the existing fields in the search results and saves them as values in new fields.
Eventstats is similar to the stats command, except that the aggregation results are added inline to each event and only if the aggregation is pertinent to that event. The eventstats command computes requested statistics, much like how stats do, but aggregates them to the original raw data.

Question 18

Who are the top direct competitors to Splunk?

Answer 18

Logstash, Loggly, LogLogic, Sumo Logic, etc. are some of the top direct competitors to Splunk.

Question 19

What do Splunk licenses specify?

Answer 19

Splunk licenses specify how much data we can index per calendar day.

Question 20

What is the command for restarting Splunk web server?

Answer 20

This is another frequently asked Splunk commands interview question. Get a thorough idea of commands We can restart the Splunk web server by using the following command:

1
splunk start splunkweb

Question 21

What is the command for restarting the Splunk Daemon?
Splunk Deamon can be restarted with the below command

Answer 21

splunk start splunkd

Question 22

What is the command used to check the running Splunk processes on Unix/Linux?

Answer 22

if we want to check the running Splunk Enterprise processes on Unix/Linux, we can make use of the following command:

1
ps aux | grep splunk

Question 23

What is the command used for enabling Splunk to boot start?

Answer 23

To boot start Splunk, we have to use the following command:

1
$SPLUNK_HOME/bin/splunk enable boot-start

Question 24

How to disable Splunk boot-start?

Answer 24

1
$SPLUNK_HOME/bin/splunk disable boot-start