Splunk Architecture 101 Flashcards

1
Q

What are some Splunk basic ports?

A
8000 - Splunk Web Port
9997 - Splunk Indexing Port
8191 - Splunk KV Store Port
8080 - Splunk Replication Port
8089 - Splunk Management Port
8088 - Splunk HEC
5514 - Syslog
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What for is the 8000 port used in Splunk?

A

Used to access Splunk through web (Splunk Web Port)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What for is the 9997 port used in Splunk?

A

Used for Indexers to recieve data from forwarders (Splunk Recieving Port)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What for is the 8191 port used in Splunk?

A

Splunk KV Store Port

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What for is the 8080 port used in Splunk?

A

Used for indexers to replicate data. Splunk Replication Port

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What for is the 8089 port used in Splunk?

A

Management Port. E.g used to making components clients of a deployment server.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What for is the 8088 port used in Splunk?

A

Splunk HEC

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What for is the 5514 port used in Splunk?

A

Syslog

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What ports would you add to security group when setting up basic splunk environment?

A
8000 - Splunk Web Port
9997 - Splunk Indexing Port
8191 - Splunk KV Store Port
8080 - Splunk Replication Port
8089 - Splunk Management Port
8088 - Splunk HEC
5514 - Syslog
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

How would you install splunk enterprise/splunk UF on a completely new instance?

A
  1. Prep system with group/user permissions
  2. Download WGET/Install splunk with RPM.
  3. Enable boot-start etc, turn off systemd, accept license, specify auto-ports ( ./splunk enable boot-start -systemd-managed 0 -user splunk –accept-license –auto-ports )
  4. Change ownership of $SPLUNK_HOME
  5. Create admin credenstials
    Either:
    - Turn on Splunk (it will prompt you for new admin credenstials)
    -or $SPLUNK_HOME/etc/system/local/user-seed.conf:
    [user_info]
    USERNAME = admin
    PASSWORD = password
  6. Change the host name of server, and minimum amount of space before splunkd is halted:

$SPLUNK_HOME/etc/system/local/server.conf

[general]
serverName = xyz

[diskUsage]
minFreeSpace = 500 (or other value)

  1. Change how host names are reflected in the logs:
    $SPLUNK_HOME/etc/system/local/inputs.conf:
    [default]
    host = xyz
  2. Enable HTTPS:
    $SPLUNK_HOME/etc/system/local/web.conf
    [settings]
    enableSplunkWebSSL
  3. Start Splunk, confirm that it is indeed working
  4. Configure license master and license slaves:
    - Add license through GUI
    - ./splunk edit licenser-localslave -master_uri https://(IP):managementport
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Dissect the “bootstart” command

A

./splunk enable boot-start -systemd-managed 0 -user splunk –accept-license –auto-ports

enable boot-start: enable automatic boot of splunk whenever server is running

  • systemd-managed 0: turn off system d to manage this application
  • user splunk: specifies, that the user splunk will run the program whenever this automatic boot procedure is happening

–accept-license - Accept the Splunk license automatically when starting for the first time

–auto-ports use default ports specified by Splunk

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is systemd?

A

First, anything that ends in a “d” is a daemon. That means is a process that works in the background.

Systemd it’s the daddy of processes. Process ID number 1. It starts all other processes/daemons at boot. It has integrated tools that manage, for example, wifi, bluetooth, suspend/shutdown, etc.

Some time ago the norm was to use a simpler process called init to start any other process. It did not include the managing of those system parts I just mentioned, so you needed a separate program for each one of those. Systemd kind of standardized these shits, like a central process. Some people don’t like it because it may have a lot of bugs (I don’t know), others just don’t like the idea of centralizing processes too much, it has gotten very big and monstrous apparently.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

How to get server’s public ip using a command?

A

curl https://ipinfo.io/ip

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What for is the app.conf file in splunk? And what are some of it’s attributes?

A
# This file maintains the state of a given app in the Splunk platform. It can
# also be used to customize certain aspects of an app.
#
# An app.conf file can exist within each app on the Splunk platform.
#
# You must restart the Splunk platform to reload manual changes to app.conf.

[install]

state = disabled | enabled

  • Determines whether an app is disabled or enabled on the Splunk platform.
  • If an app is disabled, its configurations are ignored.
  • Default: enabled

is_configured =
* Stores an indication of whether the application’s custom setup has been
performed.
* Default: false

[launcher]
Settings in this stanza determine how an app appears in the Launcher in the Splunk
platform and online on Splunkbase.

author=
description=
version=

[package]
* This stanza defines upgrade-related metadata that streamlines app upgrade
to future versions of Splunk Enterprise.

id =
* Omit this setting for apps that are for internal use only and not intended
for upload to Splunkbase.
* id is required for all new apps that you upload to Splunkbase. Future versions of
Splunk Enterprise will use appid to correlate locally-installed apps and the
same app on Splunkbase (e.g. to notify users about app updates).
* id must be the same as the folder name in which your app lives in
$SPLUNK_HOME/etc/apps.
* id must adhere to these cross-platform folder name restrictions:
* must contain only letters, numbers, “.” (dot), and “_” (underscore)
characters.
* must not end with a dot character.

[ui]
* This stanza defines UI-specific settings for this app.

is_visible =

  • Indicates if this app is visible/navigable as an app in Splunk Web.
  • Apps require at least one view to be available in Splunk Web.
label = 
* Defines the name of the app shown in Splunk Web and Launcher.
* Recommended length between 5 and 80 characters.
* Must not include "Splunk For" prefix.
* Label is required.
* Examples of good labels:
    IMAP Monitor
    SQL Server Integration Services
    FISMA Compliance
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What for is $SPLUNK_HOME/etc/system/default directory?

A

The default directory contains preconfigured versions of the configuration files with default settings. The location of the default directory in a Splunk Enterprise installation is $SPLUNK_HOME/etc/system/default.

“all these worlds are yours, except /default - attempt no editing there” – duckfez, 2010

You should never change a configuration file that’s located in the $SPLUNK_HOME/etc/system/default directory. The Splunk Enterprise upgrade process overwrites the contents in that folder automatically, which will remove any changes. If you want to retain a setting you’ve changed through an upgrade, place your configuration file into a local folder path such as $SPLUNK_HOME/etc/system/local or $SPLUNK_HOME/etc/apps/$app_name/local as described below.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

How does splunk read configuration files?

A

Splunk software uses configuration files to determine nearly every aspect of its behavior. A Splunk platform deployment can have many copies of the same configuration file. These file copies are usually layered in directories that affect either the users, an app, or the system as a whole.

When editing configuration files, it is important to understand how Splunk software evaluates these files and which ones take precedence.

When incorporating changes, Splunk software does the following to your configuration files:

It merges the settings from all copies of the file, using a location-based prioritization scheme.
When different copies have conflicting attribute values (that is, when they set the same attribute to different values), it uses the value from the file with the highest priority.
It determines the priority of configuration files by their location in the directory structure, according to the rules described in this topic.

17
Q

How Splunk determines precedence order within global context?

A

Precedence within global context
When the file context is global, directory priority descends in this order:

  1. System local directory – highest priority
  2. App local directories
  3. App default directories
  4. System default directory – lowest priority

When consuming a global configuration, such as inputs.conf, Splunk software first uses the attributes from any copy of the file in system/local. Then it looks for any copies of the file located in the app directories, adding any attributes found in them, but ignoring attributes already discovered in system/local. As a last resort, for any attributes not explicitly assigned at either the system or app level, it assigns default values from the file in the system/default directory.

18
Q

How Splunk determines precedence order within global contest, but for indexer cluster peers?

A

There is an expanded precedence order for indexer cluster peer configurations, which are considered in the global context. This is because some configuration files, like indexes.conf, must be identical across peer nodes.

To keep configuration settings consistent across peer nodes, configuration files are managed from the cluster manager node, which pushes the files to the slave-app directories on the peer nodes. Files in the slave-app directories have the highest precedence in a cluster peer’s configuration. These directories exist only on indexer cluster peer nodes.

Here is the expanded precedence order for cluster peers:

  1. Slave-app local directories – highest priority
  2. System local directory
  3. App local directories
  4. Slave-app default directories
  5. App default directories
  6. System default directory – lowest priority
19
Q

How Splunk determines precedence order within app or user context?

A

For files with an app/user context, directory priority descends from user to app to system:

  1. User directories for current user – highest priority
  2. App directories for currently running app (local, followed by default)
  3. App directories for all other apps (local, followed by default) – for exported settings only
  4. System directories (local, followed by default) – lowest priority

An attribute in savedsearches.conf, for example, might be set at all three levels: the user, the app, and the system. Splunk will always use the value of the user-level attribute, if any, in preference to a value for that same attribute set at the app or system level.

20
Q

How to change the settings to some configuration file?

A

To change the settings in a particular configuration file, you must first create a new file of the same name in a non-default directory (/etc/system/local), and add the required settings and changed values to your new configuration file. A setting with a new value defined in a non-default directory will take precedence over a setting defined in the default directory.

When changing a default setting using a new configuration file, you only need to define the stanza category, the setting, and update the value. Do not make a complete copy of the configuration file from the default directory into another folder, as the settings in that copy will take precedence and override changes made during an upgrade.

21
Q

Tell us about the Stanza scope

A

Stanza scope
Configuration files frequently have stanzas with varying scopes, with the more specific stanzas taking precedence. For example, consider this example of an outputs.conf configuration file, used to configure forwarders:

[tcpout]
indexAndForward=true
compressed=true

[tcpout:my_indexersA]
compressed=false
server=mysplunk_indexer1:9997, mysplunk_indexer2:9997

[tcpout:my_indexersB]
server=mysplunk_indexer3:9997, mysplunk_indexer4:9997

Note that this example file has two levels of stanzas:

The global [tcpout], with settings that affect all tcp forwarding.
Two [tcpout:] stanzas, whose settings affect only the indexers defined in each target group.
The setting for compressed in [tcpout:my_indexersA] overrides that attribute’s setting in [tcpout], for the indexers in the my_indexersA target group only.

22
Q

What for is server.conf in Splunk, and what are some of it’s attributes?

A

[general]
serverName =
* The name that identifies this Splunk software instance for features such as
distributed search.
* Cannot be an empty string.
* Can contain environment variables.
* After any environment variables are expanded, the server name
(if not an IPv6 address) can only contain letters, numbers, underscores,
dots, and dashes. The server name must start with a letter, number, or an
underscore.
* Default: -

pass4SymmKey =
* Authenticates traffic between:
* License master and its license slaves.
* Members of a cluster.
* Deployment server (DS) and its deployment clients (DCs).
* When authenticating members of a cluster, clustering might override the
passphrase specified in the clustering stanza. A clustering searchhead
connecting to multiple managers might further override in the
[clustermaster:stanza1] stanza.
* When authenticating deployment servers and clients, by default, DS-DCs
passphrase authentication is disabled. To enable DS-DCs passphrase
authentication, you must also add the following line to the [broker:broker]
stanza in the restmap.conf file: requireAuthentication = true
* In all scenarios, every node involved must set the same passphrase in
the same stanzas. For example in the [general] stanza and/or
[clustering] stanza. Otherwise, the respective communication does not proceed:
- licensing and deployment in the case of the [general] stanza
- clustering in case of the [clustering] stanza)
* Unencrypted passwords must not begin with “$1$”. This is used by
Splunk software to determine if the password is already encrypted.

[diskUsage]

minFreeSpace = |
* Minimum free space for a partition.
* Specified as an integer that represents a size in binary
megabytes (ie MiB) or as a percentage, written as a decimal
between 0 and 100 followed by a ‘%’ sign, for example “10%”
or “10.5%”
* If specified as a percentage, this is taken to be a percentage of
the size of the partition. Therefore, the absolute free space required
varies for each partition depending on the size of that partition.
* Specifies a safe amount of space that must exist for splunkd to continue
operating.
* Note that this affects search and indexing
* For search:
* Before attempting to launch a search, Splunk software requires this
amount of free space on the filesystem where the dispatch directory
is stored, $SPLUNK_HOME/var/run/splunk/dispatch
* Applied similarly to the search quota values in authorize.conf and
limits.conf.
* For indexing:
* Periodically, the indexer checks space on all partitions
that contain splunk indexes as specified by indexes.conf. Indexing
is paused and a ui banner + splunkd warning posted to indicate
need to clear more disk space.

23
Q

What for is the web.conf files in Splunk? Name some of it’s attributes

A

It’s used to configure the Splunk Web interface.

[settings]
* Set general Splunk Web configuration options under this stanza name.
* Follow this stanza name with any number of the following setting/value
pairs.
* If you do not specify an entry for each setting, Splunk Web uses the
default value.

startwebserver = [0 | 1]

  • Set whether or not to start Splunk Web.
  • 0 disables Splunk Web, 1 enables it.
  • Default: 1

enableSplunkWebSSL =

  • Toggle between http or https.
  • Set to true to enable https and SSL.
  • Default: false
24
Q

What for is the user-seed.conf files in Splunk? Name some of it’s attributes

A

Allows configuration of Splunk’s initial username and password. Currently, only one user can be configured
with user-seed.conf
Use this file to create an initial login.

NOTE: When starting Splunk for first time, hash of password is stored in
$SPLUNK_HOME/etc/system/local/user-seed.conf and password file is seeded
with this hash. This file can also be used to set default username and password, if $SPLUNK_HOME/etc/passwd is not present. If the $SPLUNK_HOME/etc/passwd file is present, the settings in this file (user-seed.conf) are not used.

USERNAME =

      * Username you want to associate with a password.
      * Default is Admin.

PASSWORD =

      * Password you wish to set for that user.
      * Password must meet complexity requirements.

HASHED_PASSWORD =
* Password hash you wish to set for that user.

[user_info]
USERNAME = admin
PASSWORD = password

25
Q

How to configure indexers to ingest data from forwarders?

A

inputs.conf -»
[splunktcp://9997]
disabled = 0

26
Q

What for is the inputs.conf files in Splunk? Name some of it’s attributes

A

Enable monitoring etc., allow indexers to ingest data from forwarders.

[splunktcp://9997]
* This input stanza is the same as [splunktcp://[]:], but accepts connections from any server.
disabled = 0

[default]
host = value in quotes. Use host=foo, not host=”foo”.
* If you set ‘host’ to “$decideOnStartup”, you can further control how splunkd
derives the hostname by using the ‘hostnameOption’ setting in server.conf.
* For example, if you want splunkd to use the fully qualified domain
name for the machine, set “host = $decideOnStartup” in inputs.conf and
“hostnameOption = fullyqualifiedname” in server.conf.
* More information on hostname options can be found in the server.conf
specification file.
* If you remove the ‘host’ setting from $SPLUNK_HOME/etc/system/local/inputs.conf
or remove $SPLUNK_HOME/etc/system/local/inputs.conf, the setting changes to
“$decideOnStartup”. Apps that need a resolved host value should use the
‘host_resolved’ property in the response for the REST ‘GET’ call of the
input source. This property is set to the hostname of the local Splunk
instance. It is a read only property that is not written to inputs.conf.
* Default: “$decideOnStartup”, but at installation time, the setup logic
adds the local hostname, as determined by DNS, to the
$SPLUNK_HOME/etc/system/local/inputs.conf default stanza, which is the
effective default value.

27
Q

What is SSL?

A

Secure Socket Layer is a security protocol that was developed by Netscape. SSL used for to establish an encrypted link between a web server and web browser. All the data will be passed between a web server and browser through SSL encrypted link, remains private and secure. SSL allows to protect your confidential information like credit card, debit card information, login credential, security numbers and financial transaction over the web.