Monitoring Console Flashcards
Where should the monitoring console be running?
Should be set up on dedicated host
Where should the Monitoring Console never be set on?
Production Search Heads SHC members Indexers Deployment server with more than 50 clients Deployer sharing with CM
Monitoring console should be a member of:
a Member of all indexer cluster
Monitoring Console should be a search peer of
- All Search Heads (clustered or non-clustered)
- All indexers that are not members of clusters
All other enterprise instances (deployer, deployment server, license master)
Which index contains Operational Data?
_internal
Which index contains Resource Usage
_introspection
Who is able to access the Monitoring Console?
The MC is only visible to users with an administrative role
How does the MC work?
MC utilizes RESTfu(snapshot) and log(historical) searches to check system health.
True or False:
MC is considered a single-purpose monitoring box for keeping track of the state of the Splunk deployment
True
How would you add a MC?
Add the MC as a search head of the cluster
Does the MC directly connect to universal forwarders? What about Heavy Forwarders?
UF: NO
HF: YES
Forwarder Monitoring relies upon log-based metrics and saved searches. On which indexes are these logs stored?
_internal provides info about operational things
_introspection provides info about resource usage
How are roles for Splunk instances determined?
An instance is queried for a list of its current roles. The MC focuses searches/dashboards based upon ITS OWN VERSION of the instance’s “role”
If the MC is peere to a bunch of newly created nodes before their full configuration has been provided, what is it identified as?
indexer
nodes may identify as “indexer” that are not actually doing so, e.g. search heads, before they are given an outputs.conf.
Is there a forwarder role in the MC?
There is no forwarder role. Forwarder information is gathered about them by examining their logs.
What is the REST endpoint to view server roles?
rest /services/server/info
What is the MC role process?
Indexing locally? INDEXER
Other hosts searching it? Search Peer
Splunk started with a serverclass.conf? Deployment Server
Bundle contents created from $SPLUNK_HOME/etc/hcluster/apps? SHC Deployer
How do you convince a host that it is an SHC deployer?
Run this command:
$SPLUNK_HOME/bin/splunk apply shcluster-bundle -action stage
How do you take away the SHC deployer role?
delete this directory and restart splunk:
$SPLUNK_HOME/var/run/splunk/deploy
What are the default search groups provided?
dmc_group_cluster_master: any CMs in the environment
• dmc_group_deployment_server: deployment server
• dmc_group_indexer: any full instance not having an outputs.conf
• dmc_group_kv_store: hosts, typically SH, running KV store
• dmc_group_license_master: any full instance with “self” as the license master
• dmc_group_search_head: any host that is peered to another
• dmc_group_shc_deployer: any SHC deployers in the environment
Where are roles configured for the Monitoring Console?
distsearch.conf
What are the provided roles for Clustering?
dmc_indexerclustergroup_
- All member of an indexer cluster (CM and indexers)
- If a label is provided, it will be show instead of the GUID of the CM
dmc_searchheadclustergroup_
- all members of a search head cluster
- If a label is provided, it will be show instead of the GUID of the SHC
What are the three pieces of information needed for a custom server group?
- Name of the server group
- List of servers
- Default state (true or false)
Example:
[distributedSearch: NYC]
default = false
servers = 192.168.1.1:8089, 192.168.1.2:8089
What field can you use to search a mc group on the monitoring console?
splunk_server_group=
How do you identify that a node was misconfigured in the monitoring console?
check the search.log for the string “optimized out”
Are Health Checks extensible?
Yes
Can checks provided in checklist.conf be RESTful or log-driven
Yes
