INTERVIEW PREP Flashcards
What’s minimum replication/search factor for idx cluster and sh cluster? What is replication and search factor?
sh default 2
Default ports?
8000 - web port 8191 - KV store port 8080 - Replication port (idx cluster) 8181 - search head cluster replication 9997 - receiving porg 514 - syslog port 8089 - mgmt port 8088 - hec
How to set up search head cluster, idx cluster, multisite cluster?
IDX cluster:
set up replication factor, search factor, security key, and cluster label in the server.conf of the cluster master
set up manager ip, mode, and pass4symmkey on the peer nodes and search head
SH cluster:
on the deployer set up pass4symmkey, and also label sh cluster;
initialize splunk init shcluster-config command, or use server.conf to set it up
select captain;
How to push configurations from deployers and indexer clusters?
IDX cluster: splunk validate cluster-bundle --check-restart splunk apply cluster-bundle SH clusteR: splunk apply shcluster-bundle
Where to place the configuration bundle on the deployer, DS, and CM?
$SPLUNK_HOME/etc/shcluster/apps
$SPLUNK_HOME/etc/deployment-apps
$SPLUNK_HOME/etc/master-apps/
What is the comapny name?
Symphtech
Splunk web not working and the KV store has errors how do you troubleshoot?
- see mongod.log and splunkd.log, remove mongod.lock file, see permissions for splunk.key
Architect the growth of the cluster while ensuring the
necessary storage requirements are met and retention
policies are set appropriately.
1
Schedule cronjobs using Cron Utility to automate deletion
of syslog data
1
Build syslog servers for network log management
1
Manage Logical Volume Manager (LVM)
1
Troubleshoot disk usage and memory issues on Splunk
indexers
2
Monitoring system health via Monitoring Console and
custom searches and visualizations.
2
Build custom TAs and Apps to ingest and visualize data
from various data sources
2
Review onboarded logs during last phases to ensure no PII
exist, using regex and transforms.conf
2
Manage ingestion on our license by reviewing verbose logs
for necessity of indexing; eliminating extraneous logs by
sending them to null queue or truncating logs
2
Developed 12 dashboards for Financial Board members
annual presentation
2
Built a series of Windows Security Dashboards that included
but are not limited to, Disabled AD Accounts Re-enabled,
AD password change attempts, Brute Force Attacks, Policy
Changes, Changes to Windows Firewalls, Privilege Use, etc.
2
Use advanced dashboard XML and CSS customizations to
achieve specific look on reports
2
Wrote SOPs and established runbooks to document
procedures during incidents
2
Manage the use and deployment of knowledge objects like
macros, tags, eventtypes, calculated fields, field aliases to fit
specific standards of use and avoid duplication
2
Ongoing tuning of the CIM for new data onboarding from
various branches of the organization and develop
preliminary data model searches to kick-off the utilization
of the data when handing off to Security Operations team
2
Actively working with end-users to aid in building efficient
search queries
2
Upgraded entire Splunk infrastructure in both dev and prod -
uction environments
2
Administrating Splunk servers for optimal performance
2
Leading log ingestion efforts; formatting back end configurations
2
Standardizing configurations across Splunk environment
2
Cleaning up custom apps on deployment server; implementing naming standards, app.conf, etc.
2
Leading code reviews to deploy new configurations on a weekly basis; granting approvals and
denials
2
Staggered crons to ensure that alerting is not backed up or skipped
2
Optimizing search head; rewriting SPL to improve run-time performances
2
Setting retention periods as a standard within company
2
Troubleshooting failed ingestion, missing logs and outage of Splunk servers
2
Installing forwarders on new servers as part of ingestion work
2
Designing and leading onboarding training for new hires
2
Reassign knowledge objects upon off-boarding Splunk users
2
Configure inputs and drilldowns on dashboards
2
User assistance on query development, troubleshooting, optimization
2
Upgraded entire Splunk infrastructure in both dev and production environments
2
Troubleshooted missing logs
2
Completed wide-scale project to ingest more than 300+ network devices through syslog filtration
ensuring each host is appropriately filtered into correct directories; utilized custom
configurations to bring the data into Splunk; made new ingested data CIM compliant, and
remedied existing non-compliant data
2
What is your experience with ES?
2
Part of Implementation Team building out several Splunk clusters
2
Ingested data to ensure monitoring of network devices, applications, including deployment of Nix
and Windows TAs to collect server logs
2
Performed the ingestion of several security tools, including Nessus Vulnerability scanner,
Acunetix, Malwarebytes and Bluvector; including modifying out of the box apps for use in
destination environment
2
Worked on props, transforms, inputs, authentication, authorize and severclass configurations on a daily basis
2
Performed field extractions, look ups and macros for various data sources
2
Troubleshooted data transference issues within Splunk; troubleshoot outages
2
Addressed misuse of platform to prevent search head failures, rewrite multi-paneled dashboards
to use base searches, remedy resource intensive dashboards, and shorten long running searches
2
Addressed misuse of platform to prevent search head failures, rewrite multi-paneled dashboards
to use base searches, remedy resource intensive dashboards, and shorten long running searches
2
Principal engineer responsible for designing and maintaining production-quality dashboards,customizing XML via Source Code, including development of a collection of dashboards to
support field tech teams on monitoring the 9,000+ cameras, sensors and alert systems for
building access
2
Create savedsearches to aid Splunk non-technical users in simplifying complex queries
2
Create scheduled reports for summary indexing for 90 day trends of alert failures.
2
Created and monitored Logical Volume Manager (LVM) and Swap Space
2
Utilized Vagrant to provision servers for Developers in Testing Environment
2
Configured and administered LDAP, DNS, DHCP, TLS/SSL on Linux servers
2
Administered SSH, NFS and FTP on Linux servers
2
Administered and configured Apache
2
Perform server updates, patching, upgrades and package installations using rpm, yum and wget
2
Disk Partitioning
2
Deployed and managed virtualization technologies; KVM and VMWare
2
Deployed virtual servers using templates
2
Performed RPM and YUM package installation.
2
Checked System Logs to diagnose errors and resolve them
2
Performed system backup and compression using tar, gzip and bzip
2
Provisioned accounts: added new user accounts, removed users, changed ownerships of groups
using chown, chgrp commands. Modified file permissions using chmod, and set special
permissions using ACL.
2
Automated and scheduled jobs using Cron utility
2
Monitored and resolved service desk tickets using Jira Ticketing system
2
Created and updated Process Documentation for future reference
2
Do you have any experience migrating Splunk?
2
What are macros, and how have you used them?
2
Regex: How to match a digit? How to match a word character?
2
Where do Splunk buckets and databases reside?
2
How would you perform index time field extracion?
2
Tell us about yourself
2
What is splunk’s smart store?
2
Could you tell me something about the default configuration files?
2
When heavy forwarder is down but you have access to SSH what do you do? How would you troubleshoot this?
2
Why could a search be inefficient?
2
How would you use existing syslog server to gather the data into Splunk?
2
How would you fix a slow running search?
2
What do you know about props.conf?
2
Difference between hot and warm buckets
2
How would you create an index?
2
Splunk has stopped ingesting the data and we need to fin the crash logs. Which internal logs files would you check and what path does it have?
2
Which takes precedence, local, or default, and why?
2
I see architecting on your resumee, can you talk about your architecting experience?
3
Do you have any questions for us?
3
What is splunk’s smart store?
3
what is splunk s2 s3
3
How would you describe how your environment uses indexes?
3
Experience with data migration?
3
append vs appendcols
3
Can you tell me benefits of a multiside indexer cluster
3
Scenario: How would you fix a problem when all your universal forwarders went down?
3
Why do you lose two IP addresses in a subnet?
3
How to edit a file in Vi mode?
3
How many IPs in a /23 and /24?
3
Have you ever costumized the dashboard and explain how you did it?
3
What is your experience with regex?
3
precedence orders etc
3
What is better: search time or index time field extracion?
3
How would you find big files in Linux?
3
What is TCP and UDP? Why use one over the other?
3
How to determine which process are using the most resources?
3
What is significant about ports under 1024?
3
What are some splunk default indexes?
3
splunk internal logs
3
splunk important paths
3
retention policies?
3
What is the OSI model?
3
API data examples
3
security data examples
3
How to do find and replace in VI mode?
3
ES experience
3
How would you send internal data to the indexers?
3
Difference between tstats and stats. When would you use tstats?
3
What are the retention policies (by attribute names)?
3
How to troubleshoot a dashboard that was once bringing in results but stopped?
3
commands for summary indexing
3
What are the benefits of indexer cluster?
3
What is an alert?
3
Splunk web not working and the KV store has errors how do you troubleshoot?
see mongod.log and splunkd.log, remove mongod.lock file, see permissions for splunk.key
config precedence
3
What is RAID? What is RAID 10?
3
System is full on disk, how do you troubleshoot?
3
What are the transforms.conf deleting specific events?
3
How would you upgrade Splunk?
3
What attribiutes would you use to configure retention policies?
3
How would you check storage on your CLI?
3
How would you add an scripted input?
3
How would you add a network input?
3
Give an example of character types that you can use in regex and what do they mean?
3
Houw would you troubleshoot splunk configuration files?
3
How do you use clean up command in your SPL?
3
What kind of administrative duties you find yourself doing on the GUI?
3
What is a scripted inputs, and what is your experience with them?
3
You’re ingest data through API, how would you do that?
3
What kind of internal fields are in splunk?
_raw, _time, _indextime, _cd, _bkt
Tell me about your experience with knowledge objects. How have you worked with them?
3
What summary index is, and what you ised it for?
3
What syslog data have you onboarded?
3
What is click name and click value etc.?
3
How to restore a frozen bucket?
- Copy your archive bucket into the thawed directory:
- Execute the splunk rebuild command on the archive bucket to rebuild the indexes and associated files
- splunk restart
How do you add index to indexer?
3
What are some commands you often use in creaing content?
3
Have you worked with Splunk ES?
3
What is indexer discovery and how would you configure it?
3
How would you install splunk?
3
What is a distributed search?
3
What are the benefits of a search head cluster?
3
How would you set up a search head cluster?
3
How would you set up an indexer cluster?
3
How does licensing work in splunk?
3
Do you have a process when it comes to working on dashboards for clients?
3
What kind of client facing experiences you’ve had, and what were some challenges that you’ve overcome and how?
3
What is your biggest challenge when it comes to managing search heads?
3
Do you have a process for alert creation, and how do you manage them to make sure they are being responend to?
3
Do you have any experience migrating splunk?
3
How did I initiate script?
3
Preferred flavor of Linux?
3
How would you find big files in Linux?
3
/opt is full how would you deep deeper to diagnosis?
3
How to recall the last command you ran(2 ways)?
3
What is TCP and UDP? Why use one over the other?
3
What is the purpose of a subnet?
3
How to test connectivity on a specific port or remote server?
3
Why do you lose two IP addresses
3
Does your environment currently run multi-side clustering, and what is it, and what’s your experience with it?
3
How would you set up a new syslog server to onboard data into Splunk?
3
What are the transforms.conf for changing host name?
3
What splunk configurations require restart and which do not?
3
How to list all processes?
3
Can you share with use your experience with troubleshooting? (Troubleshooting scenarios)
3
Can you elaborate on some of the command that you use regularly at your job?
3
Wxplain metadata command and give examples on how you have used it?
3
What is the eval command, and how have you used it at your work?
3
How do you limit schedule searches for users?
3
Three ways to perform field extractions in SPL and 3 ways to extract with search head
3
Difference between .CSV and key valuee?
3
Explain transaction command
3
How would you reindex the data?
3
How would you set up file monitoring?
3
What is the btool and how would you use it?
3
What are some dashboards that you have worked on?
3
What was the most recent scripted input that you have added to your environment?
3
- Can you tell me what tools have you used to bring data in?
3
- If timezone is incorrectly configured where would you go to fix it?
3
- How would you know if your configurations are correct?
3
How would you install an app from Splunkbase?
3
When onboarding data how would you brin all you have done from dev to prod?
3
data migration
3
upgrading splunk
3
What is a logical grouping, capture group, and a named capture group?
3
What are some security best practices?
- Educating employees
- principle of least privilage
- updating systems and software often
- documentation,
What is splunk DB connect
Splunk DB Connect is an Add-on which allows you to query RDMBs for data and index the result sets.
Who are the top direct competitors to Splunk?
Logstash, Loggly, LogLogic, Sumo Logi
What is the command used for enabling Splunk to boot start?
$SPLUNK_HOME/bin/splunk enable boot-start
