Splunk Stream Tips and Tricks
Introduction
Splunk Stream lets you capture, filter, index, and analyze streams of network event data.
A "stream" is a grouping of events defined by a specific network protocol and set of fields. When combined with logs, metrics, and other information, the streams that you capture with Splunk Stream can provide valuable insight into activities and suspicious behavior across your network infrastructure.
Use Splunk Stream to:
Passively capture live streams of network event data.
Capture metadata and full packet streams for multiple network protocols.
Collect NetFlow protocol data.
Apply aggregation methods for statistical analysis of event data.
Apply filters to minimize indexer requirements.
Extract content from strings and generate hashes.
Extract files from network traffic.
Monitor network trends and app performance in pre-built dashboards.
Deploy independent Stream forwarder to capture data on remote linux machines.
Scale rapidly and unobtrusively with no need for tagging or instrumentation.
Splunk Stream is a purpose-built wire data collection and analytics solution from Splunk. Splunk Stream can be one of the most robust products Splunk offers as a free addition to your Splunk Enterprise environment.
However, some of us know that Splunk Stream can be sometimes really complex to setup and utilize to its full potential. With that in mind, let’s jump into some tips and tricks of the trade for working with the Splunk Stream.
1. One Simple REST Call
The Stream REST API is a powerful function, and one simple REST command can help you power through configuring Stream Forwarders. One of the most common errors that is seen when deploying a Stream Forwarder is “Unable to ping server.” At times it can become difficult in determining whether this issue lays within your configuration or a network configuration.
Utilizing the following curl command helps determine whether you have the correct App location: curl http://<stream_app_server>:8000/en-US/custom/splunk_app_stream/ping
Using this command before deploying the Stream Add-on, or Independent Stream Forwarder, can help determine if the Stream Forwarder can access the Stream App within your deployment.
2. Independent Stream Forwarder or Stream Add-on?
Planning a new deployment, or the addition of a forwarder can spring the above question, should I install an ISF or the Stream TA on a Universal Forwarder? The answer to this can vary by environment and collection method. But as with any Splunker, I love my data!
Click here to access the Universal Forwarder and Stream TA.
Click here to access the Independent Stream Forwarder.
From the above charts you can start to compare the performance benefits of the ISF. Although your environment may never reach the ingestion rate at which you start to see dropped events from the Universal Forwarder, it is a peace of mind knowing that your forwarder can handle considerable amounts of data
3. Hunting Down Suspicious Subdomains using URL Toolbox
You can perform some simple Stream hunting just utilizing DNS data. With DNS data from Stream you can start to investigate suspicious DNS queries and subdomains from within your environment. You can empower your investigations by utilizing this URL Toolbox link.
For instance, if you perform a Splunk search for your stream:dns data, then after populating the query value you can pass the queries to the URL Toolbox. This allows you to filter out URLs that you know are not suspicious and ones that don’t have a Top Level Domain. You can take this a step further by utilizing the URL Toolbox to calculate entropy values of the subdomains, and sort to see the highest scores. (The higher the score, the more randomized the URL is) Taking these scores into account, you can start digging into specific IP investigation.
4. Splunk Stream on a Raspberry Pi
Of course it can work! One Splunk engineer put the Independent Stream Forwarder to the test to see how light-weight it really is. The Raspberry Pi is a cheap and easy way to play around with the possibilities of Splunk Stream. You could even implement this at home environment to add even more capabilities to your own lab environment. In fact, here is a link to the Splunk forwarder for Linux ARM download, which is installed on the Raspberry Pi for Splunk forwarder capabilities.
This is a great example of the power of a Stream Independent Forwarder. The Raspberry Pi in my home environment is currently running as a Pi Hole, but I am going to implement the Streamfwd to run some searches and create dashboards of the queries and how the Pi Hole handles them.
TRADEMARK LEGAL NOTICE All product names, logos, and brands are property of their respective owners in the Austria or other countries.All company, product and service names used on this website are for identification purposes only. Pheniix is notaffiliated with or an official partner of Cisco, CompTIA,Dimension Data, VMware, Amazon, Microsoft, Certified Ethical Hacker, (ISC)², Juniper, Wireshark, Offensive Security,Google, GNS3, F5, Python, Linux, Java, Openstack, Vagrant, Ansible, Docker, GIT, , Blockchain or other companies.Use of these names, logos, and brands does not imply endorsement.The opinions expressed on pheniix are personal perspectives and not those of Cisco , Dimension Data or any other company. Pheniix runs as an independent blog.
#DNS #Pheniix #Splunk #Bigdata #UniversalForwarder #RaspberryPi #SplunkStream #RESTAPI #Linux #SplunkEnterprise