Splunk is the SIEM platform I have been working the most with recently. It is a joy to work with because it has embraced the development paradigm, and organizes standard and user-developed content into apps. And it lets you write code in proper languages like Python or Java to process the data in ways that go beyond their own query language.
Which brings me to how it is that you can develop and test an app for security devices and environments you don't personally have access to. Actual sample log data is not easy to come by. And even if you do find it some Splunk apps expect the log data to be ingested in particular ways (e.g. syslog), and replaying them from a file on disk is not always trivial.
This is a concern both if you are writing a new app that will be used by others, or if you want to ensure that your internal organization's apps are able to correctly handle log entries that are infrequent or that you might not have encountered before. For example, the Network Intrusion CIM model does not specify a finite list of possible values for the
action field. And until you see some test examples you won't know that the Check Point OPSEC TA field extractions will have the service name instead of the port number in the
dest_port field of the Network Traffic CIM model. So you'll need to get a representative set of events and find what out values the TAs actually generate if you want to ensure your content will adequately survive its first encounter with with real events out there in the wilderness.
Looking into this for the Niddel app I stumbled upon a very useful tool called eventgen. Basically what I have found is that most Splunk-developed apps, and many developed by third parties, are shipped with configurations that allow you to easily generate sample log data that you can then use to test your content.
The eventgen tutorial is very thorough and the tool is very well documented. However the documentation is almost exclusively focused on app developers which want to support event generation in their apps. Very little is said about how to simply use the functionality the app developers already built-in. So let me try to help fill in the gaps here with the TL;DR version, which I only recommend for development-only Splunk instances:
First, confirm if the particular app or TA you are interested in has that configuration: go to its directory under
$SPLUNK_HOME/etc/apps(or download it from SplunkBase and check within the archive) for a
samplesdirectory and a
default/eventgen.conffile. If it's there, you're in luck.
Then, ensure that the apps or TA for which you want to generate events os configured to share their content globally, or the eventgen app (installed later) won't be able to access its sample data and configurations. The simplest way to do this is on Splunk Web is to navigate to
Manage Appsand click on
Sharingcolumn for the app you want events from. Make sure the
Sharing for config file-only objectsoption is set to
The new eventgen app does not have a setup page when you go to it under
Apps, just statistics on the generated events. So the only way I found to enable and disable it is to go to
Manage Appsand click on
Set Upon the row of the eventgen app. Mark and unmark the
Enable ./bin/eventgen.pycheckbox to enable/disable event generation.
Once this is done, eventgen will inspect all other installed apps it can access (hence the permissions step above) and generate fake events for them according to their configurations, usually directly to the
main index. No easy way to choose which apps to generate events from at this time, unfortunately.