It basically transforms data into events, stores and adds them to an index, which in turn enhances searchability. The Splunk Indexer is used for indexing and storing the data that is received from the Splunk Forwarder. In carrying out all these activities, the Splunk Forwarder consumes less processing power than other traditional monitoring tools. The Splunk Forwarder collects all of the log’s data and sends it to the indexer. The Splunk Forwarder is used to collate real-time data so as to enable real-time data analysis by the users. Now let us understand the meaning of all these components so as to better understand the entire Splunk Architecture. The Splunk Architecture comprises three main components. Splunk takes valuable machine-generated data and converts it into powerful operational intelligence by delivering insights through reports, charts, and alerts. Splunk can be used as a monitoring, reporting, analyzing, security information, and event management tool among other things. You can troubleshoot any issue by resolving it with instant results and doing an effective root cause analysis. Splunk is an absolutely fast engine and provides lightning-fast results. The best part of Splunk is that it does not need any database to store its data as it extensively makes use of its indexes to store the data.

Splunk can effectively handle big data with no decrease in performance. It can be thought of as a data mining tool for big data applications. Splunk is used for extracting value out of machine-generated data. Comparison criteriaĮLK (ElasticSearch, Logstash, and Kibana) Hope that helps.First, let us compare Splunk with the ELK (Elastic, Logstash, Kibana) stack which also does a similar kind of job. In my personal experience the cost/effort involved in setting up splunk is not worth it for smaller workloads, whereas the AWS Cloudtrail/Glue/Athena would be less expensive setup(comparatively).Īlternatively you could look at something like sumologic, which has better integration with cloudtrail as opposed to splunk. But analysing the log could require you setup Glue crawlers and you might have to use AWS Athena to run SQL Like query.

Cloudtrail on the other had is available out of the box from AWS, the setup is quite simple and straight forward. Splunk definitely is superior in terms of proactively monitoring your logs for unusal events, but getting the cloudtrail logs across to splunk would require some not so straight forward setup (Splunk has a blueprint for this setup which uses AWS kinesis/Firehose).

Well there are clear advantages of using either tools, it all boils down to what exactly are you trying to achieve with this i.e do you want to proactive monitoring or do you want debug an incident/issue. Lastly, if you already use Splunk Cloud in your enterprise and are looking for a consolidated view then, AWS Config is supported by Splunk Cloud as per their documentation too. If you have multiple AWS Account in your organization and want to detect changes there: You can now publish the configuration of third-party resources, such as GitHub repositories, Microsoft Active Directory resources, or any on-premises server into AWS Config using the new API. Here is a list of supported AWS resources types and resource relationships with AWS ConfigĪlso as of Nov, 2019 - AWS Config launches support for third-party resources. Config continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of recorded configurations against desired configurations. For continuous monitoring and detecting unusual configuration changes, I would suggest you look into AWS Config.ĪWS Config enables you to assess, audit, and evaluate the configurations of your AWS resources.

Splunk logs meaning