Jun 20
2008Splunk Ninja & Processing Distributed Logs
Filed Under (Cool Sites) by AWS Editor on 20-06-2008
Tagged Under : amazon, cloud, rss, s3
Early this morning, Ilya Grigorik, founder of AideRSS, sent me a short note via Twitter to tell me about his latest blog post.
In the post, he described his use of a single instance of Splunk to process application log files from several dozen Amazon EC2 instances. He also included a bit of Ruby code which illustrates the process of logging data to Splunk over socket connection.
Splunk is a very cool analysis tool for system and application log files. It indexes the logs, makes it easy to search them, lets you create alerts, and even generates some spiffy-looking reports, among other things.
Minutes later, one of my colleagues sent me another blog post related to Splunk. In that post, the Splunk Ninja (motto: "All batbelt. No tights.") demonstrates (in video form) his use of EC2 and S3 to demonstrate Splunk and its log processing tools. The Ninja likes the fact that EC2 offers quick provisioning and scaling, and that he doesn't have to buy anything or to wait for it to be delivered. He does complain that there's no pretty GUI for EC2, so I'll have to tell him about ElasticFox.
– Jeff;
ABOUT