Offloading Splunk log-streams to Hadoop
Overview
Industry
Location
Financial services
Poland
Technology Used
Ambari
Ambari Metrics
Ansible
Apache Hive
Apache Impala
Apache Spark
Git
Hadoop
Jenkins
Kudu
Ranger KMS
Scala
ZooKeeper
Show more
Project overview
To reduce the costs of Splunk usage in its organization, our client decided to offload parts of log traffic via forwarder and store it in Hadoop in a queryable manner. Our team developed a reusable solution to handle different types of logs, ingest them, parse them and store them in a selected Big Data storage system, according top lanned data utilization.
Client background
A Polish bank that provides retail, corporate, investment banking, and other financial services. Our client is frequently recognized as one of the most innovative banks in Poland.
Goals
Creating a solution that handles large traffic volumes. Processing a wide variety of logs. Dealing with constantly changing log patterns across different log streams. Focusing on delivering a reliable solution and avoiding data loss.
Results
Reduced costs
Reducing the expenses of Splunk usage in the organization by building a Splunk log archiving system for specialized engines.
Enhanced data quality
Identifying and eliminating redundant or duplicated data and guaranteeing completeness of information.
We'd love to hear from you!
Fill out the form - we'll get back to you as soon as possible
