--- title: What is ELK stack? description: Learn more about ELK, which is quickly overtaking existing proprietary solutions and increasingly the first choice for log analysis and management. image: https://www.heady.io/hubfs/Assets%20June%202020/Blog%20Images/Group%20169@2x.png --- **NEW LAUNCH: **Weill Cornell Medicine's [Maya app](https://www.heady.io/work/maya) is now available for Android! ![Close Bar](https://www.heady.io/hubfs/Group%2040042.svg) [![Heady_Logo_Web-01 (1)](https://www.heady.io/hs-fs/hubfs/Heady_Logo_Web-01%20(1).png?width=130&height=78&name=Heady_Logo_Web-01%20(1).png) ](https://www.heady.io/) The ELK Stack (Elasticsearch, Logstash, and Kibana) is the world’s most popular open-source log analysis platform. ELK is quickly overtaking existing proprietary solutions.. [Software Engineering](https://www.heady.io/blog/tag/software-engineering) # ELK Stack Explained and How to Configure Logging ![Deepak Poojari](https://www.heady.io/hubfs/Assets%20June%202020/Blog%20Images/Oval@2x.png) Author ** Deepak Poojari ** Published On ** Apr 30, 2020 ** Share this article - - - - ![featured_ELK_stack](https://www.heady.io/hubfs/Assets%20June%202020/Blog%20Images/Group%20169@2x.png) Share this article - - - - ## What is ELK Stack? The[ ELK Stack](https://www.elastic.co/what-is/elk-stack) (Elasticsearch, Logstash, and Kibana) is the world’s most popular open-source log analysis platform. ELK is quickly overtaking existing proprietary solutions and becoming The first choice for companies shopping for log analysis and management solutions. **ELK stack** - **Elasticsearch**: Stores and indexes transformed data from Logstash. - **Logstash**: Collect logs and events data. Also parses and transforms data and sends it to Elasticsearch. - **Kibana**: A visualization tool that runs alongside Elasticsearch to allow users to analyze data and build powerful reports.   ## Why ELK Stack? There are many [logging tools](https://stackify.com/best-log-management-tools/). But options like Loggly, Sumo Logic, and others are expensive when compared with maintaining an ELK Stack, and they don’t necessarily have more or better features. [Top companies](https://logz.io/blog/15-tech-companies-chose-elk-stack/) that use ELK Stack include Netflix, LinkedIn, Stack Overflow and others. This shouldn’t be surprising, if we consider all of the critical capabilities and services that that this one stack provides: 1. A central logging system for all microservices, with real-time logging analytics and alerting system. 2. Simplified Scales deployment, vertically and horizontally. 3. A data visualization tool to capture and display analytics, i.e. new customer acquired in a day, API fails after a new release and more.   ## Why is logging so important? With the growth of microservices and server data, logging is increasingly important. It’s critical for diagnosing and troubleshooting issues for optimal application performance. Plus, many tools make it possible to get critical business metrics and data from logs. Logging is no longer just for finding issues. It’s also for monitoring your systems.   ## Configuring and using ELK Stack If you’re ready to optimize your logging and log analysis processes, here’s what you need to do to get started with ELK Stack. ** Requirements** Java 8 Homebrew 2.X.X ** Installation** 1. Elasticsearch brew install elasticsearch && brew services start elasticsearch 2. Logstash brew install logstash && brew services start logstash 3. Kibana brew install kibana && brew services start kibana ![1_BlTVuYs_yzh9Dh3gZaAZow](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_BlTVuYs_yzh9Dh3gZaAZow.jpeg?width=900&name=1_BlTVuYs_yzh9Dh3gZaAZow.jpeg) System Architecture for Logger service **Configure Kibana to start visualizing logs. ** Open the Kibana configuration file and uncomment server.port and elasticsearch.hosts for Kibana to start listening on 5601. `sudo vi /usr/local/etc/kibana/kibana.yml` If you have successfully installed ELK Stack, you should see the Kibana status as green on [http://localhost:5601/status](https://www.heady.io/blog/elk-stack-explained-and-configuring-logging#). ![1_ccUkzkjUWvQqs8gtVdo2JA](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_ccUkzkjUWvQqs8gtVdo2JA.png?width=900&name=1_ccUkzkjUWvQqs8gtVdo2JA.png) **Start Sending data to elasticsearch using logstash** Copy this file to */usr/local/Cellar/logstash/7.6.1/libexec/config/syslog.conf.* This is the path of the configuration file for Logstash where you can configure the file path for all your log files, service logs, system logs and ngnix logs—and then transfer or send as-is to Elasticsearch. ![1_zeZZo1grrze1mafjS03ASQ](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_zeZZo1grrze1mafjS03ASQ.png?width=1000&name=1_zeZZo1grrze1mafjS03ASQ.png) ELK Stack ![1_4DqKluawgUMn2vUiJZd13w](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_4DqKluawgUMn2vUiJZd13w.png?width=900&name=1_4DqKluawgUMn2vUiJZd13w.png) Next, verify your configuration file. `/usr/local/Cellar/logstash/7.6.1/bin/logstash — config.test_and_exit -f /usr/local/Cellar/logstash/7.6.1/libexec/config/syslog.conf` To reflect new config file, restart Logstash. `/usr/local/Cellar/logstash/7.6.1/bin/logstash -f /usr/local/Cellar/logstash/7.6.1/libexec/config/syslog.conf` **Viewing and querying logs in Kibana** On the management tab in Kibana, you’ll see the new index syslog-demo created by Logstash. ![1_w499uMe3ntdJ26s3os2_1A](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_w499uMe3ntdJ26s3os2_1A.png?width=900&name=1_w499uMe3ntdJ26s3os2_1A.png) To define the index pattern to receive data from Elasticsearch, we will use the timestamp, as it can be helpful to visualize logs in chronological order. ![1_4HCG6dwfNq4UE_A67nzurA](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_4HCG6dwfNq4UE_A67nzurA.png?width=900&name=1_4HCG6dwfNq4UE_A67nzurA.png) Click on Create index pattern, and you are done. You can visualize all your log files. You can use text search on your logs, by adding filters or direct text-based search. ![1_DC9x7iyYb1cDK1NzxOhy0A](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_DC9x7iyYb1cDK1NzxOhy0A.png?width=900&name=1_DC9x7iyYb1cDK1NzxOhy0A.png) On top of these logs, you can create various visualization for monitoring critical business metrics like daily/weekly/monthly signups, registrations, subscriptions and more on your server. You can also monitor any anomalies in your server. ![1_uWpbKay8nCSOeLwZ_0Cc4g](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_uWpbKay8nCSOeLwZ_0Cc4g.png?width=900&name=1_uWpbKay8nCSOeLwZ_0Cc4g.png) **Next steps** 1. Dockerize ELK Stack. 2. Use Kafka for stream processing logs. 3. Use [ElastAlert](https://github.com/Yelp/elastalert) for alert notification on Elasticsearch. ![1_3XLwyS-0wp_qWag7U4JZ7g](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/1_3XLwyS-0wp_qWag7U4JZ7g.png?width=900&name=1_3XLwyS-0wp_qWag7U4JZ7g.png) What’s next [Load Testing ELK Stack with 1M Hits](https://medium.com/make-it-heady/load-testing-elk-stack-with-1m-hits-a05e6bc4d216). You can also read more about ELK Stack on [Github](https://github.com/deepak6446/elk_stack).   ![Group 169@2x](https://www.heady.io/hs-fs/hubfs/Assets%20June%202020/Blog%20Images/Group%20169@2x.png?width=732&height=601&name=Group%20169@2x.png) ## Interested in a career at Heady? Excellent! We are always looking for great talent. Book a call ![app-clips_1](https://www.heady.io/hubfs/Assets%20June%202020/Blog%20Images/app-clips_1.png) ![app-clips_2](https://www.heady.io/hubfs/Assets%20June%202020/Blog%20Images/app-clips_2.png) ## Interested in a career at Heady? Excellent! We are always looking for great talent. Book a call ###### LET'S TALK IT OUT. ## Figuring out your next step? We're here for you. ## Or drop us a line. [hello@heady.io](mailto:hello@heady.io) Our emails are *(almost)* as cool as our digital products. [![Heady_Logo_Web-01 (1)](https://www.heady.io/hs-fs/hubfs/Heady_Logo_Web-01%20(1).png?width=75&height=45&name=Heady_Logo_Web-01%20(1).png) ](https://www.heady.io/) Your phone will break before our apps do. ###### Brooklyn 20 Jay St Suite 1018 Brooklyn, NY 11201 ###### Mumbai WeWork Vikhroli Mumbai, India ###### Online - - - ###### Hit us up! [hello@heady.io](mailto:hello@heady.io) [646.480.0489](tel:6464800489) - - - ![Badges-24](https://www.heady.io/hubfs/Badges-24.png) ![Badges-24](https://www.heady.io/hubfs/Badges-24.png) ###### Brooklyn 20 Jay St Suite 1018 Brooklyn, NY 11201 ###### Mumbai WeWork Vikhroli Mumbai, India ###### Online - - - --- © 2026, Heady LLC. [About](https://www.heady.io/about)[Careers](https://www.heady.io/careers)[Newsroom](https://www.heady.io/newsroom)[Media Kit](https://www.heady.io/media-kit)[Privacy Policy](https://www.heady.io/privacy-policy)