In this blog we are going to cover how to improve the performance of your Helical Insight instance.
- Optimal hardware: Based on the number of users who are accessing the application concurrently, the amount of data that is being transferred as well as the number of users who are using export the hardware sizing can be optimally decided. The hardware sizing will also depend on whether any middleware like Apache Drill, Dremio, etc are being used. People who are exporting the data also consume a huge amount of memory since with export of raw data al the dataset needs to be exported.
- JVM: Based on the hardware allocated to Helical Insight it is also important to allocate the appropriate JVM to the application. Try to give as much as possible JVM to the application. How to Increase JVM Memory in Helical Insight Application
- Caching: We are having a caching functionality with cache timeout defined as 24 hours. If you think that yoru reporting db/DW is having new data refresh even latter than 24 hours then you can increase the timeout further. Cache Memory Customization
- Raw Data Export: Charts generally have aggregated dataset, so exporting them does not lead to a lot of load on the BI server or DB server. Whereas there might be tabular reports as well which can have really huge amount of data with no aggregation applied. Though when being viewed at the frontend there is pagination which only shows first 10 records whereas the other set of records is being fetched and loaded in the subsequent pages.
- Load Balancing: Helical Insight application also supports Load balancing. You can have multiple servers between which you can distribute your load. You can refer to the below link to understand how to implement load balancing across multiple servers. Implementing High Availability And Load Balancing In Helical Insight
- Database Tuning: Generally whenever a person is viewing the report we are depending on the performance of the DB to give us the aggregated response to show the data at the frontend. Hence having a data warehouse / an optimized reporting database with Indexes etc will automatically help in making sure that the queries are very fast, thus the reports will also be rendered very fast.
- Inmemory Mode: Helical Insight can also be used in an In-memory mode wherein it is possible to load the dataset required for reporting from your transactional DB into a local columnar fast DB (like Presto, Spark, etc) and in turn Helical Insight will connect to this one. Since everything is present locally in RAM the performance is extremely fast here.
First thing which should be done is to increase the hardware sizing in conjunction with the amount of data and the number of users. Please make sure that the server allocated is dedicated hardware. A Linux server is always more preferable than a windows server since it is not having a GUI and thus automatically the performance is faster.
You can always reach out to email@example.com to help you understand what could be the correct hardware sizing for your use case.
Even at report level also you are having cache timeout refresh option. Usage of Cache Refresh Script
Note: Disabling cache memory, or having cache refresh for each and every report at a very short span of time will lead to performance degrading.
Whereas if a person clicks on the export of this report, the entire data is to be downloaded from the DB server and then downloaded to the local laptop. Hence in those cases, this can result in performance issues. Hence the data export should be limited to only a few users / should be scheduled for a time when the load is lowest via HWF (Helical Workflow).
You can reach out to us at firstname.lastname@example.org to know more about any of the steps mentioned above.