Data-driven growing is exactly what it sounds like: utilizing and analyzing data for an optimized cultivation process.  After collecting data for more than 5 years, we’ve learned a thing or two about data storage. How do you effectively store billions of data points whilst still being able to search through and visualization this data in real-time? With Christmas on the doorstep we wanted to share some of our learnings as a present to you. Partner-up on data management? Plan a meeting with Jasper. 

Use the right technology 

When offering data solutions it’s a fine line between data storage costs and lightning fast accessible data for clients. One of the key elements is using cutting-edge technology to store and search data. In the dashboards it is possible to search through and visualize millions of data points milliseconds. With Elasticsearch we’re able to deliver this functionality to our users at scale. 

Move data to the right tier

While optimizing cloud storage costs there are some obvious wins. Think of regularly monitoring active machine instances or a recurring (automated) database cleanup process. Besides, there are other nuanced strategies to enhance storage savings. One of which is the use of the data tiers. There are four different elastic data tiers: Hot, Warm, Cold and Frozen.  When storage gets “colder” the costs drop, but the query speed also diminishes.

This comes back to the fine line between reducing storage costs and fast searchable data. We’re constantly monitoring this trade-off to find the right balance between pricing and query speed. In the end our clients should experience fast data visualizations, without any discernible impact on speed.

Want to partner-up on data management? Schedule that meeting with Jasper or reach out via j.vanderauweraert@30mhz.com.