When handling large amounts of distributed data sources, having a well-defined concept to store and manage all your measurement data is important. In a modern manufacturing business, many sources of technical measurement data must be acknowledged: Verification and validation data from R&D labs, as well as prototypes’ test data from the field. Last but not least, the continuous stream of End-of-Line tester data from the production floor. Unfortunately, data visibility does not scale linearly with number of devices or nodes or across departments. Instead, large amounts of data tend to become overwhelming rather easily. Trying to run big data analyses on un-curated data sets will most likely not solve these problems right away. As we discussed in earlier posts, understanding what to measure will always be the first priority. Once that has been established, it is time to choose a data management platform. These can range from historians to databases, to big file servers with a traditional folder structure or SCADA systems in industrial production environments.
While a single test fixture’s data in a lab will be easy to handle even without a proper measurement data management system in place, things look very different when attempting to manage the data of an entire production floor. When you need to integrate multiple production sites or interface to third-party tools, measurement data visibility quickly becomes a challenging task. Implementing new ways to visualize and manage your measurement data are medium to long-term projects and therefore should be part of your strategic investments.
The hardest part is to define and limit the scope of a data visibility project before getting started. We all get excited about new tools and ideas to try out. However, thinking big right from the start makes it hard to ever get started in the first place. Therefore, a multi-phased approach is typically the wisest choice. Pick a smaller example application (for instance, start with monitoring one test fixture in the lab or on the production floor) and work your way up from there. Even when larger goals are defined from the start, such as visualizing all production data, implementing a pilot system first will allow valuable insight helping to finalize the scope and shape of the overall solution over time.
Implementing a data management solution is all about the technical details: Whether you would like to visualize your data with a traditional SCADA system, an online monitoring solution or custom software, you will have to select a solution provider and a platform. Here, another decision to make is whether to host the data in-house or in a secure cloud service, which largely depends on your company’s IT strategy. One important question to ask here, is how much flexibility the platform will allow: Does it run in the cloud only? Can it be hosted locally? Can it use a hybrid cloud approach?
Another aspect to observe is the solution’s underlying architecture: Is it based on a traditional SQL database? Does it only index flat files and serve them? Is it a document-oriented database that allows organic evolution of the data storage platform? These questions become important when your internal customer (i.e. end user) needs to work with a specific client to access the data. This could be a browser-only thin client, an app or a rich thick client that might include additional analysis or processing functionality. Make sure you know what requirements the data management solution has towards the visualization client. Can it be run in a browser? Does it need certain software, plugins or runtimes? Modern measurement data platforms will have a well defined RESTful interface that allows to integrate other data analysis and processing services or data archiving platforms. For instance, when using specialty software such as NI LabVIEW or NI DIAdem, make sure your chosen solution offers a plugin for that software or is easy to integrate.
These considerations raise the question of how flexibly your chosen platform can be used. Lastly, does your integration partner provide the know-how on how to fit all the pieces of the puzzle together?
Implementing a new strategy to visualize your measurement data will not be an easy task and it should not be underestimated. Consulting with experienced professionals should be the first step of many. As mentioned before, implementing such a strategy will take endurance. Make sure you can establish a proof-of-concept before taking on the biggest challenges. Start with that one test fixture, then scale up.
Professional consulting partners with expertise in data management and visualization can help integrate existing platforms, devices, testers or databases. They can also help define a new strategy from scratch and implement a stable and scalable storage platform. This will provide you the benefits of better data insight, quicker time-to-market, more transparency in failure-analysis and an improvement in overall performance monitoring and tracking.