Using Open Source Software to improve Streaming on the Edge

Author: Apache Pulsar Neighborhood
Published: November 2, 2021

IoT devices are expected to number in the billions, each device enabled by one or more sensors, generating data about the device and its surrounding environment. Analyzing this data using machine learning (ML)/Artificial Intelligence (AI) models, for example, frequently requires transferring this data to the cloud. However for many applications the latency and throughput is too slow. And given that the amount of data being transferred and required custom solutions the cost would be prohibitive.

However, what if you could stream this data using Open Source Solutions, such as Apache Pulsar and Apache Arrow, to compute nodes that were optimized for IoT/Industry 4.0? You would be able to harness the power of Apache Arrow’s throughput and data format, which is understood everywhere (no translation penalty) and it is done “On Wire”; with Apache Pulsar’s end to end encryption, multi-tenant, and geo-replication. Robert Morrow, CEO and founder of SigmaX will walk us through how they combine open source software and optimized computing to give IIoT 20x faster throughput.

Robert is the CEO and Founder of SigmaX.ai. SigmaX.ai has a combined hardware and software approach when it comes to solving Enterprise Data Management problems at-scale. SigmaX builds and extends Apache Open Source Software so that their customers can benefit from the combined development efforts of open source without the obligation of vendor lock-in.

This was a joint event with IoTHub Meetup. You can check them out at https://www.meetup.com/IoT-Hub/