#FACEBOOK TECH STACK SOFTWARE#
The term is sometimes applied to marketing services (martech stacks) or sales services (sales stacks), but it originated in the software development community. This is Facebook’s ‘tech stack.’ĭevelopers talk about tech stacks because it makes it easy to communicate lots of information about how an application is built. The social site Facebook, for example, is composed of a combination of coding frameworks and languages including JavaScript, HTML, CSS, PHP, and ReactJS. DefinitionĪ technology stack, also called a solutions stack, technology infrastructure, or a data ecosystem, is a list of all the technology services used to build and run one single application. Each tool in your stack creates, analyzes, or ingests data, and to run most efficiently, those data sources need to link to one another. Examples of popular services that consume from Kafka include ksqlDB, Materialize.io, and any system capable of consuming from Kafka with open-source Kafka connectors.Developers can’t manage a technology stack unless they know what’s going on, which is why an analytics platform, like Mixpanel, is such an important part of the tech stack. This is a great way to build alerting functionality, power event-driven architectures, spin up an integration, or simply back up your data. This data is also processed and stored securely within seconds.Īfter data is processed and enriched by Keen, it can be streamed to any external system in real time via the Kafka Outbound Cluster and a standard Kafka Consumer. This all happens within seconds.Īlternatively, for teams that already have a Kafka-based event pipeline or systems that are compatible with open-source Kafka connectors, it will be easier to stream data to Keen via our Kafka Inbound Cluster. Events are validated, queued, and optionally enriched with additional metadata like IP-to-geo lookups. On the top rows (the ingestion side), there are 2 methods of sending data in: the HTTP Stream API and via our Kafka Inbound Cluster.įor the HTTP Stream API, load balancers can handle billions of incoming post requests a week as events stream in from apps, web sites, connected devices, servers, billing systems, etc. Our customers capture billions of events and query trillions of data points daily.Īlthough a typical developer using Keen would never need to know what’s happening behind the scenes when they send an event or run a query, here’s what the architecture looks like that processes their requests. With APIs for streaming, storing, querying, and presenting event data, we make it relatively easy for any developer to run world-class event data architecture, without having to staff a huge team and build a bunch of infrastructure. It provides big data infrastructure as a service to thousands of companies. Keen.io is an event data platform that my team built. Here’s a simplified view of their data architecture from the aforementioned post, showing Apache Kafka, Elastic Search, AWS S3, Apache Spark, Apache Hadoop, and EMR as major components.
They employ over 100 people as data engineers or analysts. At peak hours, they’ll record 8 million events per second. As their engineering team describes in the Evolution of the Netflix Data Pipeline, they capture roughly 500 billion events per day, which translates to roughly 1.3 PB per day. With 93 million MAU, Netflix has no shortage of interactions to capture. But, if you’re curious about what it would be like to be a giant, continue on for a collection of architectures from the best of them. We built Keen.io so that most software engineering teams could leverage the latest large-scale event data technologies without having to set up everything from scratch. They’ve invested millions into their data architectures, and have data teams that outnumber the entire engineering departments at most companies. Their work sets new standards for what software and businesses can know.īecause their products have massive adoption, these teams must continuously redefine what it means to do analytics at scale. We continue to be amazed by the data engineering teams at Facebook, Amazon, Airbnb, Pinterest, and Netflix. That certainly seems to be the case at the world’s leading tech companies. Here at Keen.io, we believe that companies who learn to wield event data will have a competitive advantage. This article was originally published in April 2017 and has been updated.
0 kommentar(er)
